blog.s11n.net

s11n and C++0x

A couple weeks ago i came across PEGTL (http://code.google.com/p/pegtl/) and that got me really interested in some of the new C++0x features. gcc 4.3 has beta support for 0x, and i’ve decided that i will start adding some optional parts to libs11n 1.3.x. While i’m not yet entirely sure that variadic templates will be useful in s11n, i’ve put together some code which shows some of the potential:

#include "s11n.net/s11n/s11nlite.hpp"
#include "s11n.net/s11n/s11n_debuggering_macros.hpp"
#include "s11n.net/s11n/proxy/pod/int.hpp"
#include "s11n.net/s11n/proxy/pod/double.hpp"
#include "s11n.net/s11n/proxy/pod/string.hpp"
#include "s11n.net/s11n/0x/ohex.hpp"

int main(int argc, char *argv[])
{
    int vi = 42;
    double vd = 42.42;
    std::string vs(”i’m a std::string”);
    try
    {
	s11nlite::node_type node;
	bool ret = s11n::cpp0x::serialize_v(node, vi, vd, vs);
        // ^^^ we can pass any number of Serializables here
	if( ret )
	{
	    s11nlite::save( node, std::cout );
	}
    }
    catch(std::exception const & ex)
    {
	CERR < < "EXCEPTION: " << ex.what() << '\n';
	return 1;
    }
    return 0;
}

That outputs:

#SerialTree 1
s11n_node class=s11n::s11n_node
{
        int class=int
        {
                v 42
        }
        double class=double
        {
                v 42.42
        }
        string class=string
        {
                v i'm a std::string
        }
}

This approach has some serious limitations (mainly involving naming of the children serialized this way), but as i get more comfortable with 0x i hope to find new ways to use it to empower s11n more. Your ideas are of course welcomed…

[A short while later...]

i’ve refined the above to de/serialize_group(), which can be used like this:

int main(int argc, char *argv[])
{
    int vi = 42;
    double vd = 42.42;
    std::string vs(”i’m a std::string”);
    try
    {
	s11nlite::node_type node;
	bool ret = s11n::cpp0x::serialize_group(node, “pods”, vi, vd, vs);
	if( ret )
	{
	    s11nlite::save( node, std::cout );
	}
	else
	{
	    throw s11n::s11n_exception( S11N_SOURCEINFO, “serialize_group() failed” );
	}

	int di = -1;
	double dd = -1;
	std::string ds = “error”;
	ret = s11n::cpp0x::deserialize_group(node, “pods”, di, dd, ds);
	if( ret )
	{
	    COUT < < "Deserialized group: ("<< di <<", "<< dd <<", ["<< ds <<"])\n";
	}
	else
	{
	    throw s11n::s11n_exception( S11N_SOURCEINFO, "deserialize_group() failed" );
	}
    }
    catch(std::exception const & ex)
    {
	CERR << "EXCEPTION: " << ex.what() << '\n';
	return 1;
    }
    return 0;
}

In addition to the output shown above, it shows us the results of the deserialization:

0x.cpp:36 : Deserialized group: (42, 42.42, [i'm a std::string])

[And a while later ...]

These variadics are cool. We can now easily serialize multiple child objects more flexibly than the above example:

serialize_subnodes( node, "myInt", di, "myDouble", dd, "myString", ds );

That takes pairs of (string,Serializable) and serializes each object into a subnode with the given name.
We can then deserialize with:

int myi; double myd; std::string mystr;
deserialize_subnodes( node, "myInt", myi, "myDouble", myd, "myString", mystr );

And the order of the (string,Serializable) pairs passed to deserialize_subnodes() is independent of the order passed to serialize_subnodes(). The first functions shown above do not have that property - they require the args in the same order. With deserialize_subnodes() we also have the option of partial deserialization - fetching a subset of the serialized data.

s11n offers “hope” to one developer

Earlier today i came across a blog entry from the 3rd April 2008 at http://www.bloodyaxes.com/ where the author, Rob Stoddard, had some encouraging words regarding s11n:

“I started looking for a serializer recently. … I came upon S11n. S11n has a hefty 121 page manual attached to it, as well as numerous code examples that are free for your viewing. I was starting to think that the OO crowd was hopeless, but S11n offers a good deal more than a shred of hope.”

Finally someone who appreciates library manuals!!!

:D

New s11n APT/.deb repository

At long last i’ve been tinkering with the .deb packaging tools. After some degree of effort i’ve been able to generate .deb packages for Ubuntu Gutsy and Nexenta/GnuSolaris.

Earlier today i set up an APT repository for distributing s11n via the APT tools. It’s over at apt.s11n.net - just browse over there to get the instructions (e.g. the list of APT repo URLs).

The repo currently has s11n 1.3.0 for Ubuntu Gutsy and Nexenta/GnuSolaris. The plan is to create packages for the latest s11n 1.2.x, and possibly add separate packages for the docs (API + manual).

Contributions of .deb packages for platforms i don’t have access to is always welcomed. Just get in touch with me via http://s11n.net/home/stephan/.

Update 26 Apr 2008: i’ve gone ahead and released 1.2.6, including .debs for Gutsy and GnuSolaris.

libs11n .deb packages for *buntu and GnuSolaris

The first .deb packages for libs11n 1.3, targeting i386 *buntu/Debian systems and i386 Nexenta Gnu/Solaris have been released: http://s11n.net/download/. Contributions of new precompiled binaries/packages for other systems are always welcomed!

These releases contain the headers, the DLLs, and the s11nconvert binary. They don’t contain the doxygen API docs nor the library manual - those might be shipped in their own packages someday.

libs11n on OpenSolaris

Yesterday i came across NexentaOS, which is an OpenSolaris kernel in an environment hosting the GNU toolset and the Debian apt tools (which are, IMO, the best package management tools out there). Nice stuff, and Nexenta is currently the only OpenSolaris distro which i would personally consider installing (mainly because of its GNU tools base and the apt tools).

Anyway… it turns out that libs11n will compile as-is on NexentaOS (which uses an antiquated gcc 4.0.x). i guess it’s about time to start building Debian-format source packages for s11n.

The GNU GPL: Free as in Will?

[Achtung: if you are easily offended when people bad-mouth the GPL, and claim that anyone who speaks agains the GPL "doesn't get it", then please do not read this post. i'm sick to no end of people claiming that i "don't get what the GPL is about" just because i speak out against it. They're getting to be as bad as Apple fanatics have been for years.]

Let’s talk for a moment about the GNU General Public License (GPL), arguably the most common license applied to Open software. Without it, the whole Free/Open Source Software (FOSS) movements and cultures which we know today very probably would never have arisen. Instead, the world of free licenses, e.g. the venerable BSD- and MIT-style licenses, would probably be common within academic circles but would probably never have been adopted by [m]any mainstream projects. Why not? Quite simply because they are not viral, as the GPL is. Once the GPL is used as a license, it is neigh impossible to separate that code from the GPL, short of reimplementing it from scratch (except in the case of single-author projects, where the author is free to change the license on future versions but cannot retroactively change it on versions released under the GPL).

For me, the GPL is kind of like comedian Sacha Baron Cohen - an object of both fascination and disgust. On one hand, the GPL is a critical element in the whole FOSS framework. Without it we would probably see lots more fragmentation of software projects, particularly with companies adopting free code and then keeping it closed. On the other hand, however, what the GPL preaches and what it delivers are two completely different things. In any case, without the GPL we would likely be 10 to 15 years behind where we are now in mainstream computing environments, as we would still lie completely in the grips of commercial enterprises.

The GPL lays down a set of rules which are both harsh and formidable enough to be able to keep self-serving commercial entities off of our collective backs while we (”the royal we”) get down to the serious business of advancing the field of computer science (and, indirectly, many other fields) to the benefit of humankind as a whole.

But that’s where my praise of the GPL ends. Why? Because what it delivers - in its harshness - is quite at odds with what it claims to deliver. That is not to say the that the GPL does not deliver what it should deliver. It does indeed do that. But its Holy Book (the text of the GPL) herds us in with dreams of freedom. Once we’re emotionally involved it then slaps on the shackles. And everyone we ever touch afterwards will get those shackles slapped on to them, whether or not their beliefs are consistent with the GPL’s underlying philosophies.

The GNU GPL is based upon the so-called “four freedoms”. They are (taken verbatim from the GNU web site):

• The freedom to run the program, for any purpose (freedom 0).
• The freedom to study how the program works, and adapt it to your needs (freedom 1). Access to the source code is a precondition for this.
• The freedom to redistribute copies so you can help your neighbor (freedom 2).
• The freedom to improve the program, and release your improvements to the public, so that the whole community benefits (freedom 3). Access to the source code is a precondition for this.

(Source: http://www.gnu.org/philosophy/free-sw.html)

As anyone who has anything to do with [non-Windows] software development knows, GNU’s definition of “free” is “free as in freedom.” But what dictionary did the GPL author(s) use when looking up a definition for “freedom”?

My philosophical problems with the GPL’s concept of freedom lie specifically in freedoms 2 and 3, the freedoms to redistribute the code with or without changes (those two points are conceptually the same thing - I’m not sure why they are considered to be separate points in that list. Perhaps because “three freedoms” doesn’t have the same ring to it.).

I’ve said this before (and I’ve upset people by saying it) and I’ll say it again: freedom implies choice, and the GPL strictly removes any choice from the user. The fact is, if you modify code which has the GPL license, and you give anyone a copy of your program (that is, the program is not strictly used for one’s own internal use), then you are forced to give the source code to your program out as well (though you may defer doing so until someone asks for it). Forced. There is no Freedom in Forced.

Now hold on a minute - I’m not at all saying that the release of source code should be restricted! Every line of source code for every project I release is released for all to read, modify, and redistribute as they see fit. As they see fit. They have the freedom of choice.

The vast majority of my own source code is released without copyright, into the Public Domain. Some of my code uses the less-restrictive LGPL, which I actually am a big fan of, and which I find to be a wonderful compromise between the GPL and the real world. Some of my code uses MIT- or BSD-style licenses, mainly in cases where I have adopted code released under those licenses. In my opinion, MIT/BSD-style licenses are about as close to Public Domain as one can get while still having a license.

It is technically true that the idea of Public Domain is not legally valid in all jurisdictions (if I’m not mistaken, Germany (where I live) is one of those jurisdictions). But is is also true that some jurisdictions do not accept any concept of “Intellectual Property”. In such jurisdictions, we can effectively call all software Public Domain. Thus the GPL is, in some contexts, as invalid as Public Domain is in other contexts. So don’t harp about how legally enforceable the GPL is. Just as with the GPL, I’ve never heard of a valid legal case where some known Public Domain resource was challenged and subsequently given over to a single entity. Legalities, in any case, are local conventions and not unbreakable divine laws.

Now back to the point… let’s look over the list of the Four Freedoms, this time with the following differences in our perception:

1. A reasonable, conventional definition of Freedom (i.e. one which includes choice). Or, “Free as in Will,” if I may now coin a phrase.
2. Instead of the context of resources licensed under the GPL, consider the context to be Public Domain resources.

Go on and take a look, then please continue…

Did you notice the difference? Namely, the difference that the Four Freedoms, as listed above, apply to resources released into the Public Domain. They also apply to a number of other FOSS-compatible licenses. But they do not, given any reasonable definition of Freedom, apply to the GPL.

Does this mean the GPL is somehow evil? No, of course not. The purpose of this is simply to point out that the GPL does not practice what it preaches. That said, what is practices is an essential part of the FOSS culture(s), but what is practices is not what it claims, in its manifesto, to practice.

Over the years I’ve received several mails from people new to the FOSS concepts, and they’ve asked me questions like, “what license should I use for my code?” The answer of course, depends on what you want to achieve. For many projects, the GPL is certainly suitable (given the caveat of the perverted definition of “freedom”), or arguably even essential (e.g. in my opinion, the Linux kernel). For some other purposes the GPL will forever limit the audience of the code and hinder its adoption, as commercial entities often don’t want to touch GPL code, whereas they are less reserved in their adoption of Public Domain, MIT, BSD, Apache, or similarly licensed code. (Even MSIE contains, according to its About dialog box, code from the original Mosaic browser.) Some real-world examples of this are the sqlite and OpenOffice projects, the former being Public Domain and the latter using the LGPL. (In my humble opinion, the LGPL was a perfect choice for OpenOffice, allowing the unrestricted creation of plugins without forcing plugin developers to conform to OO’s license.)

If there’s anyone out there who can explain to me how GNU’s definition of Freedom really is aligned with the common definition of the word, I would love to hear it. For those interested in flame wars, feel free to flame, but I won’t be drawn in to them.

:-D

Geeks come out of the cracks to pay tribute to Gygax

Gary Gygax, co-father of an entire genre of gaming, died last week, as you have certainly seen widely reported all over the net.

This post is largely inspired by one of the tribute postings, where the author “admits” to having played Dungeons & Dragons in his younger years.

Here i’ll jump on that bandwagon…

i first got involved in D&D with the classic “red-box” Basic Set. Two friends of mine in the 5th grade got me interested in it right after i moved to Houston in the first half of 1984. i still remember my step-dad taking me to the store down the street to buy a copy (on his red-metalflake Honda Goldwing cycle). That night i read excitedly and played through the introductory solitaire adventure that very evening, keen on getting a better understanding of the game my friends were already so engrossed with.

i continued playing it until about the 10th grade, where i was so involved with school and extra-curricular activities that i simply didn’t find the time to play it. (i was a varsity cheerleader for 3 years, believe it or not, and that took up quite a lot of time.)

Despite not actually playing RPGs after about 10th grade, i have never stopped collecting games. While i no longer have my older collections (thanks to my ex-wife, who sold them at a garage sale when i wasn’t there to defend them), i still have quite a few books and whatnot for GURPS, Tunnels and Trolls, TWERPS, and Fudge, as well as a number of strategy and board games. Aside from collecting gaming materials, i still design boardgames, strategy/simulation games, and roleplaying games in my spare time (see http://wanderinghorse.net/gaming/). All of that started with my first adventures in D&D.

But back to the point - as far as D&D went, my favourite character was Eaglebeak, a fighter who eventually got up to the 8th level or so. He was my favourite not because he was the most powerful (we all had level-30+ characters who got that way through the whims of over-active GMs), but because he was the one who made it to the 8th level “fair and square”. That is, not by simply gaining enough gold to get him the experience points. Eaglebeak married the lovely Princess Lynn, built a fort, had twin children (during the birth the GM rolled 1d20 to determine the gender and rolled a 20, which he ruled to be twin boys, who i named Tigerbeak and Rattlebeak), and lived happily ever after.

That’s all the stuff of fantasy, of course, and Gygax was largely responsible for setting us upon the path to enjoying roleplaying at home, instead of just reading books about fantasy adventures.

As for Gygax… The king is dead. Long live the king.

If you’d like to find out more about Gary Gygax, Wired magazine has an excellent article about him.

Back to C…

The past couple of months i have been experimenting with a new version control software called Fossil, available from http://www.fossil-scm.org. While it’s not as mature as, say, CVS or Subversion, it has some really killer features which make it stand out (e.g. can run as a standalone server, a command-line app, or a CGI app, it has a built-in wiki, and a built-in bug/ticket system is in the works). It is my firm belief that in a year or so Fossil will be in an outstanding position to sweep the market for distributed version control for small projects. The fact that it can run as a CGI service opens it up to many users who currently can’t run their own Subversion or CVS servers (e.g. me).

So far the only significant feature which Fossil doesn’t have, which i truly miss, is that it doesn’t store access rights for files (e.g. it can’t store the executable bit for scripts). Dr. D. Richard Hipp, Fossil’s author (and author of sqlite), wrote a few days ago in the fossil mailing list post that early versions of Fossil had that feature but that it “didn’t work out”, so he pulled it.

After sending several mails and a bug fix or two to Dr. Hipp he gave me write access to the Fossil repo, so i’ve been hacking on it the past week or two. It’s been a wonderful chance to get back into C programming. While C was my 3rd computer language (after BASIC and Pascal), i haven’t actually used it for anything significant since 1995, so my C is a bit rusty. While C++ is, by far, my language of choice, some of the techniques which Fossil uses internally are downright cool and have inspired me to take a fresh look at C. Part of that “fresh look” has meant hacking around in C, and that hacking is itself stored in a public Fossil repository:

http://wanderinghorse.net/cgi-bin/fossil-c-snippets.cgi

Probably my favourite part of that code so far is the vappendf() family of functions, which work similarly to printf() but can send their output to arbitrary channels, instead of just to stdout or a FILE handle. i found a Public Domain implementation of printf() (from the sqlite3 source tree) and modified it to send its output to a callback function. This makes it possible to send printf-processed output to, e.g., C++ strings or streams, GUI widgets, arbitrary file handles, etc. (Note that printf() itself is not used in the implementation, but appendf() and friends are functionally equivalent to [f]printf(), plus a couple of extensions.) Together with the “clob” (C blobs) API (also in that repo, hacked together last weekend) they provide a complete framework for buffering/accumulating string output, e.g. like Fossil does when it generates web pages. (In fact, the clob API is a reimplementation of the Blob support in Fossil.)

My current long-term goal (i.e. “fantasy”) is to, bit by bit, reimplement or refactor the Fossil source base into a generic application platform. That would allow really rich applications can be written in C/C++ for use as CGIs, standalone web servers, or console applications. Fossil’s engine is based largely on sqlite3, which means that such apps would have a significant amount of built-in serialization support.

Anyway… back to hacking…