blog.s11n.net

JSON API in the Fossil SCM

Goodbye, Ubuntu!

Achtung: this is one of those bad-mood posts i prefer to avoid writing, but the Ubuntu project needs a bit of a verbal thrashing…

i’ve been a happy, dedicated Ubuntu user since i left Kubuntu when they switched from KDE 3 to KDE 4 (which was, and still is, utterly useless as a work environment). i had been a happy, dedicated Kubuntu user since switching from Suse. i had been a happy, dedicated Suse user since the summer of 1998, shortly after i moved to Germany, and only left them because their package management tools simply got atrociously bad (i’m sure they have since gotten better, but once you’ve used apt-get and friends you can’t ever go back to an RPM-based system). Before Suse i would use various distros and had no real favourite (i started using Linux in 1994). i think i was using Slackware most of the time back then.

When Ubuntu 11.04 came out i diligently upgraded. “Upgrade,” he says! Upgrade!!! Right! It was supposed to be an upgrade! But the Ubuntu project went with some eye-candy desktop environment with the same core properties as KDE4: it looks beautiful but is absolutely useless as a work environment. i need a WORKstation, not a PLAYstation (that’s what my GameBoy is for). So i back-revved (upgraded!) 11.04 to 10.10 and was once a happy, dedicated Ubuntu user.

And then came 11.10, and i felt compelled to try it out. BAD IDEA! Not only does it have that same goddamned Mac-like user interface, they’ve moved the app menu to the top of the screen. Steve Jobs isn’t dead - he went to work for Ubuntu.

I HATE HAVING THE MENU ON THE TOP OF MY SCREEN. After reading several blog posts about how to remove it, nothing worked. So i installed the LXDE desktop on it, which is actually quite nice but has the fatal flaw that it cannot properly do multi-head (or it has no tool for setting it up like Ubuntu and friends have). My netbook (my only PC) gets continually moved between two different external monitors and i need a work environment which can do that. Ubuntu handled that beautifully, without any restarting or other Windows/Firefox-like behaviour. So LXDE is ruled out.

At this very moment i’m running the LiveCD version of Mint Linux, and it’s installing as i’m writing this (try doing that in Windows or Mac!).

GOODBYE, UBUNTU! i once loved you, but this betrayal is too much. i cannot live with a menu stuck at the top of my screen and a desktop with which i cannot work. Even in “Gnome Classic” mode that goddamned menu sits at the top of my screen, continually taunting me. i can’t have my PC taunting me, so off you go, back to the toybox where your Unity desktop belongs.

The installer is finished now, so it’s now time to go get all my packages installed and whatnot. What a waste of time. This type of thing was fun 10 or 12 years ago, but i’m too old for that - i want a system which just works and keeps working. Kinda like Ubuntu did up until version 11.04.

15 Billion? Apple is lying out its applehole…

Hello, all!

Some of you might know that i absolutely despise Apple (the company). i’ve boycotted them actively since around 1994 and i will continue to do so until (at the very least) other companies are allowed to build Apple hardware clones (without paying exuberant licensing fees to Apple, that is).

i don’t allow Apple products in my flat (leave them at the door or don’t come in!) and i refuse to lay my hands on them (except once, i must ashamedly admit, to compare an iPhone’s weight against my mobile phone’s). i am so strongly against Apple that i would not hesitate to quit my job if i was told that i had to work on an Apple-based project.

The list of reasons i boycott Apple is about as long as my hair (i.e. it stretches from my head to below my belt), but i’m not here today to talk about that list. Today i’m here to show that Apple believes that it can magically pull impossible numbers from its butt and get away with marketing them as truths.

Let’s have a little lesson in math…

According to this post (and others like it):

http://www.businessweek.com/ap/financialnews/D9OB0A280.htm

Apple’s App Store has reported 15 billion sales/downloads. Apple has sold, according to that same article, some 200 million devices worldwide which could be used for such sales.

Now do the math: 15b/200m = 75 apps per device on average. And its an average of roughly 2 apps per person on the planet, despite the fact that only a very small percentage of human beings own an Apple iProduct (approximately 1 in 3500 people worldwide, if we can believe the 200m number from Apple and a current worldwide population of 7b).

While there might be a small handful of people who actually buy that many apps for their iDevice (e.g. for product reviews and whatnot), there is no way in hell which Apple can convince me that the average Apple user is buying 75 applications from the App Store. (But the Apple zombies will certainly find a way to justify it, as that’s what they instinctively do every time they swallow what Apple places in front of them.)

The only ways, in my opinion, which these numbers could be explained are:

1. They’re fictional. Some finance guy/gal simply punched in the number in a spreadsheet program (or otherwise made it up).
2. Apple (or one of its 200m zombies) uses bots whose sole purpose is to just keep downloading apps over and over again to increase the official download counts. (Hell, there might be such bots built in to every iProduct, for all we know.)
3. There was an integer underflow in their download counter.

:-P

whiki: a “different sort” of Wiki back-end

Hi, all!

Despite the fact that the world has tons of wiki managers/back-ends, i felt compelled hack one of my own together. The initial inspiration for this came from the Fossil, an SCM which integrates source control, a wiki, and bug-tracking into a single application binary which can be used from the command line, a standalone web server, or as a CGI on an arbitrary web hoster. (i host many of my own projects using Fossil CGI over on fossil.wanderinghorse.net.) Through my use of fossil, i’ve come accustomed to maintaining non-API project documentation in a wiki, and love the ease-of-documentation this gives me (i’m a documentation maniac). Despite my love for fossil, however, i always felt that i was betraying the only wiki syntax i really like - the one used by Google Code. Since fossil doesn’t provide the features i need to replace its wiki handler with one of my own… you guessed it (or read it in the first line of this post)… i wrote a new wiki back-end.

In April i started work on whiki (the WanderingHorse.net Wiki), a “different sort” of wiki back-end, and it is now in a functional state (hosting three live wikis as of 17 May 2011). Like most wiki back-ends, it uses a database (sqlite3 or MySQLv5) to store its pages. What differentiates whiki from other wiki back-ends is:

• It is implemented in C and runs as a CGI application (or, to some degree, from the command line). i’m not personally aware of any other wiki manager/back-end implemented in C.
• It consumes and produces only JSON data. Requests are sent in the form of simple HTTP GET or (for more complex requests) POST requests (with JSON payload data) and responses are in the form of a JSON object tree.
• It has no “native” wiki format. Instead it tags each wiki page with a “content type” label (e.g. “text/plain” or “wiki/googlecode”), and applications can use that hint to determine how they should render a given page. Client applications can, with little effort, support multiple content types in the same logical wiki (e.g. plain text, source code (to be syntax-highlighted by the renderer), HTML, or even multiple wiki formats).
• It has no “native” user interface. UIs can be written using client-side languages like JavaScript, Java, or PHP, which interact with the whiki back-end via JSON messages. The main whiki website is implemented entirely in JavaScript, CSS, and HTML, and primarily uses the wikiwym GoogleCode-syntax wiki parser for rendering its pages (but it also supports several other content types).
• It allows clients to attach arbitrary JSON data to each page. This can be used for any number of things, e.g. implementing “tags”, marking certain pages as “featured” or “important”, defining intra-page relationships (e.g. for rendering a hierarchical menu), or flagging a certain page as the “home page.”

whiki is not, and will never, be an industrial-strength back-end like the one used to power Wikipedia (or any number of other large wikis). It is intended to supplant my fossil-based wikis (meaning anywhere from 5 to 20 pages or so). It is also missing some features which would certainly be of critical importance to many potential users. e.g. it has no search functionality, only two access levels (all or nothing), and has no support for storing historical versions of each page. (With another contributor or three we could fix those shortcomings, but i personally have relatively little need of them.)

If the following applies to you:

• You’re an eccentric C hacker who wants to host his own small wiki but doesn’t want to use PHP or…
• You’re a lone hacker who wants to an easy way to host wiki-style documentation for one of his pet projects or…
• You currently use Fossil but aren’t quite satisfied with its wiki

… and you have the ability to compile and run C applications on your web hoster…

Then whiki just might be slightly interesting for you:

http://whiki.wanderinghorse.net

Happy Hacking!

—– stephan beal

Generating JSON output from Databases on the Command Line

Hello, JSON fans,

This morning i wrote a small C application which can be used to fetch data from sqlite3 and MySQLv5 databases and convert it to JSON. While it was originally written as a test app for my C JSON library, it certainly has other uses.

It can be found here:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/select-to-json

It “should” compile fine out of the box on any Linux-like system with gcc, bash, and GNU Make. Building on other platforms requires either converting the included Makefile logic to the platform’s build tool, or linking it against the “amalgamation build” of the underlying JSON library. The code is portable ANSI C-89 and should not require any platform-specific fixes.

Happy Hacking!

—– stephan beal

JSON-centric CGI Applications in C

Hello, fellow C hackers, (everyone else can stop reading now)

A few months ago i started writing cson (pronounced “season”), a C library for generating and consuming JSON data using an OO interface (as opposed to a printf()-like API). The past few days i’ve extended it to support a mini-framework for writing JSON-only CGI applications. It’s biggest claim to fame is that it can use HTTP cookies (or a client-provided session ID) to manage persistent application sessions, which are stored (of course) as JSON. It currently supports using files, sqlite3, or mysql5 for session storage, and can easily be extended to support new back-ends. (So far i have been unable to find any other C/C++ CGI libraries which support persistent sessions.)

If you just happen to be one of the 3 or 4 remaining C hackers who like to write CGI apps, and you are also a big fan of JSON, the code might be of interest to you:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/cson_cgi

The JSON-based session support is actually independent of the CGI bits, and can be used without HTTP/CGI:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/cson_session

Here’s a demo:

http://fossil.wanderinghorse.net/demos/cson/

Happy Hacking!

—– stephan beal

Wanted: Late-night Hacker seeks SSH Access…

Hello, fellow hackers!

i just have to ask…

As part of the cpdo project i would really like to start work on a database driver for Oracle (based on ocilib, which i quite like and is well documented). My main limiting factor here is access to a Unix/Unix-ish machine with the necessary Oracle bits.

If there is anyone out there who could give me (unprivileged!) SSH access to such a machine, you would be doing me a great service. My requirements would be:

• Some form of Linux, BSD, Solaris, etc.
• A decent C compiler and related build tools.
• Access to link to libclntsh.so (some admins lock that away in a dir only readable by the oracle user and/or dba group).
• A decent editor: xemacs (preferred) or emacs. (Note that xemacs works fine without X11, and i would not need to tunnel X11 over ssh.)
• Access to a single Oracle database with minimal space requirements (maybe a meg or two, at the very most).
• i think that i need Oracle v10 or higher, but to be honest i’m not certain.
• Far under 1% (amortized) of your total CPU capacity, with most of that coming from the C compiler and the linker.
• Initially i would need a total of probably 36-48 hours of login time, and would of course like to have further access to improve upon the driver over time (but i would settle for the initial block if that’s all i can get).

i would of course Do No Evil (or even Mischief) on your machine - my sole purpose would be to develop and test this driver. i would of course also sign any disclaimers, waivers, etc. which you would require, as long as copyrights to the source code developed there are not transfered from me (the code will remain free and open source, like the should be).

Is there someone out there who could help a brother out? If so, please get in touch (contact info is at http://wanderinghorse.net/home/stephan/).

i actually have tried to get Oracle running, but after spending a whole day trying, and never getting past the installer, i gave up. (Oracle for Linux requires truly ancient system libraries which have not been seen on my PCs since about the time of Bill Clinton’s presidency.)

Thanks once again for reading, and Happy Hacking!

—– stephan beal

New Project: cpdo database abstraction API for C

Hello, C hackers!

Inspired by a recent project at work, where i had to write a DB abstraction layer for wrapping MySQL and Oracle, i’ve started a new database abstraction library for C:

http://fossil.wanderinghorse.net/repos/cpdo/

cpdo gets its name from the PHP PDO API, after which it is modeled.

It comes with a driver for sqlite3. Work started on a MySQL driver, but their prepared statements API is essentially unusable from such an abstraction layer (as best as i can determine, anyway), so i gave up. (Update: and later got it working - see below.)

The original wrapper (part of a commercially-supported add-on for the Nagios monitoring framework) only needed “plain SQL” support, not prepared statements, and thus worked fine with MySQL and Oracle. (We replaced their usage of libdbi so that they can support Oracle.) With cpdo, i decided to “implement it properly” and internally use only prepared statements for all SQL. This, however, this appears to be essentially impossible using the MySQL API, so we’re left with an abstraction layer which abstracts only sqlite3 for the time being.

If there are any MySQL-loving C gurus out there with too much energy on their hands who would like to help implement the MySQL binding (or for some other database), please get in touch. i’ve implemented the driver skeleton and maybe 1/3rd of the code (e.g. connection, disconnection, and the first step of preparing statements), but can’t proceed because the MySQL prepared statements API is so insane. Update 2011-Jan-31 @ 00:17: The MySQL driver is now basically working. There’s more testing to be done, in particular with BLOBs and large string fields, but it appears to work. Thankfully, i found a useful example (most aren’t!) of how to use the MySQL in the PHP source tree (for their mysqli bindings), and that made all the difference.

Happy Hacking!

—– stephan beal