blog.s11n.net

JSON API in the Fossil SCM

Generating JSON output from Databases on the Command Line

Hello, JSON fans,

This morning i wrote a small C application which can be used to fetch data from sqlite3 and MySQLv5 databases and convert it to JSON. While it was originally written as a test app for my C JSON library, it certainly has other uses.

It can be found here:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/select-to-json

It “should” compile fine out of the box on any Linux-like system with gcc, bash, and GNU Make. Building on other platforms requires either converting the included Makefile logic to the platform’s build tool, or linking it against the “amalgamation build” of the underlying JSON library. The code is portable ANSI C-89 and should not require any platform-specific fixes.

Happy Hacking!

—– stephan beal

JSON-centric CGI Applications in C

Hello, fellow C hackers, (everyone else can stop reading now)

A few months ago i started writing cson (pronounced “season”), a C library for generating and consuming JSON data using an OO interface (as opposed to a printf()-like API). The past few days i’ve extended it to support a mini-framework for writing JSON-only CGI applications. It’s biggest claim to fame is that it can use HTTP cookies (or a client-provided session ID) to manage persistent application sessions, which are stored (of course) as JSON. It currently supports using files, sqlite3, or mysql5 for session storage, and can easily be extended to support new back-ends. (So far i have been unable to find any other C/C++ CGI libraries which support persistent sessions.)

If you just happen to be one of the 3 or 4 remaining C hackers who like to write CGI apps, and you are also a big fan of JSON, the code might be of interest to you:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/cson_cgi

The JSON-based session support is actually independent of the CGI bits, and can be used without HTTP/CGI:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki/cson_session

Here’s a demo:

http://fossil.wanderinghorse.net/demos/cson/

Happy Hacking!

—– stephan beal

Wanted: Late-night Hacker seeks SSH Access…

Hello, fellow hackers!

i just have to ask…

As part of the cpdo project i would really like to start work on a database driver for Oracle (based on ocilib, which i quite like and is well documented). My main limiting factor here is access to a Unix/Unix-ish machine with the necessary Oracle bits.

If there is anyone out there who could give me (unprivileged!) SSH access to such a machine, you would be doing me a great service. My requirements would be:

• Some form of Linux, BSD, Solaris, etc.
• A decent C compiler and related build tools.
• Access to link to libclntsh.so (some admins lock that away in a dir only readable by the oracle user and/or dba group).
• A decent editor: xemacs (preferred) or emacs. (Note that xemacs works fine without X11, and i would not need to tunnel X11 over ssh.)
• Access to a single Oracle database with minimal space requirements (maybe a meg or two, at the very most).
• i think that i need Oracle v10 or higher, but to be honest i’m not certain.
• Far under 1% (amortized) of your total CPU capacity, with most of that coming from the C compiler and the linker.
• Initially i would need a total of probably 36-48 hours of login time, and would of course like to have further access to improve upon the driver over time (but i would settle for the initial block if that’s all i can get).

i would of course Do No Evil (or even Mischief) on your machine - my sole purpose would be to develop and test this driver. i would of course also sign any disclaimers, waivers, etc. which you would require, as long as copyrights to the source code developed there are not transfered from me (the code will remain free and open source, like the should be).

Is there someone out there who could help a brother out? If so, please get in touch (contact info is at http://wanderinghorse.net/home/stephan/).

i actually have tried to get Oracle running, but after spending a whole day trying, and never getting past the installer, i gave up. (Oracle for Linux requires truly ancient system libraries which have not been seen on my PCs since about the time of Bill Clinton’s presidency.)

Thanks once again for reading, and Happy Hacking!

—– stephan beal

New Project: cpdo database abstraction API for C

Hello, C hackers!

Inspired by a recent project at work, where i had to write a DB abstraction layer for wrapping MySQL and Oracle, i’ve started a new database abstraction library for C:

http://fossil.wanderinghorse.net/repos/cpdo/

cpdo gets its name from the PHP PDO API, after which it is modeled.

It comes with a driver for sqlite3. Work started on a MySQL driver, but their prepared statements API is essentially unusable from such an abstraction layer (as best as i can determine, anyway), so i gave up. (Update: and later got it working - see below.)

The original wrapper (part of a commercially-supported add-on for the Nagios monitoring framework) only needed “plain SQL” support, not prepared statements, and thus worked fine with MySQL and Oracle. (We replaced their usage of libdbi so that they can support Oracle.) With cpdo, i decided to “implement it properly” and internally use only prepared statements for all SQL. This, however, this appears to be essentially impossible using the MySQL API, so we’re left with an abstraction layer which abstracts only sqlite3 for the time being.

If there are any MySQL-loving C gurus out there with too much energy on their hands who would like to help implement the MySQL binding (or for some other database), please get in touch. i’ve implemented the driver skeleton and maybe 1/3rd of the code (e.g. connection, disconnection, and the first step of preparing statements), but can’t proceed because the MySQL prepared statements API is so insane. Update 2011-Jan-31 @ 00:17: The MySQL driver is now basically working. There’s more testing to be done, in particular with BLOBs and large string fields, but it appears to work. Thankfully, i found a useful example (most aren’t!) of how to use the MySQL in the PHP source tree (for their mysqli bindings), and that made all the difference.

Happy Hacking!

—– stephan beal

New Project: v8::convert

Hello, fellow v8 hackers,

(If you have no idea what a “v8 hacker” is then this post will mean nothing to you.)

i would like to announce a new v8-based project…

v8::convert is a header-only utility library for Google v8 with a very small focus:

• Converting between v8 and native values, including client-defined types.
• Converting C/C++ functions/methods to v8::InvocationCallback implementations.

With those two features we have almost everything we need for most class-binding purposes. The the ability to convert functions to have v8::InvocationCallback signatures has proven especially useful, and allows us to plug in in many C-standard or 3rd-party APIs with ease. Conversions are compile-time typesafe where possible and runtime conversion failures fail gracefully (e.g. by throwing a JS-side exception).

The core components are in no way project-specific, and can be used in conjunction with arbitrary other function/class binding mechanisms. Since the implementation is header-only, it’s particularly easy to include into other projects.

The API is trivial to use. The client API contains a good number of classes and functions, but most client code needs only one or m/ore of the 5 main function templates: CastToJS(), CastFromJS(), FunctionToInvocationCallback(), MethodToInvocationCallback(), and ConstMethodToInvocationCallback().

The missing parts are:

- Binding JS accessors to native values/functions. i have the code in v8-juice but need to port it to this API’s conventions.
- A class binding mechanism which takes care of the WeakPointer bits. There is such a mechanism in v8::convert, but it is not considered a core component because hackers tend to each have their own favourite class binding mechanism.

The code is largely a spin-off of the core-most v8-juice components, but the function-binding parts were re-implemented from scratch so that we can use function-pointer-style template parameters (which, IMO, are more readable than the v8-juice’s approach). Additionally, instead of inheriting v8-juice’s huge collection of weirdly-named function-binding templates, clients now need only 3 templates to bind functions (one each for free functions, const- and non-const member functions).

The idea of doing a header-only spin-off has been brewing for some time, but i was recently inspired to finally do the work by two other v8-users list members:

- James Pike: when i saw how vu8 takes advantage of function-pointer-style template parameters, i just had to have that feature for myself.

- Fuzzy Spoon: the separation of const- and non-const member handling was prompted by Fuzzy’s report about MSVC not being able to select the proper template specializations when the the member functions differ only in constness.

v8::convert is only about 3 nights old, but already has at least as much API documentation as v8 itself. There is an overview and example code here:

http://code.google.com/p/v8-juice/wiki/V8Convert

The source code is at:

http://code.google.com/p/v8-juice/source/browse/#svn/convert

It includes a standalone demo application and a demo v8-juice plugin.

Happy Hacking!
—– stephan beal

Tip: Code Commenting

Hello, fellow hackers!

We’re all aware of the “dangers” of adding comments to implementation code, such as:

int x = 32; // default value was decided by committee...

In practice, comments tend become disassociated from their code, especially when more than one person works on the same code. For example, the following is relatively common in real-world code:

// default value was decided by committee...
int y = 7;
int x = 32;

With that change, we’re not really sure which default value the comment is referring to.

Recently i stumbled across a commenting approach which has helped me avoid this type of problem:

int x = 32 /*default value was ...*/;

Note that the comment is inside the statement, before the semicolon. The side-effect of that is that coders who edit that line are much more likely to have to deal with the comment. For longer comments, it helps to split up the lines:

int x = 32
  /* default value was ...
     on 20101214 ... by ....
   */
;

Again, the comment is inserted as part of the statement. If someone wants to move that line, they’ll be forced to acknowledge that “something unusual” is happening, and are likely to read the comment to figure out what’s standing in their way.

It’s a simple to apply approach which i’ve started using in all my code, and i nowadays i recommend it as standard-practice over adding comments in the more conventional forms:

// before the code:
int x = ...;

int y = ...;
// ^^^^ after the code, with carets meaning "look up to the previous line"

int z = ...; // after the semicolon

Happy Hacking!

—- stephan beal

PS: there are people out there who despise any form of comments, and i was once told (on a popular open source project) that having comments in my code contributions made an “unprofessional” impression. You may disagree that comments are useful/necessary/whatever in code, and that’s fine. Let’s just agree to disagree.

Update on 2010.12.20: some of you will simply laugh and say, “my programming language of choice doesn’t use semicolons” or “my programming language uses semicolons as comment markers!” To you i say, “how sad!”

New Article: Doing OO in C

Good evening, fellow hackers!

While i tend to write absurd amounts of technical documentation, mostly in the form of API docs and other code documentation, it’s been several years since i’ve published a technical article.

For those 5 or 6 remaining aspiring C programmers in the world, i’ve got a new article which demonstrates a useful 999(shit, my left shift key just quit working…)… ahem… demonstrates a useful (in my opinion) model for implementing object-oriented data structures and APIs in C. DAMMIT, AND NOW MY ENTER KEY BROKE!

(thank god for the numeric pad’s enter key…)

Since my keyboard just died, i’ll keep this short… the article is here:

http://wanderinghorse.net/computing/papers/

happy hacking1

9you never truly appreciate the left shift key until it’s gone…0

—– stephan beal

Update:

LOL! (i’ve now got my backup keyboard…) At the very same time my keyboard broke i began being affected by this new Google Chrome bug:

http://code.google.com/p/chromium/issues/detail?id=66071

(Short version: mouse wheel cannot scroll up, but can scroll down.)

i thought, because my keyboard and mouse (in Chrome, anyway) weren’t working, that maybe my USB bus was dying.