[Pdx-pm] Designing a C library specifically for HLL binding

[chromatic]

On Thursday 30 October 2008 13:07:37 Eric Wilhelm wrote:

> Keep in mind that I don't like C well enough to convince myself that
> I "know" it.  Or rather, I don't want to spend all of my development
> time dealing with memory allocation.  And just to keep things
> interesting, almost all of the data involved is going to be arrays and
> arrays of arrays.
>
> I'm wondering if someone can give me some feedback on how this would
> work and/or point me at some examples or reading material.
>
> My thought is that the C functions would all have one of the following
> signatures:
>
>   bool gets_complex_data(source_struct, dest_struct);
>   int  gets_simple_data(source_struct);
>
> Where 'source_struct' is a pointer to a struct which essentially
> contains just the filehandle and maybe a bit of positional data (the
> input is stateful.)  Then dest_struct is block allocated. 

As long as those pointers are opaque to the bindings, this should be fine.  
Two things make life difficult for HLL extension writers:

 - memory management of shared library components
 - access to struct members of shared library components (especially if those 
structs change between *versions* of the shared library)

> The HLL 
> binding would then be doing all of the memory allocation (part of the
> plan being to minimize the memory footprint, but also to make
> a 'Perlish' API.)

This makes the HLL binding do a lot more work.  It has to know how much memory 
to allocate (which can vary from platform to platform), so you'll end up 
recompiling the HLL binding every time the shared library changes.  As well, 
you may have to keep track of what the shared library manages and what the 
bindings manage, so you free to the right place and don't leak.

If you always treat the data structures used by the shared library as opaque 
outside of the shared library, you don't have these problems.

Any decent HLL binding should be able to represent a raw C pointer as a 
pointer and pass it around appropriately, without allowing direct memory 
access to the pointer itself or what it contains.  Think inside-out objects 
where you don't have method dispatch, and you'll have the idea.

> Now remember, I only know enough C to be dangerously amusing.  But, I
> have never heard of a C library being created specifically for binding
> to an HLL interpreter, so I might be lacking the vocabulary needed to
> clarify what I think I'm trying to do. Part of the goal here would be
> to allow it to be easily bound to various other interpreters - hence
> not simply a big XS library.

Reentrancy is an issue then too.  No static data (unless it's compile time 
static).

> So, is this making any sense?  Seem like a reasonable approach?  Can
> anyone point me to a codebase which does something similar?  And
> possibly a Perl binding?

I can point you to codebases that do it wrong.  libcurl's memory management 
lifecycle for strings passed in to set URL paths is shudderworthy.

-- c