libgd bindings for go

Wed, Sep 10, 2014

tl; dr - read the godoc for gogd

In order to help a company modernize their creaking php based image resizer I started a set of cgo based libgd bindings for golang. With gogd, image resizing proxies can be built that leverage go’s awesome networking stack while delegating heavy processing to a battle tested library. An example image resizing web server is included in gogd. The rest of this post will be a few observations on making cgo bindings and some of the challenges.

But first here’s the end result (using the utah gophers logo (derived from Renee French’s work):

Challenges

I/O integration

One of the first challenges is getting data accross the go/c boundary. Libgd has 3 different I/O methods for loading and saving images. The first is using a libc FILE* using functions like the one declared below:

gdImagePtr gdImageCreateFromPng(FILE *fd);

In order to access this API from golang it’s necessary to turn an *os.File into a FILE*. This is trivial using fdopen(3) and os.File.Fd(). The technique is illustrated using some (untested) psuedocode.

import "C"
func ImageCreateFromPng(f *os.File) C.gdImagePtr {
     return C.gdImageCreateFromPng(C.fdopen(C.int(f.Fd())))
}

However this approach leaves something to be desired. Golang’s powerful io.Reader and io.Writer interfaces can’t be used. It’s impossible using this approach to load an image from an https url or write it as part of a response.

The next libgd I/O method is by passing around raw buffers. Libgd functions that work with buffers are suffixed with Ptr like this:

gdImageCreateFromPngPtr(int size, void *data);

It’s possible to map the above arguments to a golang []byte data structure using C.GoBytes but buffers don’t let us use go’s flexible io interfaces.

Finally libgd has a generalized I/O “interface” called gdIOCtx which in C land means structs of function pointers. Much like a go interface callbacks for Read and Write are defined, but the “received” is an explicit rather than implicit first parameter. The structure looks a bit like:

typedef struct gdIOCtx {
...
int (*getBuf)(struct gdIOCtx *, void *, int);
void *data;
}

This is matched up with some go code which extracts the io.Reader and converts between []byte and c arrays.

func getContext(g gdio) *IOCtx {
	return (*IOCtx)(unsafe.Pointer(&C.gdIOCtx{
		getBuf: (*[0]byte)(C.gogd_get_buf),
		data:   unsafe.Pointer(&g),
	}))
}
//export gogd_get_buf
func gogd_get_buf(ctx *C.gdIOCtx, cbuf unsafe.Pointer, l C.int) int {
	gdio := (*(*gdio)(ctx.data))
	buf := goSliceFromcString((*C.char)(cbuf), int(l))
	n, err := gdio.(io.Reader).Read(buf)
	if err != nil && err != io.EOF {
		log.Println(err)
		return 0
	}
	return n
}

Through a little bit of trickery (look at io.go and io_c.go for the details) it’s possible to create structs of C function pointers which are really exported go functions which actually cast gdIOCtx.data to a golang io.Reader/io.Writer. While there is more overhead to this approach (more cgo calls) it’s the most general and flexible approach and it’s what gogd implemented first. We will probably add FILE and buffer based interfaces for better performance later. Using gdIOCtx means that gogd integrates trivially with files, net connections, and http response writers since they all satisfy the io.Reader/io.Writer interfaces.

Memory management

When integrating garbage collected language with C, memory management can be a problem. Fortunately it’s simple (not easy, but simple) with go. Go’s garbage collector is only for memory, not really for other resources. Rather than relying on finalizers and dealing with cycles, memory management is usually as simple as:

img := decoder.Decode(resp.Body)
if !img.Valid() {
   panic("invalid image")
}
defer img.Destroy()

I’ve used runtime.SetFinalizer in the past and I know it can be indespensible when working with C code, but using defer is preferred. Finalizers aren’t always guaranteed to run, but a defer statement will always run unless using os.Exit/log.Fatal.

Conclusions

If your workflow relies on libgd for performance, image format support, or certain algorithms, this library might be for you. However it won’t run on app engine, and in the long term go libraries (and go) will be optimized enough that there’s no reason to rely on cgo based libraries. In the mean time, gogd can help ease the transition of some of your image processing infrastructure from something like php to golang.

Now you have all the tools at your disposal to make some wicked tool cat memes.