This is of course the technically best way, but the immense footprint of 18mb just for the FFI, and the constant c ABI changes in llvm was the reason we didn't do it so far also. You either ship the huge llvm libs or risk an outdated system llvm. But for proper LTS distros it's tempting.
FTR, the footprint of 18Mb is the compressed version, the uncompressed version is ~57Mb. I think there are huge improvements possible here, among them:
* compile llvm with -fvisiblity=hidden
* compile the whole thing with thin lto, which could have the effect to remove unused code
Did you consider writing a purpose-built JIT compiler? I believe the FFI use-case is narrow enough that a tiny hand-written JIT compiler would be pretty easy... like the old QEMU template based JIT.
LuaJIT’s DynASM would go a long way there; might even have reusable stubs. Or slightly crazier, if 520K isn’t too big a burden: wrap the LuaJIT cffi ...from C... and embed all of LuaJIT.
I didn't, as I really wanted to see how that would be possible using only clang/llvm. The way I see it is that it seems a shame to rewrite every possible ABI (and the more it goes the more we have (https://xkcd.com/927/)), where compilers like clang and gcc already does all the work.
One point of my llvm talk of tomorrow is to be able to discuss how we could "extract" these parts from clang to be able to do this the lightest way possible! (but that would still require the embedding of a full LLVM backend which is still huge for the only FFI case).
One good point we have embedding a full compiler is that we can really JIT C code from let's say a python interpreter. The usefulness of this is another debate :)
Said otherwise, I thought the experiment was worth the try, and it seems fun to see how far we can go from here :)
