The LuaJIT 1.x series represents the current stable branch. Only a single bug has been discovered in the last two years. So, if you need a rock-solid VM, you are encouraged to fetch the latest release of LuaJIT 1.x from the » Download page.
LuaJIT 2.0 is the currently active development branch. It has Beta Test status and is still undergoing substantial changes. It has » much better performance than LuaJIT 1.x. It's maturing quickly, so you should definitely start to evaluate it for new projects right now.
Current Status
This is a list of the things you should know about the LuaJIT 2.0 beta test:
-
Obviously there will be many bugs in a VM which has been
rewritten from the ground up. Please report your findings together with
the circumstances needed to reproduce the bug. If possible, reduce the
problem down to a simple test case.
There is no formal bug tracker at the moment. The best place for discussion is the » Lua mailing list. Of course you may also send your bug reports directly to me, especially when they contain lengthy debug output or if you require confidentiality. -
The x86 JIT compiler only generates code for CPUs with support for
SSE2 instructions. I.e. you need at least a P4, Core 2/i3/i5/i7,
Atom or K8/K10 to get the full benefit.
If you run LuaJIT on older CPUs without SSE2 support, the JIT compiler is disabled and the VM falls back to the LuaJIT interpreter. This is faster than the Lua interpreter, but not nearly as fast as the JIT compiler of course. Run the command line executable without arguments to show the current status (JIT: ON or JIT: OFF). -
The VM is complete in the sense that it should run all Lua code
just fine. It's considered a serious bug if the VM crashes or produces
unexpected results — please report this. There are only very few
known incompatibilities with standard Lua:
- The Lua debug API is missing a couple of features (return hooks for non-Lua functions) and shows slightly different behavior (no per-coroutine hooks, no tail call counting).
- Bytecode currently cannot be loaded or dumped. Note that the bytecode format differs from Lua 5.1 — loading foreign bytecode is not supported at all.
-
Some of the configuration options of Lua 5.1 are not supported:
- The number type cannot be changed (it's always a double).
- The stand-alone executable cannot be linked with readline to enable line editing. It's planned to add support for loading it on-demand.
- Most other issues you're likely to find (e.g. with the existing test suites) are differences in the implementation-defined behavior. These either have a good reason (like early tail call resolving which may cause differences in error reporting), are arbitrary design choices or are due to quirks in the VM. The latter cases may get fixed if a demonstrable need is shown.
-
The JIT compiler falls back to the
interpreter in some cases. All of this works transparently, so unless
you use -jv, you'll probably never notice (the interpreter is
» quite fast, too). Here are the known issues:
- Most known issues cause a NYI (not yet implemented) trace abort message. E.g. for calls to some internal library functions. Reporting these is only mildly useful, except if you have good example code that shows the problem. Obviously, reports accompanied with a patch to fix the issue are more than welcome. But please check back with me, before writing major improvements, to avoid duplication of effort.
- The trace compiler currently doesn't back off specialization for function call dispatch. It should really fall back to specializing on the prototype, not the closure identity. This can lead to the so-called "trace explosion" problem with closure-heavy programming. The trace linking heuristics prevent this, but in the worst case this means the code always falls back to the interpreter.
- Trace management needs more tuning: less drastic countermeasures against trace explosion and better heuristics in general.
- Some checks are missing in the JIT-compiled code for obscure situations with open upvalues aliasing one of the SSA slots later on (or vice versa). Bonus points, if you can find a real world test case for this.
- Currently some out-of-memory errors from on-trace code are not handled correctly. The error may fall through an on-trace pcall (x86) or it may be passed on to the function set with lua_atpanic (x64).
Roadmap
Rather than stating exact release dates (I'm well known for making spectacularly wrong guesses), this roadmap lists the general project plan, sorted by priority, as well as ideas for the future:
- The main goal right now is to stabilize LuaJIT 2.0 and get it out of beta test. Correctness has priority over completeness. This implies the first stable release will certainly NOT compile every library function call and will fall back to the interpreter from time to time. This is perfectly ok, since it still executes all Lua code, just not at the highest possible speed.
- The next step is to get it to compile more library functions and handle more cases where the compiler currently bails out. This doesn't mean it will compile every corner case. It's much more important that it performs well in a majority of use cases. Every compiler has to make these trade-offs — completeness just cannot be the overriding goal for a low-footprint, low-overhead JIT compiler.
- More optimizations will be added in parallel to the last step on an as-needed basis. Sinking of stores to aggregates and sinking of allocations are high on the list. More complex optimizations with less pay-off, such as value-range-propagation (VRP) will have to wait.
-
LuaJIT 2.0 has been designed with portability in mind.
Nonetheless, it compiles to native code and needs to be adapted to each
architecture. The two major work items are porting the the fast interpreter,
which is written in assembler, and porting the compiler backend.
Most other portability issues like endianess or 32 vs. 64 bit CPUs
have already been taken care of.
Several ports are already available, thanks to the » LuaJIT sponsorship program. More ports will follow in the future — companies which are interested in sponsoring a port to a particular architecture, please use the given contact address. - Documentation about the internals of LuaJIT is still sorely missing. Although the source code is included and is IMHO well commented, many basic design decisions are in need of an explanation. The rather un-traditional compiler architecture and the many highly optimized data structures are a barrier for outside participation in the development. Alas, as I've repeatedly stated, I'm better at writing code than papers and I'm not in need of any academic merits. Someday I will find the time for it. :-)
- Producing good code for unbiased branches is a key problem for trace compilers. This is the main cause for "trace explosion". Hyperblock scheduling promises to solve this nicely at the price of a major redesign of the compiler. This would also pave the way for emitting predicated instructions, which is a prerequisite for efficient vectorization.