-You can also send any questions you have directly to me:
-
-
-
-
-
-
-
-Note: I cannot reply to GMail, Google Workplace, Outlook or Office365
-mail addresses, since they prefer to mindlessly filter out mails sent
-from small domains using independent mail servers, such as mine. If you
-don't like that, please complain to Google or Microsoft, not me.
-
-Unlike Lua strings, which are constants, string buffers are
-mutable sequences of 8-bit (binary-transparent) characters. Data
-can be stored, formatted and encoded into a string buffer and later
-converted, extracted or decoded.
-
-
-The convenient string buffer API simplifies common string manipulation
-tasks, that would otherwise require creating many intermediate strings.
-String buffers improve performance by eliminating redundant memory
-copies, object creation, string interning and garbage collection
-overhead. In conjunction with the FFI library, they allow zero-copy
-operations.
-
-
-The string buffer library also includes a high-performance
-serializer for Lua objects.
-
-
-
Using the String Buffer Library
-
-The string buffer library is built into LuaJIT by default, but it's not
-loaded by default. Add this to the start of every Lua file that needs
-one of its functions:
-
-
-local buffer = require("string.buffer")
-
-
-The convention for the syntax shown on this page is that buffer
-refers to the buffer library and buf refers to an individual
-buffer object.
-
-
-Please note the difference between a Lua function call, e.g.
-buffer.new() (with a dot) and a Lua method call, e.g.
-buf:reset() (with a colon).
-
-
-
Buffer Objects
-
-A buffer object is a garbage-collected Lua object. After creation with
-buffer.new(), it can (and should) be reused for many operations.
-When the last reference to a buffer object is gone, it will eventually
-be freed by the garbage collector, along with the allocated buffer
-space.
-
-
-Buffers operate like a FIFO (first-in first-out) data structure. Data
-can be appended (written) to the end of the buffer and consumed (read)
-from the front of the buffer. These operations may be freely mixed.
-
-
-The buffer space that holds the characters is managed automatically
-— it grows as needed and already consumed space is recycled. Use
-buffer.new(size) and buf:free(), if you need more
-control.
-
-
-The maximum size of a single buffer is the same as the maximum size of a
-Lua string, which is slightly below two gigabytes. For huge data sizes,
-neither strings nor buffers are the right data structure — use the
-FFI library to directly map memory or files up to the virtual memory
-limit of your OS.
-
-
-
Buffer Method Overview
-
-
-The buf:put*()-like methods append (write) characters to the
-end of the buffer.
-
-
-The buf:get*()-like methods consume (read) characters from the
-front of the buffer.
-
-
-Other methods, like buf:tostring() only read the buffer
-contents, but don't change the buffer.
-
-
-The buf:set() method allows zero-copy consumption of a string
-or an FFI cdata object as a buffer.
-
-
-The FFI-specific methods allow zero-copy read/write-style operations or
-modifying the buffer contents in-place. Please check the
-FFI caveats below, too.
-
-
-Methods that don't need to return anything specific, return the buffer
-object itself as a convenience. This allows method chaining, e.g.:
-buf:reset():encode(obj) or buf:skip(len):get()
-
-
-
-
Buffer Creation and Management
-
-
local buf = buffer.new([size [,options]])
-local buf = buffer.new([options])
-
-Creates a new buffer object.
-
-
-The optional size argument ensures a minimum initial buffer
-size. This is strictly an optimization when the required buffer size is
-known beforehand. The buffer space will grow as needed, in any case.
-
-Reset (empty) the buffer. The allocated buffer space is not freed and
-may be reused.
-
-
-
buf = buf:free()
-
-The buffer space of the buffer object is freed. The object itself
-remains intact, empty and may be reused.
-
-
-Note: you normally don't need to use this method. The garbage collector
-automatically frees the buffer space, when the buffer object is
-collected. Use this method, if you need to free the associated memory
-immediately.
-
-
-
Buffer Writers
-
-
buf = buf:put([str|num|obj] [,…])
-
-Appends a string str, a number num or any object
-obj with a __tostring metamethod to the buffer.
-Multiple arguments are appended in the given order.
-
-
-Appending a buffer to a buffer is possible and short-circuited
-internally. But it still involves a copy. Better combine the buffer
-writes to use a single buffer.
-
-
-
buf = buf:putf(format, …)
-
-Appends the formatted arguments to the buffer. The format
-string supports the same options as string.format().
-
-
-
buf = buf:putcdata(cdata, len)FFI
-
-Appends the given len number of bytes from the memory pointed
-to by the FFI cdata object to the buffer. The object needs to
-be convertible to a (constant) pointer.
-
-
-
buf = buf:set(str)
-buf = buf:set(cdata, len)FFI
-
-This method allows zero-copy consumption of a string or an FFI cdata
-object as a buffer. It stores a reference to the passed string
-str or the FFI cdata object in the buffer. Any buffer
-space originally allocated is freed. This is not an append
-operation, unlike the buf:put*() methods.
-
-
-After calling this method, the buffer behaves as if
-buf:free():put(str) or buf:free():put(cdata, len)
-had been called. However, the data is only referenced and not copied, as
-long as the buffer is only consumed.
-
-
-In case the buffer is written to later on, the referenced data is copied
-and the object reference is removed (copy-on-write semantics).
-
-
-The stored reference is an anchor for the garbage collector and keeps the
-originally passed string or FFI cdata object alive.
-
-
-
ptr, len = buf:reserve(size)FFI
-buf = buf:commit(used)FFI
-
-The reserve method reserves at least size bytes of
-write space in the buffer. It returns an uint8_t * FFI
-cdata pointer ptr that points to this space.
-
-
-The available length in bytes is returned in len. This is at
-least size bytes, but may be more to facilitate efficient
-buffer growth. You can either make use of the additional space or ignore
-len and only use size bytes.
-
-
-The commit method appends the used bytes of the
-previously returned write space to the buffer data.
-
-
-This pair of methods allows zero-copy use of C read-style APIs:
-
-
-local MIN_SIZE = 65536
-repeat
- local ptr, len = buf:reserve(MIN_SIZE)
- local n = C.read(fd, ptr, len)
- if n == 0 then break end -- EOF.
- if n < 0 then error("read error") end
- buf:commit(n)
-until false
-
-
-The reserved write space is not initialized. At least the
-used bytes must be written to before calling the
-commit method. There's no need to call the commit
-method, if nothing is added to the buffer (e.g. on error).
-
-
-
Buffer Readers
-
-
len = #buf
-
-Returns the current length of the buffer data in bytes.
-
-
-
res = str|num|buf .. str|num|buf […]
-
-The Lua concatenation operator .. also accepts buffers, just
-like strings or numbers. It always returns a string and not a buffer.
-
-
-Note that although this is supported for convenience, this thwarts one
-of the main reasons to use buffers, which is to avoid string
-allocations. Rewrite it with buf:put() and buf:get().
-
-
-Mixing this with unrelated objects that have a __concat
-metamethod may not work, since these probably only expect strings.
-
-
-
buf = buf:skip(len)
-
-Skips (consumes) len bytes from the buffer up to the current
-length of the buffer data.
-
-
-
str, … = buf:get([len|nil] [,…])
-
-Consumes the buffer data and returns one or more strings. If called
-without arguments, the whole buffer data is consumed. If called with a
-number, up to len bytes are consumed. A nil argument
-consumes the remaining buffer space (this only makes sense as the last
-argument). Multiple arguments consume the buffer data in the given
-order.
-
-
-Note: a zero length or no remaining buffer data returns an empty string
-and not nil.
-
-
-
str = buf:tostring()
-str = tostring(buf)
-
-Creates a string from the buffer data, but doesn't consume it. The
-buffer remains unchanged.
-
-
-Buffer objects also define a __tostring metamethod. This means
-buffers can be passed to the global tostring() function and
-many other functions that accept this in place of strings. The important
-internal uses in functions like io.write() are short-circuited
-to avoid the creation of an intermediate string object.
-
-
-
ptr, len = buf:ref()FFI
-
-Returns an uint8_t * FFI cdata pointer ptr that
-points to the buffer data. The length of the buffer data in bytes is
-returned in len.
-
-
-The returned pointer can be directly passed to C functions that expect a
-buffer and a length. You can also do bytewise reads
-(local x = ptr[i]) or writes
-(ptr[i] = 0x40) of the buffer data.
-
-
-In conjunction with the skip method, this allows zero-copy use
-of C write-style APIs:
-
-
-repeat
- local ptr, len = buf:ref()
- if len == 0 then break end
- local n = C.write(fd, ptr, len)
- if n < 0 then error("write error") end
- buf:skip(n)
-until n >= len
-
-
-Unlike Lua strings, buffer data is not implicitly
-zero-terminated. It's not safe to pass ptr to C functions that
-expect zero-terminated strings. If you're not using len, then
-you're doing something wrong.
-
-
-
Serialization of Lua Objects
-
-The following functions and methods allow high-speed serialization
-(encoding) of a Lua object into a string and decoding it back to a Lua
-object. This allows convenient storage and transport of structured
-data.
-
-
-The encoded data is in an internal binary
-format. The data can be stored in files, binary-transparent
-databases or transmitted to other LuaJIT instances across threads,
-processes or networks.
-
-
-Encoding speed can reach up to 1 Gigabyte/second on a modern desktop- or
-server-class system, even when serializing many small objects. Decoding
-speed is mostly constrained by object creation cost.
-
-
-The serializer handles most Lua types, common FFI number types and
-nested structures. Functions, thread objects, other FFI cdata and full
-userdata cannot be serialized (yet).
-
-
-The encoder serializes nested structures as trees. Multiple references
-to a single object will be stored separately and create distinct objects
-after decoding. Circular references cause an error.
-
-
-
Serialization Functions and Methods
-
-
str = buffer.encode(obj)
-buf = buf:encode(obj)
-
-Serializes (encodes) the Lua object obj. The stand-alone
-function returns a string str. The buffer method appends the
-encoding to the buffer.
-
-
-obj can be any of the supported Lua types — it doesn't
-need to be a Lua table.
-
-
-This function may throw an error when attempting to serialize
-unsupported object types, circular references or deeply nested tables.
-
-
-
obj = buffer.decode(str)
-obj = buf:decode()
-
-The stand-alone function deserializes (decodes) the string
-str, the buffer method deserializes one object from the
-buffer. Both return a Lua object obj.
-
-
-The returned object may be any of the supported Lua types —
-even nil.
-
-
-This function may throw an error when fed with malformed or incomplete
-encoded data. The stand-alone function throws when there's left-over
-data after decoding a single top-level object. The buffer method leaves
-any left-over data in the buffer.
-
-
-Attempting to deserialize an FFI type will throw an error, if the FFI
-library is not built-in or has not been loaded, yet.
-
-
-
Serialization Options
-
-The options table passed to buffer.new() may contain
-the following members (all optional):
-
-
-
-dict is a Lua table holding a dictionary of strings that
-commonly occur as table keys of objects you are serializing. These keys
-are compactly encoded as indexes during serialization. A well-chosen
-dictionary saves space and improves serialization performance.
-
-
-metatable is a Lua table holding a dictionary of metatables
-for the table objects you are serializing.
-
-
-
-dict needs to be an array of strings and metatable needs
-to be an array of tables. Both starting at index 1 and without holes (no
-nil in between). The tables are anchored in the buffer object and
-internally modified into a two-way index (don't do this yourself, just pass
-a plain array). The tables must not be modified after they have been passed
-to buffer.new().
-
-
-The dict and metatable tables used by the encoder and
-decoder must be the same. Put the most common entries at the front. Extend
-at the end to ensure backwards-compatibility — older encodings can
-then still be read. You may also set some indexes to false to
-explicitly drop backwards-compatibility. Old encodings that use these
-indexes will throw an error when decoded.
-
-
-Metatables that are not found in the metatable dictionary are
-ignored when encoding. Decoding returns a table with a nil
-metatable.
-
-
-Note: parsing and preparation of the options table is somewhat
-expensive. Create a buffer object only once and recycle it for multiple
-uses. Avoid mixing encoder and decoder buffers, since the
-buf:set() method frees the already allocated buffer space:
-
-In some contexts, it's desirable to do piecewise serialization of large
-datasets, also known as streaming.
-
-
-This serialization format can be safely concatenated and supports streaming.
-Multiple encodings can simply be appended to a buffer and later decoded
-individually:
-
-while #buf ~= 0 do
- local obj = buf:decode()
- -- Do something with obj.
-end
-
-
-Since the serialization format doesn't prepend a length to its encoding,
-network applications may need to transmit the length, too.
-
-
-
Serialization Format Specification
-
-This serialization format is designed for internal use by LuaJIT
-applications. Serialized data is upwards-compatible and portable across
-all supported LuaJIT platforms.
-
-
-It's an 8-bit binary format and not human-readable. It uses e.g.
-embedded zeroes and stores embedded Lua string objects unmodified, which
-are 8-bit-clean, too. Encoded data can be safely concatenated for
-streaming and later decoded one top-level object at a time.
-
-
-The encoding is reasonably compact, but tuned for maximum performance,
-not for minimum space usage. It compresses well with any of the common
-byte-oriented data compression algorithms.
-
-
-Although documented here for reference, this format is explicitly
-not intended to be a 'public standard' for structured data
-interchange across computer languages (like JSON or MessagePack). Please
-do not use it as such.
-
-
-The specification is given below as a context-free grammar with a
-top-level object as the starting point. Alternatives are
-separated by the | symbol and * indicates repeats.
-Grouping is implicit or indicated by {…}. Terminals are
-either plain hex numbers, encoded as bytes, or have a .format
-suffix.
-
-Many of the buffer methods can throw an error. Out-of-memory or usage
-errors are best caught with an outer wrapper for larger parts of code.
-There's not much one can do after that, anyway.
-
-
-OTOH, you may want to catch some errors individually. Buffer methods need
-to receive the buffer object as the first argument. The Lua colon-syntax
-obj:method() does that implicitly. But to wrap a method with
-pcall(), the arguments need to be passed like this:
-
-
-local ok, err = pcall(buf.encode, buf, obj)
-if not ok then
- -- Handle error in err.
-end
-
-
-
FFI caveats
-
-The string buffer library has been designed to work well together with
-the FFI library. But due to the low-level nature of the FFI library,
-some care needs to be taken:
-
-
-First, please remember that FFI pointers are zero-indexed. The space
-returned by buf:reserve() and buf:ref() starts at the
-returned pointer and ends before len bytes after that.
-
-
-I.e. the first valid index is ptr[0] and the last valid index
-is ptr[len-1]. If the returned length is zero, there's no valid
-index at all. The returned pointer may even be NULL.
-
-
-The space pointed to by the returned pointer is only valid as long as
-the buffer is not modified in any way (neither append, nor consume, nor
-reset, etc.). The pointer is also not a GC anchor for the buffer object
-itself.
-
-
-Buffer data is only guaranteed to be byte-aligned. Casting the returned
-pointer to a data type with higher alignment may cause unaligned
-accesses. It depends on the CPU architecture whether this is allowed or
-not (it's always OK on x86/x64 and mostly OK on other modern
-architectures).
-
-
-FFI pointers or references do not count as GC anchors for an underlying
-object. E.g. an array allocated with ffi.new() is
-anchored by buf:set(array, len), but not by
-buf:set(array+offset, len). The addition of the offset
-creates a new pointer, even when the offset is zero. In this case, you
-need to make sure there's still a reference to the original array as
-long as its contents are in use by the buffer.
-
-
-Even though each LuaJIT VM instance is single-threaded (but you can
-create multiple VMs), FFI data structures can be accessed concurrently.
-Be careful when reading/writing FFI cdata from/to buffers to avoid
-concurrent accesses or modifications. In particular, the memory
-referenced by buf:set(cdata, len) must not be modified
-while buffer readers are working on it. Shared, but read-only memory
-mappings of files are OK, but only if the file does not change.
-
-LuaJIT adds some extensions to the standard Lua/C API. The LuaJIT include
-directory must be in the compiler search path (-Ipath)
-to be able to include the required header for C code:
-
-
-#include "luajit.h"
-
-
-Or for C++ code:
-
-
-#include "lua.hpp"
-
-
-
luaJIT_setmode(L, idx, mode)
-— Control VM
-
-This is a C API extension to allow control of the VM from C code. The
-full prototype of LuaJIT_setmode is:
-
-
-LUA_API int luaJIT_setmode(lua_State *L, int idx, int mode);
-
-
-The returned status is either success (1) or failure (0).
-The second argument is either 0 or a stack index (similar to the
-other Lua/C API functions).
-
-
-The third argument specifies the mode, which is 'or'ed with a flag.
-The flag can be LUAJIT_MODE_OFF to turn a feature off,
-LUAJIT_MODE_ON to turn a feature on, or
-LUAJIT_MODE_FLUSH to flush cached code.
-
-
-The following modes are defined:
-
-
-
luaJIT_setmode(L, 0, LUAJIT_MODE_ENGINE|flag)
-
-Turn the whole JIT compiler on or off or flush the whole cache of compiled code.
-
-This sets the mode for the function at the stack index idx or
-the parent of the calling function (idx = 0). It either
-enables JIT compilation for a function, disables it and flushes any
-already compiled code, or only flushes already compiled code. This
-applies recursively to all sub-functions of the function with
-LUAJIT_MODE_ALLFUNC or only to the sub-functions with
-LUAJIT_MODE_ALLSUBFUNC.
-
-Flushes the specified root trace and all of its side traces from the cache.
-The code for the trace will be retained as long as there are any other
-traces which link to it.
-
-This mode defines a wrapper function for calls to C functions. If
-called with LUAJIT_MODE_ON, the stack index at idx
-must be a lightuserdata object holding a pointer to the wrapper
-function. From now on, all C functions are called through the wrapper
-function. If called with LUAJIT_MODE_OFF this mode is turned
-off and all C functions are directly called.
-
-
-The wrapper function can be used for debugging purposes or to catch
-and convert foreign exceptions. But please read the section on
-C++ exception interoperability
-first. Recommended usage can be seen in this C++ code excerpt:
-
-
-#include <exception>
-#include "lua.hpp"
-
-// Catch C++ exceptions and convert them to Lua error messages.
-// Customize as needed for your own exception classes.
-static int wrap_exceptions(lua_State *L, lua_CFunction f)
-{
- try {
- return f(L); // Call wrapped function and return result.
- } catch (const char *s) { // Catch and convert exceptions.
- lua_pushstring(L, s);
- } catch (std::exception& e) {
- lua_pushstring(L, e.what());
- } catch (...) {
- lua_pushliteral(L, "caught (...)");
- }
- return lua_error(L); // Rethrow as a Lua error.
-}
-
-static int myinit(lua_State *L)
-{
- ...
- // Define wrapper function and enable it.
- lua_pushlightuserdata(L, (void *)wrap_exceptions);
- luaJIT_setmode(L, -1, LUAJIT_MODE_WRAPCFUNC|LUAJIT_MODE_ON);
- lua_pop(L, 1);
- ...
-}
-
-
-Note that you can only define a single global wrapper function,
-so be careful when using this mechanism from multiple C++ modules.
-Also note that this mechanism is not without overhead.
-
-
-The FFI library allows calling external C functions and
-using C data structures from pure Lua code.
-
-
-
-
-The FFI library largely obviates the need to write tedious manual
-Lua/C bindings in C. No need to learn a separate binding language
-— it parses plain C declarations! These can be
-cut-n-pasted from C header files or reference manuals. It's up to
-the task of binding large libraries without the need for dealing with
-fragile binding generators.
-
-
-
-The FFI library is tightly integrated into LuaJIT (it's not available
-as a separate module). The code generated by the JIT-compiler for
-accesses to C data structures from Lua code is on par with the
-code a C compiler would generate. Calls to C functions can
-be inlined in JIT-compiled code, unlike calls to functions bound via
-the classic Lua/C API.
-
-
-This page gives a short introduction to the usage of the FFI library.
-Please use the FFI sub-topics in the navigation bar to learn more.
-
-
-
Motivating Example: Calling External C Functions
-
-It's really easy to call an external C library function:
-
-② Add a C declaration
-for the function. The part inside the double-brackets (in green) is
-just standard C syntax.
-
-
-③ Call the named
-C function — Yes, it's that simple!
-
-
-Actually, what goes on behind the scenes is far from simple: ③ makes use of the standard
-C library namespace ffi.C. Indexing this namespace with
-a symbol name ("printf") automatically binds it to the
-standard C library. The result is a special kind of object which,
-when called, runs the printf function. The arguments passed
-to this function are automatically converted from Lua objects to the
-corresponding C types.
-
-
-Ok, so maybe the use of printf() wasn't such a spectacular
-example. You could have done that with io.write() and
-string.format(), too. But you get the idea ...
-
-
-So here's something to pop up a message box on Windows:
-
-Compare this with the effort required to bind that function using the
-classic Lua/C API: create an extra C file, add a C function
-that retrieves and checks the argument types passed from Lua and calls
-the actual C function, add a list of module functions and their
-names, add a luaopen_* function and register all module
-functions, compile and link it into a shared library (DLL), move it to
-the proper path, add Lua code that loads the module aaaand ... finally
-call the binding function. Phew!
-
-
-
Motivating Example: Using C Data Structures
-
-The FFI library allows you to create and access C data
-structures. Of course, the main use for this is for interfacing with
-C functions. But they can be used stand-alone, too.
-
-
-Lua is built upon high-level data types. They are flexible, extensible
-and dynamic. That's why we all love Lua so much. Alas, this can be
-inefficient for certain tasks, where you'd really want a low-level
-data type. E.g. a large array of a fixed structure needs to be
-implemented with a big table holding lots of tiny tables. This imposes
-both a substantial memory overhead as well as a performance overhead.
-
-
-Here's a sketch of a library that operates on color images, plus a
-simple benchmark. First, the plain Lua version:
-
-
-local floor = math.floor
-
-local function image_ramp_green(n)
- local img = {}
- local f = 255/(n-1)
- for i=1,n do
- img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 }
- end
- return img
-end
-
-local function image_to_gray(img, n)
- for i=1,n do
- local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue)
- img[i].red = y; img[i].green = y; img[i].blue = y
- end
-end
-
-local N = 400*400
-local img = image_ramp_green(N)
-for i=1,1000 do
- image_to_gray(img, N)
-end
-
-
-This creates a table with 160.000 pixels, each of which is a table
-holding four number values in the range of 0-255. First, an image with
-a green ramp is created (1D for simplicity), then the image is
-converted to grayscale 1000 times. Yes, that's silly, but I was in
-need of a simple example ...
-
-
-And here's the FFI version. The modified parts have been marked in
-bold:
-
-
-①
-
-
-
-
-
-②
-
-③
-④
-
-
-
-
-
-
-③
-⑤local ffi = require("ffi")
-ffi.cdef[[
-typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;
-]]
-
-local function image_ramp_green(n)
- local img = ffi.new("rgba_pixel[?]", n)
- local f = 255/(n-1)
- for i=0,n-1 do
- img[i].green = i*f
- img[i].alpha = 255
- end
- return img
-end
-
-local function image_to_grey(img, n)
- for i=0,n-1 do
- local y = 0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue
- img[i].red = y; img[i].green = y; img[i].blue = y
- end
-end
-
-local N = 400*400
-local img = image_ramp_green(N)
-for i=1,1000 do
- image_to_grey(img, N)
-end
-
-
-Ok, so that wasn't too difficult:
-
-
-① First, load the FFI
-library and declare the low-level data type. Here we choose a
-struct which holds four byte fields, one for each component
-of a 4x8 bit RGBA pixel.
-
-
-② Creating the data
-structure with ffi.new() is straightforward — the
-'?' is a placeholder for the number of elements of a
-variable-length array.
-
-
-③ C arrays are
-zero-based, so the indexes have to run from 0 to
-n-1. One might want to allocate one more element instead to
-simplify converting legacy code.
-
-
-④ Since ffi.new()
-zero-fills the array by default, we only need to set the green and the
-alpha fields.
-
-
-⑤ The calls to
-math.floor() can be omitted here, because floating-point
-numbers are already truncated towards zero when converting them to an
-integer. This happens implicitly when the number is stored in the
-fields of each pixel.
-
-
-Now let's have a look at the impact of the changes: first, memory
-consumption for the image is down from 22 Megabytes to
-640 Kilobytes (400*400*4 bytes). That's a factor of 35x less! So,
-yes, tables do have a noticeable overhead. BTW: The original program
-would consume 40 Megabytes in plain Lua (on x64).
-
-
-Next, performance: the pure Lua version runs in 9.57 seconds (52.9
-seconds with the Lua interpreter) and the FFI version runs in 0.48
-seconds on my machine (YMMV). That's a factor of 20x faster (110x
-faster than the Lua interpreter).
-
-
-The avid reader may notice that converting the pure Lua version over
-to use array indexes for the colors ([1] instead of
-.red, [2] instead of .green etc.) ought to
-be more compact and faster. This is certainly true (by a factor of
-~1.7x). Switching to a struct-of-arrays would help, too.
-
-
-However, the resulting code would be less idiomatic and rather
-error-prone. And it still doesn't get even close to the performance of
-the FFI version of the code. Also, high-level data structures cannot
-be easily passed to other C functions, especially I/O functions,
-without undue conversion penalties.
-
-This page describes the API functions provided by the FFI library in
-detail. It's recommended to read through the
-introduction and the
-FFI tutorial first.
-
-
-
Glossary
-
-
cdecl — An abstract C type declaration (a Lua
-string).
-
ctype — A C type object. This is a special kind of
-cdata returned by ffi.typeof(). It serves as a
-cdataconstructor when called.
-
cdata — A C data object. It holds a value of the
-corresponding ctype.
-
ct — A C type specification which can be used for
-most of the API functions. Either a cdecl, a ctype or a
-cdata serving as a template type.
-
cb — A callback object. This is a C data object
-holding a special function pointer. Calling this function from
-C code runs an associated Lua function.
-
VLA — A variable-length array is declared with a
-? instead of the number of elements, e.g. "int[?]".
-The number of elements (nelem) must be given when it's
-created.
-
VLS — A variable-length struct is a struct C
-type where the last element is a VLA. The same rules for
-declaration and creation apply.
-
-
-
Declaring and Accessing External Symbols
-
-External symbols must be declared first and can then be accessed by
-indexing a C library
-namespace, which automatically binds the symbol to a specific
-library.
-
-
-
ffi.cdef(def)
-
-Adds multiple C declarations for types or external symbols (named
-variables or functions). def must be a Lua string. It's
-recommended to use the syntactic sugar for string arguments as
-follows:
-
-
-ffi.cdef[[
-typedef struct foo { int a, b; } foo_t; // Declare a struct and typedef.
-int dofoo(foo_t *f, int n); /* Declare an external C function. */
-]]
-
-
-The contents of the string (the part in green above) must be a
-sequence of
-C declarations,
-separated by semicolons. The trailing semicolon for a single
-declaration may be omitted.
-
-
-Please note, that external symbols are only declared, but they
-are not bound to any specific address, yet. Binding is
-achieved with C library namespaces (see below).
-
-
-C declarations are not passed through a C pre-processor,
-yet. No pre-processor tokens are allowed, except for
-#pragma pack. Replace #define in existing
-C header files with enum, static const
-or typedef and/or pass the files through an external
-C pre-processor (once). Be careful not to include unneeded or
-redundant declarations from unrelated header files.
-
-
-
ffi.C
-
-This is the default C library namespace — note the
-uppercase 'C'. It binds to the default set of symbols or
-libraries on the target system. These are more or less the same as a
-C compiler would offer by default, without specifying extra link
-libraries.
-
-
-On POSIX systems, this binds to symbols in the default or global
-namespace. This includes all exported symbols from the executable and
-any libraries loaded into the global namespace. This includes at least
-libc, libm, libdl (on Linux),
-libgcc (if compiled with GCC), as well as any exported
-symbols from the Lua/C API provided by LuaJIT itself.
-
-
-On Windows systems, this binds to symbols exported from the
-*.exe, the lua51.dll (i.e. the Lua/C API
-provided by LuaJIT itself), the C runtime library LuaJIT was linked
-with (msvcrt*.dll), kernel32.dll,
-user32.dll and gdi32.dll.
-
-
-
clib = ffi.load(name [,global])
-
-This loads the dynamic library given by name and returns
-a new C library namespace which binds to its symbols. On POSIX
-systems, if global is true, the library symbols are
-loaded into the global namespace, too.
-
-
-If name is a path, the library is loaded from this path.
-Otherwise name is canonicalized in a system-dependent way and
-searched in the default search path for dynamic libraries:
-
-
-On POSIX systems, if the name contains no dot, the extension
-.so is appended. Also, the lib prefix is prepended
-if necessary. So ffi.load("z") looks for "libz.so"
-in the default shared library search path.
-
-
-On Windows systems, if the name contains no dot, the extension
-.dll is appended. So ffi.load("ws2_32") looks for
-"ws2_32.dll" in the default DLL search path.
-
-
-
Creating cdata Objects
-
-The following API functions create cdata objects (type()
-returns "cdata"). All created cdata objects are
-garbage collected.
-
-Creates a cdata object for the given ct. VLA/VLS types
-require the nelem argument. The second syntax uses a ctype as
-a constructor and is otherwise fully equivalent.
-
-
-The cdata object is initialized according to the
-rules for initializers,
-using the optional init arguments. Excess initializers cause
-an error.
-
-
-Performance notice: if you want to create many objects of one kind,
-parse the cdecl only once and get its ctype with
-ffi.typeof(). Then use the ctype as a constructor repeatedly.
-
-
-Please note, that an anonymous struct declaration implicitly
-creates a new and distinguished ctype every time you use it for
-ffi.new(). This is probably not what you want,
-especially if you create more than one cdata object. Different anonymous
-structs are not considered assignment-compatible by the
-C standard, even though they may have the same fields! Also, they
-are considered different types by the JIT-compiler, which may cause an
-excessive number of traces. It's strongly suggested to either declare
-a named struct or typedef with ffi.cdef()
-or to create a single ctype object for an anonymous struct
-with ffi.typeof().
-
-
-
ctype = ffi.typeof(ct)
-
-Creates a ctype object for the given ct.
-
-
-This function is especially useful to parse a cdecl only once and then
-use the resulting ctype object as a constructor.
-
-
-
cdata = ffi.cast(ct, init)
-
-Creates a scalar cdata object for the given ct. The cdata
-object is initialized with init using the "cast" variant of
-the C type conversion
-rules.
-
-
-This functions is mainly useful to override the pointer compatibility
-checks or to convert pointers to addresses or vice versa.
-
-
-
ctype = ffi.metatype(ct, metatable)
-
-Creates a ctype object for the given ct and associates it with
-a metatable. Only struct/union types, complex numbers
-and vectors are allowed. Other types may be wrapped in a
-struct, if needed.
-
-
-The association with a metatable is permanent and cannot be changed
-afterwards. Neither the contents of the metatable nor the
-contents of an __index table (if any) may be modified
-afterwards. The associated metatable automatically applies to all uses
-of this type, no matter how the objects are created or where they
-originate from. Note that predefined operations on types have
-precedence (e.g. declared field names cannot be overridden).
-
-
-All standard Lua metamethods are implemented. These are called directly,
-without shortcuts, and on any mix of types. For binary operations, the
-left operand is checked first for a valid ctype metamethod. The
-__gc metamethod only applies to struct/union
-types and performs an implicit ffi.gc()
-call during creation of an instance.
-
-
-
cdata = ffi.gc(cdata, finalizer)
-
-Associates a finalizer with a pointer or aggregate cdata object. The
-cdata object is returned unchanged.
-
-
-This function allows safe integration of unmanaged resources into the
-automatic memory management of the LuaJIT garbage collector. Typical
-usage:
-
-
-local p = ffi.gc(ffi.C.malloc(n), ffi.C.free)
-...
-p = nil -- Last reference to p is gone.
--- GC will eventually run finalizer: ffi.C.free(p)
-
-
-A cdata finalizer works like the __gc metamethod for userdata
-objects: when the last reference to a cdata object is gone, the
-associated finalizer is called with the cdata object as an argument. The
-finalizer can be a Lua function or a cdata function or cdata function
-pointer. An existing finalizer can be removed by setting a nil
-finalizer, e.g. right before explicitly deleting a resource:
-
-
-ffi.C.free(ffi.gc(p, nil)) -- Manually free the memory.
-
-
-
C Type Information
-
-The following API functions return information about C types.
-They are most useful for inspecting cdata objects.
-
-
-
size = ffi.sizeof(ct [,nelem])
-
-Returns the size of ct in bytes. Returns nil if
-the size is not known (e.g. for "void" or function types).
-Requires nelem for VLA/VLS types, except for cdata objects.
-
-
-
align = ffi.alignof(ct)
-
-Returns the minimum required alignment for ct in bytes.
-
-
-
ofs [,bpos,bsize] = ffi.offsetof(ct, field)
-
-Returns the offset (in bytes) of field relative to the start
-of ct, which must be a struct. Additionally returns
-the position and the field size (in bits) for bit fields.
-
-
-
status = ffi.istype(ct, obj)
-
-Returns true if obj has the C type given by
-ct. Returns false otherwise.
-
-
-C type qualifiers (const etc.) are ignored. Pointers are
-checked with the standard pointer compatibility rules, but without any
-special treatment for void *. If ct specifies a
-struct/union, then a pointer to this type is accepted,
-too. Otherwise the types must match exactly.
-
-
-Note: this function accepts all kinds of Lua objects for the
-obj argument, but always returns false for non-cdata
-objects.
-
-
-
Utility Functions
-
-
err = ffi.errno([newerr])
-
-Returns the error number set by the last C function call which
-indicated an error condition. If the optional newerr argument
-is present, the error number is set to the new value and the previous
-value is returned.
-
-
-This function offers a portable and OS-independent way to get and set the
-error number. Note that only some C functions set the error
-number. And it's only significant if the function actually indicated an
-error condition (e.g. with a return value of -1 or
-NULL). Otherwise, it may or may not contain any previously set
-value.
-
-
-You're advised to call this function only when needed and as close as
-possible after the return of the related C function. The
-errno value is preserved across hooks, memory allocations,
-invocations of the JIT compiler and other internal VM activity. The same
-applies to the value returned by GetLastError() on Windows, but
-you need to declare and call it yourself.
-
-
-
str = ffi.string(ptr [,len])
-
-Creates an interned Lua string from the data pointed to by
-ptr.
-
-
-If the optional argument len is missing, ptr is
-converted to a "char *" and the data is assumed to be
-zero-terminated. The length of the string is computed with
-strlen().
-
-
-Otherwise ptr is converted to a "void *" and
-len gives the length of the data. The data may contain
-embedded zeros and need not be byte-oriented (though this may cause
-endianess issues).
-
-
-This function is mainly useful to convert (temporary)
-"const char *" pointers returned by
-C functions to Lua strings and store them or pass them to other
-functions expecting a Lua string. The Lua string is an (interned) copy
-of the data and bears no relation to the original data area anymore.
-Lua strings are 8 bit clean and may be used to hold arbitrary,
-non-character data.
-
-
-Performance notice: it's faster to pass the length of the string, if
-it's known. E.g. when the length is returned by a C call like
-sprintf().
-
-
-
ffi.copy(dst, src, len)
-ffi.copy(dst, str)
-
-Copies the data pointed to by src to dst.
-dst is converted to a "void *" and src
-is converted to a "const void *".
-
-
-In the first syntax, len gives the number of bytes to copy.
-Caveat: if src is a Lua string, then len must not
-exceed #src+1.
-
-
-In the second syntax, the source of the copy must be a Lua string. All
-bytes of the string plus a zero-terminator are copied to
-dst (i.e. #src+1 bytes).
-
-
-Performance notice: ffi.copy() may be used as a faster
-(inlinable) replacement for the C library functions
-memcpy(), strcpy() and strncpy().
-
-
-
ffi.fill(dst, len [,c])
-
-Fills the data pointed to by dst with len constant
-bytes, given by c. If c is omitted, the data is
-zero-filled.
-
-
-Performance notice: ffi.fill() may be used as a faster
-(inlinable) replacement for the C library function
-memset(dst, c, len). Please note the different
-order of arguments!
-
-
-
Target-specific Information
-
-
status = ffi.abi(param)
-
-Returns true if param (a Lua string) applies for the
-target ABI (Application Binary Interface). Returns false
-otherwise. The following parameters are currently defined:
-
-
-
-
Parameter
-
Description
-
-
-
32bit
32 bit architecture
-
-
64bit
64 bit architecture
-
-
le
Little-endian architecture
-
-
be
Big-endian architecture
-
-
fpu
Target has a hardware FPU
-
-
softfp
softfp calling conventions
-
-
hardfp
hardfp calling conventions
-
-
eabi
EABI variant of the standard ABI
-
-
win
Windows variant of the standard ABI
-
-
pauth
Pointer authentication ABI
-
-
uwp
Universal Windows Platform
-
-
gc64
64 bit GC references
-
-
-
ffi.os
-
-Contains the target OS name. Same contents as
-jit.os.
-
-
-
ffi.arch
-
-Contains the target architecture name. Same contents as
-jit.arch.
-
-
-
Methods for Callbacks
-
-The C types for callbacks
-have some extra methods:
-
-
-
cb:free()
-
-Free the resources associated with a callback. The associated Lua
-function is unanchored and may be garbage collected. The callback
-function pointer is no longer valid and must not be called again
-(it may be reused by a subsequently created callback).
-
-
-
cb:set(func)
-
-Associate a new Lua function with a callback. The C type of the
-callback and the callback function pointer are unchanged.
-
-
-This method is useful to dynamically switch the receiver of callbacks
-without creating a new callback each time and registering it again (e.g.
-with a GUI library).
-
-
-
Extended Standard Library Functions
-
-The following standard library functions have been extended to work
-with cdata objects:
-
-
-
n = tonumber(cdata)
-
-Converts a number cdata object to a double and returns it as
-a Lua number. This is particularly useful for boxed 64 bit
-integer values. Caveat: this conversion may incur a precision loss.
-
-
-
s = tostring(cdata)
-
-Returns a string representation of the value of 64 bit integers
-("nnnLL" or "nnnULL") or
-complex numbers ("re±imi"). Otherwise
-returns a string representation of the C type of a ctype object
-("ctype<type>") or a cdata object
-("cdata<type>: address"), unless you
-override it with a __tostring metamethod (see
-ffi.metatype()).
-
-Calls the __pairs or __ipairs metamethod of the
-corresponding ctype.
-
-
-
Extensions to the Lua Parser
-
-The parser for Lua source code treats numeric literals with the
-suffixes LL or ULL as signed or unsigned 64 bit
-integers. Case doesn't matter, but uppercase is recommended for
-readability. It handles decimal (42LL), hexadecimal
-(0x2aLL) and binary (0b101010LL) literals.
-
-
-The imaginary part of complex numbers can be specified by suffixing
-number literals with i or I, e.g. 12.5i.
-Caveat: you'll need to use 1i to get an imaginary part with
-the value one, since i itself still refers to a variable
-named i.
-
-This page describes the detailed semantics underlying the FFI library
-and its interaction with both Lua and C code.
-
-
-Given that the FFI library is designed to interface with C code
-and that declarations can be written in plain C syntax, it
-closely follows the C language semantics, wherever possible.
-Some minor concessions are needed for smoother interoperation with Lua
-language semantics.
-
-
-Please don't be overwhelmed by the contents of this page — this
-is a reference and you may need to consult it, if in doubt. It doesn't
-hurt to skim this page, but most of the semantics "just work" as you'd
-expect them to work. It should be straightforward to write
-applications using the LuaJIT FFI for developers with a C or C++
-background.
-
-
-
C Language Support
-
-The FFI library has a built-in C parser with a minimal memory
-footprint. It's used by the ffi.* library
-functions to declare C types or external symbols.
-
-
-Its only purpose is to parse C declarations, as found e.g. in
-C header files. Although it does evaluate constant expressions,
-it's not a C compiler. The body of inline
-C function definitions is simply ignored.
-
-
-Also, this is not a validating C parser. It expects and
-accepts correctly formed C declarations, but it may choose to
-ignore bad declarations or show rather generic error messages. If in
-doubt, please check the input against your favorite C compiler.
-
-
-The C parser complies to the C99 language standard plus
-the following extensions:
-
-
-
-
The '\e' escape in character and string literals.
-
-
The C99/C++ boolean type, declared with the keywords bool
-or _Bool.
-
-
Complex numbers, declared with the keywords complex or
-_Complex.
-
-
Two complex number types: complex (aka
-complex double) and complex float.
-
-
Vector types, declared with the GCC mode or
-vector_size attribute.
-
-
Unnamed ('transparent') struct/union fields
-inside a struct/union.
-
-
Incomplete enum declarations, handled like incomplete
-struct declarations.
-
-
Unnamed enum fields inside a
-struct/union. This is similar to a scoped C++
-enum, except that declared constants are visible in the
-global namespace, too.
-
-
Scoped static const declarations inside a
-struct/union (from C++).
-
-
Zero-length arrays ([0]), empty
-struct/union, variable-length arrays (VLA,
-[?]) and variable-length structs (VLS, with a trailing
-VLA).
-
-
C++ reference types (int &x).
-
-
Alternate GCC keywords with '__', e.g.
-__const__.
-
-
GCC __attribute__ with the following attributes:
-aligned, packed, mode,
-vector_size, cdecl, fastcall,
-stdcall, thiscall.
-
-
The GCC __extension__ keyword and the GCC
-__alignof__ operator.
-
-
GCC __asm__("symname") symbol name redirection for
-function declarations.
-
-
MSVC keywords for fixed-length types: __int8,
-__int16, __int32 and __int64.
-You're encouraged to use these types in preference to
-compiler-specific extensions or target-dependent standard types.
-E.g. char differs in signedness and long differs in
-size, depending on the target architecture and platform ABI.
-
-
-The following C features are not supported:
-
-
-
-
A declaration must always have a type specifier; it doesn't
-default to an int type.
-
-
Old-style empty function declarations (K&R) are not allowed.
-All C functions must have a proper prototype declaration. A
-function declared without parameters (int foo();) is
-treated as a function taking zero arguments, like in C++.
-
-
The long double C type is parsed correctly, but
-there's no support for the related conversions, accesses or arithmetic
-operations.
-
-
Wide character strings and character literals are not
-supported.
-
-
See below for features that are currently
-not implemented.
-
-
-
-
C Type Conversion Rules
-
-
Conversions from C types to Lua objects
-
-These conversion rules apply for read accesses to
-C types: indexing pointers, arrays or
-struct/union types; reading external variables or
-constant values; retrieving return values from C calls:
-
-
-
-
Input
-
Conversion
-
Output
-
-
-
int8_t, int16_t
→sign-extint32_t → double
number
-
-
uint8_t, uint16_t
→zero-extint32_t → double
number
-
-
int32_t, uint32_t
→ double
number
-
-
int64_t, uint64_t
boxed value
64 bit int cdata
-
-
double, float
→ double
number
-
-
bool
0 → false, otherwise true
boolean
-
-
enum
boxed value
enum cdata
-
-
Complex number
boxed value
complex cdata
-
-
Vector
boxed value
vector cdata
-
-
Pointer
boxed value
pointer cdata
-
-
Array
boxed reference
reference cdata
-
-
struct/union
boxed reference
reference cdata
-
-
-Bitfields are treated like their underlying type.
-
-
-Reference types are dereferenced before a conversion can take
-place — the conversion is applied to the C type pointed to
-by the reference.
-
-
-
Conversions from Lua objects to C types
-
-These conversion rules apply for write accesses to
-C types: indexing pointers, arrays or
-struct/union types; initializing cdata objects;
-casts to C types; writing to external variables; passing
-arguments to C calls:
-
-If the result type of this conversion doesn't match the
-C type of the destination, the
-conversion rules between C types
-are applied.
-
-
-Reference types are immutable after initialization ("no re-seating of
-references"). For initialization purposes or when passing values to
-reference parameters, they are treated like pointers. Note that unlike
-in C++, there's no way to implement automatic reference generation of
-variables under the Lua language semantics. If you want to call a
-function with a reference parameter, you need to explicitly pass a
-one-element array.
-
-
-
Conversions between C types
-
-These conversion rules are more or less the same as the standard
-C conversion rules. Some rules only apply to casts, or require
-pointer or type compatibility:
-
-
-
-
Input
-
Conversion
-
Output
-
-
-
Signed integer
→narrow or sign-extend
Integer
-
-
Unsigned integer
→narrow or zero-extend
Integer
-
-
Integer
→round
double, float
-
-
double, float
→truncint32_t →narrow
(u)int8_t, (u)int16_t
-
-
double, float
→trunc
(u)int32_t, (u)int64_t
-
-
double, float
→round
float, double
-
-
Number
n == 0 → 0, otherwise 1
bool
-
-
bool
false → 0, true → 1
Number
-
-
Complex number
convert real part
Number
-
-
Number
convert real part, imag = 0
Complex number
-
-
Complex number
convert real and imag part
Complex number
-
-
Number
convert scalar and replicate
Vector
-
-
Vector
copy (same size)
Vector
-
-
struct/union
take base address (compat)
Pointer
-
-
Array
take base address (compat)
Pointer
-
-
Function
take function address
Function pointer
-
-
Number
convert via uintptr_t (cast)
Pointer
-
-
Pointer
convert address (compat/cast)
Pointer
-
-
Pointer
convert address (cast)
Integer
-
-
Array
convert base address (cast)
Integer
-
-
Array
copy (compat)
Array
-
-
struct/union
copy (identical type)
struct/union
-
-
-Bitfields or enum types are treated like their underlying
-type.
-
-
-Conversions not listed above will raise an error. E.g. it's not
-possible to convert a pointer to a complex number or vice versa.
-
-
-
Conversions for vararg C function arguments
-
-The following default conversion rules apply when passing Lua objects
-to the variable argument part of vararg C functions:
-
-
-
-
Input
-
Conversion
-
Output
-
-
-
number
→
double
-
-
boolean
false → 0, true → 1
bool
-
-
nil
NULL →
(void *)
-
-
userdata
userdata payload →
(void *)
-
-
lightuserdata
lightuserdata address →
(void *)
-
-
string
string data →
const char *
-
-
float cdata
→
double
-
-
Array cdata
take base address
Element pointer
-
-
struct/union cdata
take base address
struct/union pointer
-
-
Function cdata
take function address
Function pointer
-
-
Any other cdata
no conversion
C type
-
-
-To pass a Lua object, other than a cdata object, as a specific type,
-you need to override the conversion rules: create a temporary cdata
-object with a constructor or a cast and initialize it with the value
-to pass:
-
-
-Assuming x is a Lua number, here's how to pass it as an
-integer to a vararg function:
-
-If you don't do this, the default Lua number → double
-conversion rule applies. A vararg C function expecting an integer
-will see a garbled or uninitialized value.
-
-
-Note: this is the only place where creating a boxed scalar number type is
-actually useful. Never use ffi.new("int"), ffi.new("float")
-etc. anywhere else!
-
-
-Ditto for ffi.cast(). Explicitly boxing scalars does not
-improve performance or force int or float arithmetic! It
-just adds costly boxing, unboxing and conversions steps. And it may lead
-to surprise results, because
-cdata arithmetic on scalar numbers
-is always performed on 64 bit integers.
-
-
-
Initializers
-
-Creating a cdata object with
-ffi.new() or the
-equivalent constructor syntax always initializes its contents, too.
-Different rules apply, depending on the number of optional
-initializers and the C types involved:
-
-
-
If no initializers are given, the object is filled with zero bytes.
Valarrays (complex numbers and vectors) are treated like scalars
-when a single initializer is given. Otherwise they are treated like
-regular arrays.
-
-
Aggregate types (arrays and structs) accept either a single cdata
-initializer of the same type (copy constructor), a single
-table initializer, or a flat list of
-initializers.
-
-
The elements of an array are initialized, starting at index zero.
-If a single initializer is given for an array, it's repeated for all
-remaining elements. This doesn't happen if two or more initializers
-are given: all remaining uninitialized elements are filled with zero
-bytes.
-
-
Byte arrays may also be initialized with a Lua string. This copies
-the whole string plus a terminating zero-byte. The copy stops early only
-if the array has a known, fixed size.
-
-
The fields of a struct are initialized in the order of
-their declaration. Uninitialized fields are filled with zero
-bytes.
-
-
Only the first field of a union can be initialized with a
-flat initializer.
-
-
Elements or fields which are aggregates themselves are initialized
-with a single initializer, but this may be a table
-initializer or a compatible aggregate.
-
-
Excess initializers cause an error.
-
-
-
-
Table Initializers
-
-The following rules apply if a Lua table is used to initialize an
-Array or a struct/union:
-
-
-
-
If the table index [0] is non-nil, then the
-table is assumed to be zero-based. Otherwise it's assumed to be
-one-based.
-
-
Array elements, starting at index zero, are initialized one-by-one
-with the consecutive table elements, starting at either index
-[0] or [1]. This process stops at the first
-nil table element.
-
-
If exactly one array element was initialized, it's repeated for
-all the remaining elements. Otherwise all remaining uninitialized
-elements are filled with zero bytes.
-
-
The above logic only applies to arrays with a known fixed size.
-A VLA is only initialized with the element(s) given in the table.
-Depending on the use case, you may need to explicitly add a
-NULL or 0 terminator to a VLA.
-
-
A struct/union can be initialized in the
-order of the declaration of its fields. Each field is initialized with
-consecutive table elements, starting at either index [0]
-or [1]. This process stops at the first nil table
-element.
-
-
Otherwise, if neither index [0] nor [1] is present,
-a struct/union is initialized by looking up each field
-name (as a string key) in the table. Each non-nil value is
-used to initialize the corresponding field.
-
-
Uninitialized fields of a struct are filled with zero
-bytes, except for the trailing VLA of a VLS.
-
-
Initialization of a union stops after one field has been
-initialized. If no field has been initialized, the union is
-filled with zero bytes.
-
-
Elements or fields which are aggregates themselves are initialized
-with a single initializer, but this may be a nested table
-initializer (or a compatible aggregate).
-
-
Excess initializers for an array cause an error. Excess
-initializers for a struct/union are ignored.
-Unrelated table entries are ignored, too.
-
-
-
-Example:
-
-
-local ffi = require("ffi")
-
-ffi.cdef[[
-struct foo { int a, b; };
-union bar { int i; double d; };
-struct nested { int x; struct foo y; };
-]]
-
-ffi.new("int[3]", {}) --> 0, 0, 0
-ffi.new("int[3]", {1}) --> 1, 1, 1
-ffi.new("int[3]", {1,2}) --> 1, 2, 0
-ffi.new("int[3]", {1,2,3}) --> 1, 2, 3
-ffi.new("int[3]", {[0]=1}) --> 1, 1, 1
-ffi.new("int[3]", {[0]=1,2}) --> 1, 2, 0
-ffi.new("int[3]", {[0]=1,2,3}) --> 1, 2, 3
-ffi.new("int[3]", {[0]=1,2,3,4}) --> error: too many initializers
-
-ffi.new("struct foo", {}) --> a = 0, b = 0
-ffi.new("struct foo", {1}) --> a = 1, b = 0
-ffi.new("struct foo", {1,2}) --> a = 1, b = 2
-ffi.new("struct foo", {[0]=1,2}) --> a = 1, b = 2
-ffi.new("struct foo", {b=2}) --> a = 0, b = 2
-ffi.new("struct foo", {a=1,b=2,c=3}) --> a = 1, b = 2 'c' is ignored
-
-ffi.new("union bar", {}) --> i = 0, d = 0.0
-ffi.new("union bar", {1}) --> i = 1, d = ?
-ffi.new("union bar", {[0]=1,2}) --> i = 1, d = ? '2' is ignored
-ffi.new("union bar", {d=2}) --> i = ?, d = 2.0
-
-ffi.new("struct nested", {1,{2,3}}) --> x = 1, y.a = 2, y.b = 3
-ffi.new("struct nested", {x=1,y={2,3}}) --> x = 1, y.a = 2, y.b = 3
-
-
-
Operations on cdata Objects
-
-All standard Lua operators can be applied to cdata objects or a
-mix of a cdata object and another Lua object. The following list shows
-the predefined operations.
-
-
-Reference types are dereferenced before performing each of
-the operations below — the operation is applied to the
-C type pointed to by the reference.
-
-
-The predefined operations are always tried first before deferring to a
-metamethod or index table (if any) for the corresponding ctype (except
-for __new). An error is raised if the metamethod lookup or
-index table lookup fails.
-
-
-
Indexing a cdata object
-
-
-
Indexing a pointer/array: a cdata pointer/array can be
-indexed by a cdata number or a Lua number. The element address is
-computed as the base address plus the number value multiplied by the
-element size in bytes. A read access loads the element value and
-converts it to a Lua object. A write
-access converts a Lua object to the element
-type and stores the converted value to the element. An error is
-raised if the element size is undefined or a write access to a
-constant element is attempted.
-
-
Dereferencing a struct/union field: a
-cdata struct/union or a pointer to a
-struct/union can be dereferenced by a string key,
-giving the field name. The field address is computed as the base
-address plus the relative offset of the field. A read access loads the
-field value and converts it to a Lua
-object. A write access converts a Lua
-object to the field type and stores the converted value to the
-field. An error is raised if a write access to a constant
-struct/union or a constant field is attempted.
-Scoped enum constants or static constants are treated like a constant
-field.
-
-
Indexing a complex number: a complex number can be indexed
-either by a cdata number or a Lua number with the values 0 or 1, or by
-the strings "re" or "im". A read access loads the
-real part ([0], .re) or the imaginary part
-([1], .im) part of a complex number and
-converts it to a Lua number. The
-sub-parts of a complex number are immutable — assigning to an
-index of a complex number raises an error. Accessing out-of-bound
-indexes returns unspecified results, but is guaranteed not to trigger
-memory access violations.
-
-
Indexing a vector: a vector is treated like an array for
-indexing purposes, except the vector elements are immutable —
-assigning to an index of a vector raises an error.
-
-
-
-A ctype object can be indexed with a string key, too. The only
-predefined operation is reading scoped constants of
-struct/union types. All other accesses defer
-to the corresponding metamethods or index tables (if any).
-
-
-Note: since there's (deliberately) no address-of operator, a cdata
-object holding a value type is effectively immutable after
-initialization. The JIT compiler benefits from this fact when applying
-certain optimizations.
-
-
-As a consequence, the elements of complex numbers and
-vectors are immutable. But the elements of an aggregate holding these
-types may be modified, of course. I.e. you cannot assign to
-foo.c.im, but you can assign a (newly created) complex number
-to foo.c.
-
-
-The JIT compiler implements strict aliasing rules: accesses to different
-types do not alias, except for differences in signedness (this
-applies even to char pointers, unlike C99). Type punning
-through unions is explicitly detected and allowed.
-
-
-
Calling a cdata object
-
-
-
Constructor: a ctype object can be called and used as a
-constructor. This is equivalent
-to ffi.new(ct, ...), unless a __new metamethod is
-defined. The __new metamethod is called with the ctype object
-plus any other arguments passed to the constructor. Note that you have to
-use ffi.new inside the metamethod, since calling ct(...)
-would cause infinite recursion.
-
-
C function call: a cdata function or cdata function
-pointer can be called. The passed arguments are
-converted to the C types of the
-parameters given by the function declaration. Arguments passed to the
-variable argument part of vararg C function use
-special conversion rules. This
-C function is called and the return value (if any) is
-converted to a Lua object.
-On Windows/x86 systems, __stdcall functions are automatically
-detected, and a function declared as __cdecl (the default) is
-silently fixed up after the first call.
-
-
-
-
Arithmetic on cdata objects
-
-
-
Pointer arithmetic: a cdata pointer/array and a cdata
-number or a Lua number can be added or subtracted. The number must be
-on the right-hand side for a subtraction. The result is a pointer of
-the same type with an address plus or minus the number value
-multiplied by the element size in bytes. An error is raised if the
-element size is undefined.
-
-
Pointer difference: two compatible cdata pointers/arrays
-can be subtracted. The result is the difference between their
-addresses, divided by the element size in bytes. An error is raised if
-the element size is undefined or zero.
-
-
64 bit integer arithmetic: the standard arithmetic
-operators (+ - * / % ^ and unary
-minus) can be applied to two cdata numbers, or a cdata number and a
-Lua number. If one of them is an uint64_t, the other side is
-converted to an uint64_t and an unsigned arithmetic operation
-is performed. Otherwise, both sides are converted to an
-int64_t and a signed arithmetic operation is performed. The
-result is a boxed 64 bit cdata object.
-
-If one of the operands is an enum and the other operand is a
-string, the string is converted to the value of a matching enum
-constant before the above conversion.
-
-These rules ensure that 64 bit integers are "sticky". Any
-expression involving at least one 64 bit integer operand results
-in another one. The undefined cases for the division, modulo and power
-operators return 2LL ^ 63 or
-2ULL ^ 63.
-
-You'll have to explicitly convert a 64 bit integer to a Lua
-number (e.g. for regular floating-point calculations) with
-tonumber(). But note this may incur a precision loss.
-
-
64 bit bitwise operations: the rules for 64 bit
-arithmetic operators apply analogously.
-
-Unlike the other bit.* operations, bit.tobit()
-converts a cdata number via int64_t to int32_t and
-returns a Lua number.
-
-For bit.band(), bit.bor() and bit.bxor(), the
-conversion to int64_t or uint64_t applies to
-all arguments, if any argument is a cdata number.
-
-For all other operations, only the first argument is used to determine
-the output type. This implies that a cdata number as a shift count for
-shifts and rotates is accepted, but that alone does not cause
-a cdata number output.
-
-
-
-
Comparisons of cdata objects
-
-
-
Pointer comparison: two compatible cdata pointers/arrays
-can be compared. The result is the same as an unsigned comparison of
-their addresses. nil is treated like a NULL pointer,
-which is compatible with any other pointer type.
-
-
64 bit integer comparison: two cdata numbers, or a
-cdata number and a Lua number can be compared with each other. If one
-of them is an uint64_t, the other side is converted to an
-uint64_t and an unsigned comparison is performed. Otherwise,
-both sides are converted to an int64_t and a signed
-comparison is performed.
-
-If one of the operands is an enum and the other operand is a
-string, the string is converted to the value of a matching enum
-constant before the above conversion.
-
-
Comparisons for equality/inequality never raise an error.
-Even incompatible pointers can be compared for equality by address. Any
-other incompatible comparison (also with non-cdata objects) treats the
-two sides as unequal.
-
-
-
-
cdata objects as table keys
-
-Lua tables may be indexed by cdata objects, but this doesn't provide
-any useful semantics — cdata objects are unsuitable as table
-keys!
-
-
-A cdata object is treated like any other garbage-collected object and
-is hashed and compared by its address for table indexing. Since
-there's no interning for cdata value types, the same value may be
-boxed in different cdata objects with different addresses. Thus,
-t[1LL+1LL] and t[2LL] usually do not point to
-the same hash slot, and they certainly do not point to the same
-hash slot as t[2].
-
-
-It would seriously drive up implementation complexity and slow down
-the common case, if one were to add extra handling for by-value
-hashing and comparisons to Lua tables. Given the ubiquity of their use
-inside the VM, this is not acceptable.
-
-
-There are three viable alternatives, if you really need to use cdata
-objects as keys:
-
-
-
-
If you can get by with the precision of Lua numbers
-(52 bits), then use tonumber() on a cdata number or
-combine multiple fields of a cdata aggregate to a Lua number. Then use
-the resulting Lua number as a key when indexing tables.
-One obvious benefit: t[tonumber(2LL)]does point to
-the same slot as t[2].
-
-
Otherwise, use either tostring() on 64 bit integers
-or complex numbers or combine multiple fields of a cdata aggregate to
-a Lua string (e.g. with
-ffi.string()). Then
-use the resulting Lua string as a key when indexing tables.
-
-
Create your own specialized hash table implementation using the
-C types provided by the FFI library, just like you would in
-C code. Ultimately, this may give much better performance than the
-other alternatives or what a generic by-value hash table could
-possibly provide.
-
-
-
-
Parameterized Types
-
-To facilitate some abstractions, the two functions
-ffi.typeof and
-ffi.cdef support
-parameterized types in C declarations. Note: none of the other API
-functions taking a cdecl allow this.
-
-
-Any place you can write a typedef name, an
-identifier or a number in a declaration, you can write
-$ (the dollar sign) instead. These placeholders are replaced in
-order of appearance with the arguments following the cdecl string:
-
-
--- Declare a struct with a parameterized field type and name:
-ffi.cdef([[
-typedef struct { $ $; } foo_t;
-]], type1, name1)
-
--- Anonymous struct with dynamic names:
-local bar_t = ffi.typeof("struct { int $, $; }", name1, name2)
--- Derived pointer type:
-local bar_ptr_t = ffi.typeof("$ *", bar_t)
-
--- Parameterized dimensions work even where a VLA won't work:
-local matrix_t = ffi.typeof("uint8_t[$][$]", width, height)
-
-
-Caveat: this is not simple text substitution! A passed ctype or
-cdata object is treated like the underlying type, a passed string is
-considered an identifier and a number is considered a number. You must
-not mix this up: e.g. passing "int" as a string doesn't work in
-place of a type, you'd need to use ffi.typeof("int") instead.
-
-
-The main use for parameterized types are libraries implementing abstract
-data types
-(» example),
-similar to what can be achieved with C++ template metaprogramming.
-Another use case are derived types of anonymous structs, which avoids
-pollution of the global struct namespace.
-
-
-Please note that parameterized types are a nice tool and indispensable
-for certain use cases. But you'll want to use them sparingly in regular
-code, e.g. when all types are actually fixed.
-
-
-
Garbage Collection of cdata Objects
-
-All explicitly (ffi.new(), ffi.cast() etc.) or
-implicitly (accessors) created cdata objects are garbage collected.
-You need to ensure to retain valid references to cdata objects
-somewhere on a Lua stack, an upvalue or in a Lua table while they are
-still in use. Once the last reference to a cdata object is gone, the
-garbage collector will automatically free the memory used by it (at
-the end of the next GC cycle).
-
-
-Please note, that pointers themselves are cdata objects, however they
-are not followed by the garbage collector. So e.g. if you
-assign a cdata array to a pointer, you must keep the cdata object
-holding the array alive as long as the pointer is still in use:
-
-
-ffi.cdef[[
-typedef struct { int *a; } foo_t;
-]]
-
-local s = ffi.new("foo_t", ffi.new("int[10]")) -- WRONG!
-
-local a = ffi.new("int[10]") -- OK
-local s = ffi.new("foo_t", a)
--- Now do something with 's', but keep 'a' alive until you're done.
-
-
-Similar rules apply for Lua strings which are implicitly converted to
-"const char *": the string object itself must be
-referenced somewhere or it'll be garbage collected eventually. The
-pointer will then point to stale data, which may have already been
-overwritten. Note that string literals are automatically kept
-alive as long as the function containing it (actually its prototype)
-is not garbage collected.
-
-
-Objects which are passed as an argument to an external C function
-are kept alive until the call returns. So it's generally safe to
-create temporary cdata objects in argument lists. This is a common
-idiom for passing specific C types to
-vararg functions.
-
-
-Memory areas returned by C functions (e.g. from malloc())
-must be manually managed, of course (or use
-ffi.gc()). Pointers to
-cdata objects are indistinguishable from pointers returned by C
-functions (which is one of the reasons why the GC cannot follow them).
-
-
-
Callbacks
-
-The LuaJIT FFI automatically generates special callback functions
-whenever a Lua function is converted to a C function pointer. This
-associates the generated callback function pointer with the C type
-of the function pointer and the Lua function object (closure).
-
-
-This can happen implicitly due to the usual conversions, e.g. when
-passing a Lua function to a function pointer argument. Or, you can use
-ffi.cast() to explicitly cast a Lua function to a
-C function pointer.
-
-
-Currently, only certain C function types can be used as callback
-functions. Neither C vararg functions nor functions with
-pass-by-value aggregate argument or result types are supported. There
-are no restrictions on the kind of Lua functions that can be called
-from the callback — no checks for the proper number of arguments
-are made. The return value of the Lua function will be converted to the
-result type, and an error will be thrown for invalid conversions.
-
-
-It's allowed to throw errors across a callback invocation, but it's not
-advisable in general. Do this only if you know the C function, that
-called the callback, copes with the forced stack unwinding and doesn't
-leak resources.
-
-
-One thing that's not allowed, is to let an FFI call into a C function
-get JIT-compiled, which in turn calls a callback, calling into Lua again.
-Usually this attempt is caught by the interpreter first and the
-C function is blacklisted for compilation.
-
-
-However, this heuristic may fail under specific circumstances: e.g. a
-message polling function might not run Lua callbacks right away and the call
-gets JIT-compiled. If it later happens to call back into Lua (e.g. a rarely
-invoked error callback), you'll get a VM PANIC with the message
-"bad callback". Then you'll need to manually turn off
-JIT-compilation with
-jit.off() for the
-surrounding Lua function that invokes such a message polling function (or
-similar).
-
-
-
Callback resource handling
-
-Callbacks take up resources — you can only have a limited number
-of them at the same time (500 - 1000, depending on the
-architecture). The associated Lua functions are anchored to prevent
-garbage collection, too.
-
-
-Callbacks due to implicit conversions are permanent! There is no
-way to guess their lifetime, since the C side might store the
-function pointer for later use (typical for GUI toolkits). The associated
-resources cannot be reclaimed until termination:
-
-
-ffi.cdef[[
-typedef int (__stdcall *WNDENUMPROC)(void *hwnd, intptr_t l);
-int EnumWindows(WNDENUMPROC func, intptr_t l);
-]]
-
--- Implicit conversion to a callback via function pointer argument.
-local count = 0
-ffi.C.EnumWindows(function(hwnd, l)
- count = count + 1
- return true
-end, 0)
--- The callback is permanent and its resources cannot be reclaimed!
--- Ok, so this may not be a problem, if you do this only once.
-
-
-Note: this example shows that you must properly declare
-__stdcall callbacks on Windows/x86 systems. The calling
-convention cannot be automatically detected, unlike for
-__stdcall calls to Windows functions.
-
-
-For some use cases, it's necessary to free up the resources or to
-dynamically redirect callbacks. Use an explicit cast to a
-C function pointer and keep the resulting cdata object. Then use
-the cb:free()
-or cb:set() methods
-on the cdata object:
-
-
--- Explicitly convert to a callback via cast.
-local count = 0
-local cb = ffi.cast("WNDENUMPROC", function(hwnd, l)
- count = count + 1
- return true
-end)
-
--- Pass it to a C function.
-ffi.C.EnumWindows(cb, 0)
--- EnumWindows doesn't need the callback after it returns, so free it.
-
-cb:free()
--- The callback function pointer is no longer valid and its resources
--- will be reclaimed. The created Lua closure will be garbage collected.
-
-
-
Callback performance
-
-Callbacks are slow! First, the C to Lua transition itself
-has an unavoidable cost, similar to a lua_call() or
-lua_pcall(). Argument and result marshalling add to that cost.
-And finally, neither the C compiler nor LuaJIT can inline or
-optimize across the language barrier and hoist repeated computations out
-of a callback function.
-
-
-Do not use callbacks for performance-sensitive work: e.g. consider a
-numerical integration routine which takes a user-defined function to
-integrate over. It's a bad idea to call a user-defined Lua function from
-C code millions of times. The callback overhead will be absolutely
-detrimental for performance.
-
-
-It's considerably faster to write the numerical integration routine
-itself in Lua — the JIT compiler will be able to inline the
-user-defined function and optimize it together with its calling context,
-with very competitive performance.
-
-
-As a general guideline: use callbacks only when you must, because
-of existing C APIs. E.g. callback performance is irrelevant for a
-GUI application, which waits for user input most of the time, anyway.
-
-
-For new designs avoid push-style APIs: a C function repeatedly
-calling a callback for each result. Instead, use pull-style APIs:
-call a C function repeatedly to get a new result. Calls from Lua
-to C via the FFI are much faster than the other way round. Most well-designed
-libraries already use pull-style APIs (read/write, get/put).
-
-
-
C Library Namespaces
-
-A C library namespace is a special kind of object which allows
-access to the symbols contained in shared libraries or the default
-symbol namespace. The default
-ffi.C namespace is
-automatically created when the FFI library is loaded. C library
-namespaces for specific shared libraries may be created with the
-ffi.load() API
-function.
-
-
-Indexing a C library namespace object with a symbol name (a Lua
-string) automatically binds it to the library. First, the symbol type
-is resolved — it must have been declared with
-ffi.cdef. Then the
-symbol address is resolved by searching for the symbol name in the
-associated shared libraries or the default symbol namespace. Finally,
-the resulting binding between the symbol name, the symbol type and its
-address is cached. Missing symbol declarations or nonexistent symbol
-names cause an error.
-
-
-This is what happens on a read access for the different kinds of
-symbols:
-
-
-
-
External functions: a cdata object with the type of the function
-and its address is returned.
-
-
External variables: the symbol address is dereferenced and the
-loaded value is converted to a Lua object
-and returned.
-
-
Constant values (static const or enum
-constants): the constant is converted to a
-Lua object and returned.
-
-
-
-This is what happens on a write access:
-
-
-
-
External variables: the value to be written is
-converted to the C type of the
-variable and then stored at the symbol address.
-
-
Writing to constant variables or to any other symbol type causes
-an error, like any other attempted write to a constant location.
-
-
-
-C library namespaces themselves are garbage collected objects. If
-the last reference to the namespace object is gone, the garbage
-collector will eventually release the shared library reference and
-remove all memory associated with the namespace. Since this may
-trigger the removal of the shared library from the memory of the
-running process, it's generally not safe to use function
-cdata objects obtained from a library if the namespace object may be
-unreferenced.
-
-
-Performance notice: the JIT compiler specializes to the identity of
-namespace objects and to the strings used to index it. This
-effectively turns function cdata objects into constants. It's not
-useful and actually counter-productive to explicitly cache these
-function objects, e.g. local strlen = ffi.C.strlen. OTOH, it
-is useful to cache the namespace itself, e.g. local C =
-ffi.C.
-
-
-
No Hand-holding!
-
-The FFI library has been designed as a low-level library. The
-goal is to interface with C code and C data types with a
-minimum of overhead. This means you can do anything you can do
-from C: access all memory, overwrite anything in memory, call
-machine code at any memory address and so on.
-
-
-The FFI library provides no memory safety, unlike regular Lua
-code. It will happily allow you to dereference a NULL
-pointer, to access arrays out of bounds or to misdeclare
-C functions. If you make a mistake, your application might crash,
-just like equivalent C code would.
-
-
-This behavior is inevitable, since the goal is to provide full
-interoperability with C code. Adding extra safety measures, like
-bounds checks, would be futile. There's no way to detect
-misdeclarations of C functions, since shared libraries only
-provide symbol names, but no type information. Likewise, there's no way
-to infer the valid range of indexes for a returned pointer.
-
-
-Again: the FFI library is a low-level library. This implies it needs
-to be used with care, but it's flexibility and performance often
-outweigh this concern. If you're a C or C++ developer, it'll be easy
-to apply your existing knowledge. OTOH, writing code for the FFI
-library is not for the faint of heart and probably shouldn't be the
-first exercise for someone with little experience in Lua, C or C++.
-
-
-As a corollary of the above, the FFI library is not safe for use by
-untrusted Lua code. If you're sandboxing untrusted Lua code, you
-definitely don't want to give this code access to the FFI library or
-to any cdata object (except 64 bit integers or complex
-numbers). Any properly engineered Lua sandbox needs to provide safety
-wrappers for many of the standard Lua library functions —
-similar wrappers need to be written for high-level operations on FFI
-data types, too.
-
-
-
Current Status
-
-The initial release of the FFI library has some limitations and is
-missing some features. Most of these will be fixed in future releases.
-
C declarations are not passed through a C pre-processor,
-yet.
-
The C parser is able to evaluate most constant expressions
-commonly found in C header files. However, it doesn't handle the
-full range of C expression semantics and may fail for some
-obscure constructs.
-
static const declarations only work for integer types
-up to 32 bits. Neither declaring string constants nor
-floating-point constants is supported.
-
Packed struct bitfields that cross container boundaries
-are not implemented.
-
Native vector types may be defined with the GCC mode or
-vector_size attribute. But no operations other than loading,
-storing and initializing them are supported, yet.
-
The volatile type qualifier is currently ignored by
-compiled code.
-
ffi.cdef silently
-ignores most re-declarations. Note: avoid re-declarations which do not
-conform to C99. The implementation will eventually be changed to
-perform strict checks.
-
-
-The JIT compiler already handles a large subset of all FFI operations.
-It automatically falls back to the interpreter for unimplemented
-operations (you can check for this with the
--jv command line option).
-The following operations are currently not compiled and may exhibit
-suboptimal performance, especially when used in inner loops:
-
-
-
Vector operations.
-
Table initializers.
-
Initialization of nested struct/union types.
-
Non-default initialization of VLA/VLS or large C types
-(> 128 bytes or > 16 array elements).
-
Bitfield initializations.
-
Pointer differences for element sizes that are not a power of
-two.
-
Calls to C functions with aggregates passed or returned by
-value.
-
Calls to ctype metamethods which are not plain functions.
-
ctype __newindex tables and non-string lookups in ctype
-__index tables.
-
tostring() for cdata types.
-
Calls to ffi.cdef(), ffi.load() and
-ffi.metatype().
-This page is intended to give you an overview of the features of the FFI
-library by presenting a few use cases and guidelines.
-
-
-This page makes no attempt to explain all of the FFI library, though.
-You'll want to have a look at the ffi.* API
-function reference and the FFI
-semantics to learn more.
-
-
-
Loading the FFI Library
-
-The FFI library is built into LuaJIT by default, but it's not loaded
-and initialized by default. The suggested way to use the FFI library
-is to add the following to the start of every Lua file that needs one
-of its functions:
-
-
-local ffi = require("ffi")
-
-
-Please note, this doesn't define an ffi variable in the table
-of globals — you really need to use the local variable. The
-require function ensures the library is only loaded once.
-
-
-Note: If you want to experiment with the FFI from the interactive prompt
-of the command line executable, omit the local, as it doesn't
-preserve local variables across lines.
-
-
-
Accessing Standard System Functions
-
-The following code explains how to access standard system functions.
-We slowly print two lines of dots by sleeping for 10 milliseconds
-after each dot:
-
-
-
-①
-
-
-
-
-
-②
-③
-④
-
-
-
-⑤
-
-
-
-
-
-⑥local ffi = require("ffi")
-ffi.cdef[[
-void Sleep(int ms);
-int poll(struct pollfd *fds, unsigned long nfds, int timeout);
-]]
-
-local sleep
-if ffi.os == "Windows" then
- function sleep(s)
- ffi.C.Sleep(s*1000)
- end
-else
- function sleep(s)
- ffi.C.poll(nil, 0, s*1000)
- end
-end
-
-for i=1,160 do
- io.write("."); io.flush()
- sleep(0.01)
-end
-io.write("\n")
-
-
-Here's the step-by-step explanation:
-
-
-① This defines the
-C library functions we're going to use. The part inside the
-double-brackets (in green) is just standard C syntax. You can
-usually get this info from the C header files or the
-documentation provided by each C library or C compiler.
-
-
-② The difficulty we're
-facing here, is that there are different standards to choose from.
-Windows has a simple Sleep() function. On other systems there
-are a variety of functions available to achieve sub-second sleeps, but
-with no clear consensus. Thankfully poll() can be used for
-this task, too, and it's present on most non-Windows systems. The
-check for ffi.os makes sure we use the Windows-specific
-function only on Windows systems.
-
-
-③ Here we're wrapping the
-call to the C function in a Lua function. This isn't strictly
-necessary, but it's helpful to deal with system-specific issues only
-in one part of the code. The way we're wrapping it ensures the check
-for the OS is only done during initialization and not for every call.
-
-
-④ A more subtle point is
-that we defined our sleep() function (for the sake of this
-example) as taking the number of seconds, but accepting fractional
-seconds. Multiplying this by 1000 gets us milliseconds, but that still
-leaves it a Lua number, which is a floating-point value. Alas, the
-Sleep() function only accepts an integer value. Luckily for
-us, the FFI library automatically performs the conversion when calling
-the function (truncating the FP value towards zero, like in C).
-
-
-Some readers will notice that Sleep() is part of
-KERNEL32.DLL and is also a stdcall function. So how
-can this possibly work? The FFI library provides the ffi.C
-default C library namespace, which allows calling functions from
-the default set of libraries, like a C compiler would. Also, the
-FFI library automatically detects stdcall functions, so you
-don't need to declare them as such.
-
-
-⑤ The poll()
-function takes a couple more arguments we're not going to use. You can
-simply use nil to pass a NULL pointer and 0
-for the nfds parameter. Please note, that the
-number 0does not convert to a pointer value,
-unlike in C++. You really have to pass pointers to pointer arguments
-and numbers to number arguments.
-
-
-The page on FFI semantics has all
-of the gory details about
-conversions between Lua
-objects and C types. For the most part you don't have to deal
-with this, as it's performed automatically and it's carefully designed
-to bridge the semantic differences between Lua and C.
-
-
-⑥ Now that we have defined
-our own sleep() function, we can just call it from plain Lua
-code. That wasn't so bad, huh? Turning these boring animated dots into
-a fascinating best-selling game is left as an exercise for the reader.
-:-)
-
-
-
Accessing the zlib Compression Library
-
-The following code shows how to access the » zlib compression library from Lua code.
-We'll define two convenience wrapper functions that take a string and
-compress or uncompress it to another string:
-
-
-
-①
-
-
-
-
-
-
-②
-
-
-③
-
-④
-
-
-⑤
-
-
-⑥
-
-
-
-
-
-
-
-⑦local ffi = require("ffi")
-ffi.cdef[[
-unsigned long compressBound(unsigned long sourceLen);
-int compress2(uint8_t *dest, unsigned long *destLen,
- const uint8_t *source, unsigned long sourceLen, int level);
-int uncompress(uint8_t *dest, unsigned long *destLen,
- const uint8_t *source, unsigned long sourceLen);
-]]
-local zlib = ffi.load(ffi.os == "Windows" and "zlib1" or "z")
-
-local function compress(txt)
- local n = zlib.compressBound(#txt)
- local buf = ffi.new("uint8_t[?]", n)
- local buflen = ffi.new("unsigned long[1]", n)
- local res = zlib.compress2(buf, buflen, txt, #txt, 9)
- assert(res == 0)
- return ffi.string(buf, buflen[0])
-end
-
-local function uncompress(comp, n)
- local buf = ffi.new("uint8_t[?]", n)
- local buflen = ffi.new("unsigned long[1]", n)
- local res = zlib.uncompress(buf, buflen, comp, #comp)
- assert(res == 0)
- return ffi.string(buf, buflen[0])
-end
-
--- Simple test code.
-local txt = string.rep("abcd", 1000)
-print("Uncompressed size: ", #txt)
-local c = compress(txt)
-print("Compressed size: ", #c)
-local txt2 = uncompress(c, #txt)
-assert(txt2 == txt)
-
-
-Here's the step-by-step explanation:
-
-
-① This defines some of the
-C functions provided by zlib. For the sake of this example, some
-type indirections have been reduced and it uses the predefined
-fixed-size integer types, while still adhering to the zlib API/ABI.
-
-
-② This loads the zlib shared
-library. On POSIX systems, it's named libz.so and usually
-comes pre-installed. Since ffi.load() automatically adds any
-missing standard prefixes/suffixes, we can simply load the
-"z" library. On Windows it's named zlib1.dll and
-you'll have to download it first from the
-» zlib site. The check for
-ffi.os makes sure we pass the right name to
-ffi.load().
-
-
-③ First, the maximum size of
-the compression buffer is obtained by calling the
-zlib.compressBound function with the length of the
-uncompressed string. The next line allocates a byte buffer of this
-size. The [?] in the type specification indicates a
-variable-length array (VLA). The actual number of elements of this
-array is given as the 2nd argument to ffi.new().
-
-
-④ This may look strange at
-first, but have a look at the declaration of the compress2
-function from zlib: the destination length is defined as a pointer!
-This is because you pass in the maximum buffer size and get back the
-actual length that was used.
-
-
-In C you'd pass in the address of a local variable
-(&buflen). But since there's no address-of operator in
-Lua, we'll just pass in a one-element array. Conveniently, it can be
-initialized with the maximum buffer size in one step. Calling the
-actual zlib.compress2 function is then straightforward.
-
-
-⑤ We want to return the
-compressed data as a Lua string, so we'll use ffi.string().
-It needs a pointer to the start of the data and the actual length. The
-length has been returned in the buflen array, so we'll just
-get it from there.
-
-
-Note that since the function returns now, the buf and
-buflen variables will eventually be garbage collected. This
-is fine, because ffi.string() has copied the contents to a
-newly created (interned) Lua string. If you plan to call this function
-lots of times, consider reusing the buffers and/or handing back the
-results in buffers instead of strings. This will reduce the overhead
-for garbage collection and string interning.
-
-
-⑥ The uncompress
-functions does the exact opposite of the compress function.
-The compressed data doesn't include the size of the original string,
-so this needs to be passed in. Otherwise, no surprises here.
-
-
-⑦ The code, that makes use
-of the functions we just defined, is just plain Lua code. It doesn't
-need to know anything about the LuaJIT FFI — the convenience
-wrapper functions completely hide it.
-
-
-One major advantage of the LuaJIT FFI is that you are now able to
-write those wrappers in Lua. And at a fraction of the time it
-would cost you to create an extra C module using the Lua/C API.
-Many of the simpler C functions can probably be used directly
-from your Lua code, without any wrappers.
-
-
-Side note: the zlib API uses the long type for passing
-lengths and sizes around. But all those zlib functions actually only
-deal with 32 bit values. This is an unfortunate choice for a
-public API, but may be explained by zlib's history — we'll just
-have to deal with it.
-
-
-First, you should know that a long is a 64 bit type e.g.
-on POSIX/x64 systems, but a 32 bit type on Windows/x64 and on
-32 bit systems. Thus a long result can be either a plain
-Lua number or a boxed 64 bit integer cdata object, depending on
-the target system.
-
-
-Ok, so the ffi.* functions generally accept cdata objects
-wherever you'd want to use a number. That's why we get a away with
-passing n to ffi.string() above. But other Lua
-library functions or modules don't know how to deal with this. So for
-maximum portability, one needs to use tonumber() on returned
-long results before passing them on. Otherwise the
-application might work on some systems, but would fail in a POSIX/x64
-environment.
-
-
-
Defining Metamethods for a C Type
-
-The following code explains how to define metamethods for a C type.
-We define a simple point type and add some operations to it:
-
-① This defines the C type for a
-two-dimensional point object.
-
-
-② We have to declare the variable
-holding the point constructor first, because it's used inside of a
-metamethod.
-
-
-③ Let's define an __add
-metamethod which adds the coordinates of two points and creates a new
-point object. For simplicity, this function assumes that both arguments
-are points. But it could be any mix of objects, if at least one operand
-is of the required type (e.g. adding a point plus a number or vice
-versa). Our __len metamethod returns the distance of a point to
-the origin.
-
-
-④ If we run out of operators, we can
-define named methods, too. Here, the __index table defines an
-area function. For custom indexing needs, one might want to
-define __index and __newindexfunctions instead.
-
-
-⑤ This associates the metamethods with
-our C type. This only needs to be done once. For convenience, a
-constructor is returned by
-ffi.metatype().
-We're not required to use it, though. The original C type can still
-be used e.g. to create an array of points. The metamethods automatically
-apply to any and all uses of this type.
-
-
-Please note, that the association with a metatable is permanent and
-the metatable must not be modified afterwards! Ditto for the
-__index table.
-
-
-⑥ Here are some simple usage examples
-for the point type and their expected results. The predefined
-operations (such as a.x) can be freely mixed with the newly
-defined metamethods. Note that area is a method and must be
-called with the Lua syntax for methods: a:area(), not
-a.area().
-
-
-The C type metamethod mechanism is most useful when used in
-conjunction with C libraries that are written in an object-oriented
-style. Creators return a pointer to a new instance, and methods take an
-instance pointer as the first argument. Sometimes you can just point
-__index to the library namespace and __gc to the
-destructor and you're done. But often enough you'll want to add
-convenience wrappers, e.g. to return actual Lua strings or when
-returning multiple values.
-
-
-Some C libraries only declare instance pointers as an opaque
-void * type. In this case you can use a fake type for all
-declarations, e.g. a pointer to a named (incomplete) struct will do:
-typedef struct foo_type *foo_handle. The C side doesn't
-know what you declare with the LuaJIT FFI, but as long as the underlying
-types are compatible, everything still works.
-
-
-
Translating C Idioms
-
-Here's a list of common C idioms and their translation to the
-LuaJIT FFI:
-
-
-
-
Idiom
-
C code
-
Lua code
-
-
-
Pointer dereference int *p;
x = *p; *p = y;
x = p[0] p[0] = y
-
-
Pointer indexing int i, *p;
x = p[i]; p[i+1] = y;
x = p[i] p[i+1] = y
-
-
Array indexing int i, a[];
x = a[i]; a[i+1] = y;
x = a[i] a[i+1] = y
-
-
struct/union dereference struct foo s;
x = s.field; s.field = y;
x = s.field s.field = y
-
-
struct/union pointer deref. struct foo *sp;
x = sp->field; sp->field = y;
x = s.field s.field = y
-
-
Pointer arithmetic int i, *p;
x = p + i; y = p - i;
x = p + i y = p - i
-
-
Pointer difference int *p1, *p2;
x = p1 - p2;
x = p1 - p2
-
-
Array element pointer int i, a[];
x = &a[i];
x = a+i
-
-
Cast pointer to address int *p;
x = (intptr_t)p;
x = tonumber( ffi.cast("intptr_t", p))
-
-
Functions with outargs void foo(int *inoutlen);
int len = x; foo(&len); y = len;
local len = ffi.new("int[1]", x) foo(len) y = len[0]
-This replaces several hash-table lookups with a (faster) direct use of
-a local or an upvalue. This is less important with LuaJIT, since the
-JIT compiler optimizes hash-table lookups a lot and is even able to
-hoist most of them out of the inner loops. It can't eliminate
-all of them, though, and it saves some typing for often-used
-functions. So there's still a place for this, even with LuaJIT.
-
-
-The situation is a bit different with C function calls via the
-FFI library. The JIT compiler has special logic to eliminate all
-of the lookup overhead for functions resolved from a
-C library namespace!
-Thus it's not helpful and actually counter-productive to cache
-individual C functions like this:
-
-
-local funca, funcb = ffi.C.funca, ffi.C.funcb -- Not helpful!
-local function foo(x, n)
- for i=1,n do funcb(funca(x, i), 1) end
-end
-
-
-This turns them into indirect calls and generates bigger and slower
-machine code. Instead, you'll want to cache the namespace itself and
-rely on the JIT compiler to eliminate the lookups:
-
-
-local C = ffi.C -- Instead use this!
-local function foo(x, n)
- for i=1,n do C.funcb(C.funca(x, i), 1) end
-end
-
-
-This generates both shorter and faster code. So don't cache
-C functions, but do cache namespaces! Most often the
-namespace is already in a local variable at an outer scope, e.g. from
-local lib = ffi.load(...). Note that copying
-it to a local variable in the function scope is unnecessary.
-
-The functions in this built-in module control the behavior of the JIT
-compiler engine. Note that JIT-compilation is fully automatic —
-you probably won't need to use any of the following functions unless
-you have special needs.
-
-
-
jit.on()
-jit.off()
-
-Turns the whole JIT compiler on (default) or off.
-
-
-These functions are typically used with the command line options
--j on or -j off.
-
-jit.on enables JIT compilation for a Lua function (this is
-the default).
-
-
-jit.off disables JIT compilation for a Lua function and
-flushes any already compiled code from the code cache.
-
-
-jit.flush flushes the code, but doesn't affect the
-enable/disable status.
-
-
-The current function, i.e. the Lua function calling this library
-function, can also be specified by passing true as the first
-argument.
-
-
-If the second argument is true, JIT compilation is also
-enabled, disabled or flushed recursively for all sub-functions of a
-function. With false only the sub-functions are affected.
-
-
-The jit.on and jit.off functions only set a flag
-which is checked when the function is about to be compiled. They do
-not trigger immediate compilation.
-
-
-Typical usage is jit.off(true, true) in the main chunk
-of a module to turn off JIT compilation for the whole module for
-debugging purposes.
-
-
-
jit.flush(tr)
-
-Flushes the root trace, specified by its number, and all of its side
-traces from the cache. The code for the trace will be retained as long
-as there are any other traces which link to it.
-
-
-
status, ... = jit.status()
-
-Returns the current status of the JIT compiler. The first result is
-either true or false if the JIT compiler is turned
-on or off. The remaining results are strings for CPU-specific features
-and enabled optimizations.
-
-
-
jit.version
-
-Contains the LuaJIT version string.
-
-
-
jit.version_num
-
-Contains the version number of the LuaJIT core. Version xx.yy.zz
-is represented by the decimal number xxyyzz.
-DEPRECATED after the switch to
-» rolling releases. zz is frozen at 99.
-
-
-
jit.os
-
-Contains the target OS name:
-"Windows", "Linux", "OSX", "BSD", "POSIX" or "Other".
-
-This submodule provides the backend for the -O command line
-option.
-
-
-You can also use it programmatically, e.g.:
-
-
-jit.opt.start(2) -- same as -O2
-jit.opt.start("-dce")
-jit.opt.start("hotloop=10", "hotexit=2")
-
-
-Unlike in LuaJIT 1.x, the module is built-in and
-optimization is turned on by default!
-It's no longer necessary to run require("jit.opt").start(),
-which was one of the ways to enable optimization.
-
-
-
jit.util.* — JIT compiler introspection
-
-This submodule holds functions to introspect the bytecode, generated
-traces, the IR and the generated machine code. The functionality
-provided by this module is still in flux and therefore undocumented.
-
-
-The debug modules -jbc, -jv and -jdump make
-extensive use of these functions. Please check out their source code,
-if you want to know more.
-
-LuaJIT has an integrated statistical profiler with very low overhead. It
-allows sampling the currently executing stack and other parameters in
-regular intervals.
-
-
-The integrated profiler can be accessed from three levels:
-
-The bundled high-level profiler offers basic profiling functionality. It
-generates simple textual summaries or source code annotations. It can be
-accessed with the -jp command line option
-or from Lua code by loading the underlying jit.p module.
-
-
-To cut to the chase — run this to get a CPU usage profile by
-function name:
-
-
-luajit -jp myapp.lua
-
-
-It's not a stated goal of the bundled profiler to add every
-possible option or to cater for special profiling needs. The low-level
-profiler APIs are documented below. They may be used by third-party
-authors to implement advanced functionality, e.g. IDE integration or
-graphical profilers.
-
-
-Note: Sampling works for both interpreted and JIT-compiled code. The
-results for JIT-compiled code may sometimes be surprising. LuaJIT
-heavily optimizes and inlines Lua code — there's no simple
-one-to-one correspondence between source code lines and the sampled
-machine code.
-
-
-
-jp=[options[,output]]
-
-The -jp command line option starts the high-level profiler.
-When the application run by the command line terminates, the profiler
-stops and writes the results to stdout or to the specified
-output file.
-
-
-The options argument specifies how the profiling is to be
-performed:
-
-
-
f — Stack dump: function name, otherwise module:line.
-This is the default mode.
r — Show raw sample counts. Default: show percentages.
-
a — Annotate excerpts from source code files.
-
A — Annotate complete source code files.
-
G — Produce raw output suitable for graphical tools.
-
m<number> — Minimum sample percentage to be shown.
-Default: 3%.
-
i<number> — Sampling interval in milliseconds.
-Default: 10ms.
-Note: The actual sampling precision is OS-dependent.
-
-
-The default output for -jp is a list of the most CPU consuming
-spots in the application. Increasing the stack dump depth with (say)
--jp=2 may help to point out the main callers or callees of
-hotspots. But sample aggregation is still flat per unique stack dump.
-
-
-To get a two-level view (split view) of callers/callees, use
--jp=s or -jp=-s. The percentages shown for the second
-level are relative to the first level.
-
-
-To see how much time is spent in each line relative to a function, use
--jp=fl.
-
-
-To see how much time is spent in different VM states or
-zones, use -jp=v or -jp=z.
-
-
-Combinations of v/z with f/F/l produce two-level
-views, e.g. -jp=vf or -jp=fv. This shows the time
-spent in a VM state or zone vs. hotspots. This can be used to answer
-questions like "Which time-consuming functions are only interpreted?" or
-"What's the garbage collector overhead for a specific function?".
-
-
-Multiple options can be combined — but not all combinations make
-sense, see above. E.g. -jp=3si4m1 samples three stack levels
-deep in 4ms intervals and shows a split view of the CPU consuming
-functions and their callers with a 1% threshold.
-
-
-Source code annotations produced by -jp=a or -jp=A are
-always flat and at the line level. Obviously, the source code files need
-to be readable by the profiler script.
-
-
-The high-level profiler can also be started and stopped from Lua code with:
-
-Zones can be used to provide information about different parts of an
-application to the high-level profiler. E.g. a game could make use of an
-"AI" zone, a "PHYS" zone, etc. Zones are hierarchical,
-organized as a stack.
-
-
-The jit.zone module needs to be loaded explicitly:
-
-
-local zone = require("jit.zone")
-
-
-
zone("name") pushes a named zone to the zone stack.
-
zone() pops the current zone from the zone stack and
-returns its name.
-
zone:get() returns the current zone name or nil.
-
zone:flush() flushes the zone stack.
-
-
-To show the time spent in each zone use -jp=z. To show the time
-spent relative to hotspots use e.g. -jp=zf or -jp=fz.
-
-
-
Low-level Lua API
-
-The jit.profile module gives access to the low-level API of the
-profiler from Lua code. This module needs to be loaded explicitly:
-
-local profile = require("jit.profile")
-
-
-This module can be used to implement your own higher-level profiler.
-A typical profiling run starts the profiler, captures stack dumps in
-the profiler callback, adds them to a hash table to aggregate the number
-of samples, stops the profiler and then analyzes all captured
-stack dumps. Other parameters can be sampled in the profiler callback,
-too. But it's important not to spend too much time in the callback,
-since this may skew the statistics.
-
-
-
profile.start(mode, cb)
-— Start profiler
-
-This function starts the profiler. The mode argument is a
-string holding options:
-
-
-
f — Profile with precision down to the function level.
-
l — Profile with precision down to the line level.
-
i<number> — Sampling interval in milliseconds (default
-10ms).
-Note: The actual sampling precision is OS-dependent.
-
-
-
-The cb argument is a callback function which is called with
-three arguments: (thread, samples, vmstate). The callback is
-called on a separate coroutine, the thread argument is the
-state that holds the stack to sample for profiling. Note: do
-not modify the stack of that state or call functions on it.
-
-
-samples gives the number of accumulated samples since the last
-callback (usually 1).
-
-
-vmstate holds the VM state at the time the profiling timer
-triggered. This may or may not correspond to the state of the VM when
-the profiling callback is called. The state is either 'N'
-native (compiled) code, 'I' interpreted code, 'C'
-C code, 'G' the garbage collector, or 'J' the JIT
-compiler.
-
-This function allows taking stack dumps in an efficient manner. It
-returns a string with a stack dump for the thread (coroutine),
-formatted according to the fmt argument:
-
-
-
p — Preserve the full path for module names. Otherwise,
-only the file name is used.
-
f — Dump the function name if it can be derived. Otherwise,
-use module:line.
-
F — Ditto, but dump module:name.
-
l — Dump module:line.
-
Z — Zap the following characters for the last dumped
-frame.
-
All other characters are added verbatim to the output string.
-
-
-The depth argument gives the number of frames to dump, starting
-at the topmost frame of the thread. A negative number dumps the frames in
-inverse order.
-
-
-The first example prints a list of the current module names and line
-numbers of up to 10 frames in separate lines. The second example prints
-semicolon-separated function names for all frames (up to 100) in inverse
-order:
-
-This function starts the profiler. See
-above for a description of the mode argument.
-
-
-The cb argument is a callback function with the following
-declaration:
-
-
-typedef void (*luaJIT_profile_callback)(void *data, lua_State *L,
- int samples, int vmstate);
-
-
-data is available for use by the callback. L is the
-state that holds the stack to sample for profiling. Note: do
-not modify this stack or call functions on this stack —
-use a separate coroutine for this purpose. See
-above for a description of samples and vmstate.
-
-
-
luaJIT_profile_stop(L)
-— Stop profiler
-
-This function stops the profiler.
-
-
-
p = luaJIT_profile_dumpstack(L, fmt, depth, len)
-— Dump stack
-
-This function allows taking stack dumps in an efficient manner.
-See above for a description of fmt
-and depth.
-
-
-This function returns a const char * pointing to a
-private string buffer of the profiler. The int *len
-argument returns the length of the output string. The buffer is
-overwritten on the next call and deallocated when the profiler stops.
-You either need to consume the content immediately or copy it for later
-use.
-
-LuaJIT is also fully ABI-compatible to Lua 5.1 at the linker/dynamic
-loader level. This means you can compile a C module against the
-standard Lua headers and load the same shared library from either Lua
-or LuaJIT.
-
-
-LuaJIT extends the standard Lua VM with new functionality and adds
-several extension modules. Please note, this page is only about
-functional enhancements and not about performance enhancements,
-such as the optimized VM, the faster interpreter or the JIT compiler.
-
-
-
Extensions Modules
-
-LuaJIT comes with several built-in extension modules:
-
-
-
bit.* — Bitwise operations
-
-LuaJIT supports all bitwise operations as defined by
-» Lua BitOp:
-
-This module is a LuaJIT built-in — you don't need to download or
-install Lua BitOp. The Lua BitOp site has full documentation for all
-» Lua BitOp API functions.
-The FFI adds support for
-64 bit bitwise operations,
-using the same API functions.
-
-
-Please make sure to require the module before using any of
-its functions:
-
-
-local bit = require("bit")
-
-
-An already installed Lua BitOp module is ignored by LuaJIT.
-This way you can use bit operations from both Lua and LuaJIT on a
-shared installation.
-
-
-
ffi.* — FFI library
-
-The FFI library allows calling external
-C functions and the use of C data structures from pure Lua
-code.
-
-Unlike the standard implementation in Lua 5.1, xpcall()
-passes any arguments after the error function to the function
-which is called in a protected context.
-
-
-
load*() handle UTF-8 source code
-
-Non-ASCII characters are handled transparently by the Lua source code parser.
-This allows the use of UTF-8 characters in identifiers and strings.
-A UTF-8 BOM is skipped at the start of the source code.
-
-
-
load*() add a mode parameter
-
-As an extension from Lua 5.2, the functions loadstring(),
-loadfile() and (new) load() add an optional
-mode parameter.
-
-
-The default mode string is "bt", which allows loading of both
-source code and bytecode. Use "t" to allow only source code
-or "b" to allow only bytecode to be loaded.
-
-
-By default, the load* functions generate the native bytecode format.
-For cross-compilation purposes, add W to the mode string to
-force the 32 bit format and X to force the 64 bit format.
-Add both to force the opposite format. Note that non-native bytecode
-generated by load* cannot be run, but can still be passed
-to string.dump.
-
-
-
tostring() etc. canonicalize NaN and ±Inf
-
-All number-to-string conversions consistently convert non-finite numbers
-to the same strings on all platforms. NaN results in "nan",
-positive infinity results in "inf" and negative infinity results
-in "-inf".
-
-
-
tonumber() etc. use builtin string to number conversion
-
-All string-to-number conversions consistently convert integer and
-floating-point inputs in decimal, hexadecimal and binary on all platforms.
-strtod() is not used anymore, which avoids numerous
-problems with poor C library implementations. The builtin conversion
-function provides full precision according to the IEEE-754 standard, it
-works independently of the current locale and it supports hex floating-point
-numbers (e.g. 0x1.5p-3).
-
-An extra argument has been added to string.dump(). If set to
-true or to a string which contains the character s,
-'stripped' bytecode without debug information is generated. This speeds
-up later bytecode loading and reduces memory usage. See also the
--b command line option.
-
-
-The generated bytecode is portable and can be loaded on any architecture
-that LuaJIT supports. However, the bytecode compatibility versions must
-match. Bytecode only stays compatible within a major+minor version
-(x.y.aaa → x.y.bbb), except for development branches. Foreign bytecode
-(e.g. from Lua 5.1) is incompatible and cannot be loaded.
-
-
-Note: LJ_GC64 mode requires a different frame layout, which implies
-a different, incompatible bytecode format between 32 bit and 64 bit ports.
-This may be rectified in the future. In the meantime, use the W
-and X modes of the load* functions
-for cross-compilation purposes.
-
-
-Due to VM hardening, bytecode is not deterministic. Add d to the
-mode string to dump it in a deterministic manner: identical source code
-always gives a byte-for-byte identical bytecode dump. This feature is
-mainly useful for reproducible builds.
-
-
-
table.new(narray, nhash) allocates a pre-sized table
-
-An extra library function table.new() can be made available via
-require("table.new"). This creates a pre-sized table, just like
-the C API equivalent lua_createtable(). This is useful for big
-tables if the final table size is known and automatic table resizing is
-too expensive.
-
-
-
table.clear(tab) clears a table
-
-An extra library function table.clear() can be made available
-via require("table.clear"). This clears all keys and values
-from a table, but preserves the allocated array/hash sizes. This is
-useful when a table, which is linked from multiple places, needs to be
-cleared and/or when recycling a table for use by the same context. This
-avoids managing backlinks, saves an allocation and the overhead of
-incremental array/hash part growth.
-
-
-Please note, this function is meant for very specific situations. In most
-cases it's better to replace the (usually single) link with a new table
-and let the GC do its work.
-
-
-
Enhanced PRNG for math.random()
-
-LuaJIT uses a Tausworthe PRNG with period 2^223 to implement
-math.random() and math.randomseed(). The quality of
-the PRNG results is much superior compared to the standard Lua
-implementation, which uses the platform-specific ANSI rand().
-
-
-The PRNG generates the same sequences from the same seeds on all
-platforms and makes use of all bits in the seed argument.
-math.random() without arguments generates 52 pseudo-random bits
-for every call. The result is uniformly distributed between 0.0 and 1.0.
-It's correctly scaled up and rounded for math.random(n [,m]) to
-preserve uniformity.
-
-
-Call math.randomseed() without any arguments to seed it from
-system entropy.
-
-
-Important: Neither this nor any other PRNG based on the simplistic
-math.random() API is suitable for cryptographic use.
-
-
-
io.* functions handle 64 bit file offsets
-
-The file I/O functions in the standard io.* library handle
-64 bit file offsets. In particular, this means it's possible
-to open files larger than 2 Gigabytes and to reposition or obtain
-the current file position for offsets beyond 2 GB
-(fp:seek() method).
-
-
-
debug.* functions identify metamethods
-
-debug.getinfo() and lua_getinfo() also return information
-about invoked metamethods. The namewhat field is set to
-"metamethod" and the name field has the name of
-the corresponding metamethod (e.g. "__index").
-
-
-
Fully Resumable VM
-
-The LuaJIT VM is fully resumable. This means you can yield from a
-coroutine even across contexts, where this would not possible with
-the standard Lua 5.1 VM: e.g. you can yield across pcall()
-and xpcall(), across iterators and across metamethods.
-
-
-
Extensions from Lua 5.2
-
-LuaJIT supports some language and library extensions from Lua 5.2.
-Features that are unlikely to break existing code are unconditionally
-enabled:
-
-
-
goto and ::labels::.
-
Hex escapes '\x3F' and '\z' escape in strings.
-
load(string|reader [, chunkname [,mode [,env]]]).
-
loadstring() is an alias for load().
-
loadfile(filename [,mode [,env]]).
-
math.log(x [,base]).
-
string.rep(s, n [,sep]).
-
string.format(): %q reversible.
-%s checks __tostring.
-%a and "%A added.
-
String matching pattern %g added.
-
io.read("*L").
-
io.lines() and file:lines() process
-io.read() options.
-
os.exit(status|true|false [,close]).
-
package.searchpath(name, path [, sep [, rep]]).
-
package.loadlib(name, "*").
-
debug.getinfo() returns nparams and isvararg
-for option "u".
-
debug.getlocal() accepts function instead of level.
-
debug.getlocal() and debug.setlocal() accept negative
-indexes for varargs.
-
debug.getupvalue() and debug.setupvalue() handle
-C functions.
-Other features are only enabled, if LuaJIT is built with
--DLUAJIT_ENABLE_LUA52COMPAT:
-
-
-
goto is a keyword and not a valid variable name anymore.
-
break can be placed anywhere. Empty statements (;;)
-are allowed.
-
__lt, __le are invoked for mixed types.
-
__len for tables. rawlen() library function.
-
pairs() and ipairs() check for __pairs and
-__ipairs.
-
coroutine.running() returns two results.
-
table.pack() and table.unpack()
-(same as unpack()).
-
io.write() and file:write() return file handle
-instead of true.
-
os.execute() and pipe:close() return detailed
-exit status.
-
debug.setmetatable() returns object.
-
debug.getuservalue() and debug.setuservalue().
-
Remove math.mod(), string.gfind().
-
package.searchers.
-
module() returns the module table.
-
-
-Note: this provides only partial compatibility with Lua 5.2 at the
-language and Lua library level. LuaJIT is API+ABI-compatible with
-Lua 5.1, which prevents implementing features that would otherwise
-break the Lua/C API and ABI (e.g. _ENV).
-
-
-
Extensions from Lua 5.3
-
-LuaJIT supports some extensions from Lua 5.3:
-
-
Unicode escape '\u{XX...}' embeds the UTF-8 encoding in string literals.
-
The argument table arg can be read (and modified) by LUA_INIT and -e chunks.
-
io.read() and file:read() accept formats with or without a leading *.
-
assert() accepts any type of error object.
-
table.move(a1, f, e, t [,a2]).
-
coroutine.isyieldable().
-
Lua/C API extensions:
-lua_isyieldable()
-
-
-
-
C++ Exception Interoperability
-
-LuaJIT has built-in support for interoperating with C++ exceptions.
-The available range of features depends on the target platform and
-the toolchain used to compile LuaJIT:
-
-
-
-
Platform
-
Compiler
-
Interoperability
-
-
-
External frame unwinding
-
GCC, Clang, MSVC
-
Full
-
-
-
Internal frame unwinding + DWARF2
-
GCC, Clang
-
Limited
-
-
-
Windows 64 bit
-
non-MSVC
-
Limited
-
-
-
Other platforms
-
Other compilers
-
No
-
-
-
-Full interoperability means:
-
-
-
C++ exceptions can be caught on the Lua side with pcall(),
-lua_pcall() etc.
-
C++ exceptions will be converted to the generic Lua error
-"C++ exception", unless you use the
-C call wrapper feature.
-
It's safe to throw C++ exceptions across non-protected Lua frames
-on the C stack. The contents of the C++ exception object
-pass through unmodified.
-
Lua errors can be caught on the C++ side with catch(...).
-The corresponding Lua error message can be retrieved from the Lua stack.
-
Throwing Lua errors across C++ frames is safe. C++ destructors
-will be called.
-
-
-Limited interoperability means:
-
-
-
C++ exceptions can be caught on the Lua side with pcall(),
-lua_pcall() etc.
-
C++ exceptions will be converted to the generic Lua error
-"C++ exception", unless you use the
-C call wrapper feature.
-
C++ exceptions will be caught by non-protected Lua frames and
-are rethrown as a generic Lua error. The C++ exception object will
-be destroyed.
-
Lua errors cannot be caught on the C++ side.
-
Throwing Lua errors across C++ frames will not call
-C++ destructors.
-
-
-
-No interoperability means:
-
-
-
It's not safe to throw C++ exceptions across Lua frames.
-
C++ exceptions cannot be caught on the Lua side.
-
Lua errors cannot be caught on the C++ side.
-
Throwing Lua errors across C++ frames will not call
-C++ destructors.
-
-
diff --git a/doc/extensions/buffer.md b/doc/extensions/buffer.md
new file mode 100644
index 00000000..b628d1e3
--- /dev/null
+++ b/doc/extensions/buffer.md
@@ -0,0 +1,338 @@
+# String Buffer Library
+
+The string buffer library allows **high-performance manipulation of string-like data**.
+
+Unlike Lua strings, which are constants, string buffers are **mutable** sequences of 8-bit (binary-transparent) characters. Data can be stored, formatted and encoded into a string buffer and later converted, extracted or decoded.
+
+The convenient string buffer API simplifies common string manipulation tasks, that would otherwise require creating many intermediate strings. String buffers improve performance by eliminating redundant memory copies, object creation, string interning and garbage collection overhead. In conjunction with the FFI library, they allow zero-copy operations.
+
+The string buffer library also includes a high-performance [serializer](#serialize) for Lua objects.
+
+## Using the String Buffer Library
+
+The string buffer library is built into LuaJIT by default, but it's not loaded by default. Add this to the start of every Lua file that needs one of its functions:
+
+```lua
+local buffer = require "string.buffer";
+```
+
+The convention for the syntax shown on this page is that `buffer` refers to the buffer library and `buf` refers to an individual buffer object.
+
+Please note the difference between a Lua function call, e.g. `buffer.new()` (with a dot) and a Lua method call, e.g. `buf:reset()` (with a colon).
+
+### Buffer Objects
+
+A buffer object is a garbage-collected Lua object. After creation with `buffer.new()`, it can (and should) be reused for many operations. When the last reference to a buffer object is gone, it will eventually be freed by the garbage collector, along with the allocated buffer space.
+
+Buffers operate like a FIFO (first-in first-out) data structure. Data can be appended (written) to the end of the buffer and consumed (read) from the front of the buffer. These operations may be freely mixed.
+
+The buffer space that holds the characters is managed automatically — it grows as needed and already consumed space is recycled. Use `buffer.new(size)` and `buf:free()`, if you need more control.
+
+The maximum size of a single buffer is the same as the maximum size of a Lua string, which is slightly below two gigabytes. For huge data sizes, neither strings nor buffers are the right data structure — use the FFI library to directly map memory or files up to the virtual memory limit of your OS.
+
+### Buffer Method Overview
+
+- The `buf:put*()`-like methods append (write) characters to the end of the buffer.
+- The `buf:get*()`-like methods consume (read) characters from the front of the buffer.
+- Other methods, like `buf:tostring()` only read the buffer contents, but don't change the buffer.
+- The `buf:set()` method allows zero-copy consumption of a string or an FFI cdata object as a buffer.
+- The FFI-specific methods allow zero-copy read/write-style operations or modifying the buffer contents in-place. Please check the [FFI caveats](#ffi_caveats) below, too.
+- Methods that don't need to return anything specific, return the buffer object itself as a convenience. This allows method chaining, e.g.: `buf:reset():encode(obj)` or `buf:skip(len):get()`
+
+## Buffer Creation and Management
+
+### `local buf = buffer.new([size [,options]])` `local buf = buffer.new([options])`
+
+Creates a new buffer object.
+
+The optional `size` argument ensures a minimum initial buffer size. This is strictly an optimization when the required buffer size is known beforehand. The buffer space will grow as needed, in any case.
+
+The optional table `options` sets various [serialization options](#serialize_options).
+
+### `buf = buf:reset()`
+
+Reset (empty) the buffer. The allocated buffer space is not freed and may be reused.
+
+### `buf = buf:free()`
+
+The buffer space of the buffer object is freed. The object itself remains intact, empty and may be reused.
+
+Note: you normally don't need to use this method. The garbage collector automatically frees the buffer space, when the buffer object is collected. Use this method, if you need to free the associated memory immediately.
+
+## Buffer Writers
+
+### `buf = buf:put([str|num|obj] [,…])`
+
+Appends a string `str`, a number `num` or any object `obj` with a `__tostring` metamethod to the buffer. Multiple arguments are appended in the given order.
+
+Appending a buffer to a buffer is possible and short-circuited internally. But it still involves a copy. Better combine the buffer writes to use a single buffer.
+
+### `buf = buf:putf(format, …)`
+
+Appends the formatted arguments to the buffer. The `format` string supports the same options as `string.format()`.
+
+### `buf = buf:putcdata(cdata, len)` FFI
+
+Appends the given `len` number of bytes from the memory pointed to by
+the FFI `cdata` object to the buffer. The object needs to be convertible
+to a (constant) pointer.
+
+### `buf = buf:set(str)` `buf = buf:set(cdata, len)` FFI
+
+This method allows zero-copy consumption of a string or an FFI cdata object as a buffer. It stores a reference to the passed string `str` or the FFI `cdata` object in the buffer. Any buffer space originally allocated is freed. This is *not* an append operation, unlike the `buf:put*()` methods.
+
+After calling this method, the buffer behaves as if `buf:free():put(str)` or `buf:free():put(cdata, len)` had been called. However, the data is only referenced and not copied, as long as the buffer is only consumed.
+
+In case the buffer is written to later on, the referenced data is copied and the object reference is removed (copy-on-write semantics).
+
+The stored reference is an anchor for the garbage collector and keeps the originally passed string or FFI cdata object alive.
+
+### `ptr, len = buf:reserve(size)` FFI `buf = buf:commit(used)` FFI
+
+The `reserve` method reserves at least `size` bytes of write space in the buffer. It returns an `uint8_t *` FFI cdata pointer `ptr` that points to this space.
+
+The available length in bytes is returned in `len`. This is at least `size` bytes, but may be more to facilitate efficient buffer growth. You can either make use of the additional space or ignore `len` and only use `size` bytes.
+
+The `commit` method appends the `used` bytes of the previously returned write space to the buffer data.
+
+This pair of methods allows zero-copy use of C read-style APIs:
+
+```lua
+local MIN_SIZE = 65536;
+
+repeat
+ local ptr, len = buf:reserve(MIN_SIZE);
+ local n = C.read(fd, ptr, len);
+ if n == 0 then break end -- EOF.
+ if n < 0 then error "read error" end
+
+ buf:commit(n);
+until false;
+```
+
+The reserved write space is *not* initialized. At least the `used` bytes
+**must** be written to before calling the `commit` method. There's no
+need to call the `commit` method, if nothing is added to the buffer
+(e.g. on error).
+
+## Buffer Readers
+
+### `len = #buf`
+
+Returns the current length of the buffer data in bytes.
+
+### `res = str|num|buf .. str|num|buf […]`
+
+The Lua concatenation operator `..` also accepts buffers, just like strings or numbers. It always returns a string and not a buffer.
+
+Note that although this is supported for convenience, this thwarts one of the main reasons to use buffers, which is to avoid string allocations. Rewrite it with `buf:put()` and `buf:get()`.
+
+Mixing this with unrelated objects that have a `__concat` metamethod may not work, since these probably only expect strings.
+
+### `buf = buf:skip(len)`
+
+Skips (consumes) `len` bytes from the buffer up to the current length of the buffer data.
+
+### `str, … = buf:get([len|nil] [,…])`
+
+Consumes the buffer data and returns one or more strings. If called without arguments, the whole buffer data is consumed. If called with a number, up to `len` bytes are consumed. A `nil` argument consumes the remaining buffer space (this only makes sense as the last argument). Multiple arguments consume the buffer data in the given order.
+
+Note: a zero length or no remaining buffer data returns an empty string and not `nil`.
+
+### `str = buf:tostring()` `str = tostring(buf)`
+
+Creates a string from the buffer data, but doesn't consume it. The buffer remains unchanged.
+
+Buffer objects also define a `__tostring` metamethod. This means buffers can be passed to the global `tostring()` function and many other functions that accept this in place of strings. The important internal uses in functions like `io.write()` are short-circuited to avoid the creation of an intermediate string object.
+
+### `ptr, len = buf:ref()` FFI
+
+Returns an `uint8_t *` FFI cdata pointer `ptr` that points to the buffer data. The length of the buffer data in bytes is returned in `len`.
+
+The returned pointer can be directly passed to C functions that expect a buffer and a length. You can also do bytewise reads (`local x = ptr[i]`) or writes (`ptr[i] = 0x40`) of the buffer data.
+
+In conjunction with the `skip` method, this allows zero-copy use of C write-style APIs:
+
+```lua
+repeat
+ local ptr, len = buf:ref();
+ if len == 0 then break end
+
+ local n = C.write(fd, ptr, len);
+ if n < 0 then error "write error" end
+
+ buf:skip(n);
+until n >= len;
+```
+
+Unlike Lua strings, buffer data is *not* implicitly zero-terminated. It's not safe to pass `ptr` to C functions that expect zero-terminated strings. If you're not using `len`, then you're doing something wrong.
+
+## Serialization of Lua Objects
+
+The following functions and methods allow **high-speed serialization** (encoding) of a Lua object into a string and decoding it back to a Lua object. This allows convenient storage and transport of **structured data**.
+
+The encoded data is in an [internal binary format](#serialize_format). The data can be stored in files, binary-transparent databases or transmitted to other LuaJIT instances across threads, processes or networks.
+
+Encoding speed can reach up to 1 Gigabyte/second on a modern desktop - or server-class system, even when serializing many small objects. Decoding speed is mostly constrained by object creation cost.
+
+The serializer handles most Lua types, common FFI number types and nested structures. Functions, thread objects, other FFI cdata and full userdata cannot be serialized (yet).
+
+The encoder serializes nested structures as trees. Multiple references to a single object will be stored separately and create distinct objects after decoding. Circular references cause an error.
+
+### Serialization Functions and Methods
+
+### `str = buffer.encode(obj)` `buf = buf:encode(obj)`
+
+Serializes (encodes) the Lua object `obj`. The stand-alone function returns a string `str`. The buffer method appends the encoding to the buffer.
+
+`obj` can be any of the supported Lua types — it doesn't need to be a Lua table.
+
+This function may throw an error when attempting to serialize unsupported object types, circular references or deeply nested tables.
+
+### `obj = buffer.decode(str)` `obj = buf:decode()`
+
+The stand-alone function deserializes (decodes) the string `str`, the buffer method deserializes one object from the buffer. Both return a Lua object `obj`.
+
+The returned object may be any of the supported Lua types — even `nil`.
+
+This function may throw an error when fed with malformed or incomplete encoded data. The stand-alone function throws when there's left-over data after decoding a single top-level object. The buffer method leaves any left-over data in the buffer.
+
+Attempting to deserialize an FFI type will throw an error, if the FFI library is not built-in or has not been loaded, yet.
+
+### Serialization Options
+
+The `options` table passed to `buffer.new()` may contain the following members (all optional):
+
+- `dict` is a Lua table holding a **dictionary of strings** that commonly occur as table keys of objects you are serializing. These keys are compactly encoded as indexes during serialization. A well-chosen dictionary saves space and improves serialization performance.
+- `metatable` is a Lua table holding a **dictionary of metatables** for the table objects you are serializing.
+
+`dict` needs to be an array of strings and `metatable` needs to be an array of tables. Both starting at index 1 and without holes (no `nil` in between). The tables are anchored in the buffer object and internally modified into a two-way index (don't do this yourself, just pass a plain array). The tables must not be modified after they have been passed to `buffer.new()`.
+
+The `dict` and `metatable` tables used by the encoder and decoder must be the same. Put the most common entries at the front. Extend at the end to ensure backwards-compatibility — older encodings can then still be read. You may also set some indexes to `false` to explicitly drop backwards-compatibility. Old encodings that use these indexes will throw an error when decoded.
+
+Metatables that are not found in the `metatable` dictionary are ignored when encoding. Decoding returns a table with a `nil` metatable.
+
+Note: parsing and preparation of the options table is somewhat expensive. Create a buffer object only once and recycle it for multiple uses. Avoid mixing encoder and decoder buffers, since the `buf:set()` method frees the already allocated buffer space:
+
+```lua
+local options = { dict = { "commonly", "used", "string", "keys" } };
+local buf_enc = buffer.new(options);
+local buf_dec = buffer.new(options);
+
+local function encode(obj)
+ return buf_enc:reset():encode(obj):get();
+end
+
+local function decode(str)
+ return buf_dec:set(str):decode();
+end
+```
+
+### Streaming Serialization
+
+In some contexts, it's desirable to do piecewise serialization of large datasets, also known as *streaming*.
+
+This serialization format can be safely concatenated and supports streaming. Multiple encodings can simply be appended to a buffer and later decoded individually:
+
+```lua
+local buf = buffer.new();
+buf:encode(obj1);
+buf:encode(obj2);
+
+local copy1 = buf:decode();
+local copy2 = buf:decode();
+```
+
+Here's how to iterate over a stream:
+
+```lua
+while #buf ~= 0 do
+ local obj = buf:decode();
+ -- Do something with obj
+end
+```
+
+Since the serialization format doesn't prepend a length to its encoding, network applications may need to transmit the length, too.
+
+### Serialization Format Specification
+
+This serialization format is designed for **internal use** by LuaJIT applications. Serialized data is upwards-compatible and portable across all supported LuaJIT platforms.
+
+It's an **8-bit binary format** and not human-readable. It uses e.g. embedded zeroes and stores embedded Lua string objects unmodified, which are 8-bit-clean, too. Encoded data can be safely concatenated for streaming and later decoded one top-level object at a time.
+
+The encoding is reasonably compact, but tuned for maximum performance, not for minimum space usage. It compresses well with any of the common byte-oriented data compression algorithms.
+
+Although documented here for reference, this format is explicitly **not** intended to be a 'public standard' for structured data interchange across computer languages (like JSON or MessagePack). Please do not use it as such.
+
+The specification is given below as a context-free grammar with a top-level `object` as the starting point. Alternatives are separated by the `|` symbol and `*` indicates repeats. Grouping is implicit or indicated by `{…}`. Terminals are either plain hex numbers, encoded as bytes, or have a `.format` suffix.
+
+```
+object → nil | false | true
+ | null | lightud32 | lightud64
+ | int | num | tab | tab_mt
+ | int64 | uint64 | complex
+ | string
+
+nil → 0x00
+false → 0x01
+true → 0x02
+
+null → 0x03 // NULL lightuserdata
+lightud32 → 0x04 data.I // 32 bit lightuserdata
+lightud64 → 0x05 data.L // 64 bit lightuserdata
+
+int → 0x06 int.I // int32_t
+num → 0x07 double.L
+
+tab → 0x08 // Empty table
+ | 0x09 h.U h*{object object} // Key/value hash
+ | 0x0a a.U a*object // 0-based array
+ | 0x0b a.U h.U a*object h*{object object} // Mixed
+ | 0x0c a.U (a-1)*object // 1-based array
+ | 0x0d a.U h.U (a-1)*object h*{object object} // Mixed
+tab_mt → 0x0e (index-1).U tab // Metatable dict entry
+
+int64 → 0x10 int.L // FFI int64_t
+uint64 → 0x11 uint.L // FFI uint64_t
+complex → 0x12 re.L im.L // FFI complex
+
+string → (0x20+len).U len*char.B
+ | 0x0f (index-1).U // String dict entry
+
+.B = 8 bit
+.I = 32 bit little-endian
+.L = 64 bit little-endian
+.U = prefix-encoded 32 bit unsigned number n:
+ 0x00..0xdf → n.B
+ 0xe0..0x1fdf → (0xe0|(((n-0xe0)>>8)&0x1f)).B ((n-0xe0)&0xff).B
+ 0x1fe0.. → 0xff n.I
+```
+
+## Error handling
+
+Many of the buffer methods can throw an error. Out-of-memory or usage errors are best caught with an outer wrapper for larger parts of code. There's not much one can do after that, anyway.
+
+On the other hand, you may want to catch some errors individually. Buffer methods need to receive the buffer object as the first argument. The Lua colon-syntax `obj:method()` does that implicitly. But to wrap a method with `pcall()`, the arguments need to be passed like this:
+
+```lua
+local ok, err = pcall(buf.encode, buf, obj);
+if not ok then
+ -- Handle error in err
+end
+```
+
+## FFI caveats
+
+The string buffer library has been designed to work well together with the FFI library. But due to the low-level nature of the FFI library, some care needs to be taken:
+
+First, please remember that FFI pointers are zero-indexed. The space returned by `buf:reserve()` and `buf:ref()` starts at the returned pointer and ends before `len` bytes after that.
+
+I.e. the first valid index is `ptr[0]` and the last valid index is `ptr[len-1]`. If the returned length is zero, there's no valid index at all. The returned pointer may even be `NULL`.
+
+The space pointed to by the returned pointer is only valid as long as the buffer is not modified in any way (neither append, nor consume, nor reset, etc.). The pointer is also not a GC anchor for the buffer object itself.
+
+Buffer data is only guaranteed to be byte-aligned. Casting the returned pointer to a data type with higher alignment may cause unaligned accesses. It depends on the CPU architecture whether this is allowed or not (it's always OK on x86/x64 and mostly OK on other modern architectures).
+
+FFI pointers or references do not count as GC anchors for an underlying object. E.g. an `array` allocated with `ffi.new()` is anchored by `buf:set(array, len)`, but not by `buf:set(array+offset, len)`. The addition of the offset creates a new pointer, even when the offset is zero. In this case, you need to make sure there's still a reference to the original array as long as its contents are in use by the buffer.
+
+Even though each LuaJIT VM instance is single-threaded (but you can create multiple VMs), FFI data structures can be accessed concurrently. Be careful when reading/writing FFI cdata from/to buffers to avoid concurrent accesses or modifications. In particular, the memory referenced by `buf:set(cdata, len)` must not be modified while buffer readers are working on it. Shared, but read-only memory mappings of files are OK, but only if the file does not change.
diff --git a/doc/extensions/c-api.md b/doc/extensions/c-api.md
new file mode 100644
index 00000000..fab8abcd
--- /dev/null
+++ b/doc/extensions/c-api.md
@@ -0,0 +1,79 @@
+# Lua/C API Extensions
+
+LuaJIT adds some extensions to the standard Lua/C API. The LuaJIT include directory must be in the compiler search path (`-I`*`path`*) to be able to include the required header for C code:
+
+```c
+#include "luajit.h"
+// Or this for C++ code
+#include "lua.hpp"
+```
+
+## `luaJIT_setmode(L, idx, mode)` — Control VM
+
+This is a C API extension to allow control of the VM from C code. The full prototype of `LuaJIT_setmode` is:
+
+```c
+LUA_API int luaJIT_setmode(lua_State *L, int idx, int mode);
+```
+
+The returned status is either success (`1`) or failure (`0`). The second argument is either `0` or a stack index (similar to the other Lua/C API functions).
+
+The third argument specifies the mode, which is 'or'ed with a flag. The flag can be `LUAJIT_MODE_OFF` to turn a feature off, `LUAJIT_MODE_ON` to turn a feature on, or `LUAJIT_MODE_FLUSH` to flush cached code.
+
+The following modes are defined:
+
+### `luaJIT_setmode(L, 0, LUAJIT_MODE_ENGINE|flag)`
+
+Turn the whole JIT compiler on or off or flush the whole cache of compiled code.
+
+### `luaJIT_setmode(L, idx, LUAJIT_MODE_FUNC|flag)` `luaJIT_setmode(L, idx, LUAJIT_MODE_ALLFUNC|flag)` `luaJIT_setmode(L, idx, LUAJIT_MODE_ALLSUBFUNC|flag)`
+
+This sets the mode for the function at the stack index `idx` or the parent of the calling function (`idx = 0`). It either enables JIT compilation for a function, disables it and flushes any already compiled code, or only flushes already compiled code. This applies recursively to all sub-functions of the function with `LUAJIT_MODE_ALLFUNC` or only to the sub-functions with `LUAJIT_MODE_ALLSUBFUNC`.
+
+### `luaJIT_setmode(L, trace,` ` LUAJIT_MODE_TRACE|LUAJIT_MODE_FLUSH)`
+
+Flushes the specified root trace and all of its side traces from the cache. The code for the trace will be retained as long as there are any other traces which link to it.
+
+### `luaJIT_setmode(L, idx, LUAJIT_MODE_WRAPCFUNC|flag)`
+
+This mode defines a wrapper function for calls to C functions. If called with `LUAJIT_MODE_ON`, the stack index at `idx` must be a `lightuserdata` object holding a pointer to the wrapper function. From now on, all C functions are called through the wrapper function. If called with `LUAJIT_MODE_OFF` this mode is turned off and all C functions are directly called.
+
+The wrapper function can be used for debugging purposes or to catch and convert foreign exceptions. But please read the section on [C++ exception interoperability](./extensions#exceptions) first. Recommended usage can be seen in this C++ code excerpt:
+
+```c++
+#include
+#include "lua.hpp"
+
+// Catch C++ exceptions and convert them to Lua error messages.
+// Customize as needed for your own exception classes.
+static int wrap_exceptions(lua_State *L, lua_CFunction f) {
+ try {
+ // Call wrapped function and return result
+ return f(L);
+ }
+ catch (const char *s) {
+ // Catch and convert exceptions
+ lua_pushstring(L, s);
+ }
+ catch (std::exception& e) {
+ lua_pushstring(L, e.what());
+ }
+ catch (...) {
+ lua_pushliteral(L, "caught (...)");
+ }
+
+ // Rethrow as a Lua error
+ return lua_error(L);
+}
+
+static int myinit(lua_State *L) {
+ ...
+ // Define wrapper function and enable it.
+ lua_pushlightuserdata(L, (void *)wrap_exceptions);
+ luaJIT_setmode(L, -1, LUAJIT_MODE_WRAPCFUNC|LUAJIT_MODE_ON);
+ lua_pop(L, 1);
+ ...
+}
+```
+
+Note that you can only define **a single global wrapper function**, so be careful when using this mechanism from multiple C++ modules. Also note that this mechanism is not without overhead.
diff --git a/doc/extensions/index.md b/doc/extensions/index.md
new file mode 100644
index 00000000..d3eaa73a
--- /dev/null
+++ b/doc/extensions/index.md
@@ -0,0 +1,212 @@
+# Extensions
+
+LuaJIT is fully upwards-compatible with Lua 5.1. It supports all
+[» standard Lua library functions](https://www.lua.org/manual/5.1/manual.html#5) and the full set of [» Lua/C API functions](https://www.lua.org/manual/5.1/manual.html#3).
+
+LuaJIT is also fully ABI-compatible to Lua 5.1 at the linker/dynamic loader level. This means you can compile a C module against the standard Lua headers and load the same shared library from either Lua or LuaJIT.
+
+LuaJIT extends the standard Lua VM with new functionality and adds several extension modules. Please note, this page is only about *functional* enhancements and not about performance enhancements, such as the optimized VM, the faster interpreter or the JIT compiler.
+
+## Extensions Modules
+
+LuaJIT comes with several built-in extension modules:
+
+### `bit.*` — Bitwise operations
+
+LuaJIT supports all bitwise operations as defined by [» Lua BitOp](https://bitop.luajit.org):
+
+``` lua
+bit.tobit bit.tohex bit.bnot bit.band bit.bor bit.bxor
+bit.lshift bit.rshift bit.arshift bit.rol bit.ror bit.bswap
+```
+
+This module is a LuaJIT built-in — you don't need to download or install Lua BitOp. The Lua BitOp site has full documentation for all [» Lua BitOp API functions](https://bitop.luajit.org/api.html). The FFI adds support for [64 bit bitwise operations](./ext_ffi_semantics#cdata_arith), using the same API functions.
+
+Please make sure to `require` the module before using any of its functions:
+
+```lua
+local bit = require "bit";
+```
+
+An already installed Lua BitOp module is ignored by LuaJIT. This way you can use bit operations from both Lua and LuaJIT on a shared installation.
+
+### `ffi.*` — FFI library
+
+The [FFI library](./ext_ffi) allows calling external C functions and the use of C data structures from pure Lua code.
+
+### `jit.*` — JIT compiler control
+
+The functions in this module [control the behavior of the JIT compiler engine](./ext_jit).
+
+### C API extensions
+
+LuaJIT adds some [extra functions to the Lua/C API](./ext_c_api).
+
+### Profiler
+
+LuaJIT has an [integrated profiler](./ext_profiler).
+
+## Enhanced Standard Library Functions
+
+### `xpcall(f, err [,args...])` passes arguments
+
+Unlike the standard implementation in Lua 5.1, `xpcall()` passes any arguments after the error function to the function which is called in a protected context.
+
+### `load*()` handle UTF-8 source code
+
+Non-ASCII characters are handled transparently by the Lua source code parser. This allows the use of UTF-8 characters in identifiers and strings. A UTF-8 BOM is skipped at the start of the source code.
+
+### `load*()` add a mode parameter
+
+As an extension from Lua 5.2, the functions `loadstring()`, `loadfile()` and (new) `load()` add an optional `mode` parameter.
+
+The default mode string is `"bt"`, which allows loading of both source code and bytecode. Use `"t"` to allow only source code or `"b"` to allow only bytecode to be loaded.
+
+By default, the `load*` functions generate the native bytecode format. For cross-compilation purposes, add `W` to the mode string to force the 32 bit format and `X` to force the 64 bit format. Add both to force the opposite format. Note that non-native bytecode generated by `load*` cannot be run, but can still be passed to `string.dump`.
+
+### `tostring()` etc. canonicalize NaN and ±Inf
+
+All number-to-string conversions consistently convert non-finite numbers to the same strings on all platforms. NaN results in `"nan"`, positive infinity results in `"inf"` and negative infinity results in `"-inf"`.
+
+### `tonumber()` etc. use builtin string to number conversion
+
+All string-to-number conversions consistently convert integer and floating-point inputs in decimal, hexadecimal and binary on all platforms. `strtod()` is *not* used anymore, which avoids numerous problems with poor C library implementations. The builtin conversion function provides full precision according to the IEEE-754 standard, it works independently of the current locale and it supports hex floating-point numbers (e.g. `0x1.5p-3`).
+
+### `string.dump(f [,mode])` generates portable bytecode
+
+An extra argument has been added to `string.dump()`. If set to `true` or to a string which contains the character `s`, 'stripped' bytecode without debug information is generated. This speeds up later bytecode loading and reduces memory usage. See also the [`-b` command line option](./running#opt_b).
+
+The generated bytecode is portable and can be loaded on any architecture that LuaJIT supports. However, the bytecode compatibility versions must match. Bytecode only stays compatible within a major+minor version (x.y.aaa → x.y.bbb), except for development branches. Foreign bytecode (e.g. from Lua 5.1) is incompatible and cannot be loaded.
+
+Note: `LJ_GC64` mode requires a different frame layout, which implies a different, incompatible bytecode format between 32 bit and 64 bit ports. This may be rectified in the future. In the meantime, use the `W` and X [modes of the `load*` functions](#load-add-a-mode-parameter) for cross-compilation purposes.
+
+Due to VM hardening, bytecode is not deterministic. Add `d` to the mode string to dump it in a deterministic manner: identical source code always gives a byte-for-byte identical bytecode dump. This feature is mainly useful for reproducible builds.
+
+### `table.new(narray, nhash)` allocates a pre-sized table
+
+An extra library function `table.new()` can be made available via `require("table.new")`. This creates a pre-sized table, just like the C API equivalent `lua_createtable()`. This is useful for big tables if the final table size is known and automatic table resizing is too expensive.
+
+### `table.clear(tab)` clears a table
+
+An extra library function `table.clear()` can be made available via `require("table.clear")`. This clears all keys and values from a table, but preserves the allocated array/hash sizes. This is useful when a table, which is linked from multiple places, needs to be cleared and/or when recycling a table for use by the same context. This avoids managing backlinks, saves an allocation and the overhead of incremental array/hash part growth.
+
+Please note, this function is meant for very specific situations. In most cases it's better to replace the (usually single) link with a new table and let the GC do its work.
+
+### Enhanced PRNG for `math.random()`
+
+LuaJIT uses a Tausworthe PRNG with period 2^223 to implement `math.random()` and `math.randomseed()`. The quality of the PRNG results is much superior compared to the standard Lua implementation, which uses the platform-specific ANSI `rand()`.
+
+The PRNG generates the same sequences from the same seeds on all platforms and makes use of all bits in the seed argument. `math.random()` without arguments generates 52 pseudo-random bits for every call. The result is uniformly distributed between 0.0 and 1.0. It's correctly scaled up and rounded for `math.random(n [,m])` to preserve uniformity.
+
+Call `math.randomseed()` without any arguments to seed it from system entropy.
+
+Important: Neither this nor any other PRNG based on the simplistic `math.random()` API is suitable for cryptographic use.
+
+### `io.*` functions handle 64 bit file offsets
+
+The file I/O functions in the standard `io.*` library handle 64 bit file offsets. In particular, this means it's possible to open files larger than 2 Gigabytes and to reposition or obtain the current file position for offsets beyond 2 GB (`fp:seek()` method).
+
+### `debug.*` functions identify metamethods
+
+`debug.getinfo()` and `lua_getinfo()` also return information about invoked metamethods. The `namewhat` field is set to `"metamethod"` and the `name` field has the name of the corresponding metamethod (e.g. `"__index"`).
+
+## Fully Resumable VM
+
+The LuaJIT VM is fully resumable. This means you can yield from a coroutine even across contexts, where this would not possible with the standard Lua 5.1 VM: e.g. you can yield across `pcall()` and `xpcall()`, across iterators and across metamethods.
+
+## Extensions from Lua 5.2
+
+LuaJIT supports some language and library extensions from Lua 5.2. Features that are unlikely to break existing code are unconditionally enabled:
+
+- `goto` and `::labels::`.
+- Hex escapes `'\x3F'` and `'\z'` escape in strings.
+- `load(string|reader [, chunkname [,mode [,env]]])`.
+- `loadstring()` is an alias for `load()`.
+- `loadfile(filename [,mode [,env]])`.
+- `math.log(x [,base])`.
+- `string.rep(s, n [,sep])`.
+- `string.format()`: `%q` reversible. `%s` checks `__tostring`. `%a` and `"%A` added.
+- String matching pattern `%g` added.
+- `io.read("*L")`.
+- `io.lines()` and `file:lines()` process `io.read()` options.
+- `os.exit(status|true|false [,close])`.
+- `package.searchpath(name, path [, sep [, rep]])`.
+- `package.loadlib(name, "*")`.
+- `debug.getinfo()` returns `nparams` and `isvararg` for option `"u"`.
+- `debug.getlocal()` accepts function instead of level.
+- `debug.getlocal()` and `debug.setlocal()` accept negative indexes for varargs.
+- `debug.getupvalue()` and `debug.setupvalue()` handle C functions.
+- `debug.upvalueid()` and `debug.upvaluejoin()`.
+- Lua/C API extensions: `lua_version()` `lua_upvalueid()` `lua_upvaluejoin()` `lua_loadx()` `lua_copy()` `lua_tonumberx()` `lua_tointegerx()` `luaL_fileresult()` `luaL_execresult()` `luaL_loadfilex()` `luaL_loadbufferx()` `luaL_traceback()` `luaL_setfuncs()` `luaL_pushmodule()` `luaL_newlibtable()` `luaL_newlib()` `luaL_testudata()` `luaL_setmetatable()`
+- Command line option `-E`.
+- Command line checks `__tostring` for errors.
+
+Other features are only enabled, if LuaJIT is built with `-DLUAJIT_ENABLE_LUA52COMPAT`:
+
+- `goto` is a keyword and not a valid variable name anymore.
+- `break` can be placed anywhere. Empty statements (`;;`) are allowed.
+- `__lt`, `__le` are invoked for mixed types.
+- `__len` for tables. `rawlen()` library function.
+- `pairs()` and `ipairs()` check for `__pairs` and `__ipairs`.
+- `coroutine.running()` returns two results.
+- `table.pack()` and `table.unpack()` (same as `unpack()`).
+- `io.write()` and `file:write()` return file handle instead of `true`.
+- `os.execute()` and `pipe:close()` return detailed exit status.
+- `debug.setmetatable()` returns object.
+- `debug.getuservalue()` and `debug.setuservalue()`.
+- Remove `math.mod()`, `string.gfind()`.
+- `package.searchers`.
+- `module()` returns the module table.
+
+Note: this provides only partial compatibility with Lua 5.2 at the
+language and Lua library level. LuaJIT is API+ABI-compatible with
+Lua 5.1, which prevents implementing features that would otherwise break
+the Lua/C API and ABI (e.g. `_ENV`).
+
+## Extensions from Lua 5.3
+
+LuaJIT supports some extensions from Lua 5.3:
+
+- Unicode escape `'\u{XX...}'` embeds the UTF-8 encoding in string literals.
+- The argument table `arg` can be read (and modified) by `LUA_INIT` and `-e` chunks.
+- `io.read()` and `file:read()` accept formats with or without a leading `*`.
+- `assert()` accepts any type of error object.
+- `table.move(a1, f, e, t [,a2])`.
+- `coroutine.isyieldable()`.
+- Lua/C API extensions: `lua_isyieldable()`
+
+## C++ Exception Interoperability
+
+LuaJIT has built-in support for interoperating with C++ exceptions. The
+available range of features depends on the target platform and the
+toolchain used to compile LuaJIT:
+
+Platform | Compiler | Interoperability
+----------------------------------|------------------|------------------
+External frame unwinding | GCC, Clang, MSVC | **Full**
+Internal frame unwinding + DWARF2 | GCC, Clang | **Limited**
+Windows 64 bit | non-MSVC | **Limited**
+Other platforms | Other compilers | **No**
+
+**Full interoperability** means:
+
+- C++ exceptions can be caught on the Lua side with `pcall()`, `lua_pcall()` etc.
+- C++ exceptions will be converted to the generic Lua error `"C++ exception"`, unless you use the [C call wrapper](./ext_c_api#luajit_setmodel-idx-luajit_mode_wrapcfuncflag) feature.
+- It's safe to throw C++ exceptions across non-protected Lua frames on the C stack. The contents of the C++ exception object pass through unmodified.
+- Lua errors can be caught on the C++ side with `catch(...)`. The corresponding Lua error message can be retrieved from the Lua stack.
+- Throwing Lua errors across C++ frames is safe. C++ destructors will be called.
+
+**Limited interoperability** means:
+
+- C++ exceptions can be caught on the Lua side with `pcall()`, `lua_pcall()` etc.
+- C++ exceptions will be converted to the generic Lua error `"C++ exception"`, unless you use the [C call wrapper](./ext_c_api#luajit_setmodel-idx-luajit_mode_wrapcfuncflag) feature.
+- C++ exceptions will be caught by non-protected Lua frames and are rethrown as a generic Lua error. The C++ exception object will be destroyed.
+- Lua errors **cannot** be caught on the C++ side.
+- Throwing Lua errors across C++ frames will **not** call C++ destructors.
+
+**No interoperability** means:
+
+- It's **not** safe to throw C++ exceptions across Lua frames.
+- C++ exceptions **cannot** be caught on the Lua side.
+- Lua errors **cannot** be caught on the C++ side.
+- Throwing Lua errors across C++ frames will **not** call C++ destructors.
diff --git a/doc/extensions/jit.md b/doc/extensions/jit.md
new file mode 100644
index 00000000..510139a4
--- /dev/null
+++ b/doc/extensions/jit.md
@@ -0,0 +1,73 @@
+# `jit.*` Library
+
+The functions in this built-in module control the behavior of the JIT compiler engine. Note that JIT-compilation is fully automatic — you probably won't need to use any of the following functions unless you have special needs.
+
+### `jit.on()` `jit.off()`
+
+Turns the whole JIT compiler on (default) or off.
+
+These functions are typically used with the command line options `-j on` or `-j off`.
+
+### `jit.flush()`
+
+Flushes the whole cache of compiled code.
+
+### `jit.on(func|true [,true|false])` `jit.off(func|true [,true|false])` `jit.flush(func|true [,true|false])`
+
+`jit.on` enables JIT compilation for a Lua function (this is the default).
+
+`jit.off` disables JIT compilation for a Lua function and flushes any already compiled code from the code cache.
+
+`jit.flush` flushes the code, but doesn't affect the enable/disable status.
+
+The current function, i.e. the Lua function calling this library function, can also be specified by passing `true` as the first argument.
+
+If the second argument is `true`, JIT compilation is also enabled, disabled or flushed recursively for all sub-functions of a function. With `false` only the sub-functions are affected.
+
+The `jit.on` and `jit.off` functions only set a flag which is checked when the function is about to be compiled. They do not trigger immediate compilation.
+
+Typical usage is `jit.off(true, true)` in the main chunk of a module to turn off JIT compilation for the whole module for debugging purposes.
+
+### `jit.flush(tr)`
+
+Flushes the root trace, specified by its number, and all of its side traces from the cache. The code for the trace will be retained as long as there are any other traces which link to it.
+
+### `status, ... = jit.status()`
+
+Returns the current status of the JIT compiler. The first result is either `true` or `false` if the JIT compiler is turned on or off. The remaining results are strings for CPU-specific features and enabled optimizations.
+
+### `jit.version`
+
+Contains the LuaJIT version string.
+
+### `jit.version_num`
+
+Contains the version number of the LuaJIT core. Version xx.yy.zz is represented by the decimal number xxyyzz. **DEPRECATED after the switch to [» rolling releases](https://luajit.org/status.html#release). zz is frozen at 99.**
+
+### `jit.os`
+
+Contains the target OS name: "Windows", "Linux", "OSX", "BSD", "POSIX" or "Other".
+
+### `jit.arch`
+
+Contains the target architecture name: "x86", "x64", "arm", "arm64", "arm64be", "ppc", "mips", "mipsel", "mips64", "mips64el", "mips64r6", "mips64r6el".
+
+## `jit.opt.*` — JIT compiler optimization control
+
+This submodule provides the backend for the `-O` command line option.
+
+You can also use it programmatically, e.g.:
+
+```lua
+jit.opt.start(2); -- same as -O2
+jit.opt.start("-dce");
+jit.opt.start("hotloop=10", "hotexit=2");
+```
+
+Unlike in LuaJIT 1.x, the module is built-in and **optimization is turned on by default!** It's no longer necessary to run `require("jit.opt").start()`, which was one of the ways to enable optimization.
+
+## `jit.util.*` — JIT compiler introspection
+
+This submodule holds functions to introspect the bytecode, generated traces, the IR and the generated machine code. The functionality provided by this module is still in flux and therefore undocumented.
+
+The debug modules `-jbc`, `-jv` and `-jdump` make extensive use of these functions. Please check out their source code, if you want to know more.
diff --git a/doc/extensions/profiler.md b/doc/extensions/profiler.md
new file mode 100644
index 00000000..518afc37
--- /dev/null
+++ b/doc/extensions/profiler.md
@@ -0,0 +1,159 @@
+# Profiler
+
+LuaJIT has an integrated statistical profiler with very low overhead. It allows sampling the currently executing stack and other parameters in regular intervals.
+
+The integrated profiler can be accessed from three levels:
+
+- The [bundled high-level profiler](#high-level-profiler), invoked by the [`-jp`](#-jpoptionsoutput) command line option.
+- A [low-level Lua API](#ll_lua_api) to control the profiler.
+- A [low-level C API](#ll_c_api) to control the profiler.
+
+## High-Level Profiler
+
+The bundled high-level profiler offers basic profiling functionality. It generates simple textual summaries or source code annotations. It can be accessed with the [`-jp`](#-jpoptionsoutput) command line option or from Lua code by loading the underlying `jit.p` module.
+
+To cut to the chase - run this to get a CPU usage profile by function name:
+
+```sh
+luajit -jp myapp.lua
+```
+
+It's *not* a stated goal of the bundled profiler to add every possible option or to cater for special profiling needs. The low-level profiler APIs are documented below. They may be used by third-party authors to implement advanced functionality, e.g. IDE integration or graphical profilers.
+
+Note: Sampling works for both interpreted and JIT-compiled code. The results for JIT-compiled code may sometimes be surprising. LuaJIT heavily optimizes and inlines Lua code - there's no simple one-to-one correspondence between source code lines and the sampled machine code.
+
+### `-jp=[options[,output]]`
+
+The `-jp` command line option starts the high-level profiler. When the application run by the command line terminates, the profiler stops and writes the results to `stdout` or to the specified `output` file.
+
+The `options` argument specifies how the profiling is to be performed:
+
+- `f` - Stack dump: function name, otherwise module:line. This is the default mode.
+- `F` - Stack dump: ditto, but dump module:name.
+- `l` - Stack dump: module:line.
+- `` - stack dump depth (callee ← caller). Default: 1.
+- `-` - Inverse stack dump depth (caller → callee).
+- `s` - Split stack dump after first stack level. Implies depth ≥ 2 or depth ≤ -2.
+- `p` - Show full path for module names.
+- `v` - Show VM states.
+- `z` - Show [zones](#jit_zone).
+- `r` - Show raw sample counts. Default: show percentages.
+- `a` - Annotate excerpts from source code files.
+- `A` - Annotate complete source code files.
+- `G` - Produce raw output suitable for graphical tools.
+- `m` - Minimum sample percentage to be shown. Default: 3%.
+- `i` - Sampling interval in milliseconds. Default: 10ms. Note: The actual sampling precision is OS-dependent.
+
+The default output for `-jp` is a list of the most CPU consuming spots in the application. Increasing the stack dump depth with (say) `-jp=2` may help to point out the main callers or callees of hotspots. But sample aggregation is still flat per unique stack dump.
+
+To get a two-level view (split view) of callers/callees, use `-jp=s` or `-jp=-s`. The percentages shown for the second level are relative to the first level.
+
+To see how much time is spent in each line relative to a function, use `-jp=fl`.
+
+To see how much time is spent in different VM states or [zones](#jitzone--zones), use `-jp=v` or `-jp=z`.
+
+Combinations of `v/z` with `f/F/l` produce two-level views, e.g. `-jp=vf` or `-jp=fv`. This shows the time spent in a VM state or zone vs. hotspots. This can be used to answer questions like "Which time-consuming functions are only interpreted?" or "What's the garbage collector overhead for a specific function?".
+
+Multiple options can be combined - but not all combinations make sense, see above. E.g. `-jp=3si4m1` samples three stack levels deep in 4ms intervals and shows a split view of the CPU consuming functions and their callers with a 1% threshold.
+
+Source code annotations produced by `-jp=a` or `-jp=A` are always flat and at the line level. Obviously, the source code files need to be readable by the profiler script.
+
+The high-level profiler can also be started and stopped from Lua code with:
+
+```lua
+require "jit.p".start(options, output);
+...
+require "jit.p".stop();
+```
+
+### `jit.zone` - Zones
+
+Zones can be used to provide information about different parts of an application to the high-level profiler. E.g. a game could make use of an `"AI"` zone, a `"PHYS"` zone, etc. Zones are hierarchical, organized as a stack.
+
+The `jit.zone` module needs to be loaded explicitly:
+
+```lua
+local zone = require "jit.zone";
+```
+
+- `zone("name")` pushes a named zone to the zone stack.
+- `zone()` pops the current zone from the zone stack and returns its name.
+- `zone:get()` returns the current zone name or `nil`.
+- `zone:flush()` flushes the zone stack.
+
+To show the time spent in each zone use `-jp=z`. To show the time spent relative to hotspots use e.g. `-jp=zf` or `-jp=fz`.
+
+## Low-level Lua API
+
+The `jit.profile` module gives access to the low-level API of the profiler from Lua code. This module needs to be loaded explicitly:
+
+```lua
+local profile = require "jit.profile";
+```
+
+This module can be used to implement your own higher-level profiler. A typical profiling run starts the profiler, captures stack dumps in the profiler callback, adds them to a hash table to aggregate the number of samples, stops the profiler and then analyzes all captured stack dumps. Other parameters can be sampled in the profiler callback, too. But it's important not to spend too much time in the callback, since this may skew the statistics.
+
+### `profile.start(mode, cb)`: Start profiler
+
+This function starts the profiler. The `mode` argument is a string
+holding options:
+
+- `f` - Profile with precision down to the function level.
+- `l` - Profile with precision down to the line level.
+- `i` - Sampling interval in milliseconds (default 10ms). Note: The actual sampling precision is OS-dependent.
+
+The `cb` argument is a callback function which is called with three arguments: `(thread, samples, vmstate)`. The callback is called on a separate coroutine, the `thread` argument is the state that holds the stack to sample for profiling. Note: do *not* modify the stack of that state or call functions on it.
+
+`samples` gives the number of accumulated samples since the last callback (usually 1).
+
+`vmstate` holds the VM state at the time the profiling timer triggered. This may or may not correspond to the state of the VM when the profiling callback is called. The state is either `'N'` native (compiled) code, `'I'` interpreted code, `'C'` C code, `'G'` the garbage collector, or `'J'` the JIT compiler.
+
+### `profile.stop()`: Stop profiler
+
+This function stops the profiler.
+
+### `dump = profile.dumpstack([thread,] fmt, depth)`: Dump stack
+
+This function allows taking stack dumps in an efficient manner. It returns a string with a stack dump for the `thread` (coroutine), formatted according to the `fmt` argument:
+
+- `p`: Preserve the full path for module names. Otherwise, only the file name is used.
+- `f`: Dump the function name if it can be derived. Otherwise, use module:line.
+- `F`: Ditto, but dump module:name.
+- `l`: Dump module:line.
+- `Z`: Zap the following characters for the last dumped frame.
+- All other characters are added verbatim to the output string.
+
+The `depth` argument gives the number of frames to dump, starting at the topmost frame of the thread. A negative number dumps the frames in inverse order.
+
+The first example prints a list of the current module names and line numbers of up to 10 frames in separate lines. The second example prints semicolon-separated function names for all frames (up to 100) in inverse order:
+
+```lua
+print(profile.dumpstack(thread, "l\n", 10));
+print(profile.dumpstack(thread, "lZ;", -100));
+```
+
+## Low-level C API
+
+The profiler can be controlled directly from C code, e.g. for use by IDEs. The declarations are in `"luajit.h"` (see [Lua/C API](./ext_c_api) extensions).
+
+### `luaJIT_profile_start(L, mode, cb, data)` - Start profiler
+
+This function starts the profiler. [See above](#profilestartmode-cb-start-profiler) for a description of the `mode` argument.
+
+The `cb` argument is a callback function with the following declaration:
+
+```c
+typedef void (*luaJIT_profile_callback)(void *data, lua_State *L, int samples, int vmstate);
+```
+
+`data` is available for use by the callback. `L` is the state that holds the stack to sample for profiling. Note: do *not* modify this stack or call functions on this stack - use a separate coroutine for this purpose. [See above](#profilestartmode-cb-start-profiler) for a description of `samples` and `vmstate`.
+
+### `luaJIT_profile_stop(L)` - Stop profiler
+
+This function stops the profiler.
+
+### `p = luaJIT_profile_dumpstack(L, fmt, depth, len)` - Dump stack
+
+This function allows taking stack dumps in an efficient manner. [See above](#dump--profiledumpstackthread-fmt-depth-dump-stack) for a description of `fmt` and `depth`.
+
+This function returns a `const char *` pointing to a private string buffer of the profiler. The `int *len` argument returns the length of the output string. The buffer is overwritten on the next call and deallocated when the profiler stops. You either need to consume the content immediately or copy it for later use.
diff --git a/doc/ffi/api.md b/doc/ffi/api.md
new file mode 100644
index 00000000..23850c0b
--- /dev/null
+++ b/doc/ffi/api.md
@@ -0,0 +1,240 @@
+# `ffi.*` API Functions
+
+This page describes the API functions provided by the FFI library in
+detail. It's recommended to read through the
+[introduction](./ext_ffi) and the [FFI
+tutorial](./ext_ffi_tutorial) first.
+
+## Glossary
+
+- **cdecl**: An abstract C type declaration (a Lua string).
+- **ctype**: A C type object. This is a special kind of **cdata** returned by `ffi.typeof()`. It serves as a **cdata** [constructor](#ffi_new) when called.
+- **cdata**: A C data object. It holds a value of the corresponding **ctype**.
+- **ct**: A C type specification which can be used for most of the API functions. Either a **cdecl**, a **ctype** or a **cdata** serving as a template type.
+- **cb**: A callback object. This is a C data object holding a special function pointer. Calling this function from C code runs an associated Lua function.
+- **VLA**: A variable-length array is declared with a `?` instead of the number of elements, e.g. `"int[?]"`. The number of elements (`nelem`) must be given when it's [created](#ffi_new).
+- **VLS**: A variable-length struct is a `struct` C type where the last element is a **VLA**. The same rules for declaration and creation apply.
+
+## Declaring and Accessing External Symbols
+
+External symbols must be declared first and can then be accessed by indexing a [C library namespace](./ext_ffi_semantics#clib), which automatically binds the symbol to a specific library.
+
+### `ffi.cdef(def)`
+
+Adds multiple C declarations for types or external symbols (named variables or functions). `def` must be a Lua string. It's recommended to use the syntactic sugar for string arguments as follows:
+
+```lua
+ffi.cdef [[
+ // Declare a struct and typedef
+ typedef struct foo { int a, b; } foo_t;
+ // Declare an external C function
+ int dofoo(foo_t *f, int n);
+]]
+```
+
+The contents of the string (the part in green above) must be a sequence of [C declarations](./ext_ffi_semantics#clang), separated by semicolons. The trailing semicolon for a single declaration may be omitted.
+
+Please note, that external symbols are only *declared*, but they are *not bound* to any specific address, yet. Binding is achieved with C library namespaces (see below).
+
+C declarations are not passed through a C pre-processor, yet. No pre-processor tokens are allowed, except for `#pragma pack`. Replace `#define` in existing C header files with `enum`, `static const` or `typedef` and/or pass the files through an external C pre-processor (once). Be careful not to include unneeded or redundant declarations from unrelated header files.
+
+### `ffi.C`
+
+This is the default C library namespace — note the uppercase `'C'`. It binds to the default set of symbols or libraries on the target system. These are more or less the same as a C compiler would offer by default, without specifying extra link libraries.
+
+On POSIX systems, this binds to symbols in the default or global namespace. This includes all exported symbols from the executable and any libraries loaded into the global namespace. This includes at least `libc`, `libm`, `libdl` (on Linux), `libgcc` (if compiled with GCC), as well as any exported symbols from the Lua/C API provided by LuaJIT itself.
+
+On Windows systems, this binds to symbols exported from the `*.exe`, the `lua51.dll` (i.e. the Lua/C API provided by LuaJIT itself), the C runtime library LuaJIT was linked with (`msvcrt*.dll`), `kernel32.dll`, `user32.dll` and `gdi32.dll`.
+
+### `clib = ffi.load(name [,global])`
+
+This loads the dynamic library given by `name` and returns a new C library namespace which binds to its symbols. On POSIX systems, if `global` is `true`, the library symbols are loaded into the global namespace, too.
+
+If `name` is a path, the library is loaded from this path. Otherwise `name` is canonicalized in a system-dependent way and searched in the default search path for dynamic libraries:
+
+On POSIX systems, if the name contains no dot, the extension `.so` is appended. Also, the `lib` prefix is prepended if necessary. So `ffi.load "z"` looks for `"libz.so"` in the default shared library search path.
+
+On Windows systems, if the name contains no dot, the extension `.dll` is appended. So `ffi.load "ws2_32"` looks for `"ws2_32.dll"` in the default DLL search path.
+
+## Creating cdata Objects
+
+The following API functions create cdata objects (`type()` returns `"cdata"`). All created cdata objects are [garbage collected](./ext_ffi_semantics#gc).
+
+### `cdata = ffi.new(ct [,nelem] [,init...])` `cdata = `*`ctype`*`([nelem,] [init...])`
+
+Creates a cdata object for the given `ct`. VLA/VLS types require the `nelem` argument. The second syntax uses a ctype as a constructor and is otherwise fully equivalent.
+
+The cdata object is initialized according to the [rules for initializers](./ext_ffi_semantics#init), using the optional `init` arguments. Excess initializers cause an error.
+
+Performance notice: if you want to create many objects of one kind, parse the cdecl only once and get its ctype with `ffi.typeof()`. Then use the ctype as a constructor repeatedly.
+
+Please note, that an anonymous `struct` declaration implicitly creates a new and distinguished ctype every time you use it for `ffi.new()`. This is probably **not** what you want, especially if you create more than one cdata object. Different anonymous `structs` are not considered assignment-compatible by the C standard, even though they may have the same fields! Also, they are considered different types by the JIT-compiler, which may cause an excessive number of traces. It's strongly suggested to either declare a named `struct` or `typedef` with `ffi.cdef()` or to create a single ctype object for an anonymous `struct` with `ffi.typeof()`.
+
+### `ctype = ffi.typeof(ct)`
+
+Creates a ctype object for the given `ct`.
+
+This function is especially useful to parse a cdecl only once and then use the resulting ctype object as a [constructor](#ffi_new).
+
+### `cdata = ffi.cast(ct, init)`
+
+Creates a scalar cdata object for the given `ct`. The cdata object is initialized with `init` using the "cast" variant of the [C type conversion rules](./ext_ffi_semantics#convert).
+
+This functions is mainly useful to override the pointer compatibility checks or to convert pointers to addresses or vice versa.
+
+### `ctype = ffi.metatype(ct, metatable)`
+
+Creates a ctype object for the given `ct` and associates it with a metatable. Only `struct`/`union` types, complex numbers and vectors are allowed. Other types may be wrapped in a `struct`, if needed.
+
+The association with a metatable is permanent and cannot be changed afterwards. Neither the contents of the `metatable` nor the contents of an `__index` table (if any) may be modified afterwards. The associated metatable automatically applies to all uses of this type, no matter how the objects are created or where they originate from. Note that predefined operations on types have precedence (e.g. declared field names cannot be overridden).
+
+All standard Lua metamethods are implemented. These are called directly, without shortcuts, and on any mix of types. For binary operations, the left operand is checked first for a valid ctype metamethod. The `__gc` metamethod only applies to `struct`/`union` types and performs an implicit [`ffi.gc()`](#ffi_gc) call during creation of an instance.
+
+### `cdata = ffi.gc(cdata, finalizer)`
+
+Associates a finalizer with a pointer or aggregate cdata object. The cdata object is returned unchanged.
+
+This function allows safe integration of unmanaged resources into the automatic memory management of the LuaJIT garbage collector. Typical usage:
+
+```lua
+local p = ffi.gc(ffi.C.malloc(n), ffi.C.free);
+...
+-- Last reference to p is gone.
+p = nil;
+-- GC will eventually run finalizer: ffi.C.free(p)
+```
+
+A cdata finalizer works like the `__gc` metamethod for userdata objects: when the last reference to a cdata object is gone, the associated finalizer is called with the cdata object as an argument. The finalizer can be a Lua function or a cdata function or cdata function pointer. An existing finalizer can be removed by setting a `nil` finalizer, e.g. right before explicitly deleting a resource:
+
+```lua
+-- Manually free the memory.
+ffi.C.free(ffi.gc(p, nil));
+```
+
+## C Type Information
+
+The following API functions return information about C types. They are most useful for inspecting cdata objects.
+
+### `size = ffi.sizeof(ct [,nelem])`
+
+Returns the size of `ct` in bytes. Returns `nil` if the size is not known (e.g. for `"void"` or function types). Requires `nelem` for VLA/VLS types, except for cdata objects.
+
+### `align = ffi.alignof(ct)`
+
+Returns the minimum required alignment for `ct` in bytes.
+
+### `ofs [,bpos,bsize] = ffi.offsetof(ct, field)`
+
+Returns the offset (in bytes) of `field` relative to the start of `ct`, which must be a `struct`. Additionally returns the position and the field size (in bits) for bit fields.
+
+### `status = ffi.istype(ct, obj)`
+
+Returns `true` if `obj` has the C type given by `ct`. Returns `false`
+otherwise.
+
+C type qualifiers (`const` etc.) are ignored. Pointers are checked with the standard pointer compatibility rules, but without any special treatment for `void *`. If `ct` specifies a `struct`/`union`, then a pointer to this type is accepted, too. Otherwise the types must match exactly.
+
+Note: this function accepts all kinds of Lua objects for the `obj` argument, but always returns `false` for non-cdata objects.
+
+## Utility Functions
+
+### `err = ffi.errno([newerr])`
+
+Returns the error number set by the last C function call which indicated an error condition. If the optional `newerr` argument is present, the error number is set to the new value and the previous value is returned.
+
+This function offers a portable and OS-independent way to get and set the error number. Note that only *some* C functions set the error number. And it's only significant if the function actually indicated an error condition (e.g. with a return value of `-1` or `NULL`). Otherwise, it may or may not contain any previously set value.
+
+You're advised to call this function only when needed and as close as possible after the return of the related C function. The `errno` value is preserved across hooks, memory allocations, invocations of the JIT compiler and other internal VM activity. The same applies to the value returned by `GetLastError()` on Windows, but you need to declare and call it yourself.
+
+### `str = ffi.string(ptr [,len])`
+
+Creates an interned Lua string from the data pointed to by `ptr`.
+
+If the optional argument `len` is missing, `ptr` is converted to a `"char *"` and the data is assumed to be zero-terminated. The length of the string is computed with `strlen()`.
+
+Otherwise `ptr` is converted to a `"void *"` and `len` gives the length of the data. The data may contain embedded zeros and need not be byte-oriented (though this may cause endianess issues).
+
+This function is mainly useful to convert (temporary) `"const char *"` pointers returned by C functions to Lua strings and store them or pass them to other functions expecting a Lua string. The Lua string is an (interned) copy of the data and bears no relation to the original data area anymore. Lua strings are 8 bit clean and may be used to hold arbitrary, non-character data.
+
+Performance notice: it's faster to pass the length of the string, if it's known. E.g. when the length is returned by a C call like `sprintf()`.
+
+### `ffi.copy(dst, src, len)` `ffi.copy(dst, str)`
+
+Copies the data pointed to by `src` to `dst`. `dst` is converted to a `"void *"` and `src` is converted to a `"const void *"`.
+
+In the first syntax, `len` gives the number of bytes to copy. Caveat: if `src` is a Lua string, then `len` must not exceed `#src+1`.
+
+In the second syntax, the source of the copy must be a Lua string. All bytes of the string *plus a zero-terminator* are copied to `dst` (i.e. `#src+1` bytes).
+
+Performance notice: `ffi.copy()` may be used as a faster (inlinable) replacement for the C library functions `memcpy()`, `strcpy()` and `strncpy()`.
+
+### `ffi.fill(dst, len [,c])`
+
+Fills the data pointed to by `dst` with `len` constant bytes, given by `c`. If `c` is omitted, the data is zero-filled.
+
+Performance notice: `ffi.fill()` may be used as a faster (inlinable) replacement for the C library function `memset(dst, c, len)`. Please note the different order of arguments!
+
+## Target-specific Information
+
+### `status = ffi.abi(param)`
+
+Returns `true` if `param` (a Lua string) applies for the target ABI (Application Binary Interface). Returns `false` otherwise. The following parameters are currently defined:
+
+Parameter | Description
+----------|------------
+32bit | 32 bit architecture
+64bit | 64 bit architecture
+le | Little-endian architecture
+be | Big-endian architecture
+fpu | Target has a hardware FPU
+softfp | softfp calling conventions
+hardfp | hardfp calling conventions
+eabi | EABI variant of the standard ABI
+win | Windows variant of the standard ABI
+pauth | Pointer authentication ABI
+uwp | Universal Windows Platform
+gc64 | 64 bit GC references
+
+### `ffi.os`
+
+Contains the target OS name. Same contents as [`jit.os`](./ext_jit#jit_os).
+
+### `ffi.arch`
+
+Contains the target architecture name. Same contents as [`jit.arch`](./ext_jit#jit_arch).
+
+## Methods for Callbacks
+
+The C types for [callbacks](./ext_ffi_semantics#callback) have some extra methods:
+
+### `cb:free()`
+
+Free the resources associated with a callback. The associated Lua function is unanchored and may be garbage collected. The callback function pointer is no longer valid and must not be called again (it may be reused by a subsequently created callback).
+
+### `cb:set(func)`
+
+Associate a new Lua function with a callback. The C type of the callback and the callback function pointer are unchanged.
+
+This method is useful to dynamically switch the receiver of callbacks without creating a new callback each time and registering it again (e.g. with a GUI library).
+
+## Extended Standard Library Functions
+
+The following standard library functions have been extended to work with cdata objects:
+
+### `n = tonumber(cdata)`
+
+Converts a number cdata object to a `double` and returns it as a Lua number. This is particularly useful for boxed 64 bit integer values. Caveat: this conversion may incur a precision loss.
+
+### `s = tostring(cdata)`
+
+Returns a string representation of the value of 64 bit integers (**`"`**`nnn`**`LL"`** or **`"`**`nnn`**`ULL"`**) or complex numbers (**`"`**`re±im`**`i"`**). Otherwise returns a string representation of the C type of a ctype object (**`"ctype<`**`type`**`>"`**) or a cdata object (**`"cdata<`**`type`**`>: `**`address"`), unless you override it with a `__tostring` metamethod (see [`ffi.metatype()`](#ffi_metatype)).
+
+### `iter, obj, start = pairs(cdata)` `iter, obj, start = ipairs(cdata)`
+
+Calls the `__pairs` or `__ipairs` metamethod of the corresponding ctype.
+
+## Extensions to the Lua Parser
+
+The parser for Lua source code treats numeric literals with the suffixes `LL` or `ULL` as signed or unsigned 64 bit integers. Case doesn't matter, but uppercase is recommended for readability. It handles decimal (`42LL`), hexadecimal (`0x2aLL`) and binary (`0b101010LL`) literals.
+
+The imaginary part of complex numbers can be specified by suffixing number literals with `i` or `I`, e.g. `12.5i`. Caveat: you'll need to use `1i` to get an imaginary part with the value one, since `i` itself still refers to a variable named `i`.
diff --git a/doc/ffi/index.md b/doc/ffi/index.md
new file mode 100644
index 00000000..8c9d348c
--- /dev/null
+++ b/doc/ffi/index.md
@@ -0,0 +1,138 @@
+# FFI Library
+
+The FFI library allows **calling external C functions** and **using C data structures** from pure Lua code.
+
+The FFI library largely obviates the need to write tedious manual Lua/C bindings in C. No need to learn a separate binding language — **it parses plain C declarations!** These can be cut-n-pasted from C header files or reference manuals. It's up to the task of binding large libraries without the need for dealing with fragile binding generators.
+
+The FFI library is tightly integrated into LuaJIT (it's not available as a separate module). The code generated by the JIT-compiler for accesses to C data structures from Lua code is on par with the code a C compiler would generate. Calls to C functions can be inlined in JIT-compiled code, unlike calls to functions bound via the classic Lua/C API.
+
+This page gives a short introduction to the usage of the FFI library. *Please use the FFI sub-topics in the navigation bar to learn more.*
+
+## Motivating Example: Calling External C Functions
+
+It's really easy to call an external C library function:
+
+```lua
+local ffi = require "ffi"; -- (1)
+ffi.cdef [[
+ int printf(const char *fmt, ...);
+]]; -- (2)
+ffi.C.printf("Hello %s!", "world"); -- (3)
+```
+
+So, let's pick that apart:
+
+1. Load the FFI library.
+2. Add a C declaration for the function. The
+part inside the double-brackets (in green) is just standard C syntax.
+3. Call the named C function — Yes, it's that
+simple!
+
+Actually, what goes on behind the scenes is far from simple: (3) makes use of the standard C library namespace `ffi.C`. Indexing this namespace with a symbol name (`"printf"`) automatically binds it to the standard C library. The result is a special kind of object which, when called, runs the `printf` function. The arguments passed to this function are automatically converted from Lua objects to the corresponding C types.
+
+Ok, so maybe the use of `printf()` wasn't such a spectacular example. You could have done that with `io.write()` and `string.format()`, too. But you get the idea ...
+
+So here's something to pop up a message box on Windows:
+
+```lua
+local ffi = require "ffi";
+ffi.cdef [[
+ int MessageBoxA(void *w, const char *txt, const char *cap, int type);
+]];
+ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0);
+```
+
+Bing! Again, that was far too easy, no?
+
+Compare this with the effort required to bind that function using the classic Lua/C API: create an extra C file, add a C function that retrieves and checks the argument types passed from Lua and calls the actual C function, add a list of module functions and their names, add a `luaopen_*` function and register all module functions, compile and link it into a shared library (DLL), move it to the proper path, add Lua code that loads the module aaaand ... finally call the binding function. Phew!
+
+## Motivating Example: Using C Data Structures
+
+The FFI library allows you to create and access C data structures. Of course, the main use for this is for interfacing with C functions. But they can be used stand-alone, too.
+
+Lua is built upon high-level data types. They are flexible, extensible and dynamic. That's why we all love Lua so much. Alas, this can be inefficient for certain tasks, where you'd really want a low-level data type. E.g. a large array of a fixed structure needs to be implemented with a big table holding lots of tiny tables. This imposes both a substantial memory overhead as well as a performance overhead.
+
+Here's a sketch of a library that operates on color images, plus a simple benchmark. First, the plain Lua version:
+
+```lua
+local floor = math.floor;
+
+local function image_ramp_green(n)
+ local img = {};
+ local f = 255 / (n - 1);
+
+ for i = 1, n do
+ img[i] = { r = 0, g = floor((i - 1) * f), b = 0, a = 255 };
+ end
+
+ return img;
+end
+
+local function image_to_gray(img, n)
+ for i = 1, n do
+ local val = floor(0.3 * img[i].red + 0.59 * img[i].green + 0.11 * img[i].blue);
+ img[i].r = val;
+ img[i].g = val;
+ img[i].b = val;
+ end
+end
+
+local N = 400 * 400;
+local img = image_ramp_green(N);
+
+for i = 1, 1000 do
+ image_to_gray(img, N);
+end
+```
+
+This creates a table with 160.000 pixels, each of which is a table holding four number values in the range of 0-255. First, an image with a green ramp is created (1D for simplicity), then the image is converted to grayscale 1000 times. Yes, that's silly, but I was in need of a simple example ...
+
+And here's the FFI version. The modified parts have been marked:
+
+```lua
+local ffi = require "ffi"; -- (1)
+ffi.cdef [[
+ typedef struct { uint8_t r, g, b, a; } rgba_pixel;
+]];
+
+local function image_ramp_green(n)
+ local img = ffi.new("rgba_pixel[?]", n); -- (2)
+ local f = 255 / (n - 1);
+
+ for i = 0, n - 1 do -- (3)
+ img[i].g = i * f; -- (4)
+ img[i].a = 255;
+ end
+
+ return img;
+end
+
+local function image_to_grey(img, n)
+ for i = 0, n - 1 do -- (3)
+ local val = 0.3 * img[i].r + 0.59 * img[i].g + 0.11 * img[i].b; -- (5)
+ img[i].r = val; img[i].g = val; img[i].b = val
+ end
+end
+
+local N = 400 * 400;
+local img = image_ramp_green(N);
+for i = 1, 1000 do
+ image_to_grey(img, N);
+end
+```
+
+Ok, so that wasn't too difficult:
+
+1. First, load the FFI library and declare the low-level data type. Here we choose a `struct` which holds four byte fields, one for each component of a 4x8 bit RGBA pixel.
+2. Creating the data structure with `ffi.new()` is straightforward — the `'?'` is a placeholder for the number of elements of a variable-length array.
+3. C arrays are zero-based, so the indexes have to run from `0` to `n-1`. One might want to allocate one more element instead to simplify converting legacy code.
+4. Since `ffi.new()` zero-fills the array by default, we only need to set the green and the alpha fields.
+5. The calls to `math.floor()` can be omitted here, because floating-point numbers are already truncated towards zero when converting them to an integer. This happens implicitly when the number is stored in the fields of each pixel.
+
+Now let's have a look at the impact of the changes: first, memory consumption for the image is down from 22 Megabytes to 640 Kilobytes (400 * 400 * 4 bytes). That's a factor of 35x less! So, yes, tables do have a noticeable overhead. BTW: The original program would consume 40 Megabytes in plain Lua (on x64).
+
+Next, performance: the pure Lua version runs in 9.57 seconds (52.9 seconds with the Lua interpreter) and the FFI version runs in 0.48 seconds on my machine (YMMV). That's a factor of 20x faster (110x faster than the Lua interpreter).
+
+The avid reader may notice that converting the pure Lua version over to use array indexes for the colors (`[1]` instead of `.red`, `[2]` instead of `.green` etc.) ought to be more compact and faster. This is certainly true (by a factor of ~1.7x). Switching to a struct-of-arrays would help, too.
+
+However, the resulting code would be less idiomatic and rather error-prone. And it still doesn't get even close to the performance of the FFI version of the code. Also, high-level data structures cannot be easily passed to other C functions, especially I/O functions, without undue conversion penalties.
diff --git a/doc/ffi/semantics.md b/doc/ffi/semantics.md
new file mode 100644
index 00000000..f8fe7403
--- /dev/null
+++ b/doc/ffi/semantics.md
@@ -0,0 +1,479 @@
+# FFI Semantics
+
+This page describes the detailed semantics underlying the FFI library and its interaction with both Lua and C code.
+
+Given that the FFI library is designed to interface with C code and that declarations can be written in plain C syntax, **it closely follows the C language semantics**, wherever possible. Some minor concessions are needed for smoother interoperation with Lua language semantics.
+
+Please don't be overwhelmed by the contents of this page — this is a reference and you may need to consult it, if in doubt. It doesn't hurt to skim this page, but most of the semantics "just work" as you'd expect them to work. It should be straightforward to write applications using the LuaJIT FFI for developers with a C or C++ background.
+
+## C Language Support
+
+The FFI library has a built-in C parser with a minimal memory footprint. It's used by the [ffi.\* library functions](./ext_ffi_api) to declare C types or external symbols.
+
+Its only purpose is to parse C declarations, as found e.g. in C header files. Although it does evaluate constant expressions, it's *not* a C compiler. The body of `inline` C function definitions is simply ignored.
+
+Also, this is *not* a validating C parser. It expects and accepts correctly formed C declarations, but it may choose to ignore bad declarations or show rather generic error messages. If in doubt, please check the input against your favorite C compiler.
+
+The C parser complies to the **C99 language standard** plus the following extensions:
+
+- The `'\e'` escape in character and string literals.
+- The C99/C++ boolean type, declared with the keywords `bool` or `_Bool`.
+- Complex numbers, declared with the keywords `complex` or `_Complex`.
+- Two complex number types: `complex` (aka `complex double`) and `complex float`.
+- Vector types, declared with the GCC `mode` or `vector_size` attribute.
+- Unnamed ('transparent') `struct`/`union` fields inside a `struct`/`union`.
+- Incomplete `enum` declarations, handled like incomplete `struct` declarations.
+- Unnamed `enum` fields inside a `struct`/`union`. This is similar to a scoped C++ `enum`, except that declared constants are visible in the global namespace, too.
+- Scoped `static const` declarations inside a `struct`/`union` (from C++).
+- Zero-length arrays (`[0]`), empty `struct`/`union`, variable-length arrays (VLA, `[?]`) and variable-length structs (VLS, with a trailing VLA).
+- C++ reference types (`int &x`).
+- Alternate GCC keywords with '`__`', e.g. `__const__`.
+- GCC `__attribute__` with the following attributes: `aligned`, `packed`, `mode`, `vector_size`, `cdecl`, `fastcall`, `stdcall`, `thiscall`.
+- The GCC `__extension__` keyword and the GCC `__alignof__` operator.
+- GCC `__asm__("symname")` symbol name redirection for function declarations.
+- MSVC keywords for fixed-length types: `__int8`, `__int16`, `__int32` and `__int64`.
+- MSVC `__cdecl`, `__fastcall`, `__stdcall`, `__thiscall`, `__ptr32`, `__ptr64`, `__declspec(align(n))` and `#pragma pack`.
+- All other GCC/MSVC-specific attributes are ignored.
+
+The following C types are predefined by the C parser (like a `typedef`, except re-declarations will be ignored):
+
+- Vararg handling: `va_list`, `__builtin_va_list`, `__gnuc_va_list`.
+- From ``: `ptrdiff_t`, `size_t`, `wchar_t`.
+- From ``: `int8_t`, `int16_t`, `int32_t`, `int64_t`, `uint8_t`, `uint16_t`, `uint32_t`, `uint64_t`, `intptr_t`, `uintptr_t`.
+- From `` (POSIX): `ssize_t`.
+
+You're encouraged to use these types in preference to compiler-specific extensions or target-dependent standard types. E.g. `char` differs in signedness and `long` differs in size, depending on the target architecture and platform ABI.
+
+The following C features are **not** supported:
+
+- A declaration must always have a type specifier; it doesn't default to an `int` type.
+- Old-style empty function declarations (K&R) are not allowed. All C functions must have a proper prototype declaration. A function declared without parameters (`int foo();`) is treated as a function taking zero arguments, like in C++.
+- The `long double` C type is parsed correctly, but there's no support for the related conversions, accesses or arithmetic operations.
+- Wide character strings and character literals are not supported.
+- [See below](#status) for features that are currently not implemented.
+
+## C Type Conversion Rules
+
+### Conversions from C types to Lua objects
+
+These conversion rules apply for *read accesses* to C types: indexing pointers, arrays or `struct`/`union` types; reading external variables or constant values; retrieving return values from C calls:
+
+Input | Output | Conversion
+----------------------|------------------|----
+`int8_t`, `int16_t` | number | →sign-ext `int32_t` → `double`
+`uint8_t`, `uint16_t` | number | →zero-ext `int32_t` → `double`
+`int32_t`, `uint32_t` | number | → `double`
+`int64_t`, `uint64_t` | 64 bit int cdata | boxed value
+`double`, `float` | number | → `double`
+`bool` | boolean | 0 → `false`, otherwise `true`
+`enum` | enum cdata | boxed value
+Complex number | complex cdata | boxed value
+Vector | vector cdata | boxed value
+Pointer | pointer cdata | boxed value
+Array | reference cdata | boxed reference
+`struct`/`union` | reference cdata | boxed reference
+
+Bitfields are treated like their underlying type.
+
+Reference types are dereferenced *before* a conversion can take place — the conversion is applied to the C type pointed to by the reference.
+
+### Conversions from Lua objects to C types
+
+These conversion rules apply for *write accesses* to C types: indexing pointers, arrays or `struct`/`union` types; initializing cdata objects; casts to C types; writing to external variables; passing arguments to C calls:
+
+Input | Output | Conversion
+--------------|-------------------------|---------
+number | `double` | →
+boolean | `bool` | `false` → 0, `true` → 1
+nil | `(void *)` | `NULL` →
+lightuserdata | `(void *)` | lightuserdata address →
+userdata | `(void *)` | userdata payload →
+io.\* file | `(void *)` | get FILE \* handle →
+string | `enum` | match against `enum` constant
+string | `int8_t[]`, `uint8_t[]` | copy string data + zero-byte
+string | `const char[]` | string data →
+function | C function type | [create callback](#callback) →
+table | Array | [table initializer](#init_table)
+table | `struct`/`union` | [table initializer](#init_table)
+cdata | C type | cdata payload →
+
+If the result type of this conversion doesn't match the C type of the destination, the [conversion rules between C types](#convert_between) are applied.
+
+Reference types are immutable after initialization ("no re-seating of references"). For initialization purposes or when passing values to reference parameters, they are treated like pointers. Note that unlike in C++, there's no way to implement automatic reference generation of variables under the Lua language semantics. If you want to call a function with a reference parameter, you need to explicitly pass a one-element array.
+
+### Conversions between C types
+
+These conversion rules are more or less the same as the standard C conversion rules. Some rules only apply to casts, or require pointer or type compatibility:
+
+Input | Output | Conversion
+------------------|----------------------------|------------------
+Signed integer | Integer | →narrow\ or\ sign-extend
+Unsigned integer | Integer | →narrow\ or\ zero-extend
+Integer | `double`, `float` | →round
+`double`, `float` | `(u)int8_t`, `(u)int16_t` | →trunc `int32_t` →narrow
+`double`, `float` | `(u)int32_t`, `(u)int64_t` | →trunc
+`double`, `float` | `float`, `double` | →round
+Number | `bool` | n == 0 → 0, otherwise 1
+`bool` | Number | `false` → 0, `true` → 1
+Complex number | Number | convert real part
+Number | Complex number | convert real part, imag = 0
+Complex number | Complex number | convert real and imag part
+Number | Vector | convert scalar and replicate
+Vector | Vector | copy (same size)
+`struct`/`union` | Pointer | take base address (compat)
+Array | Pointer | take base address (compat)
+Function | Function pointer | take function address
+Number | Pointer | convert via `uintptr_t` (cast)
+Pointer | Pointer | convert address (compat/cast)
+Pointer | Integer | convert address (cast)
+Array | Integer | convert base address (cast)
+Array | Array | copy (compat)
+`struct`/`union` | `struct`/`union` | copy (identical type)
+
+Bitfields or `enum` types are treated like their underlying type.
+
+Conversions not listed above will raise an error. E.g. it's not possible to convert a pointer to a complex number or vice versa.
+
+### Conversions for vararg C function arguments
+
+The following default conversion rules apply when passing Lua objects to the variable argument part of vararg C functions:
+
+Input | Output | Conversion
+-----------------------|--------------------------|------------
+number | `double` | →
+boolean | `bool` | `false` → 0, `true` → 1
+nil | `(void *)` | `NULL` →
+userdata | `(void *)` | userdata payload →
+lightuserdata | `(void *)` | lightuserdata address →
+string | `const char *` | string data →
+`float` cdata | `double` | →
+Array cdata | Element pointer | take base address
+`struct`/`union` cdata | `struct`/`union` pointer | take base address
+Function cdata | Function pointer | take function address
+Any other cdata | C type | no conversion
+
+To pass a Lua object, other than a cdata object, as a specific type, you need to override the conversion rules: create a temporary cdata object with a constructor or a cast and initialize it with the value to pass:
+
+Assuming `x` is a Lua number, here's how to pass it as an integer to a vararg function:
+
+```lua
+ffi.cdef[[
+ int printf(const char *fmt, ...);
+]];
+
+ffi.C.printf("integer value: %d\n", ffi.new("int", x));
+```
+
+If you don't do this, the default Lua number → `double` conversion rule applies. A vararg C function expecting an integer will see a garbled or uninitialized value.
+
+Note: this is the only place where creating a boxed scalar number type is actually useful. **Never use `ffi.new("int")`, `ffi.new("float")` etc. anywhere else!**
+
+Ditto for `ffi.cast()`. Explicitly boxing scalars **does not** improve performance or force `int` or `float` arithmetic! It just adds costly boxing, unboxing and conversions steps. And it may lead to surprise results, because [cdata arithmetic on scalar numbers](#cdata_arith) is always performed on 64 bit integers.
+
+## Initializers
+
+Creating a cdata object with [`ffi.new()`](./ext_ffi_api#ffi_new) or the equivalent constructor syntax always initializes its contents, too. Different rules apply, depending on the number of optional initializers and the C types involved:
+
+- If no initializers are given, the object is filled with zero bytes.
+- Scalar types (numbers and pointers) accept a single initializer. The Lua object is [converted to the scalar C type](#convert_fromlua).
+- Valarrays (complex numbers and vectors) are treated like scalars when a single initializer is given. Otherwise they are treated like regular arrays.
+- Aggregate types (arrays and structs) accept either a single cdata initializer of the same type (copy constructor), a single [table initializer](#init_table), or a flat list of initializers.
+- The elements of an array are initialized, starting at index zero. If a single initializer is given for an array, it's repeated for all remaining elements. This doesn't happen if two or more initializers are given: all remaining uninitialized elements are filled with zero bytes.
+- Byte arrays may also be initialized with a Lua string. This copies the whole string plus a terminating zero-byte. The copy stops early only if the array has a known, fixed size.
+- The fields of a `struct` are initialized in the order of their declaration. Uninitialized fields are filled with zero bytes.
+- Only the first field of a `union` can be initialized with a flat initializer.
+- Elements or fields which are aggregates themselves are initialized with a *single* initializer, but this may be a table initializer or a compatible aggregate.
+- Excess initializers cause an error.
+
+## Table Initializers
+
+The following rules apply if a Lua table is used to initialize an Array or a `struct`/`union`:
+
+- If the table index `[0]` is non-`nil`, then the table is assumed to be zero-based. Otherwise it's assumed to be one-based.
+- Array elements, starting at index zero, are initialized one-by-one with the consecutive table elements, starting at either index `[0]` or `[1]`. This process stops at the first `nil` table element.
+- If exactly one array element was initialized, it's repeated for all the remaining elements. Otherwise all remaining uninitialized elements are filled with zero bytes.
+- The above logic only applies to arrays with a known fixed size. A VLA is only initialized with the element(s) given in the table. Depending on the use case, you may need to explicitly add a `NULL` or `0` terminator to a VLA.
+- A `struct`/`union` can be initialized in the order of the declaration of its fields. Each field is initialized with consecutive table elements, starting at either index `[0]` or `[1]`. This process stops at the first `nil` table element.
+- Otherwise, if neither index `[0]` nor `[1]` is present, a `struct`/`union` is initialized by looking up each field name (as a string key) in the table. Each non-`nil` value is used to initialize the corresponding field.
+- Uninitialized fields of a `struct` are filled with zero bytes, except for the trailing VLA of a VLS.
+- Initialization of a `union` stops after one field has been initialized. If no field has been initialized, the `union` is filled with zero bytes.
+- Elements or fields which are aggregates themselves are initialized with a *single* initializer, but this may be a nested table initializer (or a compatible aggregate).
+- Excess initializers for an array cause an error. Excess initializers for a `struct`/`union` are ignored. Unrelated table entries are ignored, too.
+
+Example:
+
+```lua
+local ffi = require "ffi";
+
+ffi.cdef [[
+ struct foo { int a, b; };
+ union bar { int i; double d; };
+ struct nested { int x; struct foo y; };
+]];
+
+ffi.new("int[3]", {}); --> 0, 0, 0
+ffi.new("int[3]", { 1 }); --> 1, 1, 1
+ffi.new("int[3]", { 1, 2 }); --> 1, 2, 0
+ffi.new("int[3]", { 1, 2, 3 }); --> 1, 2, 3
+ffi.new("int[3]", { [0] = 1 }); --> 1, 1, 1
+ffi.new("int[3]", { [0] = 1, 2 }); --> 1, 2, 0
+ffi.new("int[3]", { [0] = 1, 2, 3 }); --> 1, 2, 3
+ffi.new("int[3]", { [0] = 1, 2, 3, 4 }); --> error: too many initializers
+
+ffi.new("struct foo", {}); --> a = 0, b = 0
+ffi.new("struct foo", { 1 }); --> a = 1, b = 0
+ffi.new("struct foo", { 1, 2 }); --> a = 1, b = 2
+ffi.new("struct foo", { [0] = 1, 2 }); --> a = 1, b = 2
+ffi.new("struct foo", { b = 2 }); --> a = 0, b = 2
+ffi.new("struct foo", { a = 1, b = 2, c = 3 }); --> a = 1, b = 2 'c' is ignored
+
+ffi.new("union bar", {}); --> i = 0, d = 0.0
+ffi.new("union bar", { 1 }); --> i = 1, d = ?
+ffi.new("union bar", { [0] = 1, 2 }); --> i = 1, d = ? '2' is ignored
+ffi.new("union bar", { d = 2 }); --> i = ?, d = 2.0
+
+ffi.new("struct nested", { 1, { 2, 3 } }); --> x = 1, y.a = 2, y.b = 3
+ffi.new("struct nested", { x = 1, y = { 2, 3 } }); --> x = 1, y.a = 2, y.b = 3
+```
+
+## Operations on cdata Objects
+
+All standard Lua operators can be applied to cdata objects or a mix of a cdata object and another Lua object. The following list shows the predefined operations.
+
+Reference types are dereferenced *before* performing each of the operations below — the operation is applied to the C type pointed to by the reference.
+
+The predefined operations are always tried first before deferring to a metamethod or index table (if any) for the corresponding ctype (except for `__new`). An error is raised if the metamethod lookup or index table lookup fails.
+
+### Indexing a cdata object
+- **Indexing a pointer/array**: a cdata pointer/array can be indexed by a cdata number or a Lua number. The element address is computed as the base address plus the number value multiplied by the element size in bytes. A read access loads the element value and [converts it to a Lua object](#convert_tolua). A write access [converts a Lua object to the element type](#convert_fromlua) and stores the converted value to the element. An error is raised if the element size is undefined or a write access to a constant element is attempted.
+- **Dereferencing a `struct`/`union` field**: a cdata `struct`/`union` or a pointer to a `struct`/`union` can be dereferenced by a string key, giving the field name. The field address is computed as the base address plus the relative offset of the field. A read access loads the field value and [converts it to a Lua object](#convert_tolua). A write access [converts a Lua object to the field type](#convert_fromlua) and stores the converted value to the field. An error is raised if a write access to a constant `struct`/`union` or a constant field is attempted. Scoped enum constants or static constants are treated like a constant field.
+- **Indexing a complex number**: a complex number can be indexed either by a cdata number or a Lua number with the values 0 or 1, or by the strings `"re"` or `"im"`. A read access loads the real part (`[0]`, `.re`) or the imaginary part (`[1]`, `.im`) part of a complex number and [converts it to a Lua number](#convert_tolua). The sub-parts of a complex number are immutable — assigning to an index of a complex number raises an error. Accessing out-of-bound indexes returns unspecified results, but is guaranteed not to trigger memory access violations.
+- **Indexing a vector**: a vector is treated like an array for indexing purposes, except the vector elements are immutable — assigning to an index of a vector raises an error.
+
+A ctype object can be indexed with a string key, too. The only predefined operation is reading scoped constants of `struct`/`union` types. All other accesses defer to the corresponding metamethods or index tables (if any).
+
+Note: since there's (deliberately) no address-of operator, a cdata object holding a value type is effectively immutable after initialization. The JIT compiler benefits from this fact when applying certain optimizations.
+
+As a consequence, the *elements* of complex numbers and vectors are immutable. But the elements of an aggregate holding these types *may* be modified, of course. I.e. you cannot assign to `foo.c.im`, but you can assign a (newly created) complex number to `foo.c`.
+
+The JIT compiler implements strict aliasing rules: accesses to different types do **not** alias, except for differences in signedness (this applies even to `char` pointers, unlike C99). Type punning through unions is explicitly detected and allowed.
+
+### Calling a cdata object
+
+- **Constructor**: a ctype object can be called and used as a [constructor](./ext_ffi_api#ffi_new). This is equivalent to `ffi.new(ct, ...)`, unless a `__new` metamethod is defined. The `__new` metamethod is called with the ctype object plus any other arguments passed to the constructor. Note that you have to use `ffi.new` inside the metamethod, since calling `ct(...)` would cause infinite recursion.
+- **C function call**: a cdata function or cdata function pointer can be called. The passed arguments are [converted to the C types](#convert_fromlua) of the parameters given by the function declaration. Arguments passed to the variable argument part of vararg C function use [special conversion rules](#convert_vararg). This C function is called and the return value (if any) is [converted to a Lua object](#convert_tolua). On Windows/x86 systems, `__stdcall` functions are automatically detected, and a function declared as `__cdecl` (the default) is silently fixed up after the first call.
+
+### Arithmetic on cdata objects
+
+- **Pointer arithmetic**: a cdata pointer/array and a cdata number or a Lua number can be added or subtracted. The number must be on the right-hand side for a subtraction. The result is a pointer of the same type with an address plus or minus the number value multiplied by the element size in bytes. An error is raised if the element size is undefined.
+- **Pointer difference**: two compatible cdata pointers/arrays can be subtracted. The result is the difference between their addresses, divided by the element size in bytes. An error is raised if the element size is undefined or zero.
+- **64 bit integer arithmetic**: the standard arithmetic operators (`+ - * / % ^` and unary minus) can be applied to two cdata numbers, or a cdata number and a Lua number. If one of them is an `uint64_t`, the other side is converted to an `uint64_t` and an unsigned arithmetic operation is performed. Otherwise, both sides are converted to an `int64_t` and a signed arithmetic operation is performed. The result is a boxed 64 bit cdata object. If one of the operands is an `enum` and the other operand is a string, the string is converted to the value of a matching `enum` constant before the above conversion. These rules ensure that 64 bit integers are "sticky". Any expression involving at least one 64 bit integer operand results in another one. The undefined cases for the division, modulo and power operators return `2LL ^ 63` or `2ULL ^ 63`. You'll have to explicitly convert a 64 bit integer to a Lua number (e.g. for regular floating-point calculations) with `tonumber()`. But note this may incur a precision loss.
+- **64 bit bitwise operations**: the rules for 64 bit arithmetic operators apply analogously. Unlike the other `bit.*` operations, `bit.tobit()` converts a cdata number via `int64_t` to `int32_t` and returns a Lua number. For `bit.band()`, `bit.bor()` and `bit.bxor()`, the conversion to `int64_t` or `uint64_t` applies to *all* arguments, if *any* argument is a cdata number. For all other operations, only the first argument is used to determine the output type. This implies that a cdata number as a shift count for shifts and rotates is accepted, but that alone does *not* cause a cdata number output.
+
+### Comparisons of cdata objects
+
+- **Pointer comparison**: two compatible cdata pointers/arrays can be compared. The result is the same as an unsigned comparison of their addresses. `nil` is treated like a `NULL` pointer, which is compatible with any other pointer type.
+- **64 bit integer comparison**: two cdata numbers, or a cdata number and a Lua number can be compared with each other. If one of them is an `uint64_t`, the other side is converted to an `uint64_t` and an unsigned comparison is performed. Otherwise, both sides are converted to an `int64_t` and a signed comparison is performed. If one of the operands is an `enum` and the other operand is a string, the string is converted to the value of a matching `enum` constant before the above conversion.
+- **Comparisons for equality/inequality** never raise an error. Even incompatible pointers can be compared for equality by address. Any other incompatible comparison (also with non-cdata objects) treats the two sides as unequal.
+
+### cdata objects as table keys
+
+Lua tables may be indexed by cdata objects, but this doesn't provide any useful semantics — **cdata objects are unsuitable as table keys!**
+
+A cdata object is treated like any other garbage-collected object and is hashed and compared by its address for table indexing. Since there's no interning for cdata value types, the same value may be boxed in different cdata objects with different addresses. Thus, `t[1LL+1LL]` and `t[2LL]` usually **do not** point to the same hash slot, and they certainly **do not** point to the same hash slot as `t[2]`.
+
+It would seriously drive up implementation complexity and slow down the common case, if one were to add extra handling for by-value hashing and comparisons to Lua tables. Given the ubiquity of their use inside the VM, this is not acceptable.
+
+There are three viable alternatives, if you really need to use cdata z objects as keys:
+
+- If you can get by with the precision of Lua numbers (52 bits), then use `tonumber()` on a cdata number or combine multiple fields of a cdata aggregate to a Lua number. Then use the resulting Lua number as a key when indexing tables. One obvious benefit: `t[tonumber(2LL)]` **does** point to the same slot as `t[2]`.
+- Otherwise, use either `tostring()` on 64 bit integers or complex numbers or combine multiple fields of a cdata aggregate to a Lua string (e.g. with [`ffi.string()`](./ext_ffi_api#ffi_string)). Then use the resulting Lua string as a key when indexing tables.
+- Create your own specialized hash table implementation using the C types provided by the FFI library, just like you would in C code. Ultimately, this may give much better performance than the other alternatives or what a generic by-value hash table could possibly provide.
+
+## Parameterized Types
+
+To facilitate some abstractions, the two functions [`ffi.typeof`](./ext_ffi_api#ffi_typeof) and [`ffi.cdef`](./ext_ffi_api#ffi_cdef) support parameterized types in C declarations. Note: none of the other API functions taking a cdecl allow this.
+
+Any place you can write a **`typedef` name**, an **identifier** or a **number** in a declaration, you can write `$` (the dollar sign) instead. These placeholders are replaced in order of appearance with the arguments following the cdecl string:
+
+```lua
+-- Declare a struct with a parameterized field type and name:
+ffi.cdef([[
+ typedef struct { $ $; } foo_t;
+]], type1, name1);
+
+-- Anonymous struct with dynamic names:
+local bar_t = ffi.typeof("struct { int $, $; }", name1, name2);
+-- Derived pointer type:
+local bar_ptr_t = ffi.typeof("$ *", bar_t);
+
+-- Parameterized dimensions work even where a VLA won't work:
+local matrix_t = ffi.typeof("uint8_t[$][$]", width, height);
+```
+
+Caveat: this is *not* simple text substitution! A passed ctype or cdata object is treated like the underlying type, a passed string is considered an identifier and a number is considered a number. You must not mix this up: e.g. passing `"int"` as a string doesn't work in place of a type, you'd need to use `ffi.typeof("int")` instead.
+
+The main use for parameterized types are libraries implementing abstract data types ([» example](https://www.freelists.org/post/luajit/ffi-type-of-pointer-to,8)), similar to what can be achieved with C++ template metaprogramming. Another use case are derived types of anonymous structs, which avoids pollution of the global struct namespace.
+
+Please note that parameterized types are a nice tool and indispensable for certain use cases. But you'll want to use them sparingly in regular code, e.g. when all types are actually fixed.
+
+## Garbage Collection of cdata Objects
+
+All explicitly (`ffi.new()`, `ffi.cast()` etc.) or implicitly (accessors) created cdata objects are garbage collected. You need to ensure to retain valid references to cdata objects somewhere on a Lua stack, an upvalue or in a Lua table while they are still in use. Once the last reference to a cdata object is gone, the garbage collector will automatically free the memory used by it (at the end of the next GC cycle).
+
+Please note, that pointers themselves are cdata objects, however they are **not** followed by the garbage collector. So e.g. if you assign a cdata array to a pointer, you must keep the cdata object holding the array alive as long as the pointer is still in use:
+
+```lua
+ffi.cdef[[
+ typedef struct { int *a; } foo_t;
+]];
+
+local s = ffi.new("foo_t", ffi.new("int[10]")); -- WRONG!
+
+local a = ffi.new("int[10]"); -- OK
+local s = ffi.new("foo_t", a);
+-- Now do something with 's', but keep 'a' alive until you're done.
+```
+
+Similar rules apply for Lua strings which are implicitly converted to `"const char *"`: the string object itself must be referenced somewhere or it'll be garbage collected eventually. The pointer will then point to stale data, which may have already been overwritten. Note that *string literals* are automatically kept alive as long as the function containing it (actually its prototype) is not garbage collected.
+
+Objects which are passed as an argument to an external C function are kept alive until the call returns. So it's generally safe to create temporary cdata objects in argument lists. This is a common idiom for [passing specific C types to vararg functions](#convert_vararg).
+
+Memory areas returned by C functions (e.g. from `malloc()`) must be manually managed, of course (or use [`ffi.gc()`](./ext_ffi_api#ffi_gc)). Pointers to cdata objects are indistinguishable from pointers returned by C functions (which is one of the reasons why the GC cannot follow them).
+
+## Callbacks
+
+The LuaJIT FFI automatically generates special callback functions whenever a Lua function is converted to a C function pointer. This associates the generated callback function pointer with the C type of the function pointer and the Lua function object (closure).
+
+This can happen implicitly due to the usual conversions, e.g. when passing a Lua function to a function pointer argument. Or, you can use `ffi.cast()` to explicitly cast a Lua function to a C function pointer.
+
+Currently, only certain C function types can be used as callback functions. Neither C vararg functions nor functions with pass-by-value aggregate argument or result types are supported. There are no restrictions on the kind of Lua functions that can be called from the callback — no checks for the proper number of arguments are made. The return value of the Lua function will be converted to the result type, and an error will be thrown for invalid conversions.
+
+It's allowed to throw errors across a callback invocation, but it's not advisable in general. Do this only if you know the C function, that called the callback, copes with the forced stack unwinding and doesn't leak resources.
+
+One thing that's not allowed, is to let an FFI call into a C function get JIT-compiled, which in turn calls a callback, calling into Lua again. Usually this attempt is caught by the interpreter first and the C function is blacklisted for compilation.
+
+However, this heuristic may fail under specific circumstances: e.g. a message polling function might not run Lua callbacks right away and the call gets JIT-compiled. If it later happens to call back into Lua (e.g. a rarely invoked error callback), you'll get a VM PANIC with the message `"bad callback"`. Then you'll need to manually turn off JIT-compilation with [`jit.off()`](./ext_jit#jit_onoff_func) for the surrounding Lua function that invokes such a message polling function (or similar).
+
+### Callback resource handling
+
+Callbacks take up resources — you can only have a limited number of them at the same time (500 - 1000, depending on the architecture). The associated Lua functions are anchored to prevent garbage collection, too.
+
+**Callbacks due to implicit conversions are permanent!** There is no way to guess their lifetime, since the C side might store the function pointer for later use (typical for GUI toolkits). The associated resources cannot be reclaimed until termination:
+
+```lua
+ffi.cdef [[
+ typedef int (__stdcall *WNDENUMPROC)(void *hwnd, intptr_t l);
+ int EnumWindows(WNDENUMPROC func, intptr_t l);
+]];
+
+-- Implicit conversion to a callback via function pointer argument.
+local count = 0;
+ffi.C.EnumWindows(function(hwnd, l)
+ count = count + 1;
+ return true;
+end, 0);
+-- The callback is permanent and its resources cannot be reclaimed!
+-- Ok, so this may not be a problem, if you do this only once.
+```
+
+Note: this example shows that you *must* properly declare `__stdcall` callbacks on Windows/x86 systems. The calling convention cannot be automatically detected, unlike for `__stdcall` calls *to* Windows functions.
+
+For some use cases, it's necessary to free up the resources or to dynamically redirect callbacks. Use an explicit cast to a C function pointer and keep the resulting cdata object. Then use the [`cb:free()`](./ext_ffi_api#callback_free) or [`cb:set()`](./ext_ffi_api#callback_set) methods on the cdata object:
+
+```lua
+-- Explicitly convert to a callback via cast.
+local count = 0;
+local cb = ffi.cast("WNDENUMPROC", function(hwnd, l)
+ count = count + 1;
+ return true;
+end);
+
+-- Pass it to a C function.
+ffi.C.EnumWindows(cb, 0);
+-- EnumWindows doesn't need the callback after it returns, so free it.
+
+cb:free();
+-- The callback function pointer is no longer valid and its resources
+-- will be reclaimed. The created Lua closure will be garbage collected.
+```
+
+### Callback performance
+
+**Callbacks are slow!** First, the C to Lua transition itself has an unavoidable cost, similar to a `lua_call()` or `lua_pcall()`. Argument and result marshalling add to that cost. And finally, neither the C compiler nor LuaJIT can inline or optimize across the language barrier and hoist repeated computations out of a callback function.
+
+Do not use callbacks for performance-sensitive work: e.g. consider a numerical integration routine which takes a user-defined function to integrate over. It's a bad idea to call a user-defined Lua function from C code millions of times. The callback overhead will be absolutely detrimental for performance.
+
+It's considerably faster to write the numerical integration routine itself in Lua — the JIT compiler will be able to inline the user-defined function and optimize it together with its calling context, with very competitive performance.
+
+As a general guideline: **use callbacks only when you must**, because of existing C APIs. E.g. callback performance is irrelevant for a GUI application, which waits for user input most of the time, anyway.
+
+For new designs **avoid push-style APIs**: a C function repeatedly calling a callback for each result. Instead, **use pull-style APIs**: call a C function repeatedly to get a new result. Calls from Lua to C via the FFI are much faster than the other way round. Most well-designed libraries already use pull-style APIs (read/write, get/put).
+
+## C Library Namespaces
+
+A C library namespace is a special kind of object which allows access to the symbols contained in shared libraries or the default symbol namespace. The default [`ffi.C`](./ext_ffi_api#ffi_C) namespace is automatically created when the FFI library is loaded. C library namespaces for specific shared libraries may be created with the [`ffi.load()`](./ext_ffi_api#ffi_load) API function.
+
+Indexing a C library namespace object with a symbol name (a Lua string) automatically binds it to the library. First, the symbol type is resolved — it must have been declared with [`ffi.cdef`](./ext_ffi_api#ffi_cdef). Then the symbol address is resolved by searching for the symbol name in the associated shared libraries or the default symbol namespace. Finally, the resulting binding between the symbol name, the symbol type and its address is cached. Missing symbol declarations or nonexistent symbol names cause an error.
+
+This is what happens on a **read access** for the different kinds of
+symbols:
+
+- External functions: a cdata object with the type of the function and its address is returned.
+- External variables: the symbol address is dereferenced and the loaded value is [converted to a Lua object](#convert_tolua) and returned.
+- Constant values (`static const` or `enum` constants): the constant is [converted to a Lua object](#convert_tolua) and returned.
+
+This is what happens on a **write access**:
+
+- External variables: the value to be written is [converted to the C type](#convert_fromlua) of the variable and then stored at the symbol address.
+- Writing to constant variables or to any other symbol type causes an error, like any other attempted write to a constant location.
+
+C library namespaces themselves are garbage collected objects. If the last reference to the namespace object is gone, the garbage collector will eventually release the shared library reference and remove all memory associated with the namespace. Since this may trigger the removal of the shared library from the memory of the running process, it's generally *not safe* to use function cdata objects obtained from a library if the namespace object may be unreferenced.
+
+Performance notice: the JIT compiler specializes to the identity of namespace objects and to the strings used to index it. This effectively turns function cdata objects into constants. It's not useful and actually counter-productive to explicitly cache these function objects, e.g. `local strlen = ffi.C.strlen`. OTOH, it *is* useful to cache the namespace itself, e.g. `local C = ffi.C`.
+
+## No Hand-holding!
+
+The FFI library has been designed as **a low-level library**. The goal is to interface with C code and C data types with a minimum of overhead. This means **you can do anything you can do from C**: access all memory, overwrite anything in memory, call machine code at any memory address and so on.
+
+The FFI library provides **no memory safety**, unlike regular Lua code. It will happily allow you to dereference a `NULL` pointer, to access arrays out of bounds or to misdeclare C functions. If you make a mistake, your application might crash, just like equivalent C code would.
+
+This behavior is inevitable, since the goal is to provide full interoperability with C code. Adding extra safety measures, like bounds checks, would be futile. There's no way to detect misdeclarations of C functions, since shared libraries only provide symbol names, but no type information. Likewise, there's no way to infer the valid range of indexes for a returned pointer.
+
+Again: the FFI library is a low-level library. This implies it needs to be used with care, but it's flexibility and performance often outweigh this concern. If you're a C or C++ developer, it'll be easy to apply your existing knowledge. OTOH, writing code for the FFI library is not for the faint of heart and probably shouldn't be the first exercise for someone with little experience in Lua, C or C++.
+
+As a corollary of the above, the FFI library is **not safe for use by untrusted Lua code**. If you're sandboxing untrusted Lua code, you definitely don't want to give this code access to the FFI library or to *any* cdata object (except 64 bit integers or complex numbers). Any properly engineered Lua sandbox needs to provide safety wrappers for many of the standard Lua library functions — similar wrappers need to be written for high-level operations on FFI data types, too.
+
+## Current Status
+
+The initial release of the FFI library has some limitations and is missing some features. Most of these will be fixed in future releases.
+
+[C language support](#clang) is currently incomplete:
+
+- C declarations are not passed through a C pre-processor, yet.
+- The C parser is able to evaluate most constant expressions commonly found in C header files. However, it doesn't handle the full range of C expression semantics and may fail for some obscure constructs.
+- `static const` declarations only work for integer types up to 32 bits. Neither declaring string constants nor floating-point constants is supported.
+- Packed `struct` bitfields that cross container boundaries are not implemented.
+- Native vector types may be defined with the GCC `mode` or `vector_size` attribute. But no operations other than loading, storing and initializing them are supported, yet.
+- The `volatile` type qualifier is currently ignored by compiled code.
+- [`ffi.cdef`](./ext_ffi_api#ffi_cdef) silently ignores most re-declarations. Note: avoid re-declarations which do not conform to C99. The implementation will eventually be changed to perform strict checks.
+
+The JIT compiler already handles a large subset of all FFI operations. It automatically falls back to the interpreter for unimplemented operations (you can check for this with the [`-jv`](./running#opt_j) command line option). The following operations are currently not compiled and may exhibit suboptimal performance, especially when used in inner loops:
+
+- Vector operations.
+- Table initializers.
+- Initialization of nested `struct`/`union` types.
+- Non-default initialization of VLA/VLS or large C types (\> 128 bytes or \> 16 array elements).
+- Bitfield initializations.
+- Pointer differences for element sizes that are not a power of two.
+- Calls to C functions with aggregates passed or returned by value.
+- Calls to ctype metamethods which are not plain functions.
+- ctype `__newindex` tables and non-string lookups in ctype `__index` tables.
+- `tostring()` for cdata types.
+- Calls to `ffi.cdef()`, `ffi.load()` and `ffi.metatype()`.
+
+Other missing features:
+
+- Arithmetic for `complex` numbers.
+- Passing structs by value to vararg C functions.
+- [C++ exception interoperability](./extensions#exceptions) does not extend to C functions called via the FFI, if the call is compiled.
diff --git a/doc/ffi/tutorial.md b/doc/ffi/tutorial.md
new file mode 100644
index 00000000..f963cf1b
--- /dev/null
+++ b/doc/ffi/tutorial.md
@@ -0,0 +1,232 @@
+# FFI Tutorial
+
+This page is intended to give you an overview of the features of the FFI library by presenting a few use cases and guidelines.
+
+This page makes no attempt to explain all of the FFI library, though. You'll want to have a look at the [ffi.* API function reference](./ext_ffi_api) and the [FFI semantics](./ext_ffi_semantics) to learn more.
+
+## Loading the FFI Library
+
+The FFI library is built into LuaJIT by default, but it's not loaded and initialized by default. The suggested way to use the FFI library is to add the following to the start of every Lua file that needs one of its functions:
+
+```lua
+local ffi = require "ffi";
+```
+
+Please note, this doesn't define an `ffi` variable in the table of globals — you really need to use the local variable. The `require` function ensures the library is only loaded once.
+
+Note: If you want to experiment with the FFI from the interactive prompt of the command line executable, omit the `local`, as it doesn't preserve local variables across lines.
+
+## Accessing Standard System Functions
+
+The following code explains how to access standard system functions. We slowly print two lines of dots by sleeping for 10 milliseconds after each dot:
+
+```lua
+local ffi = require "ffi";
+ffi.cdef[[
+ void Sleep(int ms);
+ int poll(struct pollfd *fds, unsigned long nfds, int timeout);
+]]; -- (1)
+
+local sleep;
+if ffi.os == "Windows" then -- (2)
+ function sleep(s) -- (3)
+ ffi.C.Sleep(s * 1000); -- (4)
+ end
+else
+ function sleep(s)
+ ffi.C.poll(nil, 0, s*1000) -- (5)
+ end
+end
+
+for i=1,160 do
+ io.write ".";
+ io.flush();
+ sleep(0.01); -- (6)
+end
+io.write "\n";
+```
+
+Here's the step-by-step explanation:
+
+1. This defines the C library functions we're going to use. The part inside the double-brackets (in green) is just standard C syntax. You can usually get this info from the C header files or the documentation provided by each C library or C compiler.
+
+2. The difficulty we're facing here, is that there are different standards to choose from. Windows has a simple `Sleep()` function. On other systems there are a variety of functions available to achieve sub-second sleeps, but with no clear consensus. Thankfully `poll()` can be used for this task, too, and it's present on most non-Windows systems. The check for `ffi.os` makes sure we use the Windows-specific function only on Windows systems.
+
+3. Here we're wrapping the call to the C function in a Lua function. This isn't strictly necessary, but it's helpful to deal with system-specific issues only in one part of the code. The way we're wrapping it ensures the check for the OS is only done during initialization and not for every call.
+
+4. A more subtle point is that we defined our `sleep()` function (for the sake of this example) as taking the number of seconds, but accepting fractional seconds. Multiplying this by 1000 gets us milliseconds, but that still leaves it a Lua number, which is a floating-point value. Alas, the `Sleep()` function only accepts an integer value. Luckily for us, the FFI library automatically performs the conversion when calling the function (truncating the FP value towards zero, like in C). *Some readers will notice that `Sleep()` is part of `KERNEL32.DLL` and is also a `stdcall` function. So how can this possibly work? The FFI library provides the `ffi.C` default C library namespace, which allows calling functions from the default set of libraries, like a C compiler would. Also, the FFI library automatically detects `stdcall` functions, so you don't need to declare them as such.*
+
+5. The `poll()` function takes a couple more arguments we're not going to use. You can simply use `nil` to pass a `NULL` pointer and `0` for the `nfds` parameter. Please note, that the number `0` *does not convert to a pointer value*, unlike in C++. You really have to pass pointers to pointer arguments and numbers to number arguments. *The page on [FFI semantics](./ext_ffi_semantics) has all of the gory details about [conversions between Lua objects and C types](./ext_ffi_semantics#convert). For the most part you don't have to deal with this, as it's performed automatically and it's carefully designed to bridge the semantic differences between Lua and C.*
+
+6. Now that we have defined our own `sleep()` function, we can just call it from plain Lua code. That wasn't so bad, huh? Turning these boring animated dots into a fascinating best-selling game is left as an exercise for the reader. :-)
+
+## Accessing the zlib Compression Library
+
+The following code shows how to access the [» zlib](https://zlib.net/) compression library from Lua code. We'll define two convenience wrapper functions that take a string and compress or uncompress it to another string:
+
+```lua
+local ffi = require "ffi";
+ffi.cdef [[
+ unsigned long compressBound(unsigned long sourceLen);
+ int compress2(uint8_t *dest, unsigned long *destLen, const uint8_t *source, unsigned long sourceLen, int level);
+ int uncompress(uint8_t *dest, unsigned long *destLen, const uint8_t *source, unsigned long sourceLen);
+]]; -- (1)
+
+local zlib = ffi.load(ffi.os == "Windows" and "zlib1" or "z"); -- (2)
+
+local function compress(txt)
+ local n = zlib.compressBound(#txt); -- (3)
+
+ local buf = ffi.new("uint8_t[?]", n);
+ local buflen = ffi.new("unsigned long[1]", n); -- (4)
+ local res = zlib.compress2(buf, buflen, txt, #txt, 9);
+ assert(res == 0);
+
+ return ffi.string(buf, buflen[0]); -- (5)
+end
+
+local function uncompress(comp, n) -- (6)
+ local buf = ffi.new("uint8_t[?]", n);
+ local buflen = ffi.new("unsigned long[1]", n);
+ local res = zlib.uncompress(buf, buflen, comp, #comp);
+ assert(res == 0);
+
+ return ffi.string(buf, buflen[0]);
+end
+
+-- Simple test code (7)
+local txt = string.rep("abcd", 1000);
+print("Uncompressed size: ", #txt);
+
+local c = compress(txt);
+print("Compressed size: ", #c);
+
+local txt2 = uncompress(c, #txt);
+assert(txt2 == txt);
+```
+
+Here's the step-by-step explanation:
+
+1. This defines some of the C functions provided by zlib. For the sake of this example, some type indirections have been reduced and it uses the predefined fixed-size integer types, while still adhering to the zlib API/ABI.
+2. This loads the zlib shared library. On POSIX systems, it's named `libz.so` and usually comes pre-installed. Since `ffi.load()` automatically adds any missing standard prefixes/suffixes, we can simply load the `"z"` library. On Windows it's named `zlib1.dll` and you'll have to download it first from the [» zlib site](https://zlib.net/). The check for `ffi.os` makes sure we pass the right name to `ffi.load()`.
+3. First, the maximum size of the compression buffer is obtained by calling the `zlib.compressBound` function with the length of the uncompressed string. The next line allocates a byte buffer of this size. The `[?]` in the type specification indicates a variable-length array (VLA). The actual number of elements of this array is given as the 2nd argument to `ffi.new()`.
+4. This may look strange at first, but have a look at the declaration of the `compress2` function from zlib: the destination length is defined as a pointer! This is because you pass in the maximum buffer size and get back the actual length that was used. In C you'd pass in the address of a local variable (`&buflen`). But since there's no address-of operator in Lua, we'll just pass in a one-element array. Conveniently, it can be initialized with the maximum buffer size in one step. Calling the actual `zlib.compress2` function is then straightforward.
+5. We want to return the compressed data as a Lua string, so we'll use `ffi.string()`. It needs a pointer to the start of the data and the actual length. The length has been returned in the `buflen` array, so we'll just get it from there. *Note that since the function returns now, the `buf` and `buflen` variables will eventually be garbage collected. This is fine, because `ffi.string()` has copied the contents to a newly created (interned) Lua string. If you plan to call this function lots of times, consider reusing the buffers and/or handing back the results in buffers instead of strings. This will reduce the overhead for garbage collection and string interning.*
+6. The `uncompress` functions does the exact opposite of the `compress` function. The compressed data doesn't include the size of the original string, so this needs to be passed in. Otherwise, no surprises here.
+7. The code, that makes use of the functions we just defined, is just plain Lua code. It doesn't need to know anything about the LuaJIT FFI — the convenience wrapper functions completely hide it. One major advantage of the LuaJIT FFI is that you are now able to write those wrappers *in Lua*. And at a fraction of the time it would cost you to create an extra C module using the Lua/C API. Many of the simpler C functions can probably be used directly from your Lua code, without any wrappers. *Side note: the zlib API uses the `long` type for passing lengths and sizes around. But all those zlib functions actually only deal with 32 bit values. This is an unfortunate choice for a public API, but may be explained by zlib's history — we'll just have to deal with it.*
+
+First, you should know that a `long` is a 64 bit type e.g. on POSIX/x64 systems, but a 32 bit type on Windows/x64 and on 32 bit systems. Thus a `long` result can be either a plain Lua number or a boxed 64 bit integer cdata object, depending on the target system.
+
+Ok, so the `ffi.*` functions generally accept cdata objects wherever you'd want to use a number. That's why we get a away with passing `n` to `ffi.string()` above. But other Lua library functions or modules don't know how to deal with this. So for maximum portability, one needs to use `tonumber()` on returned `long` results before passing them on. Otherwise the application might work on some systems, but would fail in a POSIX/x64 environment.
+
+## Defining Metamethods for a C Type
+
+The following code explains how to define metamethods for a C type. We define a simple point type and add some operations to it:
+
+```lua
+local ffi = require "ffi";
+ffi.cdef [[
+ typedef struct { double x, y; } point_t;
+]]; -- (1)
+
+local point; -- (2)
+local mt = { __index = {} };
+
+function mt.__add(a, b) -- (3)
+ return point(a.x + b.x, a.y + b.y);
+end
+function mt.__len(a)
+ return math.sqrt(a.x * a.x + a.y * a.y);
+end
+function mt.__index.area(a) -- (4)
+ return a.x * a.x + a.y * a.y;
+end
+
+point = ffi.metatype("point_t", mt); -- (5)
+
+local a = point(3, 4); -- (6)
+print(a.x, a.y); --> 3 4
+print(#a); --> 5
+print(a:area()); --> 25
+local b = a + point(0.5, 8);
+print(#b); --> 12.5
+```
+
+Here's the step-by-step explanation:
+
+1. This defines the C type for a two-dimensional point object.
+
+2. We have to declare the variable holding the point constructor first, because it's used inside of a metamethod.
+
+3. Let's define an `__add` metamethod which adds the coordinates of two points and creates a new point object. For simplicity, this function assumes that both arguments are points. But it could be any mix of objects, if at least one operand is of the required type (e.g. adding a point plus a number or vice versa). Our `__len` metamethod returns the distance of a point to the origin.
+
+4. If we run out of operators, we can define
+named methods, too. Here, the `__index` table defines an `area`
+function. For custom indexing needs, one might want to define `__index`
+and `__newindex` *functions* instead.
+
+5. This associates the metamethods with our C type. This only needs to be done once. For convenience, a constructor is returned by [`ffi.metatype()`](./ext_ffi_api#ffi_metatype). We're not required to use it, though. The original C type can still be used e.g. to create an array of points. The metamethods automatically apply to any and all uses of this type. Please note, that the association with a metatable is permanent and
+**the metatable must not be modified afterwards!** Ditto for the `__index` table.
+
+6. Here are some simple usage examples for the point type and their expected results. The predefined operations (such as `a.x`) can be freely mixed with the newly defined metamethods. Note that `area` is a method and must be called with the Lua syntax for methods: `a:area()`, not `a.area()`.
+
+The C type metamethod mechanism is most useful when used in conjunction with C libraries that are written in an object-oriented style. Creators return a pointer to a new instance, and methods take an instance pointer as the first argument. Sometimes you can just point `__index` to the library namespace and `__gc` to the destructor and you're done. But often enough you'll want to add convenience wrappers, e.g. to return actual Lua strings or when returning multiple values.
+
+Some C libraries only declare instance pointers as an opaque `void *` type. In this case you can use a fake type for all declarations, e.g. a pointer to a named (incomplete) struct will do: `typedef struct foo_type *foo_handle`. The C side doesn't know what you declare with the LuaJIT FFI, but as long as the underlying types are compatible, everything still works.
+
+## Translating C Idioms
+
+Here's a list of common C idioms and their translation to the LuaJIT FFI:
+
+Idiom | C code | Lua code
+-------------------------------|------------------------------------|------
+Pointer dereference | `x = *p; *p = y;` | `x = p[0]; p[0] = y;`
+Pointer/array indexing | `x = p[i]; p[i] = y;` | `x = p[i]; p[i] = y;`
+`struct`/`union` dereference | `x = s.field; s.field = y;` | `x = s.field; s.field = y;`
+`struct`/`union` pointer deref | `x = s->field; s->field = y;` | `x = s.field; s.field = y;`
+Pointer arithmetic | `x = p + i; y = p - i;` | `x = p + i; y = p - i;`
+Pointer difference | `x = p1 - p2;` | `x = p1 - p2;`
+Array element pointer | `x = &a[i];` | `x = a + i;`
+Cast pointer to address | `x = (intptr_t)p;` | `x = tonumber(ffi.cast("intptr_t", p));`
+Functions with outargs | `int len = x; foo(&len); y = len;` | `local len = ffi.new("int[1]", x); foo(len); y = len[0];`
+[Vararg conversions](./ext_ffi_semantics#convert_vararg) | `printf("%g, %d", 1.0, 1);` | `printf("%g, %d", 1, ffi.new("int", 1));`
+
+## To Cache or Not to Cache
+
+It's a common Lua idiom to cache library functions in local variables or upvalues, e.g.:
+
+```lua
+local byte, char = string.byte, string.char;
+local function foo(x)
+ return char(byte(x) + 1);
+end
+```
+
+This replaces several hash-table lookups with a (faster) direct use of a local or an upvalue. This is less important with LuaJIT, since the JIT compiler optimizes hash-table lookups a lot and is even able to hoist most of them out of the inner loops. It can't eliminate *all* of them, though, and it saves some typing for often-used functions. So there's still a place for this, even with LuaJIT.
+
+The situation is a bit different with C function calls via the FFI library. The JIT compiler has special logic to eliminate *all of the lookup overhead* for functions resolved from a [C library namespace](./ext_ffi_semantics#clib)! Thus it's not helpful and actually counter-productive to cache individual C functions like this:
+
+```lua
+-- Not helpful!
+local funca = ffi.C.funca;
+local funcb = ffi.C.funcb;
+
+local function foo(x, n)
+ for i = 1,n do funcb(funca(x, i), 1) end
+end
+```
+
+This turns them into indirect calls and generates bigger and slower machine code. Instead, you'll want to cache the namespace itself and rely on the JIT compiler to eliminate the lookups:
+
+```lua
+-- Instead use this!
+local C = ffi.C;
+
+local function foo(x, n)
+ for i = 1, n do
+ C.funcb(C.funca(x, i), 1);
+ end
+end
+```
+
+This generates both shorter and faster code. So **don't cache C functions**, but **do** cache namespaces! Most often the namespace is already in a local variable at an outer scope, e.g. from `local lib = ffi.load(...)`. Note that copying it to a local variable in the function scope is unnecessary.
diff --git a/doc/img/contact.png b/doc/img/contact.png
deleted file mode 100644
index 9c73dc594efc1f47309d6c9b73d7719c3a9e04df..0000000000000000000000000000000000000000
GIT binary patch
literal 0
HcmV?d00001
literal 1340
zcmV-C1;hG@P)sQ4|K#fZOHEVa==(`aQP0%nM@di3(&I%(O)@h#$&+f^&9&U}9=sU}#xfW8dZX
zS6X4+hl`eQb9`=bdTMQUS6N_FRb1EH?oLrx)!OXR*y%bu
zKs7fzyTH%6zRkD1%O4*iwYte19Urr}$DpLK7#SM{1qGa-tpfuC0RaJph?9SUjedfR
ze1M8)YI0?0aARg}VPtGhP*&yZ{!C6(sq^PzF3k;#Avk3_bkCL7T2MCRjoQjT_h>Vwp
zij{MAe{pnuZ*qHWaCvKPcUD+m*xm0?Qd!p9?bO-p)7k3L*XYmH<~BGyy}{8mHafb$
z&bYnIARr?h9v~YW9vK=O7Z@4@1O%I(tO5c8@Y5f50008dNklZz)3AlitwxnPH%H`Uot)Wm#A~UJD{dq04A@c=B#o!5#fVO
zK9y#&^Z}?|*LsioK&(FDooZAR)tQj#4t{T51EP;48U9I=7br`Bk^O8MkeX!ZNe^m_BH=Vrj*jZZuSKj$t2j7FZ^2u4YKv8(~
zaMJ@oQZ>+`rSiG$iy;5QP}GpcaF4=ry2srsiZ0RqYjK@LQA)d@GB0_aay0Tvrp83dAL2x{13
zII2|!RUp(7P{l^2T*QHZ71!E*QcTqNtMh0|uLS!2k^8W?Ka#6$aBn3*=LUe(BpUo6
z=nkI1fP*0x3aYH^su6tQT}{)EfU4IyLBQ-;(w1i;#)>KNGk41l6I5_B2)f}c}CK-PL&a{M5Rut(jQ@VX1_p&z6vN)t
zNcp7CYSP6-mE96jwhE}y?lvr|2kRa0g^iLTaJ1y+9qs~F+W}}-FHd&oy;TeG2n6hM
z2eK~@vIu+Z%WDWZDK7wkG0}TEXW(~!Gk+0t(i)Sck-Ht%F=yzi{ZFXe-}F6T**!vM
y)$^3mN5{Eb6*vAD>S)nmP3aw7v29)6WaBSReaJM4z7xj)0000
-
-
-Installation
-
-
-
-
-
-
-
-
-
-LuaJIT is only distributed as source code — get it from the
-» git repository. This page explains how to build
-and install the LuaJIT binary and library for different operating systems.
-
-
-For the impatient (on POSIX systems):
-
-
-make && sudo make install
-
-
-
Requirements
-
-LuaJIT currently builds out-of-the box on most systems. Please check the
-supported operating systems and CPU architectures on the
-» status page.
-
-
-Building LuaJIT requires a recent toolchain based on GCC, Clang/LLVM or
-MSVC++.
-
-
-The Makefile-based build system requires GNU Make and supports
-cross-builds.
-
-
-Batch files are provided for MSVC++ builds and console cross-builds.
-
-
-
Configuring LuaJIT
-
-The standard configuration should work fine for most installations.
-Usually there is no need to tweak the settings. The following files
-hold all user-configurable settings:
-
-
-
Makefile has settings for installing LuaJIT (POSIX
-only).
-
src/Makefile has settings for compiling LuaJIT
-under POSIX, MinGW or Cygwin.
-
src/msvcbuild.bat has settings for compiling LuaJIT with
-MSVC (Visual Studio).
-
-
-Please read the instructions given in these files, before changing
-any settings.
-
-
-All LuaJIT 64 bit ports use 64 bit GC objects by default (LJ_GC64).
-For x64, you can select the old 32-on-64 bit mode by adding
-XCFLAGS=-DLUAJIT_DISABLE_GC64 to the make command.
-Please check the note about the
-bytecode format differences, too.
-
-
-
POSIX Systems (Linux, macOS, *BSD etc.)
-
Prerequisites
-
-Depending on your distribution, you may need to install a package for a
-compiler (GCC or Clang/LLVM), the development headers and/or a complete SDK.
-E.g. on a current Debian/Ubuntu, install build-essential with the
-package manager.
-
-
-
Building LuaJIT
-
-The supplied Makefiles try to auto-detect the settings needed for your
-operating system and your compiler. They need to be run with GNU Make,
-which is probably the default on your system, anyway. Simply run:
-
-
-make
-
-
-This always builds a native binary, depending on the host OS
-you're running this command on. Check the section on
-cross-compilation for more options.
-
-
-By default, modules are only searched under the prefix /usr/local.
-You can add an extra prefix to the search paths by appending the
-PREFIX option, e.g.:
-
-
-make PREFIX=/home/myself/lj2
-
-
-Note for macOS: you must set the MACOSX_DEPLOYMENT_TARGET
-environment variable to a value supported by your toolchain:
-
-
-MACOSX_DEPLOYMENT_TARGET=XX.YY make
-
-
Installing LuaJIT
-
-The top-level Makefile installs LuaJIT by default under
-/usr/local, i.e. the executable ends up in
-/usr/local/bin and so on. You need root privileges
-to write to this path. So, assuming sudo is installed on your system,
-run the following command and enter your sudo password:
-
-
-sudo make install
-
-
-Otherwise specify the directory prefix as an absolute path, e.g.:
-
-
-make install PREFIX=/home/myself/lj2
-
-
-Obviously the prefixes given during build and installation need to be the same.
-
-
-
Windows Systems
-
Prerequisites
-
-Either install one of the open source SDKs
-(» MinGW or
-» Cygwin), which come with a modified
-GCC plus the required development headers.
-Or install Microsoft's Visual Studio (MSVC).
-
-
Building with MSVC
-
-Open a "Visual Studio Command Prompt" (x86, x64 or ARM64), cd to the
-directory with the source code and run these commands:
-
-
-cd src
-msvcbuild
-
-
-Check the msvcbuild.bat file for more options.
-Then follow the installation instructions below.
-
-
-For an x64 to ARM64 cross-build run this first: vcvarsall.bat x64_arm64
-
-
Building with MinGW or Cygwin
-
-Open a command prompt window and make sure the MinGW or Cygwin programs
-are in your path. Then cd to the directory of the git repository.
-Then run this command for MinGW:
-
-
-mingw32-make
-
-
-Or this command for Cygwin:
-
-
-make
-
-
-Then follow the installation instructions below.
-
-
Installing LuaJIT
-
-Copy luajit.exe and lua51.dll (built in the src
-directory) to a newly created directory (any location is ok).
-Add lua and lua\jit directories below it and copy
-all Lua files from the src\jit directory of the distribution
-to the latter directory.
-
-
-There are no hardcoded
-absolute path names — all modules are loaded relative to the
-directory where luajit.exe is installed
-(see src/luaconf.h).
-
-
-
Cross-compiling LuaJIT
-
-First, let's clear up some terminology:
-
-
-
Host: This is your development system, usually based on a x64 or x86 CPU.
-
Target: This is the target system you want LuaJIT to run on, e.g. Android/ARM.
-
Toolchain: This comprises a C compiler, linker, assembler and a matching C library.
-
Host (or system) toolchain: This is the toolchain used to build native binaries for your host system.
-
Cross-compile toolchain: This is the toolchain used to build binaries for the target system. They can only be run on the target system.
-
-
-The GNU Makefile-based build system allows cross-compiling on any host
-for any supported target:
-
-
-
Yes, you need a toolchain for both your host and your target!
-
Both host and target architectures must have the same pointer size.
-
E.g. if you want to cross-compile to a 32 bit target on a 64 bit host, you need to install the multilib development package (e.g. libc6-dev-i386 on Debian/Ubuntu) and build a 32 bit host part (HOST_CC="gcc -m32").
-
On some distro versions, multilib conflicts with cross-compilers. The workaround is to install the x86 cross-compiler package gcc-i686-linux-gnu and use it to build the host part (HOST_CC=i686-linux-gnu-gcc).
-
64 bit targets always require compilation on a 64 bit host.
-
-
-You need to specify TARGET_SYS whenever the host OS and the
-target OS differ, or you'll get assembler or linker errors:
-
-
-
E.g. if you're compiling on a Windows or macOS host for embedded Linux or Android, you need to add TARGET_SYS=Linux to the examples below.
-
For a minimal target OS, you may need to disable the built-in allocator in src/Makefile and use TARGET_SYS=Other.
-
Don't forget to specify the same TARGET_SYS for the install step, too.
-
-
-Here are some examples where host and target have the same CPU:
-
-
-# Cross-compile to a 32 bit binary on a multilib x64 OS
-make CC="gcc -m32"
-
-# Cross-compile on Debian/Ubuntu for Windows (mingw32 package)
-make HOST_CC="gcc -m32" CROSS=i586-mingw32msvc- TARGET_SYS=Windows
-
-
-The CROSS prefix allows specifying a standard GNU cross-compile
-toolchain (Binutils, GCC and a matching libc). The prefix may vary
-depending on the --target the toolchain was built for (note the
-CROSS prefix has a trailing "-"). The examples below
-use the canonical toolchain triplets for Linux.
-
-
-Since there's often no easy way to detect CPU features at runtime, it's
-important to compile with the proper CPU or architecture settings:
-
-
-
The best way to get consistent results is to specify the correct settings when building the toolchain yourself.
-
For a pre-built, generic toolchain add -mcpu=... or -march=... and other necessary flags to TARGET_CFLAGS.
-
For ARM it's important to have the correct -mfloat-abi=... setting, too. Otherwise LuaJIT may not run at the full performance of your target CPU.
-
For MIPS it's important to select a supported ABI (o32 on MIPS32, n64 on MIPS64) and consistently compile your project either with hard-float or soft-float compiler settings.
-
-
-Here are some examples for targets with a different CPU than the host:
-
-
-# ARM soft-float
-make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabi- \
- TARGET_CFLAGS="-mfloat-abi=soft"
-
-# ARM soft-float ABI with VFP (example for Cortex-A9)
-make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabi- \
- TARGET_CFLAGS="-mcpu=cortex-a9 -mfloat-abi=softfp"
-
-# ARM hard-float ABI with VFP (armhf, most modern toolchains)
-make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabihf-
-
-# ARM64
-make CROSS=aarch64-linux-gnu-
-
-# PPC
-make HOST_CC="gcc -m32" CROSS=powerpc-linux-gnu-
-
-# MIPS32 big-endian
-make HOST_CC="gcc -m32" CROSS=mips-linux-gnu-
-# MIPS32 little-endian
-make HOST_CC="gcc -m32" CROSS=mipsel-linux-gnu-
-
-# MIPS64 big-endian
-make CROSS=mips-linux- TARGET_CFLAGS="-mips64r2 -mabi=64"
-# MIPS64 little-endian
-make CROSS=mipsel-linux- TARGET_CFLAGS="-mips64r2 -mabi=64"
-
-
-You can cross-compile for Android using the » Android NDK.
-Please adapt the environment variables to match the install locations and the
-desired target platform. E.g. Android 4.1 corresponds to ABI level 16.
-
-You can cross-compile for iOS 3.0+ (iPhone/iPad) using the » iOS SDK:
-
-
-Note: the JIT compiler is disabled for iOS, because regular iOS Apps
-are not allowed to generate code at runtime. You'll only get the performance
-of the LuaJIT interpreter on iOS. This is still faster than plain Lua, but
-much slower than the JIT compiler. Please complain to Apple, not me.
-Or use Android. :-p
-
-Building LuaJIT for consoles requires both a supported host compiler
-(x86 or x64) and a cross-compiler from the official console SDK.
-
-
-Due to restrictions on consoles, the JIT compiler is disabled and only
-the fast interpreter is built. This is still faster than plain Lua,
-but much slower than the JIT compiler. The FFI is disabled, too, since
-it's not very useful in such an environment.
-
-
-The following commands build a static library libluajit.a,
-which can be linked against your game, just like the Lua library.
-
-
-To cross-compile for PS3 from a Linux host (requires
-32 bit GCC, i.e. multilib Linux/x64) or a Windows host (requires
-32 bit MinGW), run this command:
-
-
-make HOST_CC="gcc -m32" CROSS=ppu-lv2-
-
-
-To cross-compile for the other consoles from a Windows host, open a
-"Native Tools Command Prompt for VS". You need to choose either the 32
-or the 64 bit version of the host compiler to match the target.
-Then cd to the src directory below the source code
-and run the build command given in the table:
-
-
-
-
Console
-
Bits
-
Build Command
-
-
-
PS4
-
64
-
ps4build
-
-
-
PS5
-
64
-
ps5build
-
-
-
PS Vita
-
32
-
psvitabuild
-
-
-
Xbox 360
-
32
-
xedkbuild
-
-
-
Xbox One
-
64
-
xb1build
-
-
-
Nintendo Switch NX32
-
32
-
nxbuild
-
-
-
Nintendo Switch NX64
-
64
-
nxbuild
-
-
-
-Please check out the comments in the corresponding *.bat
-file for more options.
-
-
-
Embedding LuaJIT
-
-LuaJIT is API-compatible with Lua 5.1. If you've already embedded Lua
-into your application, you probably don't need to do anything to switch
-to LuaJIT, except link with a different library:
-
-
-
It's strongly suggested to build LuaJIT separately using the supplied
-build system. Please do not attempt to integrate the individual
-source files into your build tree. You'll most likely get the internal build
-dependencies wrong or mess up the compiler flags. Treat LuaJIT like any
-other external library and link your application with either the dynamic
-or static library, depending on your needs.
-
If you want to load C modules compiled for plain Lua
-with require(), you need to make sure the public symbols
-(e.g. lua_pushnumber) are exported, too:
-
On POSIX systems you can either link to the shared library
-or link the static library into your application. In the latter case
-you'll need to export all public symbols from your main executable
-(e.g. -Wl,-E on Linux) and add the external dependencies
-(e.g. -lm -ldl on Linux).
-
Since Windows symbols are bound to a specific DLL name, you need to
-link to the lua51.dll created by the LuaJIT build (do not rename
-the DLL). You may link LuaJIT statically on Windows only if you don't
-intend to load Lua/C modules at runtime.
-
-
-
-
Additional hints for initializing LuaJIT using the C API functions:
-
-
Here's a
-» simple example
-for embedding Lua or LuaJIT into your application.
-
Make sure you use luaL_newstate. Avoid using
-lua_newstate, since this uses the (slower) default memory
-allocator from your system (no support for this on 64 bit architectures).
-
Make sure you use luaL_openlibs and not the old Lua 5.0 style
-of calling luaopen_base etc. directly.
-
To change or extend the list of standard libraries to load, copy
-src/lib_init.c to your project and modify it accordingly.
-Make sure the jit library is loaded, or the JIT compiler
-will not be activated.
-
The bit.* module for bitwise operations
-is already built-in. There's no need to statically link
-» Lua BitOp to your application.
-
-
-
Hints for Distribution Maintainers
-
-The LuaJIT build system has extra provisions for the needs of most
-POSIX-based distributions. If you're a package maintainer for
-a distribution, please make use of these features and
-avoid patching, subverting, autotoolizing or messing up the build system
-in unspeakable ways.
-
-
-There should be absolutely no need to patch luaconf.h or any
-of the Makefiles. And please do not hand-pick files for your packages —
-simply use whatever make install creates. There's a reason
-for all the files and directories it creates.
-
-
-The build system uses GNU make and auto-detects most settings based on
-the host you're building it on. This should work fine for native builds,
-even when sandboxed. You may need to pass some of the following flags to
-both the make and the make install command lines
-for a regular distribution build:
-
-
-
PREFIX overrides the installation path and should usually
-be set to /usr. Setting this also changes the module paths and
-the paths needed to locate the shared library.
-
DESTDIR is an absolute path which allows you to install
-to a shadow tree instead of the root tree of the build system.
-
MULTILIB sets the architecture-specific library path component
-for multilib systems. The default is lib.
-
Have a look at the top-level Makefile and src/Makefile
-for additional variables to tweak. The following variables may be
-overridden, but it's not recommended, except for special needs
-like cross-builds:
-BUILDMODE, CC, HOST_CC, STATIC_CC, DYNAMIC_CC, CFLAGS, HOST_CFLAGS,
-TARGET_CFLAGS, LDFLAGS, HOST_LDFLAGS, TARGET_LDFLAGS, TARGET_SHLDFLAGS,
-TARGET_FLAGS, LIBS, HOST_LIBS, TARGET_LIBS, CROSS, HOST_SYS, TARGET_SYS
-
-
-
-The build system has a special target for an amalgamated build, i.e.
-make amalg. This compiles the LuaJIT core as one huge C file
-and allows GCC to generate faster and shorter code. Alas, this requires
-lots of memory during the build. This may be a problem for some users,
-that's why it's not enabled by default. But it shouldn't be a problem for
-most build farms. It's recommended that binary distributions use this
-target for their LuaJIT builds.
-
-Finally, if you encounter any difficulties, please
-contact me first, instead of releasing a broken
-package onto unsuspecting users. Because they'll usually gonna complain
-to me (the upstream) and not you (the package maintainer), anyway.
-
-
-
diff --git a/doc/install.md b/doc/install.md
new file mode 100644
index 00000000..94dd11de
--- /dev/null
+++ b/doc/install.md
@@ -0,0 +1,303 @@
+# Installation
+
+LuaJIT is only distributed as source code — get it from the [» git repository](https://luajit.org/download.html). This page explains how to build and install the LuaJIT binary and library for different operating systems.
+
+For the impatient (on POSIX systems):
+
+```sh
+make && sudo make install
+```
+
+## Requirements
+
+LuaJIT currently builds out-of-the box on most systems. Please check the supported operating systems and CPU architectures on the [» status page](https://luajit.org/status.html).
+
+Building LuaJIT requires a recent toolchain based on GCC, Clang/LLVM or MSVC++.
+
+The Makefile-based build system requires GNU Make and supports cross-builds.
+
+Batch files are provided for MSVC++ builds and console cross-builds.
+
+## Configuring LuaJIT
+
+The standard configuration should work fine for most installations. Usually there is no need to tweak the settings. The following files hold all user-configurable settings:
+
+- `Makefile` has settings for **installing** LuaJIT (POSIX only).
+- `src/Makefile` has settings for **compiling** LuaJIT under POSIX, MinGW or Cygwin.
+- `src/msvcbuild.bat` has settings for compiling LuaJIT with MSVC (Visual Studio).
+
+Please read the instructions given in these files, before changing any settings.
+
+All LuaJIT 64 bit ports use 64 bit GC objects by default (`LJ_GC64`). For x64, you can select the old 32-on-64 bit mode by adding `XCFLAGS=-DLUAJIT_DISABLE_GC64` to the make command. Please check the note about the [bytecode format](./extensions#stringdumpf-mode-generates-portable-bytecode) differences, too.
+
+## POSIX Systems (Linux, macOS, *BSD etc.)
+
+### Prerequisites
+
+Depending on your distribution, you may need to install a package for a compiler (GCC or Clang/LLVM), the development headers and/or a complete SDK. E.g. on a current Debian/Ubuntu, install `build-essential` with the package manager.
+
+### Building LuaJIT
+
+The supplied Makefiles try to auto-detect the settings needed for your operating system and your compiler. They need to be run with GNU Make, which is probably the default on your system, anyway. Simply run:
+
+```sh
+make
+```
+
+This always builds a native binary, depending on the host OS you're running this command on. Check the section on [cross-compilation](#cross-compiling-luajit) for more options.
+
+By default, modules are only searched under the prefix `/usr/local`. You can add an extra prefix to the search paths by appending the `PREFIX` option, e.g.:
+
+```sh
+make PREFIX=/home/myself/lj2
+```
+
+Note for macOS: you **must** set the `MACOSX_DEPLOYMENT_TARGET` environment variable to a value supported by your toolchain:
+
+```sh
+MACOSX_DEPLOYMENT_TARGET=XX.YY make
+```
+
+### Installing LuaJIT
+
+The top-level Makefile installs LuaJIT by default under `/usr/local`, i.e. the executable ends up in `/usr/local/bin` and so on. You need root privileges to write to this path. So, assuming sudo is installed on your system, run the following command and enter your sudo password:
+
+```sh
+sudo make install
+```
+
+Otherwise specify the directory prefix as an absolute path, e.g.:
+
+```sh
+make install PREFIX=/home/myself/lj2
+```
+
+Obviously the prefixes given during build and installation need to be the same.
+
+## Windows Systems
+
+### Prerequisites
+
+Either install one of the open source SDKs ([» MinGW](http://mingw.org/) or [» Cygwin](https://www.cygwin.com/)), which come with a modified GCC plus the required development headers. Or install Microsoft's Visual Studio (MSVC).
+
+### Building with MSVC
+
+Open a "Visual Studio Command Prompt" (x86, x64 or ARM64), `cd` to the directory with the source code and run these commands:
+
+```sh
+cd src
+msvcbuild
+```
+
+Check the `msvcbuild.bat` file for more options. Then follow the installation instructions below.
+
+For an x64 to ARM64 cross-build run this first: `vcvarsall.bat x64_arm64`
+
+### Building with MinGW or Cygwin
+
+Open a command prompt window and make sure the MinGW or Cygwin programs are in your path. Then `cd` to the directory of the git repository. Then run this command for MinGW:
+
+```sh
+mingw32-make
+```
+
+Or this command for Cygwin:
+
+```sh
+make
+```
+
+Then follow the installation instructions below.
+
+### Installing LuaJIT
+
+Copy `luajit.exe` and `lua51.dll` (built in the `src` directory) to a newly created directory (any location is ok). Add `lua` and `lua\jit` directories below it and copy all Lua files from the `src\jit` directory of the distribution to the latter directory.
+
+There are no hardcoded absolute path names — all modules are loaded relative to the directory where `luajit.exe` is installed (see `src/luaconf.h`).
+
+## Cross-compiling LuaJIT
+
+First, let's clear up some terminology:
+
+- Host: This is your development system, usually based on a x64 or x86 CPU.
+- Target: This is the target system you want LuaJIT to run on, e.g. Android/ARM.
+- Toolchain: This comprises a C compiler, linker, assembler and a matching C library.
+- Host (or system) toolchain: This is the toolchain used to build native binaries for your host system.
+- Cross-compile toolchain: This is the toolchain used to build binaries for the target system. They can only be run on the target system.
+
+The GNU Makefile-based build system allows cross-compiling on any host for any supported target:
+
+- Yes, you need a toolchain for both your host *and* your target!
+- Both host and target architectures must have the same pointer size.
+- E.g. if you want to cross-compile to a 32 bit target on a 64 bit host, you need to install the multilib development package (e.g. `libc6-dev-i386` on Debian/Ubuntu) and build a 32 bit host part (`HOST_CC="gcc -m32"`).
+- On some distro versions, multilib conflicts with cross-compilers. The workaround is to install the x86 cross-compiler package `gcc-i686-linux-gnu` and use it to build the host part (`HOST_CC=i686-linux-gnu-gcc`).
+- 64 bit targets always require compilation on a 64 bit host.
+
+You need to specify `TARGET_SYS` whenever the host OS and the target OS
+differ, or you'll get assembler or linker errors:
+
+- E.g. if you're compiling on a Windows or macOS host for embedded Linux or Android, you need to add `TARGET_SYS=Linux` to the examples below.
+- For a minimal target OS, you may need to disable the built-in allocator in `src/Makefile` and use `TARGET_SYS=Other`.
+- Don't forget to specify the same `TARGET_SYS` for the install step, too.
+
+Here are some examples where host and target have the same CPU:
+
+```sh
+# Cross-compile to a 32 bit binary on a multilib x64 OS
+make CC="gcc -m32"
+
+# Cross-compile on Debian/Ubuntu for Windows (mingw32 package)
+make HOST_CC="gcc -m32" CROSS=i586-mingw32msvc- TARGET_SYS=Windows
+```
+
+The `CROSS` prefix allows specifying a standard GNU cross-compile toolchain (Binutils, GCC and a matching libc). The prefix may vary depending on the `--target` the toolchain was built for (note the `CROSS` prefix has a trailing `"-"`). The examples below use the canonical toolchain triplets for Linux.
+
+Since there's often no easy way to detect CPU features at runtime, it's important to compile with the proper CPU or architecture settings:
+
+- The best way to get consistent results is to specify the correct settings when building the toolchain yourself.
+- For a pre-built, generic toolchain add `-mcpu=...` or `-march=...` and other necessary flags to `TARGET_CFLAGS`.
+- For ARM it's important to have the correct `-mfloat-abi=...` setting, too. Otherwise LuaJIT may not run at the full performance of your target CPU.
+- For MIPS it's important to select a supported ABI (o32 on MIPS32, n64 on MIPS64) and consistently compile your project either with hard-float or soft-float compiler settings.
+
+Here are some examples for targets with a different CPU than the host:
+
+```sh
+# ARM soft-float
+make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabi- TARGET_CFLAGS="-mfloat-abi=soft"
+
+# ARM soft-float ABI with VFP (example for Cortex-A9)
+make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabi- TARGET_CFLAGS="-mcpu=cortex-a9 -mfloat-abi=softfp"
+
+# ARM hard-float ABI with VFP (armhf, most modern toolchains)
+make HOST_CC="gcc -m32" CROSS=arm-linux-gnueabihf-
+
+# ARM64
+make CROSS=aarch64-linux-gnu-
+
+# PPC
+make HOST_CC="gcc -m32" CROSS=powerpc-linux-gnu-
+
+# MIPS32 big-endian
+make HOST_CC="gcc -m32" CROSS=mips-linux-gnu-
+# MIPS32 little-endian
+make HOST_CC="gcc -m32" CROSS=mipsel-linux-gnu-
+
+# MIPS64 big-endian
+make CROSS=mips-linux- TARGET_CFLAGS="-mips64r2 -mabi=64"
+# MIPS64 little-endian
+make CROSS=mipsel-linux- TARGET_CFLAGS="-mips64r2 -mabi=64"
+```
+
+You can cross-compile for **Android** using the
+[» Android NDK](https://developer.android.com/ndk/). Please adapt the environment variables to match the install locations and the desired target platform. E.g. Android 4.1 corresponds to ABI level 16.
+
+```sh
+# Android/ARM64, aarch64, Android 5.0+ (L)
+NDKDIR=/opt/android/ndk
+NDKBIN=$NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin
+NDKCROSS=$NDKBIN/aarch64-linux-android-
+NDKCC=$NDKBIN/aarch64-linux-android21-clang
+make \
+ CROSS=$NDKCROSS \
+ STATIC_CC=$NDKCC \
+ DYNAMIC_CC="$NDKCC -fPIC" \
+ TARGET_LD=$NDKCC \
+ TARGET_AR="$NDKBIN/llvm-ar rcus" \
+ TARGET_STRIP=$NDKBIN/llvm-strip
+
+# Android/ARM, armeabi-v7a (ARMv7 VFP), Android 4.1+ (JB)
+NDKDIR=/opt/android/ndk
+NDKBIN=$NDKDIR/toolchains/llvm/prebuilt/linux-x86_64/bin
+NDKCROSS=$NDKBIN/arm-linux-androideabi-
+NDKCC=$NDKBIN/armv7a-linux-androideabi16-clang
+
+make \
+ HOST_CC="gcc -m32" \
+ CROSS=$NDKCROSS \
+ STATIC_CC=$NDKCC \
+ DYNAMIC_CC="$NDKCC -fPIC" \
+ TARGET_LD=$NDKCC \
+ TARGET_AR="$NDKBIN/llvm-ar rcus" \
+ TARGET_STRIP=$NDKBIN/llvm-strip
+```
+
+You can cross-compile for **iOS 3.0+** (iPhone/iPad) using the [» iOS SDK](https://developer.apple.com/ios/):
+
+Note: **the JIT compiler is disabled for iOS**, because regular iOS Apps are not allowed to generate code at runtime. You'll only get the performance of the LuaJIT interpreter on iOS. This is still faster than plain Lua, but much slower than the JIT compiler. Please complain to Apple, not me. Or use Android. :-p
+
+```sh
+# iOS/ARM64
+ISDKP=$(xcrun --sdk iphoneos --show-sdk-path)
+ICC=$(xcrun --sdk iphoneos --find clang)
+ISDKF="-arch arm64 -isysroot $ISDKP"
+make DEFAULT_CC=clang CROSS="$(dirname $ICC)/" TARGET_FLAGS="$ISDKF" TARGET_SYS=iOS
+```
+
+### Cross-compiling for consoles
+
+Building LuaJIT for consoles requires both a supported host compiler (x86 or x64) and a cross-compiler from the official console SDK.
+
+Due to restrictions on consoles, the JIT compiler is disabled and only the fast interpreter is built. This is still faster than plain Lua, but much slower than the JIT compiler. The FFI is disabled, too, since it's not very useful in such an environment.
+
+The following commands build a static library `libluajit.a`, which can be linked against your game, just like the Lua library.
+
+To cross-compile for **PS3** from a Linux host (requires 32 bit GCC, i.e. multilib Linux/x64) or a Windows host (requires 32 bit MinGW), run this command:
+
+```sh
+make HOST_CC="gcc -m32" CROSS=ppu-lv2-
+```
+
+To cross-compile for the other consoles from a Windows host, open a "Native Tools Command Prompt for VS". You need to choose either the 32 or the 64 bit version of the host compiler to match the target. Then `cd` to the `src` directory below the source code and run the build command given in the table:
+
+Console | Bits | Build Command
+-------------------------|------|---------------
+**PS4** | 64 | `ps4build`
+**PS5** | 64 | `ps5build`
+**PS Vita** | 32 | `psvitabuild`
+**Xbox 360** | 32 | `xedkbuild`
+**Xbox One** | 64 | `xb1build`
+**Nintendo Switch NX32** | 32 | `nxbuild`
+**Nintendo Switch NX64** | 64 | `nxbuild`
+
+Please check out the comments in the corresponding `*.bat` file for more options.
+
+## Embedding LuaJIT
+
+LuaJIT is API-compatible with Lua 5.1. If you've already embedded Lua into your application, you probably don't need to do anything to switch to LuaJIT, except link with a different library:
+
+- It's strongly suggested to build LuaJIT separately using the supplied build system. Please do *not* attempt to integrate the individual source files into your build tree. You'll most likely get the internal build dependencies wrong or mess up the compiler flags. Treat LuaJIT like any other external library and link your application with either the dynamic or static library, depending on your needs.
+- If you want to load C modules compiled for plain Lua with `require()`, you need to make sure the public symbols (e.g. `lua_pushnumber`) are exported, too:
+ - On POSIX systems you can either link to the shared library or link the static library into your application. In the latter case you'll need to export all public symbols from your main executable (e.g. `-Wl,-E` on Linux) and add the external dependencies (e.g. `-lm -ldl` on Linux).
+ - Since Windows symbols are bound to a specific DLL name, you need to link to the `lua51.dll` created by the LuaJIT build (do not rename the DLL). You may link LuaJIT statically on Windows only if you don't intend to load Lua/C modules at runtime.
+
+Additional hints for initializing LuaJIT using the C API functions:
+
+- Here's a [» simple example](http://lua-users.org/wiki/SimpleLuaApiExample) for embedding Lua or LuaJIT into your application.
+- Make sure you use `luaL_newstate`. Avoid using `lua_newstate`, since this uses the (slower) default memory allocator from your system (no support for this on 64 bit architectures).
+- Make sure you use `luaL_openlibs` and not the old Lua 5.0 style of calling `luaopen_base` etc. directly.
+- To change or extend the list of standard libraries to load, copy `src/lib_init.c` to your project and modify it accordingly. Make sure the `jit` library is loaded, or the JIT compiler will not be activated.
+- The `bit.*` module for bitwise operations is already built-in. There's no need to statically link [» Lua BitOp](https://bitop.luajit.org/) to your application.
+
+## Hints for Distribution Maintainers
+
+The LuaJIT build system has extra provisions for the needs of most POSIX-based distributions. If you're a package maintainer for a distribution, *please* make use of these features and avoid patching, subverting, autotoolizing or messing up the build system in unspeakable ways.
+
+There should be absolutely no need to patch `luaconf.h` or any of the Makefiles. And please do not hand-pick files for your packages — simply use whatever `make install` creates. There's a reason for all the files *and* directories it creates.
+
+The build system uses GNU make and auto-detects most settings based on the host you're building it on. This should work fine for native builds, even when sandboxed. You may need to pass some of the following flags to *both* the `make` and the `make install` command lines for a regular distribution build:
+
+- `PREFIX` overrides the installation path and should usually be set to `/usr`. Setting this also changes the module paths and the paths needed to locate the shared library.
+- `DESTDIR` is an absolute path which allows you to install to a shadow tree instead of the root tree of the build system.
+- `MULTILIB` sets the architecture-specific library path component for multilib systems. The default is `lib`.
+- Have a look at the top-level `Makefile` and `src/Makefile` for additional variables to tweak. The following variables *may* be overridden, but it's *not* recommended, except for special needs like cross-builds: `BUILDMODE, CC, HOST_CC, STATIC_CC, DYNAMIC_CC, CFLAGS, HOST_CFLAGS, TARGET_CFLAGS, LDFLAGS, HOST_LDFLAGS, TARGET_LDFLAGS, TARGET_SHLDFLAGS, TARGET_FLAGS, LIBS, HOST_LIBS, TARGET_LIBS, CROSS, HOST_SYS, TARGET_SYS `
+
+The build system has a special target for an amalgamated build, i.e. `make amalg`. This compiles the LuaJIT core as one huge C file and allows GCC to generate faster and shorter code. Alas, this requires lots of memory during the build. This may be a problem for some users, that's why it's not enabled by default. But it shouldn't be a problem for most build farms. It's recommended that binary distributions use this target for their LuaJIT builds.
+
+The tl;dr version of the above:
+
+```sh
+make amalg PREFIX=/usr && \
+make install PREFIX=/usr DESTDIR=/tmp/buildroot
+```
+
+Finally, if you encounter any difficulties, please [contact me](./contact) first, instead of releasing a broken package onto unsuspecting users. Because they'll usually gonna complain to me (the upstream) and not you (the package maintainer), anyway.
diff --git a/doc/luajit.html b/doc/luajit.html
deleted file mode 100644
index 73a1c17f..00000000
--- a/doc/luajit.html
+++ /dev/null
@@ -1,203 +0,0 @@
-
-
-
-LuaJIT
-
-
-
-
-
-
-
-
-
-
-LuaJIT is a Just-In-Time Compiler (JIT) for the
-» Lua programming language.
-Lua is a powerful, dynamic and light-weight programming language.
-It may be embedded or used as a general-purpose, stand-alone language.
-
-LuaJIT has been successfully used as a scripting middleware in
-games, appliances, network and graphics apps, numerical simulations,
-trading platforms and many other specialty applications.
-
-
-LuaJIT is part of a hundred million web sites, huge SaaS installations,
-network switches, set-top boxes and other embedded devices. You've probably
-already used LuaJIT without knowing about it.
-
-
-LuaJIT scales from embedded devices, smartphones, desktops up to server
-farms. It combines high flexibility with high performance and an unmatched
-low memory footprint.
-
-
-LuaJIT has been in continuous development since 2005. It's widely
-considered to be one of the fastest dynamic language
-implementations. It has outperformed other dynamic languages on many
-cross-language benchmarks since its first release — often by a
-substantial margin.
-
-
-For LuaJIT 2.0, the whole VM has been rewritten from the ground up
-and relentlessly optimized for performance. It combines a high-speed
-interpreter, written in assembler, with a state-of-the-art JIT
-compiler.
-
-
-An innovative trace compiler is integrated with advanced,
-SSA-based optimizations and highly tuned code generation backends.
-A substantial reduction of the overhead associated with dynamic languages
-allows it to break into the performance range traditionally reserved for
-offline, static language compilers.
-
-
-
More ...
-
-Please select a sub-topic in the navigation bar to learn more about LuaJIT.
-
-LuaJIT has only a single stand-alone executable, called luajit on
-POSIX systems or luajit.exe on Windows. It can be used to run simple
-Lua statements or whole Lua applications from the command line. It has an
-interactive mode, too.
-
-
-
Command Line Options
-
-The luajit stand-alone executable is just a slightly modified
-version of the regular lua stand-alone executable.
-It supports the same basic options, too. luajit -h
-prints a short list of the available options. Please have a look at the
-» Lua manual
-for details.
-
-
-LuaJIT has some additional options:
-
-
-
-b[options] input output
-
-This option saves or lists bytecode. The following additional options
-are accepted:
-
-
-
-l — Only list bytecode.
-
-s — Strip debug info (this is the default).
-
-g — Keep debug info.
-
-W — Generate 32 bit (non-GC64) bytecode.
-
-X — Generate 64 bit (GC64) bytecode.
-
-d — Generate bytecode in deterministic manner.
-
-n name — Set module name (default: auto-detect from input name)
-
-t type — Set output file type (default: auto-detect from output name).
-
-a arch — Override architecture for object files (default: native).
-
-o os — Override OS for object files (default: native).
-
-F name — Override filename (default: input filename).
-
-e chunk — Use chunk string as input.
-
- (a single minus sign) — Use stdin as input and/or stdout as output.
-
-
-The output file type is auto-detected from the extension of the output
-file name:
-
-
-
c — C source file, exported bytecode data.
-
cc — C++ source file, exported bytecode data.
-
h — C/C++ header file, static bytecode data.
-
obj or o — Object file, exported bytecode data
-(OS- and architecture-specific).
-
raw or any other extension — Raw bytecode file (portable).
-
-
-Notes:
-
-
-
See also string.dump()
-for information on bytecode portability and compatibility.
-
A file in raw bytecode format is auto-detected and can be loaded like
-any Lua source file. E.g. directly from the command line or with
-loadfile(), dofile() etc.
-
To statically embed the bytecode of a module in your application,
-generate an object file and just link it with your application.
-
On most ELF-based systems (e.g. Linux) you need to explicitly export the
-global symbols when linking your application, e.g. with: -Wl,-E
-
require() tries to load embedded bytecode data from exported
-symbols (in *.exe or lua51.dll on Windows) and from
-shared libraries in package.cpath.
-
-
-Typical usage examples:
-
-
-luajit -b test.lua test.out # Save bytecode to test.out
-luajit -bg test.lua test.out # Keep debug info
-luajit -be "print('hello world')" test.out # Save cmdline script
-
-luajit -bl test.lua # List to stdout
-luajit -bl test.lua test.txt # List to test.txt
-luajit -ble "print('hello world')" # List cmdline script
-
-luajit -b test.lua test.obj # Generate object file
-# Link test.obj with your application and load it with require("test")
-
-
-
-j cmd[=arg[,arg...]]
-
-This option performs a LuaJIT control command or activates one of the
-loadable extension modules. The command is first looked up in the
-jit.* library. If no matching function is found, a module
-named jit.<cmd> is loaded and the start()
-function of the module is called with the specified arguments (if
-any). The space between -j and cmd is optional.
-
-
-Here are the available LuaJIT control commands:
-
-
-
-jon — Turns the JIT compiler on (default).
-
-joff — Turns the JIT compiler off (only use the interpreter).
-
-jflush — Flushes the whole cache of compiled code.
-
-jv — Shows verbose information about the progress of the JIT compiler.
-
-jdump — Dumps the code and structures used in various compiler stages.
-The -jv and -jdump commands are extension modules
-written in Lua. They are mainly used for debugging the JIT compiler
-itself. For a description of their options and output format, please
-read the comment block at the start of their source.
-They can be found in the lib directory of the source
-distribution or installed under the jit directory. By default,
-this is /usr/local/share/luajit-XX.YY.ZZ>/jit on POSIX
-systems (replace XX.YY.ZZ by the installed version).
-
-
-
-O[level]
--O[+]flag-O-flag
--Oparam=value
-
-This options allows fine-tuned control of the optimizations used by
-the JIT compiler. This is mainly intended for debugging LuaJIT itself.
-Please note that the JIT compiler is extremely fast (we are talking
-about the microsecond to millisecond range). Disabling optimizations
-doesn't have any visible impact on its overhead, but usually generates
-code that runs slower.
-
-
-The first form sets an optimization level — this enables a
-specific mix of optimization flags. -O0 turns off all
-optimizations and higher numbers enable more optimizations. Omitting
-the level (i.e. just -O) sets the default optimization level,
-which is -O3 in the current version.
-
-
-The second form adds or removes individual optimization flags.
-The third form sets a parameter for the VM or the JIT compiler
-to a specific value.
-
-
-You can either use this option multiple times (like -Ocse
--O-dce -Ohotloop=10) or separate several settings with a comma
-(like -O+cse,-dce,hotloop=10). The settings are applied from
-left to right, and later settings override earlier ones. You can freely
-mix the three forms, but note that setting an optimization level
-overrides all earlier flags.
-
-
-Note that -Ofma is not enabled by default at any level,
-because it affects floating-point result accuracy. Only enable this,
-if you fully understand the trade-offs of FMA for performance (higher),
-determinism (lower) and numerical accuracy (higher).
-
-
-Here are the available flags and at what optimization levels they
-are enabled:
-
-
-
-
Flag
-
-O1
-
-O2
-
-O3
-
-
-
-
fold
•
•
•
Constant Folding, Simplifications and Reassociation
-
-
cse
•
•
•
Common-Subexpression Elimination
-
-
dce
•
•
•
Dead-Code Elimination
-
-
narrow
•
•
Narrowing of numbers to integers
-
-
loop
•
•
Loop Optimizations (code hoisting)
-
-
fwd
•
Load Forwarding (L2L) and Store Forwarding (S2L)
-
-
dse
•
Dead-Store Elimination
-
-
abc
•
Array Bounds Check Elimination
-
-
sink
•
Allocation/Store Sinking
-
-
fuse
•
Fusion of operands into instructions
-
-
fma
Fused multiply-add
-
-
-Here are the parameters and their default settings:
-
-
-
-
Parameter
-
Default
-
-
-
-
maxtrace
1000
Max. number of traces in the cache
-
-
maxrecord
4000
Max. number of recorded IR instructions
-
-
maxirconst
500
Max. number of IR constants of a trace
-
-
maxside
100
Max. number of side traces of a root trace
-
-
maxsnap
500
Max. number of snapshots for a trace
-
-
hotloop
56
Number of iterations to detect a hot loop or hot call
-
-
hotexit
10
Number of taken exits to start a side trace
-
-
tryside
4
Number of attempts to compile a side trace
-
-
instunroll
4
Max. unroll factor for instable loops
-
-
loopunroll
15
Max. unroll factor for loop ops in side traces
-
-
callunroll
3
Max. unroll factor for pseudo-recursive calls
-
-
recunroll
2
Min. unroll factor for true recursion
-
-
sizemcode
32
Size of each machine code area in KBytes (Windows: 64K)
-
-
maxmcode
512
Max. total size of all machine code areas in KBytes
-
-
diff --git a/doc/running.md b/doc/running.md
new file mode 100644
index 00000000..43a0026c
--- /dev/null
+++ b/doc/running.md
@@ -0,0 +1,134 @@
+# Running LuaJIT
+
+LuaJIT has only a single stand-alone executable, called `luajit` on POSIX systems or `luajit.exe` on Windows. It can be used to run simple Lua statements or whole Lua applications from the command line. It has an interactive mode, too.
+
+## Command Line Options
+
+The `luajit` stand-alone executable is just a slightly modified version of the regular `lua` stand-alone executable. It supports the same basic options, too. `luajit -h` prints a short list of the available options. Please have a look at the [» Lua manual](https://www.lua.org/manual/5.1/manual.html#6) for details.
+
+LuaJIT has some additional options:
+
+### `-b[options] input output`
+
+This option saves or lists bytecode. The following additional options
+are accepted:
+
+- `-l` — Only list bytecode.
+- `-s` — Strip debug info (this is the default).
+- `-g` — Keep debug info.
+- `-W` — Generate 32 bit (non-GC64) bytecode.
+- `-X` — Generate 64 bit (GC64) bytecode.
+- `-d` — Generate bytecode in deterministic manner.
+- `-n name` — Set module name (default: auto-detect from input name)
+- `-t type` — Set output file type (default: auto-detect from output name).
+- `-a arch` — Override architecture for object files (default: native).
+- `-o os` — Override OS for object files (default: native).
+- `-F name` — Override filename (default: input filename).
+- `-e chunk` — Use chunk string as input.
+- `-` (a single minus sign) — Use stdin as input and/or stdout as output.
+
+The output file type is auto-detected from the extension of the output
+file name:
+
+- `c` — C source file, exported bytecode data.
+- `cc` — C++ source file, exported bytecode data.
+- `h` — C/C++ header file, static bytecode data.
+- `obj` or `o` — Object file, exported bytecode data (OS and architecture-specific).
+- `raw` or any other extension — Raw bytecode file (portable).
+
+Notes:
+
+- See also [string.dump()](./extensions#stringdumpf-mode-generates-portable-bytecode) for information on bytecode portability and compatibility.
+- A file in raw bytecode format is auto-detected and can be loaded like any Lua source file. E.g. directly from the command line or with `loadfile()`, `dofile()` etc.
+- To statically embed the bytecode of a module in your application, generate an object file and just link it with your application.
+- On most ELF-based systems (e.g. Linux) you need to explicitly export the global symbols when linking your application, e.g. with: `-Wl,-E`
+- `require()` tries to load embedded bytecode data from exported symbols (in `*.exe` or `lua51.dll` on Windows) and from shared libraries in `package.cpath`.
+
+Typical usage examples:
+
+```sh
+# Save bytecode to test.out
+luajit -b test.lua test.out
+# Keep debug info
+luajit -bg test.lua test.out
+# Save cmdline script
+luajit -be "print('hello world')" test.out
+
+# List to stdout
+luajit -bl test.lua
+# List to test.txt
+luajit -bl test.lua test.txt
+# List cmdline script
+luajit -ble "print('hello world')"
+
+# Generate object file
+luajit -b test.lua test.obj
+# Link test.obj with your application and load it with require("test")
+```
+
+### `-j cmd[=arg[,arg...]]`
+
+This option performs a LuaJIT control command or activates one of the
+loadable extension modules. The command is first looked up in the
+`jit.*` library. If no matching function is found, a module named
+`jit.` is loaded and the `start()` function of the module is called
+with the specified arguments (if any). The space between `-j` and `cmd`
+is optional.
+
+Here are the available LuaJIT control commands:
+
+- `-jon`: Turns the JIT compiler on (default).
+- `-joff`: Turns the JIT compiler off (only use the interpreter).
+- `-jflush`: Flushes the whole cache of compiled code.
+- `-jv`: Shows verbose information about the progress of the JIT compiler.
+- `-jdump`: Dumps the code and structures used in various compiler stages.
+- `-jp`: Start the [integrated profiler](./ext_profiler).
+
+The `-jv` and `-jdump` commands are extension modules written in Lua. They are mainly used for debugging the JIT compiler itself. For a description of their options and output format, please read the comment block at the start of their source. They can be found in the `lib` directory of the source distribution or installed under the `jit` directory. By default, this is `/usr/local/share/luajit-XX.YY.ZZ>/jit` on POSIX systems (replace XX.YY.ZZ by the installed version).
+
+### `-O[level]` `-O[+]flag` `-O-flag` `-Oparam=value`
+
+This options allows fine-tuned control of the optimizations used by the JIT compiler. This is mainly intended for debugging LuaJIT itself. Please note that the JIT compiler is extremely fast (we are talking about the microsecond to millisecond range). Disabling optimizations doesn't have any visible impact on its overhead, but usually generates code that runs slower.
+
+The first form sets an optimization level — this enables a specific mix of optimization flags. `-O0` turns off all optimizations and higher numbers enable more optimizations. Omitting the level (i.e. just `-O`) sets the default optimization level, which is `-O3` in the current version.
+
+The second form adds or removes individual optimization flags. The third form sets a parameter for the VM or the JIT compiler to a specific value.
+
+You can either use this option multiple times (like `-Ocse -O-dce -Ohotloop=10`) or separate several settings with a comma (like `-O+cse,-dce,hotloop=10`). The settings are applied from left toright, and later settings override earlier ones. You can freely mix the three forms, but note that setting an optimization level overrides all earlier flags.
+
+Note that `-Ofma` is not enabled by default at any level, because it affects floating-point result accuracy. Only enable this, if you fully understand the trade-offs of FMA for performance (higher), determinism (lower) and numerical accuracy (higher).
+
+Here are the available flags and at what optimization levels they are enabled:
+
+Flag | -O1 | -O2 | -O3 | Description
+-------|-----|-----|-----|-------------
+fold | • | • | • | Constant Folding, Simplifications and Reassociation
+cse | • | • | • | Common-Subexpression Elimination
+dce | • | • | • | Dead-Code Elimination
+narrow | | • | • | Narrowing of numbers to integers
+loop | | • | • | Loop Optimizations (code hoisting)
+fwd | | | • | Load Forwarding (L2L) and Store Forwarding (S2L)
+dse | | | • | Dead-Store Elimination
+abc | | | • | Array Bounds Check Elimination
+sink | | | • | Allocation/Store Sinking
+fuse | | | • | Fusion of operands into instructions
+fma | | | | Fused multiply-add
+
+Here are the parameters and their default settings:
+
+Parameter | Default | Description
+-----------|---------|-------------
+maxtrace | 1000 | Max. number of traces in the cache
+maxrecord | 4000 | Max. number of recorded IR instructions
+maxirconst | 500 | Max. number of IR constants of a trace
+maxside | 100 | Max. number of side traces of a root trace
+maxsnap | 500 | Max. number of snapshots for a trace
+hotloop | 56 | Number of iterations to detect a hot loop or hot call
+hotexit | 10 | Number of taken exits to start a side trace
+tryside | 4 | Number of attempts to compile a side trace
+instunroll | 4 | Max. unroll factor for instable loops
+loopunroll | 15 | Max. unroll factor for loop ops in side traces
+callunroll | 3 | Max. unroll factor for pseudo-recursive calls
+recunroll | 2 | Min. unroll factor for true recursion
+sizemcode | 32 | Size of each machine code area in KBytes (Windows: 64K)
+maxmcode | 512 | Max. total size of all machine code areas in KBytes
-
-