Lua Tips and Tricks

Jump to ...

Metatables and Metamethods

Other special keys in a metatable

__index
__newindex
__call
__tostring
__pairs and __ipairs (Lua 5.2 and above)

setfenv and getfenv (LuaJIT and Lua 5.1)
LuaJIT

table.new
xpcall
string.dump
ffi.*, string.buf and the Lua/C API

For loops
Integer and float types for numbers (Lua 5.3 and above)
Length of a string (Lua 5.3 and above)
That’s all for now, folks!

Metatables and Metamethods

In Lua, you can use metatables and metamethods to extend the default behaviors of Lua tables. Let me show an example code usage first:

-- This table holds our metamethods.
-- This example essentially performs "operator overloading," as known in C++ for the + sign.
-- Additionally, since this function creates a new table without modifying either self or other, this function is pure.
local mt = {}
function mt:__add(other)
    return { a = self.a + other.a, b = self.b .. " " .. other.b }
end

-- setmetatable takes two arguments: the initial table and the metamethods table
local table_1 = setmetatable({ a = 7, b = "Hi!" }, mt)
-- table_1 = {
--   a = 7,
--   b = "Hi!",
--   <metatable> = {
--     __add = <function 1>
--   }
-- }

print(table_1.a, table_1.b)
-- prints: 7 Hi!

-- getmetatable returns the metatable for table_1
-- printing will only show an address, but you can always loop through it and print both the key and value
print(getmetatable(table_1))

local table_2 = setmetatable({ a = 5, b = "Goodbye!" }, mt)
-- table_2 = {
--   a = 5,
--   b = "Goodbye!",
--   <metatable> = {
--     __add = <function 1>
--   }
-- }

local result_table = table_1 + table_2
print(result_table.a, result_table.b) -- print result_table
-- prints: 12 Hi! Goodbye!
-- This doesn't have a metatable though; we didn't set it while returning a new table
-- getmetatable(result_table) will show nil

This is quite powerful, isn’t it? This is just the tip of the iceberg. In addition to your user-defined methods for a table, you can now use metamethods like __sub (for subtraction), __mul (for multiplication), __div (for division), __mod (for modulus), etc. Explore all the mathematical, bitwise, and comparison operators at lua-users wiki. Bitwise operators and __idiv are new additions to Lua 5.3. The __unm is the only unary operator, so it doesn’t take any other parameters. Remember—using Lua metamethods is akin to C++ operator overloading, allowing you to extend your table, change data state, and more. Just avoid anything unintuitive, like subtracting values in __add, for your sanity and others’. Also, note that there are no __gt, __gte, or __neq operators, as they are simply inverses/negations of __le, __lt, and __eq, respectively, and Lua handles them automatically.

Other special keys in a metatable

__index

Control ‘prototype’ inheritance. In fact, if we assign a table to the __index key in a metatable, it will work quite similarly to JavaScript’s prototype inheritance, where __index is equivalent to JavaScript’s [[Prototype]]. For example, behold ”The Inception”:

-- deepest level
local mt_1 = { __index = { d = 4 } }
-- deeper level
local mt_2 = { __index = { c = 3 } }
mt_2.__index = setmetatable(mt_2.__index, mt_1)
-- deep level
local mt_3 = { __index = { b = 2 } }
mt_3.__index = setmetatable(mt_3.__index, mt_2)
-- actual level
local actual_table = setmetatable({ a = 1 }, mt_3)
-- The actual table now looks like this:
-- actual_table = {
--   a = 1,
--   <metatable> = {
--     __index = {
--       b = 2,
--       <metatable> = {
--         __index = {
--           c = 3,
--           <metatable> = {
--             __index = {
--               d = 4
--             }
--           }
--         }
--       }
--     }
--   }
-- }

print(actual_table.a) -- prints: 1
print(actual_table.b) -- prints: 2
print(actual_table.c) -- prints: 3
print(actual_table.d) -- prints: 4
print(actual_table.p) -- prints: nil, key is not present

So yeah, each __index of a metatable is similar to JS’s prototype. Thus, whatever you can do with JS prototype, you can do with Lua’s __index. Also, __index can be used as a function. In the following example, I track only what I need. How? Well, since it looks into __index only when it doesn’t find a property in the first ”layer”, so whenever it does go into the __index, only then do I track those properties that need to be.

local needs_tracking = { track_1 = "Hi", track_2 = "Hello" }
local tracker = {}

local dont_track = setmetatable({ a = 9, b = 11 }, {
	__index = function(_, key)
		tracker[key] = (tracker[key] or 0) + 1
		return needs_tracking[key]
	end,
})

print(dont_track.a) -- prints: 9
print(tracker.a) -- prints: nil
print(dont_track.track_1) -- prints: Hi
print(tracker.track_1) -- prints: 1

We can also implement a full-fledged prototype class in Lua similar to what JS does.

local F1 = {}

function F1:new(car_name)
	local state = { car_name = car_name }
	self.__index = self
	return setmetatable(state, self)
end

function F1:introduce()
	print("This is", self.car_name)
end

local RB = F1:new("Red Bull")

function RB:race()
	print("We are FAAAAST")
end

local Ferrari = F1:new("Ferrari")

function Ferrari:radio()
	print("We are checking,", self.car_name)
end

function Ferrari:race()
	print("Today is race day, we don't perform today")
end

Ferrari:introduce() -- This is Ferrari
Ferrari:race() -- Today is race day, we don't perform today

-- since we now call Ferrari with an argument, the topmost layer has Leclerc
-- deeper layer still has Ferrari, but that won't be seen
local leclerc = Ferrari:new("Leclerc")
leclerc:introduce() -- This is Leclerc
leclerc:radio() -- We are checking, Leclerc
leclerc:race() -- Today is race day, we don't perform today

-- now I call without arguments
local red_bull = RB:new()
red_bull:introduce() -- This is Red Bull
red_bull:race() -- We are FAAAAST

If you want to bypass the __index function call, use rawget(self, key).

__newindex

The __newindex controls property assignment. If this points to a function, it gets called only when a new property is assigned. Let’s use this to make a logger for any new assignment:

local mt = {}

function mt:__newindex(key, val)
    print(key, val)
    -- You cannot do self[key] = val now; that will be recursive!
    -- So, use rawset to bypass __newindex
    rawset(self, key, val)
end

local logger = setmetatable({ x = 10 }, mt)

logger.hello = "world"
logger.foo = "bar"
logger.x = 15
logger.y = 10

-- prints:
-- hello world
-- foo bar
-- y 10
-- x 15 is not printed as x is already set in the table passed to setmetatable

__call

This makes a table act like a function. You can now call the table like you would call a function.

local mt = {}

function mt:__call(arg)
    return self.ten + arg
end

local add_10 = setmetatable({ ten = 10 }, mt)

print(add_10(7)) -- prints 17
print(add_10.ten) -- prints 10

__tostring

You can use this function to stringify a table, however you want. You can also use __tostring for debugging. Think of it somewhat like overloading the << operator in C++. For example:

local mt = {}

-- I have used varargs to show how to use them in Lua
-- but it isn't really necessary for this example
local function better_print(indent, stringify, ...)
    for i = 1, select("#", ...) do
        local o = select(i, ...)
        if type(o) == "string" then
            stringify[#stringify + 1] = string.format("%q,", o)
        elseif type(o) ~= "table" then
            stringify[#stringify + 1] = tostring(o) .. ","
        else
            stringify[#stringify + 1] = "{\n"
            for k, v in pairs(o) do
                stringify[#stringify + 1] = string.format("%s  %s = ", indent, k)
                better_print(indent .. "  ", stringify, v)
            end
            stringify[#stringify + 1] = indent .. "},"
        end
        stringify[#stringify + 1] = "\n"
    end
end

function mt:__tostring()
    local stringify = {}
    better_print("", stringify, self)
    return table.concat(stringify)
end

local my_table = setmetatable(
    { "apple", 1, 2.1, lake = "yellow", { a = 9, b = 10, c = "mango", d = { x = 9, y = 10, z = { p = "grapes", m = { n = 4.9 } } } } },
    mt
)

print(my_table) -- prints the whole table, but not necessarily in order

This is a good and dirty method to print the whole table, with all the keys and values. It doesn’t support cyclic tables, but you can add that easily. Not necessarily a great example, but you get a brief idea of how you can use __tostring.

pairs and ipairs (Lua 5.2 and above)

You can implement these functions so that you have a custom way to iterate over the table. In the following simple example, I provide a custom iterator for the table with pairs, which will print any JS, C++, and Lua filenames with their corresponding Nerd Font icons. For my example, you need Lua 5.3 and above, as it uses UTF-8 strings.

local mt = {}

local nerd_fonts = {
    js = "\u{e74e} ",
    lua = "\u{e620} ",
    cpp = "\u{e646} ",
}

function mt:__pairs()
    local function stateless_iter(tbl, k)
        local v
        k, v = next(tbl, k)

        if type(v) == "string" then
            v = (nerd_fonts[v:match("^.+%.(.+)$")] or "") .. v
        end

        if nil ~= v then
            return k, v
        end
    end

    -- return an iterator function, the table, starting point
    return stateless_iter, self, nil
end

local my_table = setmetatable({ "file.js", "file_1.lua", "file_2.cpp" }, mt)

for k, v in pairs(my_table) do
    print(string.format("%s: %s", k, v))
end

-- prints:
-- 1:  file.js
-- 2:  file_1.lua
-- 3:  file_2.cpp

Another quick and dirty example, but you get the idea. You can also declare a custom __ipairs metamethod similarly.

setfenv and getfenv (LuaJIT and Lua 5.1)

You can set the environment of a function explicitly. For example:

-- Suppose this function returns colors for an HTML page
function colors()
    return {
        root = { fg = C.fg, bg = C.bg },
        div = { fg = C.red, bg = C.black },
        p = { fg = C.green, bg = C.white },
    }
end

-- The following will error out
-- print(colors())
-- Now, set the environment for the colors function explicitly
setfenv(colors, {
    C = {
        fg = "#ffeeee",
        bg = "#555555",
        red = "#ff0000",
        green = "#00ff00",
        black = "#000000",
        white = "#ffffff",
    },
})
print(colors())
-- Prints a table that contains
-- {
--   div = {
--     bg = "#000000",
--     fg = "#ff0000"
--   },
--   p = {
--     bg = "#ffffff",
--     fg = "#00ff00"
--   },
--   root = {
--     bg = "#555555",
--     fg = "#ffeeee"
--   }
-- }
-- You can also get the environment of a function
print(getfenv(colors))
-- Prints a table that contains
-- {
--   C = {
--     bg = "#555555",
--     black = "#000000",
--     fg = "#ffeeee",
--     green = "#00ff00",
--     red = "#ff0000",
--     white = "#ffffff"
--   }
-- }

You might think: What help will it do? Can we just not send the C table to the colors function? This seems like a bad design choice, doesn’t it? Yeah, if it is your own function, then yes, this is a bad design choice. But sometimes, we work with libraries that have internal functions somewhat similar to the colors function, and you, for some reason, might want to use that. So if you can just obtain the C table from the library, then you can import the module, set the environment of one of its functions like this, and voilà, now you can use the colors function easily!

LuaJIT

LuaJIT is a Lua 5.1 interpreter that runs on the same platform as Lua but is much faster. It also has some interesting modifications and enhancements to Lua 5.1 and borrows some more from Lua 5.2 and 5.3.

table.new

table.new is an enhancement to Lua 5.1. Basically, using this, you can pre-allocate memory. This might be necessary when the size of your table is known or when table resizing is too expensive. Think of it like reserve of std::vector of C++, but a bit different. It takes two arguments, narray and nhash. For keys that have to be hashed, like for example, a string, we should count the nhash. For keys which are not hashed, like for example, consecutive numbers, we should count the narray. Why consecutive? If, for example, you put a[1], a[2], ..., a[100], and then suddenly put a[777], that 777 index would be hashed. If any of the values in a consecutive range are nil, the next key with a non-nil value will be hashed as well, though this behavior differs by implementation. For example,

local tnew = require("table.new")

local LIM = 1e8
-- I need to store consecutive numbers, thus narray should be the one set to LIM.
-- nhash should be ideally 0, but I set it to 2 as the first key inserted, 2, should be hashed
local all_nums = tnew(LIM, 2)

for i = 2, LIM do
    if all_nums[i] == nil then
        all_nums[i] = true
        for j = i * i, LIM, i do
            all_nums[j] = false
        end
    end
end

for k, v in pairs(all_nums) do
    if v then
        print(k)
    end
end

If you had to write the same Sieve of Eratosthenes code with just local all_nums = {}, it will work too, and would take some more time. But using just Lua without LuaJIT? It will take a significant amount of time.

xpcall

I mentioned this as this differs in LuaJIT and Lua 5.1. For LuaJIT, a simple example would suffice to say what it does

local function f(a, b)
    return a + b
end

local function err(x)
    print("error:", x)
    return "check your args..."
end

local ok, res = xpcall(f, err, 1, 2)
print(ok, res) -- prints: true 3

ok, res = xpcall(f, err, "a", 2)
-- error: [string ":source (no file)"]:2: attempt to perform arithmetic on local 'a' (a string value)
print(ok, res) -- prints: false check your args...

string.dump

This one is a little hard for me to explain. I cannot have examples like the others. So I am just gonna explain what it does different to the regular string.dump. Well, the regular string.dump takes a function as the first parameter, and another boolean, which mentions if the function should be stripped or not before dumping. However, the output of string.dump is not deterministic. This might not really be a problem for you, but if you ever want to hash the output to check with some previously stored hash to check whether the function has changed, that non-deterministic output might be a problem. So, LuaJIT offers you a variation. Supply a function to string.dump, as usual, but for mode, instead of just true/false, you can now use string modes. So, now you can use it like this:

-- s for strip, d for deterministic
string.dump(function() print("Hello World") end, "ds")

This might be useful for reproducible builds too.

ffi.*, string.buf and the Lua/C API

These will be covered in my next blog. Stay tuned! One little taste of it — I implemented the ‘Sieve of Eratosthenes’ using ffi.new. Just take a look at its performance, and you would understand why LuaJIT FFI is so useful. Performance of LuaJIT ffi.new

The code:

-- remember: now all_nums is 0 based!
local ffi = require("ffi")
local LIM = 1e8

-- yes, I know ffi.new doesn't take a const char*. It doesn't matter, tbh though
local all_nums = ffi.new(string.format("int[%d]", LIM + 1), { 1 })
all_nums[0] = 0
all_nums[1] = 0

for i = 2, LIM do
    if i * i > LIM then
        break
    end
    if all_nums[i] == 1 then
        for j = i * i, LIM, i do
            all_nums[j] = 0
        end
    end
end

for i = 2, LIM do
    if all_nums[i] == 1 then
        print(i)
    end
end

There are some other features of LuaJIT that I am not going to cover here, as I didn’t find them of much use for my own work. But these extensions are nice, and if you want to learn more, look at the extensions in LuaJIT documentation. Also look at the bit.* API too, at https://bitop.luajit.org/api.html, and the jit.* API at https://luajit.org/ext_jit.html If there’s something you don’t understand, feel free to ask in the comments below.

For loops

You might be asking yourself, ”You said you are not going to repeat the same silly things like ‘for-loops’ and ‘if-else,’ so why this section?” The truth is, the ‘for’ loops in Lua don’t operate the way you might think they do, specifically the ‘pairs’ and ‘ipairs’ ones. You may have had a little hint of that in the __pairs section when you saw that I was returning three values from the custom ’__pairs’ metamethod.

In Lua, the ‘for’ loop internally keeps three values instead of just the iterator: the iterator function, an invariant state, and a control variable. So, in reality, ‘k, v’ are the values returned by ‘iterator(state, control_var).’ Similarly, you can return an iterator function that returns more than two values when called. However, if the ‘control_var’ is ever nil, the loop terminates.

For example, take a look at this, which returns the values in the insertion order in the table.

Integer and float types for numbers (Lua 5.3 and above)

Type conversion for Lua, especially for numbers, is a bit unintuitive. Also, up until Lua 5.3, Lua just had only one number type, as everything was a floating-point type. That means the safe integers were between [-2⁵³, 2⁵³], and any integer beyond these limits couldn’t be represented exactly, not unlike JavaScript. However, since Lua 5.3, numbers can be both of integer type and floating-point type, and any integer between [-2⁶³, 2⁶³ - 1] can be represented safely.

-- Use math.type to distinguish between integer and float
print(type(3), type(3.0)) -- prints number number
print(math.type(3), math.type(3.0)) -- however, this prints integer float

-- Type conversion
-- Float to integer: OR with 0
local two_pow_53 = 2 ^ 53
print(math.type(two_pow_53)) -- prints float
local two_pow_53_int = two_pow_53 | 0
print(math.type(two_pow_53_int)) -- print integer
-- However, if the number has a fractional part or is out of range, it returns an error
-- So this will error out: print((2 ^ 63) | 0)
-- And so will this: print(5.3 | 0)

-- Integer to float: just add 0.0
print(math.type(two_pow_53_int + 0.0))

-- However, beware of the safe range
for i = -2, 2, 1 do
    print(string.format("%15.0f", two_pow_53 + i))
end
-- The above print statement prints: 9007199254740990 9007199254740991 9007199254740992 9007199254740992 9007199254740994
-- So, after 2 ^ 53, the integers cannot be represented exactly with floating-point numbers
-- But they sure can be represented with integer type, up to 2^63 - 1
print(two_pow_53_int + 0.0 == two_pow_53_int) -- prints true
local two_pow_53_int_plus_1 = two_pow_53_int + 1
print(two_pow_53_int_plus_1, two_pow_53_int_plus_1 + 0.0 == two_pow_53_int_plus_1) -- prints 9007199254740993 false

Length of a string (Lua 5.3 and above)

While there is string.len in Lua, it counts the number of bytes. So does the # operator. If you want the number of UTF-8 characters in a string, use utf8.len.

print(string.len("Naïve")) -- prints 6
print(utf8.len("Naïve")) -- correctly prints 5

print(string.len("👋")) -- prints 4
print(utf8.len("👋")) -- correctly prints 1

print(string.len("🎅🏼")) -- prints 8. Why? Extra 4 bytes for skin tone.
print(utf8.len("🎅🏼")) -- prints 2. Again, an extra byte for skin tone.

It turns out that for UTF-16 and UTF-32, you have to use a different library; utf8 will not suffice. However, it’s still an improvement.

That’s all for now, folks!

Let me know if you enjoyed this post or if you have any suggestions in the comment below. Stay tuned for the next part, where we’ll unravel the magic of LuaJIT FFI, Lua/C API, Lua coroutines and much more. Bye for now! 🎉

Editor: ChatGPT