Embed Lua for scriptable apps

Incorporate application scripting using a small language built for the job

The Lua programming language is a small scripting language specifically designed to be embedded in other programs. Lua's C API allows exceptionally clean and simple code both to call Lua from C, and to call C from Lua. This allows developers who want a convenient runtime scripting language to easily implement the basic API elements needed by the scripting language, then use Lua code from their applications. This article introduces the Lua language as a possible tool for simplifying common development tasks, and discusses some of the reasons to embed a scripting language in the first place.

Share:

Peter Seebach (developerworks@seebs.plethora.net), Freelance writer, Wind River Systems

Author photoPeter Seebach collects tiny devices that run on Linux. He is sick of hearing the joke about making a beowulf cluster of them.



27 January 2009

Also available in Russian Japanese

Lua is a small scripting language. How small? Lua uses a custom pattern-matching feature, rather than POSIX regular expressions, because a complete regular expression implementation is substantially larger than all the standard Lua libraries put together; the much simpler string matching provided by Lua, while not quite as powerful, is a fraction of the size.

Lua variables are not strongly typed; you can check the type of a value, but there is nothing preventing a variable's type from changing over time. Both of these choices are good fits for a scripting language. Lua's type system is reasonably simple, but quite flexible. Arrays and associative arrays are combined into a single type, called tables. Strings, numbers (floating point only), booleans, and the special nil type are basic types. Perhaps more interestingly, functions are also a basic type. You can assign functions to variables as easily as any other type; there is no special syntax. Additional support exists for custom userdata objects, which can be defined by developers to handle types beyond those of the basic system.

One of the biggest surprises for programmers coming from other languages is that, in Lua, only false and nil are considered false; any object of a non-boolean type is always considered true in tests. While this behavior can surprise people who are used to C idioms, such as using 1 and 0 for true and false, it is easy to adapt to.

Lua is written in portable C. It can also be used with C++, but the core language is extremely portable; while there are a few features that require host features, Lua runs fine without platform dependencies. There is no huge suite of autoconf tests; Lua sticks to the standard. Lua is distributed under the MIT license, and is completely free for any use, including commercial use. (This is perhaps a reason why many programmers have felt free to incorporate it in applications.)

Why embed a language?

Embedding a scripting language offers a number of advantages. I'll use the example that got me started on Lua: Blizzard's massively-multiplayer online RPG, World of Warcraft (WoW). The user interface for WoW is implemented entirely in Lua; the developers provided a few fundamental API calls to actually interact with the rendering engine and request data about the world, then used Lua for the core of the user interface code.

This makes it much easier for the user interface code to be sandboxed nicely away from the guts of the game, improving security and reliability. And that, in turn, means that Blizzard has been able to open the user interface up to players, allowing players to write custom code to change the way they interact with the game.

In general, scripting languages are easier to work in for many kinds of tasks than lower-level languages. A garbage-collected language with implicit allocation and associative arrays often lends itself to simpler code that is developed faster. It may not run faster, but in many cases, that's not an issue; user interfaces, for instance, only have to run faster than the user can type or mouse.

There are a couple of ways to use a scripting language. The first and simplest is to use it to control the behavior of a program, treating the C code as an implementation detail of a program that is really written in Lua. The second is to write a program primarily in C, then use embedded Lua as a way to store and report data and configuration. The third, and most flexible, is to mix and match these approaches, using Lua to script some actions, and code in C to manage others. The simple interface between Lua and C encourages approaches like this.


Writing an engine

Writing an engine for a program that will primarily be written in Lua makes sense if the primary use of CPU time is embedded in individual operations, and the top-level control is relatively lightweight. This helps separate implementation details from the high-level design. Implementing the core logic of a program in Lua, rather than in C, might well dramatically reduce development time.

For this type of design, you'd expect most of your Lua-to-C interface to consist of defining C functions to be called from Lua, with the expectation that once you start executing the script, all future use of your C code will be calls from the script.


Scriptable configuration files

Every programmer I know has written at least one piece of code that did nothing but try to store configuration values in a file, and restore those values later. (Of the ones I've used, Apple's property lists are probably my favorite.) However, an embedded scripting language can be used as the format for such files, giving users a spectacular array of configuration options. The Ion window manager uses Lua for its configuration files, allowing users to write powerful and flexible configurations.

What makes this excellent for users is that they are no longer restricted to simple assignments of values; configuration files expressed as Lua can have comments, conditionals, and more. You can provide a limited API for acquiring data that might influence configuration choices.


Mixing and matching

It's quite possible to bounce back and forth between Lua and C, as the Lua interpreter is reentrant. If your C program invokes the Lua interpreter on a script, which calls a C function, which then uses the Lua interpreter again, that's okay.

World of Warcraft uses essentially this model for its user interface; Lua calls in the user interface can make calls back into the engine, and the engine delivers events to user interface code that was written in Lua. The result is a flexible interface with good separation and security, which gives users a great deal of room to do things with little risk that they will trigger buffer overruns in the application or cause crashes. In a C-based API, embedded code would almost certainly be able to cause a crash; using the Lua interface, if user interface code can cause a crash, there is a bug that ought to get fixed.


Building and using Lua

Building Lua is easy; you just run make <platform>; you can fall back on posix or even ansi if you don't want to rely on platform-specific features. The build produces a library, liblua.a, which you can link into programs. Congratulations! You've embedded Lua. Of course, actually using it takes a little more work.

Lua's reentrancy comes from keeping all interpreter state in an object; there can be multiple interpreters, which do not share variables, and there are no global items shared between them. To interact with Lua, you must start by creating a Lua state:

lua_State *l;
l = lua_open();

If the lua_open() call fails, it returns a null pointer. Otherwise, you have a valid Lua interpreter state. Of course, without libraries it's not much good to you. You can add the standard libraries to it with the luaL_openlibs() function:

luaL_openlibs(l);

The Lua state is now ready to execute code. Here's a sample loop for a program that simply executes its arguments as Lua code:

Listing 1. Executing arguments as Lua code
  for (i = 1; i < argc; ++i) {
    if (luaL_loadbuffer(l, argv[i], strlen(argv[i]), "argument")) {
      fprintf(stderr, "lua couldn't parse '%s': %s.\n",
		 		 argv[i], lua_tostring(l, -1));
      lua_pop(l, 1);
    } else {
      if (lua_pcall(l, 0, 1, 0)) {
        fprintf(stderr, "lua couldn't execute '%s': %s.\n",
		 		 argv[i], lua_tostring(l, -1));
        lua_pop(l, 1);
      } else {
        lua_pop(l, lua_gettop(l));
      } 
    }
  }

The luaL_loadbuffer() function compiles a script into Lua code; if there is a syntax error, this is where the failure is observed. The error message is returned on the stack. Otherwise, the compiled code can be executed using the lua_pcall() function. Again, if there is an error, it is returned on the stack. Note that every C language call to or about Lua takes a Lua state as an argument; there is no default state.

Understanding the Lua stack

The Lua interpreter uses a stack interface to communicate with calling code. Data being sent to Lua code are pushed on the stack by the C code; responses from the Lua interpreter are also pushed on the stack. If the code passed to luaL_loadbuffer() is invalid, the error message is pushed on the stack.

Items on the stack have types and values. The lua_type() function queries the type of an object; the lua_to<type>() functions (such as lua_tostring()) yield values coerced into particular C types. Code written in Lua always strictly obeys the stack model; C code, however, can poke around on the rest of the stack, or even insert values in the stack.

This interface is simple but surprisingly powerful. Code to be executed is not treated differently; it is simply pushed on the stack, to be executed by the lua_pcall() function.

Using lua_pcall()

The lua_pcall() function takes three arguments other than the Lua state it is to operate on. The code to execute is not one of these arguments; the code is pushed on the stack by luaL_loadbuffer() or by another function which obtains code. Instead, lua_pcall() takes as arguments the number of stack arguments to pass to the code it is about to execute, the number of results to expect back, and optionally an error handler. To call a function, you push first the function, then the arguments it will take (in order). Arguments returned are pushed in the same order; the first returned value is at the bottom, and the last returned value is at the top.

Both for sending arguments and obtaining return values, Lua silently corrects the number of values to match the numbers passed to lua_pcall(); if not enough values are provided, the remainder are filled in with nil values, and if there are extras, they are silently discarded. (This is the same as Lua's behavior on multiple assignment operations.)

The error handler, if one is provided, is the index on the stack of the Lua code to handle any errors that occur. For this overview article, I'm going to omit detailed discussion of error handling; what's important to know first is that there is error handling, and second that it's done in Lua. This turns out to be very handy.


Embedding C in Lua

Writing functions in C to be used by Lua is surprisingly easy. If you've ever written code for embedding in another scripting language, you'll probably find this sort of shocking. Here's a C function which, given a number x, returns x + 1:

Listing 2. C function to be used by Lua
int
l_ink(lua_State *L) {
        int x;
        if (lua_gettop(L) >= 0) {
                x = (int) lua_tonumber(L, -1);
                lua_pushnumber(L, x + 1);
        }
        return 1;
}

The function is called with a Lua state argument; once again, all interaction between C and Lua occurs through the stack of the Lua state. The return value is the number of objects the function pushed on the stack. To make this function available to Lua, you must do two things; the first is to create a Lua object representing this function, and the second is to give it a name:

lua_pushcfunction(L, l_ink);
lua_setglobal(L, "ink");

You use lua_pushcfunction() to convert a C function pointer into an internal Lua object. This object, of course, is pushed on the stack. The lua_setglobal() function then assigns the top value on the stack to a named global variable. Because functions are just values in Lua, this creates a callable function.

The stack interface dramatically simplifies this; there is no need to define or declare the arguments that the function takes. It wouldn't matter; Lua is very flexible about how code gets called. You can, however, check some things more carefully if you want. The luaL_checknumber() function can also be used to check arguments, printing an informative error message and aborting execution of the function.


Making good use of embedded scripting

The simplicity of embedding Lua in code in other languages (but especially C) makes it an easy target for substantially improving the functionality of programs in other languages. This is a real and viable alternative to inventing your own configuration language, or writing your own expression parser.

A few things may surprise people used to working with much larger scripting languages. The first is that the cost of setting up a Lua interpreter is very small; if you want to run something in a sandbox, go right ahead and do it. Lua's got a lot of security features I haven't touched on, but you should be aware that it's quite possible to do effective sandboxing even within a single Lua state.

Functions that simply return data about the state of your program make a wonderful tool for configuration scripts and data. If you want to start writing some top-level logic in Lua, though, it's very easy and efficient to do so. While many scripting languages grudgingly accept the need to work closely with code in other languages, Lua is probably the clearest example of a language entirely designed to work closely with other languages.

Resources

Learn

Get products and technologies

  • With IBM trial software, available for download directly from developerWorks, build your next development project on Linux.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux, Open source
ArticleID=366864
ArticleTitle=Embed Lua for scriptable apps
publish-date=01272009