In this article I will discuss three methods for using Lua to build iOS apps. The techniques range from using Lua to build the entire application (Corona) to using Lua as a component for scripting your application (do it yourself or Wax). Before getting into the details, there are two main questions to answer:

1. Why use Lua?
2. Will Apple let you use Lua?

The answers to these questions are intertwined.

In case you landed here without knowing anything about Lua, let me hit the high points of the language. If you already are familiar with Lua, you can skip ahead.

About Lua

Lua is a fast, lightweight, embedded scripting language. It is similar to languages such as JavaScript, Ruby or Python. Many of its users, including myself, feel Lua is a particularly clean and elegant language.

Lua was created in 1993 by Roberto Ierusalimschy, Waldemar Celes and Luiz Henrique de Figueiredo at the Pontifical Catholic University in Rio de Janeiro, Brazil. It is used in a range of applications including Mi Casa Verde, Adobe Lightroom, Celestia, lighttpd, LuaTeX, nmap, Wireshark, Cisco’s Adaptive Security Appliance, pbLua, and a ton of games including Grim Fandango, World of Warcraft, Tap Tap Revenge, and many others. Lua’s license is a variation of the MIT License—this means that there are essentially no hurdles to including Lua in both commerical and non-commerical projects.

Lua’s main data structuring mechanism is the table—a combination resizable array and hash. Listing 1 shows a table we might use in a hypothetical application to keep track of automobiles and their gas mileage. We can store information about the automobile using string keys such as license and make. A series of mileage readings are stored using integer indices.

 
car_data = { license = 'XVW1942', make = 'Volvo',
                model = 'XC70', 30, 31.3, 32.4, 34.0 }
 
print(car_data[1])                -- 30
print(car_data['license'])       -- XVW1942
print(car_data.license)          -- XVW1942 (also!)

Out of the box, Lua is neither an object-oriented (OO) programming language nor a functional programming language. Rather, it provides a small set of mechanisms with which you can build your own higher-level features. A number of different object systems have been built in Lua including more traditional OO systems as well as classless OO systems (à la languages like Self or Io). Lua supports first-class functions and lexical closures and has a meta-programming facility (known as metatables and metamethods). Lua can be used in a functional programming style

For a gentle introduction to OO in Lua, read Programming in Lua (in fact, PIL is a very well-written book on Lua in particular and programming in general). You should also look at the several examples available on Lua wiki.

If you like drinking from a firehose, Listing 2 shows one possible implementation for a linked list class. The table stored in the variable List serves as a metatable for all linked list objects. It provides a fall-back location for looking up table indexes and thus serves as a class dispatch mechanism. The line “List.__index = List” is what allows us to create methods for our list objects. Methods are implemented as functions stored in the List metatable. When we attempt to call one of these functions on our list object, the lookup mechanism will lead us to the function defined on the List metatable and that’s what will get run.

The code shows a number of additional features of Lua: multiple assignment (and functions can return multiple results), syntactic sugar for method calls (the ‘:’ notation, this performs the common technique—used in languages ranging from Python to Objective-C—of rewriting the function call to have an additional parameter which points to self).

List = {}
List.__index = List

function List:new()
  local l = { head = {}, tail = {}, size = 0 }
  l.head.__next, l.tail.__prev = l.tail, l.head
  return setmetatable(l, self)
end

function List:first()
  if self.size > 0 then
    return self.head.next
  else
    return nil
  end
end

function List:addFirst(elem)
   local node = { prev = self.head, value = elem, 
                        next = self.head.next }
   node.next.prev = node
   self.head.next = node
   self.size = self.size + 1
end

mylist = List:new()
mylist:addFirst(12)
print(mylist:first())

I’m sure I’ve left out the really interesting/important things (such as lexical closures), but this at least gives you a little taste of Lua. There are more samplings below when we get to actual iPhone coding. For further details on Lua, see the web site.

Can I Script on iOS?

Of the two questions listed at the beginning of this article, the most important question to address is are you allowed to use Lua (or any interpreted language) on the iPhone? After all, early on the iPhone Developer Program License Agree stated that “[n]o interpreted code may be downloaded or used in an Application except for code that is interpreted and run by Apple’s Documented APIs and built- in interpreter(s).”

In fact, an earlier version of this article was tabled when Apple made changes to the developer license (circa April 2010) which forbade developing iOS applications in any language other than Objective-C and Javascript (the Javascript could be used either to build web apps or native apps via the UIWebView). Recently (September 2010), Apple again changed the developer license to allow the use of scripting languages..

There are still several important constraints in place. In particluar, although you can use Lua and other scripting languages, you cannot create an application where users could download plugins for your app from your website (in-app purchasing anyone?), nor can you allow the user to write scripts, download scripts, etc. There are (and have been, e.g. Tap Tap Revenge) quite a number of apps available on the app store that use Lua as well as other languages.

Of course, creating a plugin system and allowing users to write scripts are two major use cases for including a language like Lua in your application, so what’s left? Plenty!

Why Use Lua for iOS development?

Although you cannot expose a plugin system to the end user, nor can you give her the ability to write her own scripts, you can still develop your system using a plugin architecture! This can both speed up initial development as well as be a big help when it’s time to add functionality for the next version.

There are other benefits from using Lua. It allows you to develop using rapid prototyping (beware my pet peeve: don’t devolve into seat-of-your-pants programming), reduces/eliminates the need to worry about memory allocation, allows more of your team to participate in development (many Lua projects have non-programmers writing code), makes it easier to tune your application, and provides a powerful persistence mechanism.

In short, using Lua can reduce development time and lower entry barriers. And it’s just plain fun!

Assuming you’re sold you on the idea of using Lua, how do we go about it?

Ansca Mobile’s Corona allows you to develop your iOS app entirely in Lua. And it’s not just for iOS. You can also develop apps for Android. In fact, you can use the same source code to build both an iOS and an Android app. This adds a compelling reason to use Lua (and, in particular, to use Corona): the ability to easily build cross-platform applications.

Listing 3 is the complete source for an app.

local w, h = display.stageWidth, display.stageHeight
local dx, dy, dtheta = 5, 5, 5


local background = display.newRect(0, 0, w, h)
background:setFillColor(255, 255, 255)


local message = display.newText('Hello from Corona', w/2, h/2)
message:setTextColor(0, 0, 200)


local function update(event)
   local counter_spin = false
   message:translate(dx, dy)
   message:rotate(dtheta)
   if message.x > w or message.x < 0 then
      dx = -1 * dx
      counter_spin = true
   end
   if message.y > h or message.y < 0 then
      dy = -1 * dy
      counter_spin = true
   end
   if counter_spin then
      dtheta = -1 * dtheta
   end
end


Runtime:addEventListener('enterFrame', update)

Corona apps are developed using your favorite text editor—I use Emacs. All Lua source code and any necessary resources (images, sounds, and data) must reside in a single directory and Corona expects a Lua file, main.lua which is where your app starts executing. You test your code in Corona's simulator which runs on both Intel and PPC Macs. Figure 1 shows my Corona 'IDE': namely Emacs with two windows (a Lua file and the project directory), the Corona terminal (you can print debugging info to the terminal), and the Corona simulator.

When you are ready to run your app on actual hardware, you use the Corona Simulator's Open for Build option. For an iOS build, you must have a provisioning profile (either development or distribution)—yes, Virginia, you do need a membership in the iOS Developer Program—which is uploaded, along with your app's source and resources, to Ansca's servers. A compiled app is returned to you. For Android builds, you will need a suitable signing certificate. Again the build process is handled by uploading your source to Ansca's servers. You do not need to have the Android SDK installed.

I haven't done any in-depth poking around, but both the .apk file from the Android build process and the iOS .app bundle contain a file containing all your Lua code pre-processed in some fashion. Approximately seven seconds of examination suggests that it's not standard byte-compiled Lua code, but I'm guessing it must be a similar format.

Corona's event system lets you handle touches (including multi-touch), access the GPS and the acceleratometer, handle animation, and define custom events. There is a powerful graphics system which allows you to draw circles, rectangles and text. They've recently added polylines so you can draw regular lines and polygons. You can also display images. Corona allows you to group these objects and do transformations on them. Listing 4, an excerpt from a Solar System simulator, shows a (crude) example of grouping graphics objects. Other Corona features include playing audio and video clips, a cryptography library, networking using the LuaSocket library, and access to SQLite databases (using LuaSQLite). There is some access to native widgets including textfields, alerts and activity indicators. There's a nice feature where you can overlay a web view for doing things like login screens and a sample application provides a library for connecting to Facebook. The last thing I'll mention is that there is a (more expensive) game edition that includes the Box2D physics engine, sprites and some OpenFeint functionality such as leaderboards.

function new(params)
   local color = params.color or planet_colors[random(#planet_colors)]
   local radius = params.radius or planetRadius()
   local planet = display.newGroup()

   planet.theta = 0
   local x = params.x - ox
   local y = params.y - oy
   planet.orbital_radius = sqrt(x*x+y*y)

   local body = display.newCircle(x + ox, y + oy, radius, radius)
   body:setFillColor(unpack(color))
   planet:insert(body, true)
   planet.body = body

   planet.delta_theta = (40/planet.orbital_radius) * 0.1

   return planet
end

Corona gives you a lot. However, there are still a lot of things missing with Corona. The biggest item is the limited access to native controls. Not to mention the access that is provided is awkward to use because of limited support in Corona's simulator. In the simulator, native alerts and activity indicators are implemented using OS X equivalents, rather than the iOS widgets. However, text fields, text boxes and web popups are unusable when running in the simulator. This makes development, to say the least, painful.

Finally, there is no mechanism to access the Objective-C API except for what ANSCA has specifically provided. Not only does this mean you cannot access large portions of the standard libraries, but you cannot make use of third-party libraries like Three20, or one of the mobile ads APIs. Of course, with the release of the Android version of Corona, you may not want to access the Objective-C API since it limits (or complicates) your ability to do multi-platform applications. It would be nice, however, to be able to add in Lua extensions that use Lua's C API, as many of these are multi-platform.

I've found the ANSCA staff to be quite helpful and responsive to questions posted in the forums on their website. With the release of version 2.0 (September 2010), Corona costs $249 per developer per year. The Game Edition is $349 per developer per year. Ansca's website indicates that the game edition prices is pre-release. I assume that means it will be higher when it is officially released.

Including the Lua interpeter in your iOS app is simple. Open an Xcode project and add the Lua source files (except lua.c and luac.c—source for the command line programs). Compile. You can now make use of the standard Lua C API to create an interpreter and run some code. An example project, iLua, may be downloaded from http://github.com/profburke/ilua. iLuaShell is a simple, view-based application that presents the user with two text fields—an editable one where the user can enter Lua code and a non-editable one where the results of evaluating the Lua code are displayed.

The work is done in the method, evaluate, which is shown in Listing 5. The method retrieves the text of the first text field, hands it off to the Lua interpreter which parses and executes it, and then sets the Lua output as the text of the output field.

-(void)evaluate {
    int err;
 
    [input resignFirstResponder];
    lua_settop(L, 0);
 
    err = luaL_loadstring(L, [input.text 
                     cStringUsingEncoding:NSASCIIStringEncoding]);
    if (0 != err) {
        output.text = [NSString stringWithCString:lua_tostring(L, -1) 
                             encoding:NSASCIIStringEncoding];
        lua_pop(L, 1);
        return;
    }
 
    err = lua_pcall(L, 0, LUA_MULTRET, 0);
    if (0 != err) {
        output.text = [NSString stringWithCString:lua_tostring(L, -1) 
                             encoding:NSASCIIStringEncoding];
        lua_pop(L, 1);
        return;
    }
    int nresults = lua_gettop(L);
 
    if (0 == nresults) {
        output.text = @"<no results>";
    } else {
        NSString *outputNS = [NSString string];
        for (int i = nresults; i > 0; i--) {
            outputNS = [outputNS stringByAppendingFormat:@"%s ", 
                                               lua_tostring(L, -1 * i)];
        }
        lua_pop(L, nresults);
        output.text = outputNS;
    }
}

Note the error checking and handling is minimal. A little additional effort could yield a nice Lua shell—not that you could actually put it in the App Store...

What are the drawbacks to this approach? The biggest one is the lack of a bridge to Objective-C. Ideally we want to call Objective-C methods from Lua. Going the other direction is important also. Being able to code callbacks and delegate methods in Lua would be a big win.

Pursuing that leads us to...

Powerful interoperability between Lua and Objective-C is the star attraction of iOS Wax by Corey Johnson. Using Wax, you can easily subclass Objective-C classes in Lua! Listing 6 shows a Wax implementation of a custom view controller. In particular, this code is a port of the application described in the DIY section. Details after the listing.

waxClass{'RootViewController', UI.ViewController }

function init(self)
        self.super:init()

    self.input = UI.TextView:initWithFrame(CGRect(20, 20, 280, 114))

    self.output = UI.TextView:initWithFrame(CGRect(20, 184, 280, 225))

    local evalButton = UI.Button:buttonWithType(UIButtonTypeRoundedRect)
    evalButton:setTitle_forState('Evaluate', UIControlStateNormal)
    evalButton:setFrame(CGRect(200, 142, 100, 32))
    evalButton:addTarget_action_forControlEvents(self, 'eval:', 
                            UIControlEventTouchUpInside)
    self.evalButton = evalButton
    
    self:view():addSubview(self.input)
    self:view():addSubview(self.output)
    self:view():addSubview(self.evalButton)
    
    return self
end

function eval(self, sender)
    self.input:resignFirstResponder()

    local code, errmsg = loadstring(self.input:text())  
    if not code then
        self.output:setText(errmsg)
        return
    end
    
    local success, result = pcall(code)
    print('result is ' .. tostring(result))
    if not success then
        self.output:setText('Error: ' .. tostring(result))
    else
        self.output:setText(tostring(result))
    end 
    
end

The waxClass function essentially defines a new Objective-C class. In this instance we are defining a class named RootViewController which is a sub-class of UIViewController. (In Wax, the Objective-C classes have been put into namespaces, hence UI.ViewController, rather than UIViewController). The Lua representation of instances of this class is a table (well, really a userdata, but you can think table...), hence items like self.input are Lua table fields and not Objective-C properties. To access properties you use setters and getters, e.g. self.output:setText(). If you're like me, this will trip you up until you embarass yourself by asking about it in the mailing list. Aftewards, you won't mix it up again. (Actually, the people on the mailing list are quite nice.)

Wax classes can also implement protocols. For instance, the Wax sample project, States, demonstrates handling a UITableView with two custom UITableViewController classes. Each of which implement the UITableViewDelegate and UITableViewDataSource protocols. Listing 7 is a minor variation on this theme and presents a class that implements the UITableViewDataSource protocol for a multi-section table.

Listing 7: SortedDataSource.lua
waxClass{'SortedDataSource', NS.Object, protocols = {'UITableViewDataSource'}, }


function init(self, source_table)
    self.source_table = source_table
    return self
end


function numberOfSectionsInTableView(self, tableView)
    return #self.source_table.headers
end


function tableView_numberOfRowsInSection(self, tableView, section)
    local index = self.source_table.headers[section+1]
    return #self.source_table[index]
end


function tableView_cellForRowAtIndexPath(self, tableView, indexPath)  
    local identifier = 'TableViewCell'
    local cell = tableView:dequeueReusableCellWithIdentifier(identifier)
    cell = cell or UI.TableViewCell:initWithStyle_reuseIdentifier(UITableViewCellStyleDefault, 
                                                      identifier)

    local key = self.source_table.headers[indexPath:section()+1]
    local component = self.source_table[key]
    local player = component[indexPath:row()+1]
    cell:setText(player[1] .. ' ' .. player[2] .. ' ' .. player[3])

    return cell
end

function tableView_titleForHeaderInSection(self, tableView, section)
    return self.source_table.headers[section+1]
end

The uniform naming scheme Wax uses allows you to easily predict the name to use for accessing an Objective-C function from Lua. Wax comes with a TextMate bundle which makes it very easy to manipulate the Objective-C calls. For example, you can paste method signatures copied from Xcode's documentation and have them automatically transformed into Lua calls. (I'm debating writing Emacs functions to do this or just drink the Kool-ade and start using TextMate.)

Wax has a number of other goodies including extensions for working with SQLite, easy HTTP requests, XML and JSON handling, and working with Core Graphics transforms and gradients. In addition, recent updates include the ability to write the App delegate in Lua (rather than having a small Objective-C implementation that launches Wax), and the ability to run tests from the command line (in a headless simulator). Perhaps the most interesting (and powerful) new development is that Wax now provides an interactive console so you can telnet into the simulator (or a device!) and interact with a running application: tweak its parameters or inspect its current state.

Wax is open source and is also licensed with an MIT-style license. The project is being actively developed and used by a number of developers.

Summary

Lua-powered apps are available in the app store. The Ansca forum for listing Corona apps has over 150 topics (I haven't read them all...they may not all announce new apps). Reading through the Wax mailing list, you'll see several developers announcing apps they've written using Wax, and there are likely to be many others who have not posted to the list. And there are many apps that have taken the DIY approach.

Corona offers a very nice alternative for building iOS apps, but only if you don't need native UI elements. It's great that they've added native text fields, but the fact that you can't see them in the simulator is a real show-stopper. But throw its Android capabilities into the mix, and Corona is certainly worth considering. I like using Corona and I hope/expect to see lots of improvements over time. Just to weasel out of making an endorsement, I should add that although I have not had any major problems building apps with Corona, the dependency on Ansca and their servers to do builds is something you should seriously think about.

Of course the DIY approach is the complete opposite: you have total control. But if you need a lot of interaction between your Lua code and your Objective-C code, doing the bindings can be a fair amount of effort. Johnson's Wax does a fantastic job of bridging Lua and Objective-C. Wax also plays well with Lua C libraries—something which Corona doesn't handle.

Although they may not go as far as we'd like, the recent changes to the iOS developer agreement mean that you can now use Lua in iOS development without the stress of worrying that you're setting youself up for app rejection. I believe the range of techniques available for using Lua in your iOS project means that using Lua will often be a useful tool for successfully completing your iOS project. Given the active communities surrounding Corona and Wax, as well as the ease of plotting your own course if you want more direct control of your use of Lua, I encourage you to take advantage of this gem of a language.

References

• Lua's Website (http://www.lua.org)
• Corona Developer Portal (http://developer.anscamobile.com)
• Wax's Repository (http://github.com/probablycorey/wax/)
• Wax Syntax Tips (http://probablyinteractive.com/2009/10/19/How%20does%20iPhone%20Wax%20work.html)
• http://groups.google.com/group/iphonewax

RISC OS offers a very simple interface to user software, via the ARM’s SWI (SoftWare Interrupt) instruction. Data is passed between the user’s software and the operating system in ARM registers; that data may consist of pointers to buffers in memory that hold further data. RISC OS has always contained the BBC BASIC programming language. This offers the user means of accessing the operating system using:

• DIM — to reserve a block of memory of a given size and pass back its address;
• Indirection operators ! (for words), ? (for bytes) and $ (for strings);
• SYS — to call an SWI with given values in the registers on entry and variables to receive register values on exit;
• an inbuilt ARM assembler and a CALL command to link to assembled code.

With these facilities BBC BASIC is sufficiently powerful to enable a user to control every aspect of her machine, if she is equipped with the RISC OS PRMs (Programming Reference Manuals) - these give details, for each of the modules constituting RISC OS, the protocols and input/output details for each of the SWIs available, and of the messages by which the kernel communicates with the modules.

Acorn Computers Ltd started out as manufacturers of laboratory equipment. Their computers (with the exception of the RISCix work stations) were always intended as single-user machines over which the user has total control. Security? An irrelevance in those days. Just as today’s motorist almost certainly does not carry a set of tappet-calipers in his hip-pocket and has never disassembled a carburettor on the kitchen table, so modern computer-owners are unlikely to want to know about SWIs or programming. But RISC OS developed in an era when computers had only just become affordable, and there was a sizable minority of people who wanted not just to do things with their computer but to understand how it did them, and how they could control it. In the end it was the association with education that did for Acorn Computers Ltd and for RISC OS, after ARM Holdings Ltd had been split off.

Porting Unix or Windows software to RISC OS is fraught with problems. RISC OS uses cooperative, not preemptive, multitasking. The RISC OS filing system is fundamentally different, too, and not just syntactically (a dot is the directory separator symbol). Filer objects have a filetype attribute that is not part of the object’s name.

RISC OS filing systems were designed on the supposition that they were part of a graphical user interface – the GUI is not just something tacked on later to a commandline interface. When a user clicks on the icon of an object in a directory-window the window manager broadcasts a message to all running tasks:

An object at coordinates (x,y) with pathname obj_name and
filetype obj_type has just been clicked on. Are any of you
guys interested?

If a task is interested it sends back an acknowledgment (so that the broadcast is aborted) and has its way with the object. If no task replies, the taskmanager looks to see if a system command (a sort of environment macro) Alias$@RunType_xxx has been defined, where xxx are the hexadecimal digits of obj_type. If it has, it is executed with obj_name as a commandline argument. This command generally starts up a task and passes on its argument to it. There is a special type for command files – known as “obeyfiles”. An obeyfile always “knows” where it is because the operating system sets the variable Obey$Dir to the pathname of the directory containing it when it is executed. Textfiles constitute a special type. Holding down the SHIFT key when clicking a file will always open it in a text editor (even if the user has not installed a text editor – there is a basic multifile texteditor incorporated in the OS, so fundamental is the role of editing ASCII text – the same principle applies to graphics, both bit-mapped and vector). Each type of object is displayed by its own icon, apart from application directories.

RISC OS has two types of directory. There are ordinary directories, and there are application directories.

• Click on the icon of an ordinary directory with the left mouse button and its window will open; with the right mouse button and the currently open directory window containing it will simultaneously close – this makes navigation and the avoidance of window clutter very simple and direct – no opening of menus needed.

• Application directories behave differently. Their appearance is governed by a file inside them called !Sprites – a bit-mapped graphics file. Click an application directory and the obeyfile !Run inside it will be executed. To open an application directory like an ordinary directory you must depress the SHIFT key when you click.

The point about application directories is that they can be treated as a package. There is no such thing as installing or deinstalling or registering an application. You simply load the application directory to wherever you want in the filing system and it will be launched when you click on it. To deinstall, simply delete it. This independence of position is achieved by a file called !Boot within an application. It is executed when the window manager is first made aware of the application’s existence by the opening of the window of the directory containing it; this action can be suppressed by holding down the CTRL key when opening a directory window. The !Boot file will define system variables and macros which will inform user software about where the application is, and anything else they need to know of it. Alternatively the user can “filer_boot” and “filer_run” applications from obeyfiles that are inserted into the boot sequence. This is done by loading the obeyfiles into appropriate places in the directory !Boot that must reside at the top level of any RISC OS filing system. Only the !Boot directory is off-limits for the user’s personal whims (though even that is highly customizable). In other words, the filing system is not a thing of labyrinthine complexity that belongs to the manufacturer or to tradition. It belongs to the user. This is made possible by the relocatability of applications.

In the RISC OS GUI, the window on top of the window stack is not necessarily the window with the input focus. I often find myself inputting text even though the window containing it is partially obscured by another window that I need to be reading at the same time, to compose my input. If I drag a window with the left-hand button it will come to the top of the window stack, and so be dragged over any other windows. If I drag with the right-hand button it will retain its position in the window stack, and so be dragged behind some windows and over others. The single most exasperating and thoughtless aspect in Microsoft Windows, to my mind, is the way windows leap to the top, obscuring what you need to be reading. Real desktops do not behave in this way, unless somebody has left the door open and there is a gale blowing through. It is a small detail, but a crucial one. I have not sufficient experience to know which Linux window managers, if any, get this right.

Of course, applications in read-only filing systems may need to rent space for their configuration files on another, writable filing system. RISC OS is usually held in ROM, so startup and switchoff are immediate. I can switch on both my computers, one RISC OS, an Iyonix, and the other Windows XP, an Advent 4211 notebook, at the same time, and by the time the Advent is ready to use I have read my emails, replied to them and switched off the Iyonix.

I mention all these details so that readers who are only familiar with Windows or Unix will understand that there are other operating systems which present a very different user experience. Many of the worries and tools to deal with them (defragmenting the hard disc, installing and removing packages, viruses, rootkits) that the Windows or Unix user must take into consideration do not exist for the RISC OS user. The other side of the coin is that RISC OS cannot deliver much that is now taken for granted – its user base is too small to catch up.

As BASICs go, BBC BASIC is very good. There is a version for Windows, I believe, with many improvements. It is almost unfair to call it BASIC, because the facilities I cited above give it a whiff of BCPL. Recall that BCPL was developed in Cambridge and was the grand-daddy of C. BCPL was untyped. Although I am an admirer of BBC BASIC, I am not blind to its limitations as a programming language. For a long time I wondered if it were possible to have the best of both worlds: something that incorporated the advances in programming language design since 1964, BASIC’s year of birth, and yet retained the integration with the operating system and the ease of use which BBC BASIC can boast. I spent many years on different attempts to realize this dream. Then I came across Lua.

Lua is portable to any platform with an ANSI C compiler. Lua is a modern phenomenon, if we take the development of C as the watershed dividing ancient from modern. Lua is safe, and it can make no assumptions about its host platform; it certainly cannot peek or poke memory. RISC OS is unsafe, tied to the ARM architecture, and uses memory references. It is written mostly in ARM assembler; it conflates 32-bit integers with addresses, with file-handles, task-handles, task-manager messages, … . It is in some ways a relic of the age before C. To marry Lua with RISC OS, to produce what I called in an unimaginative moment, RiscLua, these contradictions had to be squared.

The first problem is that ARM CPUs tend not to have floating-point hardware. Continual conversion between doubles and 32-bit integers would be a burden. This problem is solved by #defining LUA_NUMBER to int and leaving out the math library. I also extended the Lua VM with bit operations, putting them on an equal footing with the other arithmetic operations. For those that have to use floating point numbers I put in a library implementing doubles in the heap.

To integrate Lua with RISC OS, something along the lines of BBC BASIC’s DIM,!,?,$ and SYS was needed. To begin with I used userdata for holding the values of addresses. Then I could reserve blocks of garbage-collectible memory, and I could check that references were within set bounds. One downside was that the error-checking slowed things down. Much worse was the problem of how to convert the data returned in a register from an SWI call. To what sort of Lua value should it be converted? An integer, a userdatum holding an address, a file-handle, a string, …. ? Eventually I realised that this problem had no universal solution. The only answer was to play as dirty as BBC BASIC; to abandon userdata and simply conflate integers with addresses. This made RiscLua less safe, but faster, and allowed a syntax for indirection that was very close to that of BBC BASIC – an advantage for users more accustomed to BBC BASIC than to anything else. The latest versions of RiscLua have a riscos library containing dim, sys, !, ? and $ (these are allowable in variable names in RiscLua). Where BBC BASIC might have x!4 RiscLua can have ![x+4]. The symbols !,? and $ in the riscos environment are just empty tables with metamethods: __index peeks and __newindex pokes. The sys function has one little twist that SYS does not have: a nil argument for a register value means “use the value returned for that register by the previous call to sys”. This is useful because the register usage for SWI calls in RISC OS has been carefully optimized to minimize shuffling data between registers, something which can be exploited by this syntax.

There are a few other extensions: backslash is sugar for function and ‘=>’ is sugar for return.

This is a texteditor window displaying a short RiscLua script that uses Roberto’s lpeg library to run code enclosed in <lua> … </lua> tags (with the remainder of the text taking the value “text”). The point of the picture is the third icon with the green arrow pointing down. If you shift-drag the icon of a scriptfile onto the green arrow of a texteditor window, the script is run (with arg[1] holding the name of a pseudofile containing the contents of the window) and the contents of the window are replaced by the standard output of the script. This provides the texteditor with a convenient facility - any scripting language can be used so long as the texteditor has been configured to understand the right command line syntax for it. In this case, shift dragging “untangle” onto the green arrow of a window containing “<lua>print(text:upper())</lua>hello” would convert its contents to “HELLO”. Yes, I know it is contrived, but I hope it shows the general principle.

This shows the final dialogue box when the following program is run.

-- Fred, tiny wimp program using error dialogue
require "wimp.task"
do
 local dim,! in riscos
 local buffer, wimp_msgs = dim(256), dim(4)
 local title$ = dim "Fred\r"
 local ask = "Count is %d. CANCEL to stop now?"
 local goodbye = "You reached %d. Goodbye."
 local flags = 23  -- for dialogue window
 ![wimp_msgs] = 0  -- no wimp messages
 fred = task.new(title$,wimp_msgs,buffer)   -- create task "fred"
 in fred do
  count = 0                      -- give it a counter
  handler[0] = \ (self)      -- give it a null-event handler
    count = count + 1
    local click = self:report(ask:format(count),flags)
    => (click ~= 1)          -- OK button to continue
  end -- handler
  preclosedown = \ (self)     -- give it a farewell action
     self:report(goodbye:format(count))
  end -- preclosedown
 end -- do
 fred:init()                   -- register it
 fred:run()                    -- run it
end -- do

This is a toy example of a RiscLua wimp program. It puts up a window with two buttons, CANCEL for stopping, OK for carrying on, and it increments a counter showing the number of times OK is clicked. I hope the program is not too opaque. Note the use of Peter Shook’s patch in local dim,! in riscos and the convenient syntax provided by lexical environments in in fred do … end. The wimp.task library provides a single function task.new which returns a table, with methods “init”, “run”, “preclosedown”, etc and a table, “handler” of event-handlers, indexed by the event-codes sent by the taskmanager. Nothing much surprising about this approach.

I will not pretend that RiscLua can do all the things that BBC BASIC can do. It has no inbuilt ARM assembler, it does not have extensive commands for sound or graphics. But there are things that make RiscLua, for me, much more satisfying to use. It is more expressive, more modular, more consistent and more concise. It fulfils my quest, and still offers plenty of opportunities for further development.

RiscLua and its sources are available at http://lua.riscos.org.uk/ .

Gavin Wraith

Sputnik is a second-generation extensible wiki engine written in Lua. First generation wikis (like the original WikiWIki) opened our eyes to the possibility of easy collaborative content generation, with automatic revision control. However, anybody who has been involved with a Wiki knows that they are not self-organizing, and so behind any Wiki is a core of busy elves correcting and ‘refactoring’ content. Plus, all content is usually in the form of marked-up text, plus uploaded binary data like images. For instance, Wiki pages often turn into discussions, but there is usually no way to structure these discussions and no support for creating them.

A second-generation wiki allows for ‘virtual’ pages, pages with explicit data fields, structured discussions, complete control of permissions, and namespace management. The site designer can choose the right balance between freedom and structure that is appropriate for the community and its common purpose.

Frameworks and Libraries

There are two basic approaches to software development; either start small, pulling as much functionality in using libraries, or to start with a framework, which does most of the job, and customize it to fit your functionality. Frameworks have a bad reputation because framework developers can get just a little mad in the process of writing them. But if you want something that can do wiki-like things, manage the content and control the revisions, handle authentication, then it’s time to get a Wiki framework, because these are not easy applications to get right. As the Sputnik git page says, “Sputnik provides a good foundation for anything that’s kind of like a wiki but not quite.”

It is true that the first trade-off is the freedom to do things your way, since you must learn how the framework does things, and cooperate with it. In a way, it is like collaboration with another developer on a project which you have just joined. I am assuming here that you want to get something done, which is not a million miles from a Wiki, and don’t want to rewrite a whole bunch of wheels, just maybe update the hubcaps. A better analogy would be this: if you want to equip a kitchen, then using a framework that includes the kitchen sink is appropriate (and not just a joke). Maybe you just want different fittings for your kitchen sink.

Getting Sputnik

Installing Sputnik is straightforward, providing you are on a Unix-like platform (like Linux or OS X) although it does also run on Windows. In this introduction, I assume that it is an Unix environment (although out of convenience, not religious fervour; it is very easy to get a Linux virtual machine and set it up for Sputnik testing.) After following instructions, you will have a Sputnik install in ~/sputnik, no special permissions necessary. Starting the webserver is then just:

  ~/sputnik$ ./bin/sputnik.lua start-xavante sputnik.ws

Sputnik comes with the Kepler stack, including the Lua webserver Xavante. This is quite good enough for testing and experimentation.

A useful change is to first edit sputnik.ws and set BASE_URL to ‘/’ and to add SHOW_STACK_TRACE = true. Sputnik will then give you Lua error traces when some hitch occurs. Then just open http://localhost:8080 in your favourite browser and start playing. Create a special user Admin to view and edit the configuration nodes.

All Sputnik configuration is via nodes with Lua content. For instance, sputnik/config assigns a set of fields to values. If you are editing this file (as Admin) and make a syntax error (such as ‘Sputnik”, mismatching string delimiters) the edit field will turn pink. sputnik/navigation controls the navigation bar menu:

  NAVIGATION = {
     {id="index", title="Start",
       {id="snippets", title="Snippets"},
       {id="tags", title="Tags"},
       {id="sputnik",title="Configure"},
     },
     {id="News", title="Timeline",
       {id="News"},
       {id="Future Plans"},
       {id="history", title="Recent Wiki Edits"},
       {id="history/edits_by_recent_users", title="Edits by Recent Users"},
       {id="history.rss", title="RSS Feed"},
     },
  }

Creating a new node is easy; if you ask for a node test then it will tell you that this node does not exist, but then will give you several types to choose from. The ‘Basic’ type is a plain Wiki page, and the link provided is something like http://localhost:8080/?p=test.edit&prototype=. In general, a Sputnik request has a node id (‘test’), an action (‘edit’) and parameters (‘prototype=’)

If you click on this link you can edit your new Wiki node. The markup used is Markdown which has a clean, readable syntax. In addition, Sputnik supports Wiki links; a link to your new page would be [[test]] and a link with some text would be [[test|My First Page]] . There is a convenient toolbar providing the most common operations when editing.

Customizing Sputnik

A lot can be done with basic Sputnik, just by editing the configuration nodes. (And, yes, the configuration nodes are Wiki nodes, so you can revert to an earlier version.) Also, like any modern Web framework, style and functionality are kept separate as CSS and HTML; these Sputnik sites show that you can get just about any look and feel.

Since the configuration is done with Lua data, it technically involves programming, but not in any serious sense of the word. Lua is particularly well-suited to expressing configuration, since it was originally conceived as a data-description language. This data is converted into Lua table structures using Lua itself, using a sandbox so that it will not execute any dangerous stuff.

The first ‘real’ customization we will do is make a whole set of pages default to a particular node type. That is, any node like pages/GeneralInfo or pages/Introduction will be created as ‘Basic’ nodes.

If Sputnik cannot find a node, it will first attempt to find a node default with that name. The request for ‘pages’ results in Sputnik attempting to load a Lua module like so: require “sputnik.node_defaults.pages”. So we have to create a file pages.lua in a directory so that Sputnik can resolve this module.

Sputnik itself is distributed as a LuaRocks application. In my system, the Sputnik package sits at ~/sputnik/rocks/sputnik/9.03.16-0/lua – call this $SPUTNIK. The relevant package directory structure is:

    sputnik
        actions
        hooks
        node_defaults
        ....

It is possible to customize Sputnik by putting a suitable pages.lua in $SPUTNIK/sputnik/node_defaults, but that is a road of misery that will end in tears. However, it’s useful to acquaint yourself with the contents of this directory because it defines the standard namespaces.

The best quick solution is to add directories to the LUA_PATH environment variable before launching Sputnik:

  ~/sputnik$ export LUA_PATH=";;$HOME/sputnik/examples/?.lua"

Then create a directory ~/sputnik/examples/sputnik/node_defaults containing this file, pages.lua:

  module(..., package.seeall)
 
  NODE = {
     title="Pages",
     content = "",
     child_defaults = [[
          any='prototype = ""'
      ]],
  }

It is structured as a Lua module, which contains a single exported table, NODE . The definition of the field child_defaults may appear a little hairy at first, but it’s now time to get the three different ways to do string literals in Lua straight. Whether you use single or double quotes for a string, does not matter; it is a convenience so that you can embed the other kind of quotes: ‘prototype = “”‘. This string is then further embedded in a Lua ‘long string’ literal.

After restarting Sputnik, go to pages: nothing much to see at this point – but note that Sputnik can find the node. If you go to pages/Introduction you will get another blank page with a title, which you can edit directly as a Wiki page. The child_defaults field tells Sputnik that any ‘child’ of pages like pages/Introduction has a particular prototype value which corresponds to the ‘Basic’ node type.

If you enter this text

    Some text for the Introduction node. See [[pages/Basic]]

and save, the result will have a link to a new page, which you can in turn edit.

To see how Sputnik saves pages by default, look at the ~/sputnik/wiki-data directory. There will be a subdirectory pages%2FIntroduction with files 000001 and index. (The subdirectory is just the URL-encoded form of ‘pages/Introduction’ ). index will contain something like this:

  add_version{
  version   = "000001",
  timestamp = "2009-11-01 14:33:07",
  author    = "",
  comment   = "",
   ["minor"] = "",
   ["ip"] = "127.0.0.1",
  }

And 000001 contains the node text:

  title          = "pages/Introduction"
  category       = ""
  content        = [=[Some text for the Introduction node. See [[pages/Basic]]
 
  ]=]
  breadcrumb     = ""

As the node is edited, each revision is saved in a similar format, 000002, etc. In this way, edit history is managed in a simple way, easily readable by humans, as opposed to being stored in some relational database. However, using a database for storage is also supported by the Sputnik content manager, which is called Saci.

Custom Output

Before actually starting with code, it’s useful to have a handy shortcut to the Lua executable used by Sputnik. I suggest creating a little executable script like this on your path:

  $> cat ~/bin/slua
  ~/sputnik/bin/lua -lluarocks.require $*

Before we actually get round to generating dynamic content, here is a quick review of Cosmo, which is a powerful template engine used by Sputnik.

  ~$ slua

  Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
  > require "cosmo"
  > template = "$rank of $suit"
  > values = {rank="Ace",suit="Spades"}
  > = cosmo.fill(template,values)
  Ace of Spades

That’s useful, although only a little more friendly than Lua’s string.format function. However, Cosmo goes way beyond simple string interpolation.

  -- testcosmo.lua
  require "cosmo"
  template = [==[
  <h1>$list_name</h1>
  <ul>
   $do_items[[<li>$item</li>]]
  </ul>
  ]==]
 
  print(cosmo.fill(template, {
      list_name = "My List",
      do_items  = function()
          for i=1,5 do
             cosmo.yield { item = i }
          end
      end
  }
  ))

  ~$ slua testcosmo.lua

  <h1>My List</h1>
  <ul>
   <li>1</li><li>2</li><li>3</li><li>4</li><li>5</li>
  </ul>

Subtemplates are a powerful feature which makes generating HTML straightforward.

Now, we create a node which creates custom HTML output. First, put Memory.lua in your node_defaults directory:

  module(..., package.seeall)
 
  NODE = {
      title = "Lua Memory",
      content = "",
      actions = [[
          show="Memory.show_memory"
      ]],
  }

Create a directory actions (that is, ~/sputnik/examples/sputnik/actions) and put this Memory.lua in it:

  module(..., package.seeall)
 
  actions = {}
 
  local template = [=[
     <h2>Memory used by Lua is $mem kb</h2>
  ]=]
 
  function actions.show_memory (node, request, sputnik)
      node.inner_html = cosmo.f(template) {
          mem = ('%6.0f'):format(collectgarbage('count')),
      }
      return node.wrappers.default(node, request, sputnik)
  end

The convention is that any action functions are in the actions directory; these functions must be in a nested actions table. Visiting the node Memory will show the memory managed by Lua (as returned by the collectgarbage(‘count’) call).

The node’s inner_html is the source for the node’s display area, that is, not including the menu and all other frame decorations. node.wrappers.default is the actual function which generates the source for the whole page.

So, we now have a customized node with generated output. Please note that it does not appear in wiki-data; the node is fully ‘virtual’ and has no storage associated with it.

To take this example further, let us wrap the output of the ps command:

  ~$ ps aux --cols 256
  USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  ...
  sdonovan  2867  0.8  2.3  41972 15932 pts/1    Sl   10:47   3:57 /home/sdonovan/lua/scite/SciTE
  sdonovan  2902  1.2 12.4 179152 83160 ?        Sl   10:48   5:19 /usr/lib/iceweasel/firefox-bin -a firefox
  sdonovan  5993  0.0  0.4   5360  2844 pts/2    Ss   16:13   0:00 bash
  sdonovan  7118  0.0  0.3   5360  2024 pts/0    S+   18:03   0:00 bash
  sdonovan  7119  0.4  1.1   8384  7380 pts/0    S+   18:03   0:01 /home/sdonovan/sputnik/bin/lua -lluarocks.require /home/sdonovan/sputnik/rocks/sputnik/9.03.16-0/bin/sputnik.lua start-xavante sputnik.ws

(The –cols flag is necessary to prevent ps from thoughtfully truncating the line length to fit the terminal screen)

Chopping lines up is easy using sputnik.util.split – notice that it returns multiple values, not a table:

  ~$ slua

  Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
  > util = require 'sputnik.util'
  > = util.split('one two three','%s+')
  one     two     three

With this, a more exciting version of actions/Memory.lua can be written:

  module(..., package.seeall)
  local util = require 'sputnik.util'
  actions = {}
 
  local template = [=[
     <h2>Memory used by Lua is $mem kb</h2>
     <h2>Processes</h2>
     <table>
     <tr>
      <th>User</th><th>CPU</th><th>Mem</th><th>Command</th>
     </tr>
     $do_processes[[
          <tr>
          <td>$user</td><td>$cpu</td><td>$mem</td><td>$cmd</td>
          </tr>
     ]]
     </table>
  ]=]
 
  function actions.show_memory (node, request, sputnik)
      node.inner_html = cosmo.f(template) {
          mem = ('%6.0f'):format(collectgarbage('count')),
          do_processes = function()
              local user = os.getenv 'USER'
              -- this works on LInux, may need some mods for BSD/OS X.
              local f = io.popen('ps --cols 256 aux','r')
              f:read() -- not interested in column headers
              for line in f:lines() do
                  local fields = {util.split(line,'%s+')}
                  if fields[1] == user then
                      cosmo.yield {
                          user = fields[1], cpu = fields[3], mem = fields[4], cmd = fields[11],
                      }
                  end
              end
          end
      }
      return node.wrappers.default(node, request, sputnik)
  end

Now the node Memory is actually useful for the administrator of the website – but probably not a good idea to expose to the world!

Permissions

The node Memory should be restricted; only the Admin user should be able to view it. Also, it does not make sense to edit it, even as an administrator. actions/Memory.lua needs to have a permissions field:

  module(..., package.seeall)
 
  NODE = {
      title = "Lua Memory",
      content = "",
      actions = [[
          show="Memory.show_memory"
      ]],
      permissions = [[
          deny(all_users,all_actions)
          allow(Admin,show)
      ]]
  }

We start by prohibiting everything, and then only let Admin view the page. Notice that all of the usual little action icons on the right-hand side have disappeared.

To get back to the pages example; we may insist that only authenticated users can edit the pages. There are two kinds of solution to this, make it site-wide policy or only for children of pages. The first solution requires no code; as Admin, go to the @Root node, and choose the ‘configure’ action (which is usually the little gear icon next to ‘edit’ on the actions toolbar) Now open the ‘Advanced Fields’ section, and you can then edit the ‘Permissions’ field. By default, Sputnik comments out this line:

  -- deny(Anonymous, edit_and_save)

Remove the comment and save. Now anonymous users have lost their power to make edits anywhere on the site.

We’ve already seen with Memory how to enforce permissions for a single node. But how to do this for a group of nodes? Sputnik prototypes provide the solution. A prototype acts as the ‘type’ of a node. When a node is retrieved from storage, Sputnik copies default values from its prototype node. The basic prototype of all nodes is @Root.

Previously, the pages node has insisted that its children have an ‘empty’ prototype. The idea is to create an explicit prototype that will apply to the children, which will be called @pages. (By convention, all prototypes begin with ‘@’)

node_defaults/pages.lua becomes:

  module(..., package.seeall)
 
  NODE = {
     title="Pages",
     content = "",
     child_defaults = [[
          any='prototype = "@pages"'
      ]]
  }

and create a new file node_defaults/@pages.lua to define the @pages prototype:

  module(..., package.seeall)
 
  NODE = {
     content = "",
     permissions=[[
          deny(Anonymous,edit_and_save)
     ]]
  }

It does very little, just explicitly overrides the permissions.

Hooks

Continuing with the pages example, it would be useful if we could track the author of a particular page, defined simply as the user that first edited it. Also, we would like to track the creation date. So the prototype for all pages must include these new_fields and define a save hook which will be called when the node is saved:

  -- node_defaults/@pages.lua
  module(..., package.seeall)
 
  NODE = {
     content = "",
     permissions=[[
          deny(Anonymous,edit_and_save)
     ]],
     fields = [[
        author = {1.1}
        creation_time = {1.2}
    ]],
     save_hook = "pages.save_page"
  }

Create a directory ‘sputnik/hooks’ as before, and put pages.lua in it:

  -- hooks/pages.lua
  module(..., package.seeall)
 
  function save_page(node, request, sputnik)
      if not node.creation_time then
         local params = {}
         params.author = request.user
         params.creation_time = tostring(os.time())
         node = sputnik:update_node_with_params(node, params)
      end
      return node
  end

When a node’s save_hook field is set to MODULE.FUNCTION , then Sputnik will try to load sputnik.hooks.MODULE and use the FUNCTION defined by that module. In this case, the save hook is only interested in a new node, where the author and creation time fields have not been assigned yet. It uses the update_node_with_params method to write the new key/value pairs into the node.

Now, after saving pages/fred, the revision in the pages%2Ffred directory will look like this:

  title          = "pages/fred"
  category       = ""
  prototype      = "@pages"
  content        = [[Some content!
  ]]
  breadcrumb     = ""
  author         = "sdonovan"
  creation_time  = "1257157113"

This shows that we are saving the new information, although these fields are not displayed yet.

Wrapping up

Currently, pages is blank, and like Memory we can output something sensible by defining a ‘show’ action that will display a table of the existing pages.

If you are accustomed to regular Web frameworks, you are probably tempted to maintain and use a relational database at this point. Sputnik does not exclude that, you are free to store things as you wish, but it’s best to work with the underlying ‘document-oriented’ database machinery provided by Saci.

The existing pages can be found by querying Saci, which provides a get_nodes_by_prefix method. The prefix identifies the namespace for the nodes, which is pages in this case. This method returns a table where the keys are the identifiers (e.g. pages/fred) and the values are the node objects. Here is code that creates a list of nodes from this table, and sorts it by creation time.

  local function pages_in_order (sputnik)
      local pages = sputnik.saci:get_nodes_by_prefix 'pages'
      local res = {}
      for id,page in pairs(pages) do
          res[#res+1] = page
      end
      table.sort(res,function(p1,p2) return p1.creation_time < p2.creation_time end)
      return res
  end

node_defaults/pages.lua gets an actions field, just as with node_defaults/Memory.lua:

  actions=[[
      show="pages.show_pages"
  ]]

And actions/pages.lua will be:

  module (...,package.seeall)
 
  actions = {}
 
  local template = [=[
     <h2>Existing Pages</h2>
     <table>
     <tr>
      <th>Page</th><th>Author</th><th>Created</th>
     </tr>
     $do_pages[[
          <tr>
          <td><a href="?p=$id">$name</a></td><td>$author</td><td>$created</td>
          </tr>
     ]]
     </table>
  ]=]
 
  local function pages_in_order(sputnik)
  ....
  end
 
  function actions.show_pages(node, request, sputnik)
      node.inner_html = cosmo.f(template) {
          do_pages = function()
              local pages = pages_in_order(sputnik)
              for _,page in ipairs(pages) do
                  cosmo.yield{
                      id = page.id,
                      name = page.id:gsub('pages/',''),
                      author = page.author,
                      created = sputnik:format_time(page.creation_time,"%d %b %Y")
                  }
              end
          end
      }
      return node.wrappers.default(node, request, sputnik)
  end

Beyond the Basics

If you are curious about how Sputnik creates the whole page, note that node.wrappers.default is usually implemented by the wrappers.default function in sputnik/actions/wiki.lua which does a Cosmo expansion of the node.html_main template (see NODE.html_main in sputnik/node_defaults/@Root.lua)

If you don’t like a visual feature, then it is often easy to remove it. For instance, editing @Root as Admin (using @Root.configure as before) you can open up the ‘HTML Fields’ and modify the templates directly. Removing the menu bar is as easy as clearing out the text in the ‘Menu’ field.

For a good overview of Sputnik permissions, see this list question and its reply.

The next part of this tutorial will deal with further customizations, like internationalization, providing a custom form for editing a node’s fields, and using the built-in @Collection prototype to simplify the common pattern of groups of content nodes. And Sputnik’s ability to create custom actions will change the way you look at file extensions forever.