Learning Erlang, a UNIX developer's perspective

Erlang is destined for great things in the age of multi-core, due to its unique process loving architecture and functional nature. In this article, you can learn the basics about programming in Erlang.

Share:

Noah Gift , Associate Director Engineering, ATT Interactive

Photo of Noah Gift

Noah is an experienced technical leader and software developer. He solves interesting problems in a variety of languages including Python/Iron Python, Erlang, F#, C#, and Javascript, and is a member of the Python Software Foundation. He is the associate director of engineering at ATT Interactive and has worked at a variety of companies around the world, including: Caltech, Disney Feature Animation, Sony Imageworks, and Weta Digital.

He is the co-author of Python For UNIX and Linux System Administration, and numerous technical articles for IBM developerWorks, O'Reilly, Red Hat Magazine, and Mac Tech. In his free time, he composes piano music and runs marathons and ultramarathons. Links to much of his writing can be found at his personal site at http://noahgift.com. He has a bachelor's of science in nutritional science from Cal Poly San Luis Obispo, a master's of science in computer information systems from CSULA, and is an MBA candidate at UC Davis specializing in business analytics, finance, and entrepreneurship.



21 June 2011

Also available in Chinese

Introduction

If you are a UNIX developer or systems administrator, you probably have a favorite tool or set of tools. Perhaps your go to tool is Perl, Python, sed and awk, or Ruby. Alternately, you might belong to the Java™ or C++ clan. Once you gain a certain familiarity with a set of tools, it can become very difficult to justify spending the time to learn a new language. There needs to be a compelling reason to justify the effort. Erlang is compelling enough to learn. In this article, I am going to tell you why it is so compelling and how to get started.

The age of multi-core systems has already happened, and largely, we aren't ready for it. Many of our languages are dependent on fragile concurrency mechanisms, like threads with shared state, or possibly worse, a global lock that limits execution to a single thread on a single core. A typical commodity nix server has 24 cores. It is easy to write overly complex and error prone threaded code or code that only uses one core on a 24 core box, yet is still CPU bound. One solution to this problem is to use a functional language designed to scale. Erlang doesn't have mutable state, and this is a dramatically different decision then the majority of popular languages used by nix professionals. In addition, concurrency is in it's blood. In fact, many other languages are built around objects, where Erlang is built around processes. Spawning thousands or millions of processes is quite normal, because the languages controls the concurrency mechanism not the operating system. In Erlang, things are different, and it happens to be different in exactly the right way for the computing challenges of today.

Interactive Erlang shell

Command line gurus will appreciate Erlang's interactive shell. With the interactive shell you have the ability to type expressions, compile code, and communicate with processes. It contains the same type of elegance in design I feel when administering a *nix machine with a Lights Out Management card, and a sophisticated virtualization system. I will begin by assuming you have Erlang installed on your system. If you don't you will need to consult the Resources section for information on how to download and install it. Once Erlang is installed, you will find "erl" in your path. When you type in "erl", you will drop into the interactive prompt, where you can type commands.

Interactive shell: Part A
lion% erl
Erlang R14B 
(erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.1  (abort with ^G)
1>

To get a Help menu, execute the help command as shown below. Note, you must always end each expression with a period. This tells the interpreter to execute it. Also, as a quick note, you execute at any time by typing "q().", which is shown in the help menu output below.

Interactive shell: Help output
Eshell V5.8.1  (abort with ^G)
1> help().
** shell internal commands **
b()        -- display all variable bindings
e(N)       -- repeat the expression in query <N>
f()        -- forget all variable bindings
f(X)       -- forget the binding of variable X
h()        -- history
history(N) -- set how many previous commands to keep
results(N) -- set how many previous command results to keep
catch_exception(B) -- how exceptions are handled
v(N)       -- use the value of query <N>
rd(R,D)    -- define a record
rf()       -- remove all record information
rf(R)      -- remove record information about R
rl()       -- display all record information
rl(R)      -- display record information about R
rp(Term)   -- display Term using the shell's record information
rr(File)   -- read record information from File (wildcards allowed)
rr(F,R)    -- read selected record information from file(s)
rr(F,R,O)  -- read selected record information with options
** commands in module c **
bt(Pid)    -- stack backtrace for a process
c(File)    -- compile and load code in <File>
cd(Dir)    -- change working directory
flush()    -- flush any messages sent to the shell
help()     -- help info
i()        -- information about the system
ni()       -- information about the networked system
i(X,Y,Z)   -- information about pid <X,Y,Z>
l(Module)  -- load or reload module
lc([File]) -- compile a list of Erlang modules
ls()       -- list files in the current directory
ls(Dir)    -- list files in directory <Dir>
m()        -- which modules are loaded
m(Mod)     -- information about module <Mod>
memory()   -- memory allocation information
memory(T)  -- memory allocation information of type <T>
nc(File)   -- compile and load code in <File> on all nodes
nl(Module) -- load module on all nodes
pid(X,Y,Z) -- convert X,Y,Z to a Pid
pwd()      -- print working directory
q()        -- quit - shorthand for init:stop()
regs()     -- information about registered processes
nregs()    -- information about all registered processes
xm(M)      -- cross reference check a module
y(File)    -- generate a Yecc parser
** commands in module i (interpreter interface) **
ih()       -- print help for the i module
true
2>

It would be very instructive to type out all of these commands and see what it is they do. In moving on to the actual aspect of writing code, remember that state is not mutable in Erlang, so it may be surprising to see the output below when setting a variable:

Interactive Shell: Part B
Eshell V5.8.1  (abort with ^G)
1> Var = 1.
1
2> Var = 2.
** exception error: no match of right hand side value 2
3>

Of course, a "Hello World" example is always in order when learning a new language:

Interactive Shell: Part C
4> io:format("Hello World~n").
Hello World
ok

There are some more things to cover in a comprehensive lesson on Erlang, but let's finish off with the workhorse data type list:

Interactive Shell: Part D
5> List = [1,2,3].
[1,2,3]
6> [Head|Tail] = List.
[1,2,3]
7> Head.
1

In that example, pattern matching was used to extract both the head and the tail of the list. Pattern matching plays a big role in Erlang, it would be useful to read more about it in some of the Resources provided at the end of the article.

Writing and compiling an Erlang Module

Writing an application in Erlang involves writing actual modules, not just the expressions shown in the interactive shell. First a quick note about a development environment. If you plan on getting into heavy duty Erlang development you may want to consider using Eclipse with the Erlide plugin, or Emacs. Alternately, for lightweight programming with Erlang, vim works, quite well.

Eclipse with Erlide plugin
Screen shot of Eclipse with Erlide plugin

If you are using an IDE like Erlang with Erlide, it will automatically compile your modules code into beam files as you write it, which is quite convenient. Alternately, compilation can take place inside of the Erlang interactive interpreter or through the erlc command line compiler. Below is an example of a simple module, which compiles down to a beam file:

fingerprint.erl
%% This is a module that gets the operating system type
-module(fingerprint).
-export([get_os/0]).

get_os ()->
    os:cmd("uname").

To compile this interactively, and run it, run erl again:

Compiling module interactively
lion% erl                                    
Erlang R14B 
(erts-5.8.1) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.1  (abort with ^G)
1> c(fingerprint).
{ok,fingerprint}
2> fingerprint:
get_os/0       module_info/0  module_info/1  
2> fingerprint:get_os().
"Darwin\n"
3> q().
ok

If you do a direct listing from the shell you see that a fingerprint.beam file was created. Alternately, the beam file could have been created by running:

lion% erlc fingerprint.erl

Unit testing Erlang with EUnit

If you are writing Erlang code, chances are you want to build a high availability system. If that is the case, it is important to pay attention to detail from the beginning and write unit tests for your code. In building from the previous contrived module, here is what a test would look like:

EUnit example
%% This is a module that gets the operating system type
-module(fingerprint_with_test).
-include_lib("eunit/include/eunit.hrl").
-export([get_os/0]).


get_os ()->
    os:cmd("uname").

get_os_test ()->
    get_os().

The main take-aways are that to test a module with EUnit, you need to include the header, as previously shown, and name a function that matches the pattern "_test". If you do this, all your test functions will be automatically exported and run via the test function shown below.

EUnit interactively compiled and tested
Eshell V5.8.1  (abort with ^G)
1> c(fingerprint_with_test).
{ok,fingerprint_with_test}
2> fingerprint_with_test:test().
  Test passed.
ok
3>

Using Erlang as a scripting language or from the command line

It is also worth noting that you can run Erlang in a script mode without needing to first compile it. To do this, you will need to run a copy of Erlang at R11B4 or greater. We can slightly modify the example from above and turn it into a script by adding a shebang line and a main function:

EUnit interactively compiled and tested
#!/usr/local/bin/escript

get_os ()->
    os:cmd("uname").

main(_) ->
    io:format("OS Platform: ~p~n", [get_os()]).

Here is the output:

lion% chmod +x fingerprint.escript
lion% ./fingerprint.escript  
OS Platform: "Darwin\n"

Finally, you can also write one liners from the command line, just like Perl or Ruby. That same example looks like this as a one liner:

lion% erl -eval 'io:format("OS Platform: ~p~n", [os:cmd("uname")])' -noshell -s init stop
OS Platform: "Darwin\n"

Distributed computing made easy: Remote messaging between Mac OS X and Ubuntu

Cloud computing: A compelling new paradigm

Cloud computing has already proven to be a compelling new paradigm, and one of its advantages is in leveraging abstractions that are designed to scale. Erlang has something called Location Transparency that applies a similar benefit. What this means is that sending a message to a process on a local machine is no different then sending a message to a process on another machine. This is one of Erlang's killer competitive advantages. Location Transparency makes it irrelevant, and transparent, where a process lives. This is worth a pause. Really think about that for a second.
One of the ways this is achieved is through asynchronous message passing, where state is never shared between processes. To send a message to a local process on your machine you would type:

Pid ! "Interprocess Messaging is Fun"

To send a message to a remote process is the exact same code, because all of the information necessary to send a message is embedded in the message itself.

With some of the fundamentals out of the way, we can get into the really fun stuff…building a cluster of processes that talk to each other. Because of Location Transparency, it is easy to test a cluster on a single machine, and then move it into a large distributed cluster where machines could live all over the world! In the following, I start Erlang on both Ubuntu (via an Virtual Machine) and Mac OS X, on my laptop and run code remotely in just a couple lines of code. Note, for this example, I am assuming issues like local DNS and firewall settings, if any, are configured correctly.

First start Erlang on both Ubuntu and OS X nodes:

mac% erl -name mac -setcookie test
ubuntu% erl -name mac -setcookie test

Once you do this setup communication from the OS X machine and verify you can see the remote node by calling the nodes function:

(mac@lion.local)1> net_adm:ping('ubuntu@ubuntu.localdomain').
pong
(mac@lion.local)2> nodes(). 
['ubuntu@ubuntu.localdomain']

Now you can execute arbitrary code:

(mac@lion.local)15> rpc:call('ubuntu@ubuntu.localdomain', os, cmd, ['uname -a']). 
"Linux ubuntu.local 
2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 01:41:57 UTC 2010 i686 GNU/Linux\n"

It takes a bit to sink in how powerful this actually is, and how trivial it is to do Interprocess Communication in Erlang. There is much more to go into past this trivial example, but this should give you a good idea of the true power of Erlang in action.


Conclusion

This article went into the theory, and practice of working with Erlang, but there is much, much more to learn. The increase of large scale production systems written in Erlang is bound to grow dramatically in coming years. In a recent Harvard Business Review article, they mention that "scarcity" is the number one key to innovation. What is scarce now, is people experienced in dealing with highly current and scalable systems. This is the time to gain a competitive advantage and become an expert in the innovative Erlang language.

If you are interested in learning more Erlang, I would recommend the following advice. Read one, or all preferably, of the Erlang books, there are three newly published books to choose from. They are all quite good. In addition, work your way through the interactive tutorial at tryerlang.org, which is referenced in the resources section. Finally, try out some of the Erlang projects that are running in the wild, like CouchDB, RabbitMQ, and Ejabbrd.


Download

DescriptionNameSize
Learning Erlang codehttp://public.dhe.ibm.com/software/dw/aix/learningerlangcode.zip2KB

Resources

Learn

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=681700
ArticleTitle=Learning Erlang, a UNIX developer's perspective
publish-date=06212011