I also can’t help viewing those VMware ‘appliance’ things with suspicion, and I’m starting to put real data on these things now, so I went for a clean install from debian-40r1-i386-netinst.iso. Doing the initial setup via the VMware Server console, viewed via VNC, is unbelievably painful but all that was needed after the actual install was to reconfigure the networking and install sshd before getting back to a nice clean SSH login.

I’m slightly cross with myself for not having scripted the rest of the setup yet, but on the other hand I’m pleased I was anal enough to document it all in the first place, because all I had to do was copy the commands out of my previous posts here and paste them into the SSH window to get back to where I was. I also set up Debian to use sudo, because I prefer that to using su.

The only differences in all the setup steps from the previous ‘doing X on Ubuntu’ posts due to the change of distribution were:

• The m4 package was already installed.
• Instead of libssl-dev, I needed openssl.
• I left out the detail of how to fix ejabberdctl. (added as a comment to that post)

I didn’t go as far as the Mnesia data import stage, stopping instead after installing and testing Erlyweb, for reasons that will become apparent later. For the record, the steps were:

• Erlang
• Ejabberd
• Jabberlang
• YAWS
• ErlyWeb

I stayed up far too late last night playing with Erlang and friends – much longer than planned due to a few hitches. I did manage to complete more or less what I’d planned in the end though, so here is the documentation.

The initial environment is that built up over my previous posts with one slight change – I had to switch to revision 171 of ErlyWeb from SVN, as the Mnesia driver is temporarily broken in more recent revisions.

Step one was to create the Mnesia database and import some data I already had in XML format. After a bit of head-scratching and cursing, I came up with djimport.erl. It’s not all pleasant (see the last function, which was the source of most of the cursing until I gave in and committed that horrific crime), but it did suck in 6714 ‘record’ records quickly and without even complaining about the awful choice of record name. I used the following Erlang command-line to run it:

erl -s djimport -sname dj -mnesia dir '"db"'

Step two was to extend the ErlyWeb app created in the previous post by adding a new component, as follows:

erl -pa /home/ciaran/erlyweb/ebin
1> erlyweb:create_component("record", "/home/ciaran/dj").

Finally, all I had to do was to update my dj startup script to include the same -sname dj -mnesia dir ‘”db”‘ on the command-line, and put the following two lines at the beginning of dj.erl:

mnesia:start()
mnesia:create_table(counter, [{disc_copies, [node()]},
              {attributes, [key, counter]}])

The end result was the ability to navigate and edit the database in a browser (see screenshot), albeit with a few issues that I’ll sort out next time.

All I’m documentating at this stage is the initial setup of a skeleton app, which starts with getting and building ErlyWeb:

svn co http://erlyweb.googlecode.com/svn/trunk/ erlyweb
cd erlyweb
mv Emakefile Emakefile.old
sed -e "s*/usr/local/src*/usr/local/lib*" Emakefile.old >Emakefile
make
cd ..

Note that I amended the Erlang makefile to point at the location of my installed YAWS includes from an earlier step. Now, back in the home directory, start Erlang with the following command line:

erl -pa erlyweb/ebin

In the Erlang shell, do this to create the skeleton application:

1> erlyweb:create_app("dj","/home/ciaran").
2> halt().

A ‘dj’ directory was created above with the skeleton ErlyWeb application structure in place. Add two extra files in that directory as follows. Firstly a small Erlang program which runs the app with YAWS in embedded mode – dj.erl. It’s just a slight extension of the embeded YAWS test code from last time:

-module(dj).
-export([start/0]).
 
-include("/usr/local/lib/yaws/include/yaws.hrl").
 
start() ->
    erlyweb:compile("/home/ciaran/dj",[{erlydb_driver, mnesia}]),
    application:start(yaws),
    GC = yaws_config:make_default_gconf(false,"dj"),
    SC = #sconf{port = 8001,
                servername = "ubuntu01",
                listen = {0,0,0,0},
                docroot = "www",
                appmods = [{"/", erlyweb}],
                opaque = [{"appname","dj"}] },
    yaws_api:setconf(GC, [[SC]]).

And secondly a script to compile and start it, called simply dj:

#! /bin/sh
erlc dj.erl
if [ $? -ne 0 ]; then exit; fi;
erl -pa /usr/local/lib/yaws/ebin /home/ciaran/erlyweb/ebin -yaws embedded true -s dj

Now to start things up:

chmod 770 dj
./dj

All being well, hitting port 8001 with a web browser should show the skeleton application, which doesn’t do very much at this stage. The choice of the name ‘dj’ for this first application indicates that I’m planning to replace the back-end of my inexplicable DJ software with this. It’s unlikely to be anything to get excited over, unless you’re me.

Another dependency first:

sudo apt-get install libpam0g-dev

Now we can get, build and install YAWS from SVN. Note that as before I’m grabbing a specific revision so I have a stable and repeatable setup:

svn -r 1129 co https://erlyaws.svn.sourceforge.net/svnroot/erlyaws/trunk/yaws
cd yaws
./configure
make
sudo make install

To use YAWS in embedded mode, you need to start up Erlang with something like the following:

erl -pa /usr/local/lib/yaws/ebin -yaws embedded true

Finally, here’s a very small test program that just serves static files from the ‘wwwtest’ subdirectory of the directory you run from. Before running this, you need an empty directory .yaws/yaws/yawstest off your home directory. To be honest I don’t know why yet as the parameter is newly added and (surprise surprise) undocumented, but the final part corresponds to the second parameter to make_default_gconf.

-module(yawstest).
-export([main/0]).
 
-include("/usr/local/lib/yaws/include/yaws.hrl").
 
main() ->
    application:start(yaws),
    GC = yaws_config:make_default_gconf(false,"yawstest"),
    SC = #sconf{port = 8001,
		servername = "ubuntu01",
		listen = {0,0,0,0},
		docroot = "wwwtest"},
    yaws_api:setconf(GC, [[SC]]).

Stick some documents in wwwroot, run the above and test away.

w3cdtf(GregSecs) ->
	Date=calendar:gregorian_seconds_to_datetime(GregSecs),
	[UDate|_]=calendar:local_time_to_universal_time_dst(Date),
	USecs=calendar:datetime_to_gregorian_seconds(UDate),
	{{Y, M, D},{HH, MM, SS}} = Date,
	io_lib:format("~4..0b-~2..0b-~2..0bT~2..0b:~2..0b:~2..0b~s",
		[Y,M,D,HH,MM,SS,tzid(GregSecs-USecs)]).
 
tzid(Secs) when Secs==0 ->
	"Z";
tzid(Secs) ->
	{HH,MM,_}=calendar:seconds_to_time(abs(Secs)),
	io_lib:format("~c~2..0b:~2..0b",
		[case Secs >= 0 of false -> $-; true -> $+ end,HH,MM]).

As well as being shorter and clearer still, it doesn’t use the calendar:time_difference like the original did. That’s good because not only is that function very silly, it turns out it is marked as obsolete in the documentation. I suspect those two points are not unrelated.

Anyway, I’ll probably give the above some more rigourous testing and submit a patch. It may seem rather trivial, but actually it was a good exercise in getting to know Erlang a bit better. Half an hour of practical problem is usually worth ten of reading TFM.

less_shit(GregSecs) ->
	Date=calendar:gregorian_seconds_to_datetime(GregSecs),
	{{Y, Mo, D},{H, Mi, S}} = Date,
	[UDate|_] = calendar:local_time_to_universal_time_dst(Date),
	{DiffD,{DiffH,DiffMi,_}}=calendar:time_difference(UDate,Date),
	i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ tzid(DiffD,DiffH,DiffMi).
 
tzid(DiffD,DiffH,DiffMi) when DiffH < 12,  DiffH /= 0 ->
	"+" ++ add_zero(DiffH) ++ ":" ++ add_zero(DiffMi);
tzid(DiffD,DiffH,DiffMi) when DiffH > 12,  DiffD == 0 ->
	"+" ++ add_zero(DiffH) ++ ":" ++ add_zero(DiffMi);
tzid(DiffD,DiffH,DiffMi) when DiffH > 12,  DiffD /= 0, DiffMi /= 0 ->
	"-" ++ add_zero(23-DiffH) ++ ":" ++ add_zero(60-DiffMi);
tzid(DiffD,DiffH,DiffMi) when DiffH > 12,  DiffD /= 0, DiffMi == 0 ->
	"-" ++ add_zero(24-DiffH) ++ ":" ++ add_zero(60-DiffMi);
tzid(DiffD,DiffH,DiffMi) when DiffH == 0 -> "Z".

The point to note is that no complex expression is repeated multiple times, and nor is the actual logic of generating the timezone identifier buried within one. It’s much clearer, and much shorter.

Another point to note is that it is still shit, and I make no apologies for that, as firstly I wanted to keep the methodology the same, and secondly I am still getting to grips with Erlang.

%%%
%%% Create W3CDTF (http://www.w3.org/TR/NOTE-datetime) formatted date
%%% w3cdtf(GregSecs) -> "YYYY-MM-DDThh:mm:ssTZD"
%%%
w3cdtf(GregSecs) ->    Date = calendar:gregorian_seconds_to_datetime(GregSecs),
    {{Y, Mo, D},{H, Mi, S}} = Date,
    [UDate|_] = calendar:local_time_to_universal_time_dst(Date),
    {DiffD,{DiffH,DiffMi,_}}=calendar:time_difference(UDate,Date),
    w3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi). 
 
%%%  w3cdtf's helper function
w3cdtf_diff(Y, Mo, D, H, Mi, S, _DiffD, DiffH, DiffMi) when DiffH < 12,  DiffH /= 0 ->
    i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ "+" ++ add_zero(DiffH) ++ ":"  ++ add_zero(DiffMi);
 
w3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi) when DiffH > 12,  DiffD == 0 ->
    i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ "+" ++ add_zero(DiffH) ++ ":"  ++
        add_zero(DiffMi);
 
w3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi) when DiffH > 12,  DiffD /= 0, DiffMi /= 0 ->
    i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ "-" ++ add_zero(23-DiffH) ++
        ":" ++ add_zero(60-DiffMi);
 
w3cdtf_diff(Y, Mo, D, H, Mi, S, DiffD, DiffH, DiffMi) when DiffH > 12,  DiffD /= 0, DiffMi == 0 ->
   i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ "-" ++ add_zero(24-DiffH) ++
        ":" ++ add_zero(DiffMi); 
 
w3cdtf_diff(Y, Mo, D, H, Mi, S, _DiffD, DiffH, _DiffMi) when DiffH == 0 ->
    i2l(Y) ++ "-" ++ add_zero(Mo) ++ "-" ++ add_zero(D) ++ "T" ++
        add_zero(H) ++ ":" ++ add_zero(Mi) ++ ":"  ++
        add_zero(S) ++ "Z".

I’m sure I’m would be offending a number of people, were they to read this, but nonetheless I will say it – that’s absolutely shocking, and it’s fairly typical of the project in general. At some points in the source, there are slight redeeming features of some actual documentation headers above some of the functions, but they’re few and far between and incomplete when they do occur. All in all I’d say it either needs some dramatic clean-up and refactoring, or a good sprinkling with parmesan and black pepper to make a meal for a tramp’s dog.

I very much doubt that this is a YAWS issue, because the underlying OTP framework smells very similar, but I do live in hope of seeing a “well engineered” Erlang project. On the other hand, perhaps my assessment of the YAWS code is wrong. Disagreements are welcome.

Anyway, for now I am sticking with my opinion that it’s the “Erlang way of doing things” that’s interesting (and I mean the high-level design concepts obviously), and not Erlang itself.

*It’s somewhat difficult to verify the ‘exactly’ actually – suffice to say they ought to do the same thing. The above example came from yaws_rss.erl. See yaws.erl and yaws_log.erl for other examples.

Erlang is a functional programming language designed specifically to deal with the issues of concurrent and fault-tolerant distributed systems. It was conceived and developed at Ericsson, and to this day finds its primary application in telephony switches and the like. However, the characteristics that made it useful in this area are rapidly becoming more relevant to ‘mainstream’ computing with the rise of technologies such as the internet and multi-core CPUs, for example.

In general, the main thing that surprised me is the very strong smell of ad-hoc ‘hackishness’ around the language. In the past, I’ve created countless domain-specific languages for various purposes and something about Erlang reminds me of many of these. This isn’t really a criticism, I find it rather endearing, but it doesn’t really bode well for the longevity of the language (yes, I know it’s 20 years old already) or its widespread adoption.

I like the terse elegance of it as well. The more you get into the language, the more elegant it becomes. I would love to unleash this language on developers I’ve worked with who favour a complex class structure (“We should use the MultiTentacledSquid Pattern” ; “Why?” ; “Dunno”) over a ‘few lines’ (ish!) of straightforward code and think that ThisIsASensibleWayToNameAVariable.

However, while I’m sure Erlang is addressing the needs of the future, and was doing so way before its time, I am not at all convinced it is a development platform of the future. I tend to think it will remain more or less within its current niche, although attempting to contradict me are several conspicuous counterexamples* to this already.

The defining features of Erlang though, will definitely be part of those future platforms, and I see these as being the cheap processes and lightweight interprocess messaging. To my mind, much of the rest of the touted benefits of Erlang come for free with these in place. I’m talking about fault-tolerance, hot-swapping (clumsy in Erlang if you ask me), scalability and distributability (if that’s even a word), for example. That’s assuming you make use of your cheap processes properly of course – it’s more about methodology than anything else I think. There’s no reason you can’t take any language/platform and develop a system in ‘the Erlang way’, but it would be likely to perform very badly. It’s the presence of those two key features that make the methodology practical.

Anyway, that’s enough for now. I will continue to play with Erlang a bit more, but I’m pretty close to the point where I would need to develop something substantial and purposeful to learn much more, which will definitely have to wait for a less busy time.

*One good example is probably ejabberd, a Erlang-based XMPP server. I run a few installations of OpenFire (Java-based) currently, whcih is very well featured and stable, but I use it under relatively small loads. Despite having no evidence to back this up, I’d be willing to bet ejabberd would totally wipe the floor with it when it comes to dealing with heavy traffic.

Update

Here is an example with some evidence. Yaws is an Erlang web server, and here are the results of a face-off between Apache and Yaws. Rather convincing I think. I’ll be looking more closely at Yaws at some point.