Thursday, March 22, 2007

Why use a standard builsystem? (Well structured OTP applications)

Why use a standard build system? Over here at Erlware we have been using Erlang for quite some time now. I personally was introduced to Erlang in 1999, I think the first version of ERTS that I ran was R6. My first project was layed out according to my best guess at a good directory structure. The project contained a home grown supervisor of sorts, which if I recall correctly was about 1000+ lines of Erlang doing all kinds of app specific stuff. There was not a gen_server to be found in the whole project and starting the app required compiling C code that fork exec'd ERTS (I am proud to say that the app ran better than 99 percent of the C code at that company). Upon completion of that first application more were commissioned, they all had different directory structures, each marginally better than the first, each had varying degrees of OTP compliance. After about 3 or 4 of these one-offs the reality of the situation started to sink in, this was a mess. Each application had to be started a different way, each application had its artifacts such as source, object files, and configuration in a different place, and each application required support personnel to be aware of all these differences. The more apps that were written the harder support became, a standard was needed - the question was 'what standard?'. The anwser was OTP, that IS the standard.

OTP is an entire specification, handed down from those who know Erlang the best, detailing how to structure an application, start it, stop it, ensure fault tolerance, etc. OTP covers aspects from directory structure, to application startup and configuration, right down to actual abstractions used in the code. Often times you will hear the following reasons for why the gen_server behaviour should be used:

1. All the generic code has been abstracted away and well tested. This simplifies things and provides the additional stability afforded by well tested code.
2. It is a standard way of structuring code, anyone that knows gen_server can more readily understand code structured this way than they could free form code.
3. It follows OTP idioms and so hooks into the OTP framework which affords you even more well tested functionality for free.

The above reasons are not just reasons for using a single behaviour, they are reasons for writing well structured OTP applications period. Well structured OTP applications afford you the benefit of having much of the generic behavior of your system abstracted away in a well written and well tested framework. OTP applications allow others that are familiar with the idioms that OTP employs the ability to quickly understand your application's structure and behavior. OTP structuring also offers consistency between projects which allows others to depend on that structure, whether they be internal framework designers, other developers, or application support personnel. Consistency is key.

Ensuring that applications are well structured are where standard build systems come into play. A standard build system will automatically generate and validate the structure of an application or collection of cooperating applications (project). This means that a good system not only ensures consistency but it enables consistency by making it easy to create the structure of an application or project.

The combination of well structured OTP applications, releases and a good build system that enables their creation and maintenance from development to production is a massive effort saver. Whether you are a single developer or an enterprise with 20 teams these standards are a huge win. That's why Ericsson created OTP, and why Erlware provides build tools.

At Erlware we have been working to get the next generation of our software out the door. OTP Base has been at the center of our software offerings. This June things will change dramatically. In an effort to make OTP standards more accessible and OTP development even simpler we will be releasing a trio of open source functionality:

1. Sinan, an all Erlang build system complete with support for builds, dependency management, static code analysis, and edoc (to name a few) out of the box.
2. A community driven back end repository for hosting community OTP applications and releases.
3. Faxien, a release system that allows users to publish, download, and install OTP applications and releases.

In this way we hope to extend standardization and OTP practices with improvements on our build systems and through functionality that extends further into the life cycle of applications. As we lead up to our release in June we will be publishing more information.


Masklinn said...

Thanks for the post, it was interesting, but I had a questions:

When you're talking about "a well structured OTP application", do you mean "an application following the OTP Design Principles (" (especially from chapter 7 onwards I guess) or is there more to it?

Martin said...

Yes, that is correct, though the chapters that precede 7 are very valuable and as much a part of OTP as those that follow it. Those standards have been used with success in some huge applications inside and outside of Ericsson. The next release of Erlware takes those standards and uses them as the groundwork for building as well as deploying and maintaining OTP structured applications and releases. Whether you choose to use tools like that or not adhering to proven standards and using proven code will enhance the quality of your erlang software. I can definitely attest to that.

Anonymous said...

It is gr8 But the tutorial on the site is so out dated that it is impossible to understand how it should work

Anonymous said...
This comment has been removed by a blog administrator.