[ | smalltalk dot org ].
Community and industry meet inventing the future.
The Case for First Class Messages
posted by Peter William Lount
version 1, 20040518

Patrick Logan wrote
"I am not sure where the line is with Smalltalk. What is programming vs. metaprogramming in Smalltalk? I don't know. It's Smalltalk all the way down for Smalltalk-80 based systems, until you get to a fairly thin layer of magic that varies by implementation."

If you are used to special and distinct "meta" grammar and syntax in the languages you are using I can see why you'd be confused where the line is. The line is very clear in Smalltalk.

Programming and meta programming in Smalltalk are essentially the same with the differences being which classes you are working with. This is a result of the fact that you use the regular Smalltalk syntax to perform metaprogramming as all the accessible meta data are first class objects.

This also has the effect of making meta programming very accessible to all programmers using the language. All you need to do is learn and use a few additional classes and you are on your way. No special or cryptic syntax to learn.

I assume by "thin layer of magic" you are referring to the virtual machine? Most modern languages have virtual machines or a "model of execution" that implements the lowest level operations required to execute the language.

Yes, the many varied versions of Smalltalk have different implementations of the virtual machine and the class libraries - they do so for various reaons, the primary of which is different target auidences and purposes.

Do language such as C, C++, Java, Phython, PERL, ML, LISP, Forth, and others have "thin layers of magic"? Yes, of course they do, although for some of these langauges the "thin layer" is a rather "thick layer of magic" that is more often hidden than not.

One reason for "hiding" these "lower level layers" is that much software is still closed source and proprietary. Another is that often these lower layers are written in a different lanugage, i.e. C, for portability reasons.

What does this all mean? It seems, needless to say, very obvious that any language needs some means of having the lowest level operations that enable it to do it's work expressed somehow. There are different approaches to this.

While there are some hand written C files in the Squeak Smalltalk virtual machine the bulk of the virtual machine is written in Smalltalk itself and translated into C. This enables development of the virtual machine to actually be debugged in the system itself. Not many languages can say that. This also points to how "thin" the "layer of agic" is in Smalltalk's case.

Other Smalltalk's take a different approach. IBM's VisualAge Smalltalk is entierly written in Smalltalk (although there might be a bit of hand written assembly for each version) with their own native code compilier generating the binary code.

Have you ever tried understanding the GNU GCC compiler source code for C and C++? It's a mountain of obscured complexity. It would certainly not be easy to classify it as a thin layer, it's more of a very thick and twisted layer of "uber magic" for these languages. Yet it works.

Computer languages "embed" special "magical layers" of core operations into their "compilier" and the "run time library". What other choice is there? The key is how big is this lower area?

Smalltalk, and other systems, have attempted to keep this "thin layer of magic" small for various design reasons. Many langauges are now written in themselves and bootstrapped into existance.

Slate Smalltalk is attempting to eliminate the idea of a virtual machine altogether. This might have the effect of making this thin layer of magic even thiner.

One of the advantages of the Smalltalk syntax is that it's minimal. The complexity of many special operations and special grammatical structures are simply not present. Either they don't exist since they are not needed or they are shifted to the system library of objects often as first class meta data. Two examples are "binary operators" and "control structures".

Binary operations in Smalltalk are simply "binary message sends" and are thus "first class message objects". This enables all of the "magic" that implements these operations to be shifted from the hidden compiled code of the language to the visible code of the library of objects.

All, yes ALL, control structures in Smalltalk are implemented using Block Contexts (closures). This includes all conditionals, loops, breaks, and exceptions. This shifts the "magic" needed to implement these "language structures" from the compiler to the library of objects. The innovative approach of using "closures" for all control structures means that ANY user can extend the available set of control structures at any time simply by implmeenting methods in, usually, just a few places in the system library of objects.

In the eighties I added "nil" control structures to the langauge since I was tired of using "anObject isNil ifTrue: [...] ifFalse: [...]" all the time. I wanted to say "anObject ifNil: [...] ifNotNil: [...]" which is much easier on my brain (and my fingers) since I'm not having to switch from "existance logic (i.e. nil or not nil)" to "boolean truth logic" all the time. This "existance" control structure improves the clarity of programs immensely by reducing the amount a human brain has to comprehend to read it.

It's interesting to note that human nature being what it is many other Smalltalkers resisted this change for a long time. Finally all Smalltalk implmentations have added this nil control stucture, so I don't have to add it in my self when using a different version.

There were four methods in this new "nil constrol structure" protocol and they only needed to be added to the classes Object and UnknownObject (the class for nil) for a total of eight new methods! No recompilation of the system was needed and I started using them immediately! I could also see the source code and modify it at any time.

Subsequently I've added many new and powerful control structures to "Smalltalk" that extends it's expressive power. Sometimes I've thought of new and useful control structures and added them in the midst and fury of writing other code. On of the primary advantages of these new control structures is that they provide "linguistic" and "semantic" meaning that can be debugged and then used later with increased assurance of reliablity.

An example of a powerful control structure that increases reliability is looping over a list of items and needing to do some action between the elements in the list. Sure this is straight forward to implement most languages but it can also lead to subtle bugs so why not have a control structure for it? Well, that's what I created! It only took a couple of hours to get it the way I wanted and presto all of the appropriate collection classes now understood a protocol of messages for performing actions between elements in lists. Since I wanted maximal flexibility and had to deal with some other cases I also added the ability to deal with the following situations or cases: perform a specific block of code when the list is empty, when the list has one element, when the list has many elements, when an error occurs during processing of the elements, perform a specific block for the first, last and "middle" elements of the block. I've already mentioned the "between" elements case. For completeness I also added pre and post blocks although those are not really needed. The advantage of this protocol of methods is that "collections" can now easily deal with these many cases thus simplifiying and increasing the reliability of my other code.

In reality I'm adding these "control structures" to the class library of objects, but it is the reality of the design of Smalltalk's syntax and virtual machine that enables me to have this expressive power. Even if these control structures never become "standard" in the various implementations I can easily add them as needed.

In effect "control structures" become "first class messages" and thus control structures become first class objects in Smalltalk.

Yes, it's true that a few of these operations and control structures have "primitives" that root the system in the virtual machine, but that's an implmentation detail, not a langauge detail. That can't be said of the majority of langauges that embed their "binary operators" and "control strucutres" into the language specification and the compilier.

Smalltalk isn't perfect and neither are the implementations, but they've come a long way in the 32+ years that Smalltalk has been around (24+ years since Smalltalk-80 which the above pertains to). There are over twenty implementations of Smalltalk with five recent additions in the last few years and a couple of new ones in the works. Smalltalk is no longer simply "Smalltalk-80" but a family of languages that enable an "eco-system" that has adapted to various end user system requirements.

It is a result of Smalltalk's approach to minimal grammar that enables it's "thin layer of magic" to be one of the thinest layers out there.

I'm working on a new variant of Smalltalk, known as Zoku, that attempts to reduce this "thin layer" even further.
Copyright 1999-2010 by Smalltalk.org"!, All Rights Reserved.