Smalltalk.org | ZokuTalk Lisp, Smalltalk, + Others Hybrid
... reinventiing computer languages ...
... booting ...
... parser and compiler in progress ...
ZokuTalk's Meta Protocol
by Peter William Lount
Copyright 1997-2019, all rights reserved.
ZokuTalk EBNF Keywords:
Singleton-Instances EBNF Keywords
Pseudo-Variables EBNF Keywords:
The global namespace object zoku
The obsolete Smalltalk keyword/pseudo-variable "thisContext" is replaced by the super powerful metaprogramming keyword/pseudo-variable "meta" in ZokuTalk;
of course you can still access the current stack context by using "meta thisContext" or "meta executionContext."
The ZokuTalk keyword/pseudo-variable "meta" empowers metaprogramming with the full power of the full general purpopse
programming language so both have the same full expressive powers. All too often in programming languages
meta programming is hobbled by special language syntax or a very restricted subset of a language.
As in the case of the other keywords/pseudo-variables meta refers to an instance of the Meta class of objects, although it's not necessarily
a singleton as the keywords nil, true, or false object instances are; it's a pseudo-variable self or super keywords/psudo-variables
that refer to different objects instances or classes at different times. Any given execution context might have a different
instance of the Meta class due to various factors such as, for example, a security execution context that might restrict low level operating
An example of obsolete restricted meta programming in Smalltalk a special syntax is required for "primitives" using an archaic non-message passing
syntax for accessing a virtual machine primitive function using the ick syntax of "<primitive number>" or
some other modern variant thereof. The same non-message passing syntax is true of the so called "foreign function interface" (FFI) calls
to Dynamic Link Libraries or Shared Libraries. This "special" primitive syntax really is primitve in that it
also forces every known Smalltalk variant to have only one primitve call in a method and crippled exception handling
syntax that again is not standard general purpose programming syntax.
A core observation is that everytime you need special syntax the language loses genericity and makes life difficult for programmers, users, and power users alike.
The core design principle to come out of this observation is that in ZokuTalk there is only one language syntax for all forms of programming, no little languages,
just the full power of the MOBS syntax. Messages as the big idea as Alan Kay imagined. Objects that fuse data, relationships and functions aka methods into discrete related pieces of
code and data. One MOBS syntax to rule them all.
The EBNF based Grammar Syntax definition of ZokuTalk has three main areas to implement the core MOBS syntax. Code is Data is Objects is Code ... is an important #Homoiconic Principle
in the design of the ZokuTalk grammar. This has lead to a reduction of the grammar to its essence to support source code, data atoms and literals, blocks and objects. A uniform grammar is essential
to keeping a language mean and lean for maximal computational capbilities. It turns out that decluttering computer language grammars makes them far more powerful.
ZokuTalk's meta unifies all meta programming capabilities including all primitive activations, DLL or FFI calls
by using the exact same language syntax as general purpose programming using unary, binary and keyword message passing
and block syntax. To support sane mapping of foreign functions, that only use a single function name yet have multiple parameters, to ZokuTalk method names "Function Keywords"
were added to the ZokuTalk grammar.
In addition all standard exception processing behavior is as one would expect in general purpose programming.
Furthermore there are no limitations on how many low level primitive or foreign function calls one can make in any given method
although standard object oriented sytle guidelines for reuse and clarity using short methods should still be followed.
"meta primitive add64Integer: aInteger as64BitInteger to: bInteger as64BitInteger" is all it takes to access a low level function "primitive" in ZokuTalk,
and you can wrap standard exception handling code around it as desired or required for defensive programming 101.
Due to the nature of most primitive or FFI or DLL calls specific class types for parameters are often required to avoid runtime errors so a range
of class/type conversion methods exist, in addition to the extensive ZokuTalk Verification Library of classes.
"meta module win64Api someFunctionCallMessageAndParamtersGoHere: aParameter" is all it takes to access an already loaded
DLL, and methods exist to check for the existence of and load any given DLL according to the proper Microsoft or Unix style APIs.
Obviously differences in operating systems need to be dealt with but that can happen in the higher level ZokuTalk message passing syntax
rather than using an iCk C based language.
meta is an access doorway into the internal world of the ZokuTalk language meta environment giving access to all kinds
of objects that make up the ZokuTalk runtime environment of the running application. There are dozens of already thought of uses
for the meta keyword in ZokuTalk, and as it's a generic doorway implemented as classes in ZokuTalk it is open to expansion into new areas.
In traditional Smalltalk systems there is a dictionary instance, often of class SystemDictionary, that contains as keys the names of Classes (and other global variables)
accessible to the current running Smalltalk programs. All class names start with an uppercase letter as do "global variables". When a class name is referenced
the Smalltalk compiler looks up the name in the "Smalltalk" dictionary. In ZokuTalk this lookup occurs through the "meta classes dictionary lookup, which is part
of the ZokuTalk namespace system, meta namespace, that includes global variables and class names that are accessible within the current meta instance. In this way
as part of the ZokuTalk security system the available classes in any given execution context can be customized, expanded, filtered, restricted, or selected. For example,
in a Security Sandbox you might not want to give full access to all the classes in the full ZokuTalk execution context so that "foreign code" being executed isn't able
to read or write to the local storage device.
In ZokuTalk the base namespace is called "zoku". Elements of this system "dictionary" or rooted namespace can be accessed using the namespace syntax. For example, zoku::core lets you access the
core namespace inside the zoku namespace. Note that "::" is a binary message that namespaces understand and which are used to navigate the name space tree. Namespacs are still a work in progress
and will likely change or be refined for ZokuTalk 1.0.
To support executing code in a Security Sandbox blocks of code can be executed with a new meta context instance that specifies the various security restrictions or
permissions, for example: "[:someParameter ... some code you don't trust ... ] value: aParameter metaContext: Meta secureSandboxMaximumRestriction" will execute the block
of code from some foreign source you don't trust with an instance of the Meta configured to be a maximum security sandbox that doesn't allow for reading or write to the
file system and omits all but the most basic set of system library classes. In this way code from outside souces, can in theory, be executed in a safe execution context
with limited resources to prevent actions that virus authors would love to take to infiltrate your system.
In summary, ZokuTalk adds the powerful keyword/pseudo-variable meta maintining the functionality of thisContext while providing a wide range of new
capabilities at compiletime and runtime of Zoku(tm) based programs including but not limited to:
ZokuTalk has many other capabilities that distinguish it from Smalltalk systems. ZokuTalk is based upon an evolution of Smalltalk syntax that aims to make it a much more pure
unary, binary, keyword, block & object based message syntax than other languages. Elimination of dedicated non-message passing syntax such as primitive syntax and legacy
file in chunk formats from ZokuTalk syntax helps to clean up and purify the minimal elegance of the ZokuTalk EBNF design so that one unified and uniform unary,
binary, keyword, block & object based message syntax rules them all.
- Access to the current closure execution context is provided by the convenience method, thisContext via meta thisContext;
- Primitives using Standaried Unified Message Passing Object Block Syntax, aka MOBS syntax;
- Foreign Function Interface (FFI) using Standaried Unified Message Passing Object Block Syntax;
- Uniform Execption Handling for all code in the ZokuTalk system;
- Security Sandbox Contexts for executing code with specific access restrictions;
- Namespace Management and Integration using a unified approach;
- Access to all First Class Objects in the ZokuTalk Execution Environment, unless restricted by a Security Sandbox, enabling powerful development systems
not to mention amazing application prowess at dealilng with runtime enviroment issues;
- Access all things not ZokuTalk, all things below the language, or not of the language, or of inside the language, all things from others, ... all things meta;
- Many other purposes not yet disclosed.
It should be noted that while ZokuTalk improves upon Smalltalk in many ways including by purifying the syntax all legacy Smalltalk capabilities are preserved.
In fact many if not most Smalltalk classes should work just fine with minimal or no porting effort; a Smalltalk language specific compiler EBNF will most likely
be provided for importing your existing Smalltalk codebase with minimal changes.
As ZokuTalk unfolds into realized implementation more details will be disclosed. Most aspects of ZokuTalk a quite fixed and locked down at this point, however some aspects are not yet
fixed in stone and are likely to change.
Current progress towards ZokuTalk 1.0
2017 and 2018 saw significant advancements in the design and implmentation of ZokuTalk. Implementation work began on the Parser and Compiler impmenentation in 2018. A large amount of time
has been committed to learning what to change and what not to when it comes to the ZokuTalk EBNF Grammar Syntax. Also learning parser writing at the same time as attempting to pin down
a new language syntax was a bit of a challenge. After a number of false starts the parser is about 1/2 done.
In 2017 the ZokuTalk Language
EBNF was nailed down and frozen with only a few minor improvements made to the ZokuTalk EBNF Grammar since then. Foreign Function Keyword message syntax was added to support mapping of
C, C++, Java, ... and other "round parenthesis" languages which have a single "identifier" for their function names while having zero, one or many unnamed parameters.
were improved so that they can have more than one statement in them including temporary variable definitions, of course the last statement in the expression list of statements is the one whose
object value is result of the immediate execution of the expression to be compatible with existing more limited expression syntax in Smalltalk systems.
Single colons, ":", as
"binary operator message sends" are being considered for inclusion in ZokuTalk's grammar; this is an influence of Cuis Smalltalk but was also considered before, it seems that Cuis is using
the binary operator message ":" to number to construct instances of the Time object which is rather nice. As this usage would obliviate a special syntax for time data it is worthwhile including
the generic ":" binary operator in ZokuTalk.
Double colons, "::", without any characters in between them are also being considered as binary operator message characters. The first usecase for this would be Namespaces. While many
Smalltalk's use the period "." as a name space seperator which is also how a number of non-Smalltalk languages write namespaces the period is already used for a number of grammar elements and
wouldn't be using a generic message send approach. As much as possible ZokuTalk aims to make use of message sending to implment capabilities as it is far more flexible and more powerful usually.
The use of the double colon binary message send for namespaces would go something like this, "zoku::core::Object subclass: #Point" to create a subclass. The first send of "::" to the namespace
"zoku::core" gets one the "core" namespace where the class Object lives, to obtain "Object" one uses the second message send of "::". This might be how it's done, we'll see, this part is open to
change. Fortunately if it's implmented this way it's implmenetation is essentially entirely in the object library and can be altered easily.
20190112 ZokuTalk Source Code Project Managment Milestone Arhieved!
ZokuTalk now has a Project Manager that can do two things, (1) In the middle of January 2019 the ProjectManager class in the Pharo Smalltalk development environment was enhanced to support the latest Pharo Smalltalk, this enables the project manager
instance to emit the ZokuTalk code base in Legacy Smalltalk Source Code Chunk Format for moving it forward to newer versions of Pharo Smalltalk (version 7 is on the horizon). In addition the project manager
was enhanced to emit the ZokuTalk code base in ZokuTalk Source Code MOBS Format (Message, Object & Data, Block Syntax). Two files are emitted, a Smalltalk version and a ZokuTalk version of the
code base the project manager is looking at. I generally have it emit the source code at every image save.
Another project manager instance was also set up to collect up all the source code that will be exported from Pharo Smalltalk under the MIT License for incorporation as code into the ZokuTalk system.
This is an ongoing project as the methods need to be minimal Smalltalk syntax to be compatible with the ZokuTalk minimal syntax as there are many syntaxes eliminated from ZokuTalk as they are
done a different way or not used. For example, Smalltalk primitive syntax is entirely eliminated and a more modern approach taken at the Execution Engine level.
In any event the Bonai pruning of any MIT Licensed Pharo Smalltalk source code will take time. All primitives referred to will need to be rewritten. Any classes not being exported will need to
not be referred to. For the most part the classes being borrowed are base classes inherited from Smalltalk-80 excluding all GUI classes.
ZokuTalk 1.0 2019 Target
ZokuTalk 1.0 is being worked on for a 2019 release date; fingers crossed. The first version of the ZokuTalk compilier is currently about 1/3rd completed.
Pharo Smalltalk is currently
being used as one of the tools for the development of ZokuTalk as it's a mature and stable Smalltalk. In addition many objects and their methods are being carefully selected and added to a
Bonsai trimmed set of classes and methods being ported over to form a core set of seed code for ZokuTalk. However all primitives required will be rewritten from scratch as the legacy primitive
syntax and virtual machine designs have been abandonded for a direct ZokuTalk BareMetal Runtime Execution Engine for the purposes of raw performance and more powerful capabilities.