™ | ZokuTalk™ Lisp, Smalltalk, + Others Hybrid

... reinventiing computer languages ...
... booting ...
... parser and compiler in progress ...

ZokuTalk™'s Meta Protocol
by Peter William Lount
Copyright 1997-2019, all rights reserved.

ZokuTalk™ EBNF Keywords:

Singleton-Instances EBNF Keywords

Pseudo-Variables EBNF Keywords:

The global namespace object zoku

The obsolete Smalltalk keyword/pseudo-variable "thisContext" is replaced by the super powerful metaprogramming keyword/pseudo-variable "meta" in ZokuTalk; of course you can still access the current stack context by using "meta thisContext" or "meta executionContext."

The ZokuTalk keyword/pseudo-variable "meta" empowers metaprogramming with the full power of the full general purpopse programming language so both have the same full expressive powers. All too often in programming languages meta programming is hobbled by special language syntax or a very restricted subset of a language.

As in the case of the other keywords/pseudo-variables meta refers to an instance of the Meta class of objects, although it's not necessarily a singleton as the keywords nil, true, or false object instances are; it's a pseudo-variable self or super keywords/psudo-variables that refer to different objects instances or classes at different times. Any given execution context might have a different instance of the Meta class due to various factors such as, for example, a security execution context that might restrict low level operating system access.

An example of obsolete restricted meta programming in Smalltalk a special syntax is required for "primitives" using an archaic non-message passing syntax for accessing a virtual machine primitive function using the ick syntax of "<primitive number>" or some other modern variant thereof. The same non-message passing syntax is true of the so called "foreign function interface" (FFI) calls to Dynamic Link Libraries or Shared Libraries. This "special" primitive syntax really is primitve in that it also forces every known Smalltalk variant to have only one primitve call in a method and crippled exception handling syntax that again is not standard general purpose programming syntax.

A core observation is that everytime you need special syntax the language loses genericity and makes life difficult for programmers, users, and power users alike. The core design principle to come out of this observation is that in ZokuTalk there is only one language syntax for all forms of programming, no little languages, just the full power of the MOBS syntax. Messages as the big idea as Alan Kay imagined. Objects that fuse data, relationships and functions aka methods into discrete related pieces of code and data. One MOBS syntax to rule them all.

The EBNF based Grammar Syntax definition of ZokuTalk has three main areas to implement the core MOBS syntax. Code is Data is Objects is Code ... is an important #Homoiconic Principle in the design of the ZokuTalk grammar. This has lead to a reduction of the grammar to its essence to support source code, data atoms and literals, blocks and objects. A uniform grammar is essential to keeping a language mean and lean for maximal computational capbilities. It turns out that decluttering computer language grammars makes them far more powerful.

ZokuTalk's meta unifies all meta programming capabilities including all primitive activations, DLL or FFI calls by using the exact same language syntax as general purpose programming using unary, binary and keyword message passing and block syntax. To support sane mapping of foreign functions, that only use a single function name yet have multiple parameters, to ZokuTalk method names "Function Keywords" were added to the ZokuTalk grammar.

In addition all standard exception processing behavior is as one would expect in general purpose programming. Furthermore there are no limitations on how many low level primitive or foreign function calls one can make in any given method although standard object oriented sytle guidelines for reuse and clarity using short methods should still be followed.

"meta primitive add64Integer: aInteger as64BitInteger to: bInteger as64BitInteger" is all it takes to access a low level function "primitive" in ZokuTalk, and you can wrap standard exception handling code around it as desired or required for defensive programming 101. Due to the nature of most primitive or FFI or DLL calls specific class types for parameters are often required to avoid runtime errors so a range of class/type conversion methods exist, in addition to the extensive ZokuTalk Verification Library of classes.

"meta module win64Api someFunctionCallMessageAndParamtersGoHere: aParameter" is all it takes to access an already loaded DLL, and methods exist to check for the existence of and load any given DLL according to the proper Microsoft or Unix style APIs. Obviously differences in operating systems need to be dealt with but that can happen in the higher level ZokuTalk message passing syntax rather than using an iCk C based language.

meta is an access doorway into the internal world of the ZokuTalk language meta environment giving access to all kinds of objects that make up the ZokuTalk runtime environment of the running application. There are dozens of already thought of uses for the meta keyword in ZokuTalk, and as it's a generic doorway implemented as classes in ZokuTalk it is open to expansion into new areas.

In traditional Smalltalk systems there is a dictionary instance, often of class SystemDictionary, that contains as keys the names of Classes (and other global variables) accessible to the current running Smalltalk programs. All class names start with an uppercase letter as do "global variables". When a class name is referenced the Smalltalk compiler looks up the name in the "Smalltalk" dictionary. In ZokuTalk this lookup occurs through the "meta classes dictionary lookup, which is part of the ZokuTalk namespace system, meta namespace, that includes global variables and class names that are accessible within the current meta instance. In this way as part of the ZokuTalk security system the available classes in any given execution context can be customized, expanded, filtered, restricted, or selected. For example, in a Security Sandbox you might not want to give full access to all the classes in the full ZokuTalk execution context so that "foreign code" being executed isn't able to read or write to the local storage device.

In ZokuTalk the base namespace is called "zoku". Elements of this system "dictionary" or rooted namespace can be accessed using the namespace syntax. For example, zoku::core lets you access the core namespace inside the zoku namespace. Note that "::" is a binary message that namespaces understand and which are used to navigate the name space tree. Namespacs are still a work in progress and will likely change or be refined for ZokuTalk 1.0.

To support executing code in a Security Sandbox blocks of code can be executed with a new meta context instance that specifies the various security restrictions or permissions, for example: "[:someParameter ... some code you don't trust ... ] value: aParameter metaContext: Meta secureSandboxMaximumRestriction" will execute the block of code from some foreign source you don't trust with an instance of the Meta configured to be a maximum security sandbox that doesn't allow for reading or write to the file system and omits all but the most basic set of system library classes. In this way code from outside souces, can in theory, be executed in a safe execution context with limited resources to prevent actions that virus authors would love to take to infiltrate your system.

In summary, ZokuTalk adds the powerful keyword/pseudo-variable meta maintining the functionality of thisContext while providing a wide range of new capabilities at compiletime and runtime of Zoku(tm) based programs including but not limited to: ZokuTalk has many other capabilities that distinguish it from Smalltalk systems. ZokuTalk is based upon an evolution of Smalltalk syntax that aims to make it a much more pure unary, binary, keyword, block & object based message syntax than other languages. Elimination of dedicated non-message passing syntax such as primitive syntax and legacy file in chunk formats from ZokuTalk syntax helps to clean up and purify the minimal elegance of the ZokuTalk EBNF design so that one unified and uniform unary, binary, keyword, block & object based message syntax rules them all.

It should be noted that while ZokuTalk improves upon Smalltalk in many ways including by purifying the syntax all legacy Smalltalk capabilities are preserved. In fact many if not most Smalltalk classes should work just fine with minimal or no porting effort; a Smalltalk language specific compiler EBNF will most likely be provided for importing your existing Smalltalk codebase with minimal changes.

As ZokuTalk unfolds into realized implementation more details will be disclosed. Most aspects of ZokuTalk a quite fixed and locked down at this point, however some aspects are not yet fixed in stone and are likely to change.

Current progress towards ZokuTalk 1.0
2017 and 2018 saw significant advancements in the design and implmentation of ZokuTalk. Implementation work began on the Parser and Compiler impmenentation in 2018. A large amount of time has been committed to learning what to change and what not to when it comes to the ZokuTalk EBNF Grammar Syntax. Also learning parser writing at the same time as attempting to pin down a new language syntax was a bit of a challenge. After a number of false starts the parser is about 1/2 done.

In 2017 the ZokuTalk Language EBNF was nailed down and frozen with only a few minor improvements made to the ZokuTalk EBNF Grammar since then. Foreign Function Keyword message syntax was added to support mapping of C, C++, Java, ... and other "round parenthesis" languages which have a single "identifier" for their function names while having zero, one or many unnamed parameters.

Expressions recently were improved so that they can have more than one statement in them including temporary variable definitions, of course the last statement in the expression list of statements is the one whose object value is result of the immediate execution of the expression to be compatible with existing more limited expression syntax in Smalltalk systems.

Single colons, ":", as "binary operator message sends" are being considered for inclusion in ZokuTalk's grammar; this is an influence of Cuis Smalltalk but was also considered before, it seems that Cuis is using the binary operator message ":" to number to construct instances of the Time object which is rather nice. As this usage would obliviate a special syntax for time data it is worthwhile including the generic ":" binary operator in ZokuTalk.

Double colons, "::", without any characters in between them are also being considered as binary operator message characters. The first usecase for this would be Namespaces. While many Smalltalk's use the period "." as a name space seperator which is also how a number of non-Smalltalk languages write namespaces the period is already used for a number of grammar elements and wouldn't be using a generic message send approach. As much as possible ZokuTalk aims to make use of message sending to implment capabilities as it is far more flexible and more powerful usually. The use of the double colon binary message send for namespaces would go something like this, "zoku::core::Object subclass: #Point" to create a subclass. The first send of "::" to the namespace "zoku::core" gets one the "core" namespace where the class Object lives, to obtain "Object" one uses the second message send of "::". This might be how it's done, we'll see, this part is open to change. Fortunately if it's implmented this way it's implmenetation is essentially entirely in the object library and can be altered easily.


20190112 ZokuTalk Source Code Project Managment Milestone Arhieved!
ZokuTalk now has a Project Manager that can do two things, (1) In the middle of January 2019 the ProjectManager class in the Pharo Smalltalk development environment was enhanced to support the latest Pharo Smalltalk, this enables the project manager instance to emit the ZokuTalk code base in Legacy Smalltalk Source Code Chunk Format for moving it forward to newer versions of Pharo Smalltalk (version 7 is on the horizon). In addition the project manager was enhanced to emit the ZokuTalk code base in ZokuTalk Source Code MOBS Format (Message, Object & Data, Block Syntax). Two files are emitted, a Smalltalk version and a ZokuTalk version of the code base the project manager is looking at. I generally have it emit the source code at every image save.

Another project manager instance was also set up to collect up all the source code that will be exported from Pharo Smalltalk under the MIT License for incorporation as code into the ZokuTalk system. This is an ongoing project as the methods need to be minimal Smalltalk syntax to be compatible with the ZokuTalk minimal syntax as there are many syntaxes eliminated from ZokuTalk as they are done a different way or not used. For example, Smalltalk primitive syntax is entirely eliminated and a more modern approach taken at the Execution Engine level.

In any event the Bonai pruning of any MIT Licensed Pharo Smalltalk source code will take time. All primitives referred to will need to be rewritten. Any classes not being exported will need to not be referred to. For the most part the classes being borrowed are base classes inherited from Smalltalk-80 excluding all GUI classes.

ZokuTalk 1.0 2019 Target
ZokuTalk 1.0 is being worked on for a 2019 release date; fingers crossed. The first version of the ZokuTalk compilier is currently about 1/3rd completed.

Pharo Smalltalk is currently being used as one of the tools for the development of ZokuTalk as it's a mature and stable Smalltalk. In addition many objects and their methods are being carefully selected and added to a Bonsai trimmed set of classes and methods being ported over to form a core set of seed code for ZokuTalk. However all primitives required will be rewritten from scratch as the legacy primitive syntax and virtual machine designs have been abandonded for a direct ZokuTalk BareMetal Runtime Execution Engine for the purposes of raw performance and more powerful capabilities.