See (p3,cmdlang) for something currently available.

{arcane}{nonref}Command Language Interpreter: A Preliminary Modularization. Some ideas on the structure of a command language for Gnosis. The command language will consist of the following components:

Command Language Interpreter.
This program takes the commands of the command language and interprets them calling on the other functions to cause user domains to be created and executed. It specifically does not recognize abbreviations for commands or other syntactic entities.

Power Typer.
The power typer uses a definition of the commands to provide the facilities of Command completion, Command recognition and expansion, File name expansion, etc. It may exist in several versions for different terminals or user preferences.

Its inputs are a CML like definition of the commands, and a terminal capability.

Its output is a complete command as will be acceptable to the command language interpreter. This output may be routed directly to the command language interpreter or to a file while building command files.

Command Grammar Merge.
This component takes the CMS like grammar for a set of commands {e.g., a database query language} and merges it with the currently active set of command grammars. It modifies the information base used by both the power typer and the command language interpreter. This allows the user to extend the command language and preserve the single layer image of the command system.

Command Grammars.
These define the commands. They describe what the user has to type, what output is produced from the power typer, what defaults will be provided for unspecified parameters, what help messages will be provided for the equivalent of the TENEX ? convention, and what actions are to be taken to invoke the command.

Directory Scanner.
This part of the command language parses file names. It owns a directory (the root directory for this user) and returns either lists of file names (via wildcard expansion) or the capability associated with some entity in a directory.

Filedef.
This program provides the link between Gnosis capabilities and the OS simulator. Hopefully users will not need to know about this part of the command system unless they are using OS simulation.

Command Domain Creator.
This program creates an environment for a discreet command to execute in. It may be identical to the factory {(p2,factory)}.

I think that the factory is merely another facility available via the CLI. The CLI should know nothing special about factories.

This may be a bad if either :
The frequency that the factory facility is needed causes a noisome burden on the user or

It is determined that we need to "make using the factory easier than not using it" {seat belts?}.

This should perhaps be a test of any "language extensibility" properties of the CLI. There are three that come to mind:
Calling a subroutine in a domain for each extension,

Begin (yet another) programming language in the language,

Make the CLI a sub-language of an existing {good} language.

I think that I have already displayed a prejudice against the new programming language idea.

Commands.
N.B. Not all domains will be usable as commands. Commands must follow a specific (as yet undefined) protocol as to parameters accepted and returned.

Commands recognized by the command system are in the following distinct flavors:

Discreet commands.
These commands specify a creator that is designed to meet the factory protocol. They can not steal data from the user {even the data that is passed to the command}.

General Capabilities.
These commands are invoked by the command language interpreter directly (via either Fork or Call) passing the capabilities specified in the command line. They are the only place to hide a Trojan horse.

Message Generator.
The message generator is a program that takes a set of message skeletons and a reference to one of them and some replacement data and produces a formatted message that may be displayed for a user.

There are several requirements perceived that might be met by one language. This is a collection of ideas and requirements brought together here in the hope of fostering some commonality.

Needs

A command language
I here mean a language that is executed as it is entered.

Another possible definition of "command language" is a language in which to instruct the system what jobs you want it to do and when.

Lets play with this definition a bit:

Informing the system of what to do for you during the coming evening is really programming. Typically such programs are simple and it is important to get the instructions right, as the consequences of an error might set some project back a day. All of the needs of general programming are present but the emphasis is different. The only differences that I can see, however, is that performance is unimportant and clarity is extra important(?).

In traditional command languages files are the most frequent operands and programs fetched from files are the most frequent operators. I believe that keys will play the role of files in this regard in Gnosis.

A command language must do simple things simply. In particular it must do the common operations on the common objects in one short line.

A language in which supports the key as a type

A language in which programs are very easy to debug

Tymshare got its start in providing languages such as Super Basic and CAL. These languages supported direct execution conveniently, consistently and efficiently. Tymshare currently has no packages that are nearly as convenient. EXPRESS has some of these features but it is not viewed as a general purpose programming language.

A simple language in which to instruct domains for those instances where performance is not critical

Bill's example
Bill Frantz has proposed a specific task that might be posed to a command system. UNIX handles this task very well.

The task is to build a spell checker from a few simple components and to do so with the assurances that motivated the design of factories. We should also address the issues of space banks and meters.

We postulate the following components: Each of these programs consumes its input and produces its output according to the byte stream protocol. Each of these three programs are provided via factories.

A program CL that reads the text and replaces all inter-word character sequences by one space. The order of the words is unchanged.

A program SORT that reads a space-separated word list and sorts it and outputs the sorted list.

A program BOOL that reads two word lists and produces three sorted streams, one stream with the words that occur in both inputs and one stream for words in the first bit not the second and vice versa. The BOOL factory accepts its co-routine parameters, one at a time, along with an order code indicating which of its three outputs is required.

Assume the following specs:
CL_FACTORY(0,"#"^^str;IN,OUT ==> c) creates a list maker that functions by replacing punctuation (any character not in str) strings by a "#". IN and OUT are co-gates for the input and output of the routine.

SORT_FACTORY(0,"#";IN,OUT==>c) produces a sorter of records that are delimited by "#" in the input stream and likewise on output.

BOOL_FACTORY(2;IN1,IN2,OUT==>c;) produces a unit that takes two lists and yields the list of words that are in the first list but not in the second list.

UNIX style pipes
We describe here the semantics of the UNIX pipe joint operator "^" in conjunction with the proposed co-routine conventions (p3,costart).

We wish to explain the meaning of "a^b^c" where the letters represent the creators of units with one input and one output. We consider "^" to be an associative binary operator. It is clear that the MODE of x^y must be the same as the MODE of x. Let us call this MODE UF for "unit factory".

According to our conventions an object of mode UF is a thing waiting for two co-gates upon receipt of which it will begin. Thus MODE UF = PROC(EXIT, EXIT)VOID.

MODE EXITPAIR = STRUCT(EXIT in, out); OP ^ = (UF left, right)UF: (EXIT a, b)VOID: (EXITPAIR ep = bscr #bscr is the byte stream creator.#; PAR BEGIN left(a, in OF ep), right(out OF ep, b) END)

UNIX - ALGOL68 solution
PROC(STRING s, EXIT in, out)VOID parse = SKIP #parser routine here#;

PROC(EXIT in, out)VOID sort = SKIP #sort routine here#;

PROC(INT b, EXIT in1, in2, out)VOID bool = SKIP #bool routine here#;

PROC read file = (FILEKEY x)EXIT: SKIP #This routine produces a producing cogate for the file#;

PROC write file = (REF KEY out file key slot)EXIT: SKIP #return an consuming cogate that produces a file.#;

#The job! {with intermediate storage variables}#

PROC two arg bool = (EXIT a, b)VOID: bool(2,a,read file(#file-spec for spelling word list# SKIP), b);

PROC two arg parse = (EXIT a, b)VOID:
parse(",abcdefghijklmnopqrstuvwxyz",a, b);

UF ultimate pipe = two arg parse ^ sort ^ two arg bool;

ultimate pipe(#file-spec for file to be checked# SKIP, sok key for terminal);

Implementation ideas
It seems that an interpretive language provides the right tradeoffs here. Interpretive languages support immediate execution naturally. They also support debugging better than compiled languages.

We should try to use existing language ideas and ideally an existing language.

Better yet we might find an interpreter that runs on the 370 that might be adapted to Gnosis. We list those interpreters for the 370 which we know of:

BRUIN
Bruin was once on our machine. It was a UCLA implementation of JOSS like ideas. I don't remember much about it except that it provided much less than the original JOSS which in turn provided less function than our CAL on the 940.

I have heard nothing of BRUIN for at least several years.

PLIC
IBM's checkout compiler for PL/I is admirable in many ways. Unfortunately it seems to have been designed by intelligent people who had read about interactive higher level language debuggers but had not used them. There is evidence that experience with the checkout compiler has not been allowed to influence its design. The result is there are many obscure and exasperating restrictions that keep you thinking about the logic of the compiler instead of the problem under consideration.

PLIC is slow at direct execution. I suspect that it would still be slow in Gnosis.

PL/I is a poor language for writing programs that are expected to run correctly the first time. It is not a simple language for the nonprofessional programmer. It is difficult to type constants in PL/I and the simplest use of a command language uses no variables, only constants.

FORTH
I hear that there is a FORTH interpreter. I know little of it except that it would be a amazing if the forth language itself would be appropriate for a command language. (Remember we are looking for an existing language to serve as a command language; not a language in which to write a command language interpreter.)

FORTH is more a philosophy then a language. There are many languages that share the philosophy. Key areas of the philosophy:

Stack oriented reverse polish. {If you liked your HP calculator you'll love FORTH.}

There should be no extra overhead for checking such things as arithmetic overflow as the language itself allows you to re-define the operators to perform such checks.

Programs should be built in small units. {I have never seen a FORTH word definition take more then one 1024 byte "screen".}

Forth makes a good command language if you can take the reverse polish. Not everyone can.

Chion's Algol 68
I have heard rumors that this compiler is "semi-interpretive". There might be hope that it might have or be adaptable to an immediate mode. I will check. {They don't return my calls.}

EXEC and EXEC2
These languages have been used as the programming extension to the CMS {and other systems} command languages. They have been successful at allowing non-programmers to get their start in programming. They lack a number of things needed for writing large, complex systems.

The UNIX(r) Shell
This language provides a way to connect what in Gnosis would be co-routines together in a simple and natural way. It also provides structured IF and LOOP constructs. It should be looked at further.

BASIC
Don't shudder

I conclude that we must look further than the above systems. There may be others that are unknown to us.

Using the domain as the {only} subroutine construct.

If we have a language in which we can simply create and instruct a new domain and also a language which can simply invoke keys, we will have ipso-facto a language which may be extended by definition.

We will also have a language in which we can construct simply the most powerful and must fundamental object in Gnosis -- a key to a domain with custom code inside.

The important point here is that a subroutine so constructed could be used not only in the context of other programs written in our interpretive language, but in any environment where keys may be invoked.

Language features
Control over Space Banks
The current design of space banks makes it necessary to choose an appropriate space bank for various activities. For example, a compiler should use one guarded bank for its temporary storage but another for its end product.

See (scope-rules) about connections between space bank problems and the scope problems of classical programming languages.

It is common for command languages to support concatenation of command components during execution. Interpretive languages get a lot of mileage from this idea because it provides for an easily understood structure function.

While programs using this feature are easy to construct they are frequently difficult to analyze. This may be a good trade-off here.

Preliminary (Premature?) Conclusions
I think that we should wait until we have a algorithmic language that approaches Super Basic or CAL in human interface quality. I think that the name space of such a language should be unified with the user's permanent directory (if he wishes).

In the meantime we should have a direct execution language that has no macro facility, no conditionality, no arithmetic ability, no character substitution, but has a name space defined by a record collection, a command to invoke a key using key names as functions and arguments. Some facility is also necessary for constructing, passing, receiving and examining the data of key invocations.

Unless programming at all levels, and by this I mean from simple, throw-away programs to large, complex systems with hundreds of person years of effort and large budgets, is made as easy as possible, I believe that Gnosis will fail. We have a number of professional languages for large, complex systems, e.g., Pascal, PL/I, FORTRAN, COBOL etc. What we lack are languages that make combining existing function in novel ways easy. EXEC and the UNIX(r) shell have gone a long way beyond what these professional languages have done in making it simple to combine things. We should not make the mistake of thinking that one language can do everything.

One way to look at the command language problem is to ask "How can we make creator construction and invocation easy?" However, doing everything as gate jumps has the problem that people can't do gate jumps directly. They must use some language to instruct the system. The next thing they want to do is combine the facilities of that language to build custom constructs. A good example is receiving the Adventure application. It currently takes 5 or 10 CL1 commands, where the only real information the user needs to present is "Install Adventure, the transfer mechanism password is 'xxxx'." Commands files would solve this particular problem.

I would prefer to support the programming-like functions of the CMS exec language by writing PL/I programs and making sure that small jobs, so expressed, are truly small. By small I mean:

Not more than two or three lines of compiler input overhead.

Not more than two pages and 10 nodes in the resulting "compiled object".

Not more than 1000 gate jumps in the creation of the object, including the editing, compilation, etc.

Not more than 20 gate jumps of overhead in the execution.

Not more than an editing session and a single command to create the object.

The issues of the amount of overhead on language constructs are important and I think essentially correct. However, PL/I is a poor language for a command language because it is too complex. By this I mean it has too many datatypes and too many basic concepts. It is also hard to type constants in PL/I. This may be acceptable for programs which will last for many years, but not for commands that will be gone as soon as the "execute" button is pressed. I don't think you could even make the case that it does a new user a greater service to start with PL/I rather then a language that is as bad as, e.g., BASIC.

There are several kinds of CLI's:
PROXY: Proxies are designed to be completely loyal to the users at the other end of the circuit. They have no more authority than is intended to be wieldable by people who know the Tymnet password associated with the user name that located the entry in the LUD.
A proxy may refuse to wield some of its authority in order to protect its own integrity. The distinction here is that no one but the proxy is depending on the proxy for this protection.

Proxies may talk to programmers in terms of the capabilities that they hold for the programmer. Proxies may talk to reservation agents about reservations.

GUARD: The guard, on the other hand, may be designed to execute a policy restricting the information and effects the user is intended to have.
I think that it is a bad policy to build programs such as guards. Such programs integrate the functions of proxies and security modules. Gnosis was invented in order to separate security code from other code. Such integration assumes that the role of the user is not likely to be played by a program. Such assumptions typically turn out to be false for successful systems.

AUDITOR: The auditor does what ever you want but remembers what you did. It knows enough system semantics to report how objects were created. It understands banks and may forget certain facts about objects when they no longer exist. If the auditor works with a graphics interface it might serve as an elementary "monkey-see monkey-do" command file function.

Some CLI's may be designed to talk to people at terminals while others are designed for programs at the other end of the circuit. We may want to change the nomenclature here, but we need a name for CLI's for programs.

CLI's are also likely to be tailored for terminal types.

See (screen).

Of the three ways of de-allocating storage {(p3,garbage)} programming languages have used all three.

Viewed as a language, the command system must cope with the same dilemmas.

Current Art

Each of our current 370, 940 and 10 operating systems rely on explicit freeing by the user to reclaim file space. Removing a user from the system is tantamount to freeing a sub-pool.

The EXEC language in CMS is easy to use and subroutines therein are represented as separate files. This means that after several years of use one's directory holds hundreds of routines in the form of files and there are no tools to help the user remember which routines indirectly support what functions. CMS thus puts the burden on the user without giving him the tools to do the job.

Algol-68 gives the programmer two choices when allocating new {core} storage, with one choice, storage is freed as the current block is exited. With the other, storage stays until the "end of the job" unless garbage collection can manage to free it. This is obviously a compromise leaning toward what is easily implemented rather than what is needed.

We already have sub-directories. I would like to see if the ideas of blocks in block structured languages can be applied to our sub-directories. This means that a subdirectory has an explicitly namable space bank that can also be conveniently implicitly named in some commands. Banks associated with other (higher?) directories could also be explicitly named.

Commands packaged as Domains
I would also like for all but a small fixed number of the commands to be implemented within domains and that the user supplied commands be supplied by providing such domains.

Some of the fixed commands that I can think of are: Jump to another directory, Logout, Show directory, Delete directory entry,

Of the domain packaged commands a pervasive problem is whether the command system is responsible for remembering whether such a command is discreet. Whether, indeed, such commands have their own memory.

A mail system interface would, most likely, be a packaged domain which is certainly not discreet.

Some of the people involved with the DOD security process have expressed the idea of a proxy that would help the user sort out what information was allowed where. We are not here talking about the proxy being the policeman, but merely the policeman's assistant. This need is over and beyond the "absolute need" for mandatory security.

We assume some syntax, different perhaps from the general keycall, that is designed for calling those entries designed to be called as commands. We term this the "general command style key call" in distinction to the "general keycall command" that is able to issue an arbitrary key invocation.

Of the commands of our current system {(p3,com)} lets see which ones cause any problems and what the form of the mold should be:

AUXDEF
The domain behind this command takes sysno and the EBCDIC text following sysno as data parameters.

The key designated by keyname2 is passed as a key parameter and the returned key would be installed in the user's directory as filename1.

You can not greatly improve on the general command style key call to improve the syntax of calling AUXDEF.

AUXFILE
There is a funnyness here: "directory-name" is either a key source or a key sink depending on the choice of SEND or GET.

I would replace this command with two, one for get and one for send. Then the general command style key call does fine again.

CMDFILE
I think that I would like to see this function disappear because it is redundant with other ways of programming the system. If it is needed, however it must be one of the fixed commands {unless we do a UNIX shell trip}.

COPYKEY
I would do without this command out of parsimony and use either the general keycall or the command style key call to RETURNER.

CPUTIME & START & STOP
The underlying semantics of this will probably change. I won't consider this case.

DEBUG
This raises only the issue of calling conversational programs. More on that later.

DIRECTORY
This may be another fixed command. The proxy may wish not to give the user the key to the directory proper.

EDIT
Two issues here:

Do we support the CMS function of producing a new file?

And then the old issue of a conversational program.

EXECUTE
I shall presume that this command is on its way out.

FACTORY
No special problems except that it would be easier, in some ways, to speak to the builder's key directly with the general keycall command. That way you could specify the components in any order.

FORK
As in execute.

KEYCALL
A fixed command

LOAD
Obsolescent

LOG & LOGOFF
Fixed.

ZAP
OK but general keycall does well here too.

Gleaned syntax requirements:
This is a collection of requirements for the syntax, that was gleaned from the above exercize.

The general call syntax should make manifest which names provide keys to send and which provide names for returned keys.

Provisions must be made for conversations with talkative and inquisitive commands such as DDT in DEBUG.