Byte Stream to Segment Converter Creator

BSTSCC(0;SB,M,N ==> c;) "Create Byte Stream to Segment Converter"
N is a node or fetch key to a node containing:
slot 0 - a key to a segment with write access

slot 1 - a key to a byte stream producer

A byte stream to segment converter is created. It reads the byte stream and stores the bytes into the segment as soon as they are produced. The user retains a key to the segment and can examine the bytes as soon as they are received.

The first 8 bytes of the segment will have the number of bytes read so far. This number is stable while the byte stream is in the producing state. The bytes from the byte stream are stored consecutively beginning at byte 8. Up to 2**48 -8 bytes can be received.

The segment can be a fresh segment, that is, the pages can be defined in the address segment in response to store instructions from a domain. However, once defined, a page should not be dematerialized.

If the last byte received occupies the last byte of a page, the next page is materialized also.

When and if the byte stream signals EOF, the byte stream to segment converter will destroy itself, leaving the segment with all the bytes received. There is no notification that EOF has happened.

C is zero if successful, 1 if SB is not a prompt official space bank, 2 if not enough space in SB, 4 if N is not a node or fetch key.

BSTCC(kt;==>c;) returns X'D0D' in c.

STRCCC(0;PSB,M ==> c1,,,K) "Create Segment to Record Collection Converter"
PSB is an Official Prompt space bank. M is a meter.

If c1 = 0 Then K is a key to be called to perform the conversion as follows:

K(n;S,R ==> c2;) "Perform the Conversion". R is a key to the record collection to which the records are to be added . It must be an entry sequenced collection {(p2,rcc)}. S is a [read only] segment which holds the records in the file transfer protocol format {(p2,record-file-format)}. The maximum length of each record is 4090 bytes. It is the caller's responsibility to ensure that the keys passed as the Segment and the Record Collection are actually of those types. Outcomes:
c2 = 0 - Successful completion

c2 = 3 - Error during record collection write operation.

c2 = 4 - Supplied record collection created with names.

c2 = 5 - A segment record exceeds the maximum legal length.

If c1 = 1 Then the space bank is not a prompt space bank or If c1 = 2 Then there was a problem creating the domain. In either case K is a zero data key.

STRCCC(kt;==>c;) Returns x'18' in c.

ESTONSCF(0;PSB,M ==> c1;ESTONSC) "Create Entry Sequence to/from Name Sequence Record Collection Converter"
PSB is an Official Prompt space bank. M is a meter.

If c1 = 0 Then a new ESTONSC is returned.

If c1 = 1 Then the space bank is not a prompt space bank or If c1 = 2 Then there was insufficient space.

ESTONSCF(kt;==>c;) Returns x'30F' in c.

ESTONSC(0;IN,OUT ==> c2;) "Convert".

Case 1: IN is an entry sequenced record collection (ESRC) {(p2,rcc)} and OUT is a name sequenced record collection (NSRC).
The records of IN are copied to OUT. The ESRC names are discarded. The records must be formatted as: (1,nlen),(nlen,name),(string).

Case 2: IN is a name sequenced record collection (NSRC) {(p2,rcc)} and OUT is an entry sequenced record collection (ESRC).
The records of IN are copied to OUT. They are appended.

Outcomes:
c2 = 0 - Successful completion

c2 = 3 - IN is ESRC but OUT is not NSRC.

c2 = 4 - IN is NSRC but OUT is not ESRC.

c2 = 5 - IN is neither ESRC nor NSRC.

c2 = 6 - An error occurred writing NSRC OUT.

c2 = 7 - An NSRC record was too large to convert to ESRC.

c2 = 6 - An error occurred writing ESRC OUT.

ESTONSC(kt+4; ==> 0;) destroys the ESTONSC.

ESTONSC(kt; ==> X'20F')

PROTCC(akt;PSB,M,sb ==> c;PROTC)
This key creates a PROTC {(protc)}. PROTCC(kt; ==> X'011F')

PROTC(lrecl;STRU1,STRU2 ==> c)
This key uses STRU1 as input and STRU2 as output. LRECL is used to determine the maximum length record expected. Since the OS simulator supports long records (greater than 4094) by appending them to a "green word" it is necessary to know in advance if this is the case. This number is actually used to fill in the LRECL field of an OS JFCB before a DCB is opened.

If LRECL is negative, STRU1 is appended to the end of STRU2.

STRU1 and STRU2 may be any structure supported by the OS simulator. The OS simulator supports Record Keepers, Byte and Record Streams, File transfer segments, remote files, and tapes.

Return codes
1 - input filedef failed

2 - output filedef failed

3 - input open failed

4 - output open failed

5 - input SYNAD

6 - output SYNAD

PROTC(kt; ==> x'1f')

SYSPC(0;bank,meter,switcher==>c;CR) returns an entry CR that behaves as a byte stream consumer. Consumed bytes will appear on a new branch of switcher. Being an entry, it may be reused. Each reuse of CR will create a new branch on the switcher {(switcher)}. If a call on CR signals EOF then no branch is created. You can find SYSPC in sub-directory SYSTEM of your directory.

SYSPC(1;bank,meter,switcher==> c;,,,EX); EX(0;((sl,sig))==>c2;SYSP) returns SYSP where the first output on any branch created by SYSP is the ASCII string sig. The string sig will be truncated to 16 characters.

This object is a SYSP replacement (p2,sysp) that does not use branches on a multiplexor for sharing access to a "terminal". In fact, it is possible to avoid output to the terminal altogether which is desirable when using Plipacker to build things as part of a script. The SINK key is an entry key that can be used by several domains as if they were using a SOK key.

SINKF(0;sb,meter,[sok] ==>c,SINK,SOURCE) create a sink.

The SOK key is optional. If present the SOK key will be called for each record passed to the SINK key. This is useful as the equivalent to SPOOL CONSOLE START TERM.

SINK is an entry key that behaves like a SOK key.

SOURCE is a file transfer segment which contains all the data from calls to SINK.

Return codes - standard factory return codes.

SINK(0,(string); ==> 4096;,,,SINK) put a string into the segment and optionally to the SOK key.

SINK(255; ==> c+,,,SOK0 Empty the segment and return the SOK key. This is equivalent to SPOOL CONSOLE STOP PURGE.

SINK(kt; ==> x'1a',(1,0))

SINK(kt+4; ==>) destroy SOURCE and SINK

Introduction
This is a cleverer facility that fills a need similar to SYSP. It is for one-way flow and switching and is controlled by the presence of data instead of switcher commands.

There are funnels and funnel branches. A funnel holds a SOK. A funnel branch consists of a label which is either an ASCII or EBCDIC string, and a reference to a funnel.

Like an SYSP, a funnel branch may be conveniently mistaken for a SOK co-routine. If X is a slot holding a funnel branch key then X(0,string;==>c;,,,X) delivers the string to the funnel branch just as if X were a co-routine and X is unchanged.

Delivering a string to a funnel branch results in that string emerging from the referenced funnel's SOK suitably identified with the funnel branch's label.

FUNNELC(c,SB,M,SOK;==>c;FUN) returns a funnel key FUN. If c=0 then FUN's working storage is a page and if c=1 it two pages. FUN holds SOK. The yield of SYSPC {(sysp)} has been observed to work well here.

FUN(0,LAB;==>c;,,,FB) returns a funnel branch FB referring to FUN and with label LAB. c is the initial sok limit.

There is a limit of about 250 funnel branches per funnel. Another limit arises from the fact that FUN's working storage must hold each branch label. Transmitted strings must fit in FUN's working storage not devoted to branch labels.

FB is a pseudo SOK which passes its input to FUN's SOK with FB's label prepended when the output from FUN's SOK switches from another branch.

FUN(kt+4;==>0;) Deletes the funnel and its branches.

Design Note

Programs such as the PLIPACKER create a funnel at the beginning of an operation, attach it to the user's switcher, distribute funnel branches to the various subcontractors who might want to report, and delete the funnel at the end of the operation.

BSCR(F;SB,M ==> c;BSC,,,BSP) where F is 0, 1, or 2 {"Create Byte Stream"}
M is a meter. c is 0 if successful, 1 if SB is not a prompt official space bank, 2 if not enough space in SB. BSC and BSP are associated co-gates. Either one may be called first. The domain(s) that they designate will be reclaimed when a transfer begins. F is the data format number {(p3,fn)} that defines the calls BSC(kt...) and BSP(kt...).

BSCR(kt; ==> X'1B';).

If you hold a memory key and you are in a position to invoke keys but not to access segments then a CALLSEG object allows you to access the segment.

A CALLSEG object has a memory slot. The object's "segment" is the segment designated by the key in its memory slot.

If C is a CALLSEG object then:

C(0;==>0;M) returns the key, M, from C's memory slot.

C(1;M==>0) places M in C's memory slot {even if M isn't a memory key}.

C(2,(6,addr),(2,n); ==> c,str;) requests n bytes from C's segment starting at addr in the segment. n must be <= 4096.

If c = 0 the access was successful and the results are returned in str. See (callseg-error) if c isnt 0.

C(3,(6,addr),data; ==> c;) requests data to be placed in C's segment starting at addr. Length of data must be <= 4000.
If c = 0 then the operation succeeded. Otherwise c is the segment access trap code. In this case some of the data may have been stored. See (callseg-error) if c isnt 0.

C(kt+4;==>;) deletes the object.

C(kt==>X'110D').

For orders 2 and 3 the return code is not 0 when there was either a non-zero return code from the keeper of the segment or when there was no keeper. In either of these cases c is not 0.

If there is no segment keeper defined for the specified address in C's segment and that address is not defined, then str = ((4,1),(e_code,1)) and c = e_code. The value of e_code is described at (p1,err-codes).

If the segment keeper returns with a non-zero return code then c=that code and str = X'0200'. See (segrc) for conventions regarding such codes.

CALLSEGF is a CALLSEG object factory. It creates CALLSEG objects whose memory key is DK(0).
CALLSEGF(0;PSB,M,SB==>0;CALLSEG)
Where PSB is a prompt segment, M is a meter, and SB is a spacebank. The return codes are as from the factory.

CALLSEGF(kt==>X'120D')

Use of the TP bit in memory keys.
The introduction of the TP bit memory keys presents an oportunity for and a obligation on CALLSEG. The oportunity is to put in one piece of code a kludge that prevents unnecessary page population even when running on a terminating machine. Such logic can be put in applications when performance dictates, but it may otherwise be centralized in callseg.

Suppose that CALLSEG did all writes directly via the memory key in its memory slot, but does reads via a RO key that it interposes.

The benefit of this scheme arises for segments with a keeper that creates new pages upon write exceptions, while running on a machine that may terminate writes. In such cases the new page is not produced since the kepper is not notified.

There is a significant cost however: When CALLSEG is used to alternately read and write a page of a segment it will do so via addresses with different authority. This causes the kernel to remap the page upon each reference. This takes 625 microseconds per complete cycle on a 4341 (1 mip).

An adaptive scheme
There are two modes called 1 and 2. In mode 1 all reads and writes are via one NOCALL window. In mode 2 reads are via a RO window and writes are via an unfiltered window.

Reactions:

When CALLSEG's keeper is invoked in mode 1, switch to mode 2.

When CALLSEG's keeper is invoked in mode 2, reply to CALLSEG's client.

When CALLSEG causes several kernel remaps in mode 2 since being in mode 1, switch to mode 1.

Observations:
The machine model can change during CALLSEG's execution, but the model cannot change during one memory access.

Either mode is safe on either machine.

Headway is impossible in mode 1 when either:

The segment's keeper must be invoked,

A read must be performed from a page that is (TP, RO) in the segment and the model terminates.

Mode 1 is inefficient when the segment keeper is invoked.

Mode 2 is inefficient when there is much switching between read and write.

Analysis for the non terminating machine
When CALLSEG is producing many invocations of the segment keeper, mode 2 will predominate.

When CALLSEG is switching rapidly between read and write mode 1 will predominate.

Analysis for the terminating machine
There is one extra reason to go into mode 2 on a machine that terminates. Otherwise the situation is the same.

An elaboration of the above adaptive scheme.
The above worries about threshing of page table states. It does poorly when there are several tables, infact it may be counter productive. The following may bemuch more efficient and not much more complex.

Imagine a cache of six entries. Each entry would remember a one word offset (high 32 bits of 48 bit offset), an offset state that traverses a state map described below, and an ennumeration value, fmt from {NOCALL, RO, PLAIN}, and slot pair name.

Logical assertion is:

The six slot pair names are distinct and are each one of (4, 6, 8, 10, 12, 14).

For each cache entry, the offset in the window keys located by the slot pair name are that of the offset field in the cache entry and the window key's control bits match the ennumeration value.

The performance expectations are:
The array of cache entries are in LRU order.

For each cache entry, its offset probably corresponds to a produced page table whose format "matches" fmt.

Searching the Cache
A cache entry with fmt=NOCALL or fmt=RO suits a read operation. A cache entry with fmt=NOCALL or fmt=PLAIN suits a write operation.

As you begin to search the cache you know the offset and whether you need to read or to write. You scan the entries in order trying to match the offset. If the offset matches but the state does not suit, you remember the state and resume the search. When you find a zot with matching offset and suitable state you compute a new state from the remembered state (earlier in the cache).

This scheme means that there is one effective state per offset, the first one in the cache.

The state diagram
The events are: read, write, fault while accessing thru NOCALL segment.

TYPE(0;PSB,M,SB,OBJECT,SOK6==>c) - "Type object in EBCDIC format"

TYPE(1;PSB,M,SB,OBJECT,SOK6==>c) - "Type object in hex with EBCDIC translation"

PSB,M, and SB are prompt space bank, meter, and space bank respectivly

OBJECT is the data storage object to be typed. It may be a record collection (either name or entry sequenced) or a segment in file transfer format.

SOK is an output key. It may be a SOK6 key or a SOK3 key in EBCDIC mode.

Return codes:

0 - All OK, object typed

1 - PSB not prompt space bank

2 - Not enough space

3 - Error message printed

Error messages:
Return code nnn, X'xxxxxxxx' from object
The record collection OBJECT returned the unexpected return code printed.

Return code nnn, X'xxxxxxxx' from callseg
Callseg returned the unexpected return code printed. This message is most frequently caused if object is not a segment or not in file transfer format.

Record length greater than 4092, terminating
A segment contained a record longer than 4092 bytes, this exceeds an implementation limit in type. No further records will be typed.

TYPE(kt==>X'040B5C65') (67853413 in decimal)