File I/O support

The support for file I/O is by far the most complex and potentially confusing portion of the OS simulation support. The goal of the simulation is to make it possible for any OS program to read from and write into any Gnosis data structure.

Note, this is much more general than what OS permits:

For example, Gnosis record collections may be read through the BDAM, BPAM, BSAM, ISAM, or QSAM access methods. In OS, data structures can be accessed only by the appropriate access method. For example, the BDAM access method cannot access ISAM files. Since there is not a one-to-one correspondence between access methods and data structures, (i.e., BDAM and BSAM can access the same data structures) the user must be aware of the access method-data structure matrix when writing programs. Fortunately many high level languages, specifically PL/I, hide this problem from the user.

The Gnosis model of OS I/O
In Gnosis the OS simulator maps OS data structures into Gnosis data structures in such a way as to keep the essential parts of the OS structure and discard the unessential parts. Thus Gnosis records are unblocked and all variable length allowing mapping with OS blocked or unblocked fixed or variable length records. While this mapping may lead to some confusing behavior to the Assembly language programmer (Short records are padded with blanks and long records are truncated) the PL/I programmer will not be aware of this behavior because the PL/I run time libraries do not rely heavily on the behavior of the access methods to control errors (the PL/I libraries check for long records and short variables). For assembly language programmers an extensive description of the mapping assumptions and restrictions can be found in later sections.

Gnosis File Technologies
Named Sequence Record Collections
Named sequence record collections can be used for any OS access method. Because of the overhead involved in searching the directory, named sequence collections should be avoided for sequential access methods. Of course, named sequence collections are required for OS access methods that require keys, such as ISAM, keyed BSAM, and some forms of BDAM. For sequential files the simulator generates names (sticky line numbers) on output and strips off the names on input. The capability associated with each record is ignored except when the record collection is really the directory of simulated PDS (p3,bpam) when the capability is the key to the record collection of the member. Programs which need to access the capabilities associated with record collections must call the record collections directly, rather than through the OS simulator.

Entry Sequence record collections
Entry sequence record collections can be used only for sequential access methods and members of PDS's. Like OS sequential data structures there are restrictions on the ability to change or delete existing records. Because there is no directory search involved in accessing records in these collections efficiency is much higher. Entry sequence collections can also be used for non-keyed BDAM (regional-1) with reasonable efficiency.

The simulator will deal with either entry or name sequence collections where possible and treat those few cases were named sequence collections are required as OPEN failures.

Standard Co-routines
Sequential, uni-directional I/O is supported via standard format co-routines. Co-routines are most likely used to connect the output of one program with the input of another program. Three protocols are supported by the simulator, and the choice of protocol is made externally to the program when the co-routine is defined to connect two domains. Each protocol has advantages in certain environments.

With the standard byte stream {(p3,sbs)} protocol, one co-routine call is made for each OS record. The OS record is stripped of all trailing blanks and a new-line character is appended. Standard byte streams closely resemble UNIX pipes and are specifically designed to communicate with terminals. The connection of a domain with a terminal via a user defined co-routine is not possible at this time.

With the standard record stream (p3,srs) protocol each co-routine call transmits 1 OS record without modification (the record is first converted into a standard Gnosis record which is variable length and unblocked). As with record collections padding and truncation take place on input but attempts to write records larger than the consumer is willing to accept results in a SYNAD. Since the simulator interface with co-routines sets up a consumer that will accept 4096 bytes, this is not a problem when connecting two OS programs.

With the standard blocked record stream (p3,sbrs) protocol multiple OS records are blocked until the total co-routine buffer is filled. Each logical record is preceded by a halfword length. Although a logical record may span co-routine calls, the simulator generates no such records unless the logical record will not fit on one page (see section on long records). Because of the blocking this is the most efficient of the three protocols. However, since the transmission of a record will not occur unless a buffer is full, it should not be used in situations were the results of an operation need to be immediately apparent to a user (i.e., records come in bunches after long delays. A message may get stuck in a buffer for a long time).

Segments
Segments are supported as sequential files. More exotic uses of segments is dependent on methods of identifying the segment type to the simulator. At this time only the file transfer segment {(p2,record-file-format)} is supported. The segment is mapped into the program's address space {above 800000} and records are moved into and out of the segment sequentially. On input EOF will be signalled when all of the records have been transferred {as indicated by the record count word at the beginning}, and on output the record count word will be kept up to date at all times. Record lengths up to 65535 will be supported.

Tapes
Tapes are supported as strictly sequential files. The information that controls the appearance of the tape (tape block size, format, etc) is taken from the OS DCB at the time the tape file is opened. Tapes are standard IBM labelled tapes unless specific action is taken to make them unlabelled. Records are limited to 4096 bytes in length (excluding the record length word in variable records) and block sizes of 32768.

Mappings between OS access methods and Gnosis File Technologies
BSAM
The strictly sequential operations of BSAM are supported by any Gnosis data structure. Non-sequential operations such as BSP (backspace) and rewrite require Record collections. For Entry Sequence record collections each record has a record number assigned by the Record Keeper. For Name Sequence record collections each record is assigned a record number (sticky line number) by the simulator. This record number is an eight digit number incremented by 10 for each output record. When data is added to the end of the data set (by specifying DISP = MOD on the FILEDEF) the last record is read to determine the correct record number for records that are added. For co-routines, records are simply passed to and from the co-routine domain. For segments records are transferred to and from the segment and the user buffer. For Tapes records are transferred to and from the tape and the user buffer.

QSAM
QSAM mappings are exactly like BSAM. The interface to the OS program is slightly different in that in QSAM records are not blocked as they are in BSAM. This is insured by special treatment of the DCB. The DCB buffer pointers always indicate a buffer full condition so that attempts by programs to "help" QSAM by doing their own buffering (specifically PL/I libraries) are defeated. If the DCB specifies blocking, records will be blocked when written to tape. This may seem to be somewhat cumbersome, but the innards of the OS access methods need not be aware that the data is going to tape and will be blocked.

BPAM
In OS PDS's are really BSAM files with a directory at the beginning. Each member is terminated by a disk End of File. The act of FIND is to locate the correct TTR for the beginning of the member and POINT the DCB to that TTR. From that point on BSAM rules apply. This is not true in Gnosis.

In Gnosis each member is a separate record collection {only record collections are created by the simulator but a member may be any Gnosis data structure} and the directory is a separate record collection that is exactly like a Gnosis "directory". In this way each member may be treated separately with any OS access method {even ISAM can be used, though when the member is read as part of a PDS the keys will be stripped and the file will be accessed sequentially} and treated as an aggregate as a PDS. This unusual treatment leads mostly to advantages as any Gnosis facility can be applied to members of a PDS without knowledge that it is in fact a member of a PDS. Catalogs can be processed by OS programs as if they were PDS's. Dissimilar things can be assembled together as a PDS.

Tapes may not be members of a PDS.

However, some OS programs will not work on PDS's as defined in Gnosis. Programs that read the directory {IEHMOVE, IEBCOPY, etc.} will not work. There is at present no way to have an ALIAS since OS treats the TTR of a member as its "true name" and there is no equivalent in Gnosis. The only practical program that will fail is the OS link editor. It is not likely that this will pose a long term problem.

ISAM
Indexed files are mapped into Name Sequence record collections only. The OS key is extracted (embedded or not) and used as the Gnosis Name. The OS record is written as is (key and data) as the data portion of the Gnosis record. Only unblocked (or blocked 1 record per block) records are supported. Records are limited to 4095 bytes minus the length of the key. Since named collections can be processed as BSAM and QSAM files, files created by ISAM can be processed as sequential files.

BDAM
BDAM comes in two flavors, keyed and non-keyed. Keyed files have two subtypes, fixed records and variable records. Each of the three types is handled differently in Gnosis. Keyed BDAM is very similar to ISAM in that the key is extracted from the block and used as a name for a Named Sequence record collection. Therefore, keyed BDAM requires a Name Sequence record collection. Unlike ISAM the key is appended to a TTR (three bytes) so that the BDAM search operations will work the same as in OS.

In OS BDAM is very dependent on the behavior of Count, Key, Data disk devices. Each track represents an area similar to a VSAM control interval and can contain multiple records with keys. Several tracks can have records with the same key though keys must be unique within a track. In Gnosis the key is combined with the TTR to simulate this behavior. The TTR used by the simulator is calculated based on the Record per Track information that is supplied by the DEVTYPE macro. In Gnosis all simulated devices are 3330's. Since the number of records per track is dependent on the record size, the user must not change this parameter between uses of the file. This is consistent with usage in OS. There is a certain amount of overhead involved in this approach and keyed BDAM should be avoided.

Non-keyed fixed format files can be mapped into either Named or Entry Sequence record collections. The Block Number supplied by the application program is simply the Gnosis record number. In Named Sequence collections each block is simply written as a record with the block number as the name. In Entry Sequence collections the missing records are filled in as needed with dummy records so that the record written will have the correct record number. Records that are longer than 4091 bytes require very special treatment in BDAM and will be discussed here. For a complete discussion of long records see the section Long Records. With Named Sequence collections records cannot exceed 4087 bytes. With Entry Sequence collections records can be up to 32767 bytes in length and therefore will span several Gnosis records. In this case the record number will be a fixed multiple of the block number. Since all blocks are of the same length all OS operations are preserved.

Keyed fixed format files can be mapped only into Named Sequence collections. Here the Block Number supplied by the application program corresponds to a disk TTR that is the nominal place to write the record. If the TTR is currently empty (contains a dummy record) the record is written there. If the TTR is in use the next available one is used. In Gnosis the TTR is appended to the key and a unique name is generated. If there is a name in the collection with this TTR as its beginning (determined by searching on a 3 byte name) then the TTR is incremented to the next valid value (Gnosis simulates 3330 devices) and the first available TTR is used.

Keyed variable format files can only be mapped into Named Sequence collections. Here the Block Number supplied by the application corresponds to a track number and the first available TTR on that track is used for the record. As in fixed format files the TTR is added to the beginning of the name and a short key read is used to find the first unused TTR on the track. If the track is full the next track is used.

A Mapping Matrix
BSAM QSAM ISAM BDAM BDAM BPAM non-keyed keyed Entry Sequence x x x x Named Sequence x x x x x x Co-routines x x x Segments x x
x Terminals x x x

Logical records and Blocked Records

The OS access method portions of the simulator behave the same regardless of the Gnosis protocol. Each Gnosis data structure is handled by a structure dependent routine chosen and OPEN time. The OS access method portions of the simulator convert OS records into a simulator standard record which has two parts, the data itself and a length word. These two parts are transmitted to a data structure translator module that communicates with the Gnosis data protocols.

The Gnosis record collection protocols support logical records of approximately 1 page in length. For Entry Sequence collections the maximum record is 4091 bytes as 5 bytes are required for record identifier. Name Sequence collections support records of 4095 bytes minus the length of the name. When sequential non-keyed records are mapped into Name Sequence collections, the OS simulator generates an 8 byte name so the maximum record length is 4087 bytes.

Longer records are supported only with Entry Sequence collections and co-routines. Attempts to support long records with Name Sequence collections would destroy the "sticky line number" concept. With Entry Sequence collections logical records up to 65535 bytes are supported by breaking each logical record into 4087 byte chunks each preceded by a record control word. The first chunk has a control word that contains the length of the logical record, the middle chunks contain control words of zero, and the last chunk contains a control word with the left most bit on. If the record is in fact 4087 bytes or less the first word will contain the length and have the left most bit on. For co-routines the record is simply sent with as many co-routine calls as necessary. Either byte stream or block record streams can be used.

The information that a record collection contains long logical records is in the record collection user data area. The first 4 bytes of the user data contain flags describing various attributes of the collection that might interfere with standard processing

With the exception of BDAM all logical records stored in Gnosis data structures are unblocked. The BSAM simulation module contains code that unblocks the user records on output and provides short blocks on input. QSAM is normally entered once for each logical record and unblocking is not necessary. Since PL/I run time libraries attempt to cut down on QSAM overhead by blocking records based on information in the DCB, the simulator maintains that information so that PL/I believes the block is always full. For BDAM blocks are written as provided by the user as there is no Gnosis facility that can process these records. For ISAM files blocking is prohibited except for fixed block 1 record per block (required by COBOL). Attempts to open an ISAM file with blocking will result in an OPEN failure.

High performance data transfer that takes advantage of blocking can be achieved by using blocked record co-routines (p3,sbrs).

Spanned Records

OS supports logical records that are longer than the specified block size. This format is necessary when records are longer than a single track on the disk. Since the OS simulator handles long records transparently to the user, the need to support OS spanned records does not exist. FTNH, however, requires this format for unformatted I/O. Therefore, BSAM does support VS and VBS record formats. QSAM treats VBS and VS as an error.

For BSAM spanned records it is necessary to include a span byte with each record. This byte is normally part of the OS "green word" but Gnosis does not have this spare byte as part of the record length. For spanned records a byte is added to the beginning of the OS record and the OS record plus span byte is transmitted to the Gnosis data structure (record collection or co-routine). There is a flag bit in the record collection user data describing this condition.

Deleted records
Gnosis and OS handle record deletion in very different manners. In Gnosis, a deleted record is removed from the record collection; while in OS deleted records remain in the dataset, but are recognized by the application program as deleted if the record contains 1-bits (x'FF') in the first byte of the key. In PL/I the detection of deleted records takes place in the run time libraries so the PL/I programmer is not aware of the difference {ISAM only}. A program which reads sequentially through a file which contains "deleted" records will get different results if it counts records.

Deleted records do not exist, and cannot be read. OPTCD=L is assumed for all operations for which deleted records are allowed. Fixed records with RKP=0 and Variable records with RKP=4 may not have deleted records. {This restriction was carried over from OS for consistency.}

Record padding and truncation
Since all Gnosis records are variable in length and much data that is input into application programs will originate in Gnosis utilities (such as the EDITOR) the simulator pads all input records with trailing blanks when the application specifies Fixed format records. Likewise all long records are truncated. PL/I record conditions are not affected by this operation as these conditions are detected in the libraries when variables are transferred to and from the I/O buffers. These conditions are based on the relations of LRECL and the variable length. Of course for Variable length records there is no problem.

Locate/Move mode
Locate mode operations are handled as they are in OS. An I/O buffer is obtained at OPEN time and data is moved there for locate mode operations. Since the I/O is actually performed in another domain (if at all) data is moved either to the users area for move mode or the I/O buffer for locate mode. There is no advantage to the use of Locate mode I/O. PL/I does all QSAM I/O in locate mode regardless of what the user specifies.

Positioning of files
OS rules are followed for all file positioning at OPEN time. DISP=MOD must be used if data is to be added to an old dataset. For record collection files all run time positioning operations are supported. For co-routines, segments, and terminals all non-sequential operations will result in SYNAD conditions.

Concatenation of files
Concatenation is handled by FILEDEF. The OS simulator always asks FILEDEF if there are concatenated data structures when the end of each data structure is reached. Thus any number of any kind of structures can be concatenated into a single logical file. Record keepers, Co-routines, Segments, and Terminals can be mixed in any way that is supported by FILEDEF. BPAM members may not be concatenated as this is an unwarranted extension of OS facilities.

Multiple usage of a file
FILEDEF is described in the section (p2,filedef).

DCB parameters are filled in in much the same was as in OS. The simulator keeps information about the last use of a file in the user data portion of the record keeper. This information corresponds to the DCSB of OS and can be used to specify fully the DCB information for an old file. Unlike OS this information is totally unrelated to the structure of the data and is kept for programmer convenience. If the information is overridden when the file is OPENed the new information about the suggested use of the file will replace the old information.

The JFCB built by FILEDEF is first filled from the DSCB information (only for fields not specified on the FILEDEF command) and then the DCB is filled from the JFCB (only for fields not specified in the DCB). When the file is Closed the information in the DCB is written into the DSCB to be used the next time the file is opened.

When OS programs open multiple DCB's for the same file {DDNAME} the simulator copies the capability key for that data structure and sets up a completely separate control path for the data. For record keepers this works with no restrictions because all of the memory about position within the file is kept in the simulator control blocks associated with each data path. Since terminal protocols are handled specially there is no trouble, but with co-routines the results are a user call of a zero data key. For segments, as with Record keepers, multiple DCB's may result in odd looking files. No provision is made to control the order of entry into the segment. It is up to the user to imagine creative uses for this feature.

Output to record keepers with multiple DCB's will lead to an orderly distribution of records as the records will be interspersed as each data path is used to output data. OS handles this problem by warning the user that the dataset may be ruined by doing this.

DDNAME'S {FILEDEF and such}
The OS simulator has an association of DDNAME's with keys. This is done with an object called a "FILEDEF". FILEDEF is consulted when the OS programs refers to a DDNAME. See (newfdef) about the initiation of this mechanism and (p2,filedef) about the nature of this mechanism.

Files that are undefined by FILEDEF default to the terminal system {(terminal)}. The DCB parameters are filled in as 80 byte Fixed length records. This allows considerable freedom from OS labors as many programs need not have any files defined by FILEDEF at all.

If there is no information about the format of a file then Fixed length 80 byte records is the default.

Private Communication from Charlie Landau

There are two places where DDNAME's are stored. One is in the OS simulator itself through the OSFD key. The manual needs to say that a fresh OSSIM has no DDNAME's initially.

The second place is in FILEDEF through the FILEDEFQUERY key you {sometimes} supply {type A only}. Whatever you defined appears here. I think FILEDEFQUERY is searched after OSFD.

The support for terminals is a bit kludgy at present. If a file is a terminal, either through filedef or default, it is the one and only terminal, found in slots 3,4,5. Suggestions will be entertained by Alan or me.

Terminal communication
Both the OS Operators terminal {the WTO macros} and the application program terminal {the TGET macros} are supported by transmitting to the co-routine key found in cache slot 4 and receiving via the co-routine key in cache slot 3.

A PL/I callable interface to terminal I/O which is much more efficient than the standard OS interface has been provided. The subroutines save 4 gate jumps per terminal transaction. Someday there will be a section in the manual about $TIN and $TOUT. See Bill Frantz about this. It was done for the Adventure and Midas applications.

If a DDNAME is not not declared as a terminal to FILEDEF then the BSAM access method will use co-routine keys {such as to terminals} without incuring the OS domain overhead.

Dynamic loading
The LINK, LOAD, XCTL and DELETE SVC's are only partially implemented at this time. The services provided are limited to locating the desired component in the application program domain, and returning the address of it {or in the case of XCTL, branching to it}. All components which are requested via the dynamic loading SVC's must be included in the memory tree of the domain requesting the service.

The addresses are determined by the OS simulator scanning tables of components which are included within the simulator itself, the PL/I library, and the application program.

If it is desired to execute an OS program which uses dynamic loading SVC's under OS, it is necessary to define a CSECT with the entry point name GNODLOAD and entries in the following format:

GNODLAD CSECT , DC CL8'LINKPACK' header DC CL8'SEGMENTNAME' an identifier for debugging DC CL16'&SYSDATE &SYSTIME' time and date of assembly DC CL8'&name',v(&module),v(&entry) one entry/component DC X'FF' end of table indication

Normally, the module origin (&module) and the entry point (&entry) will be the same external symbol name. &name is the name which the module will be referred to in the dynamic linkage SVC. It is also often the same name as the module name.

The modules described in GNODLOAD must be linkedited with the application program and GNODLOAD under CMS. When the Gnosis initialization program {GNOIBM} is invoked as the application program begins execution, it will complete the linkage between the OS simulator and the list of supplied "dynamic loading" components.

Since components are not physically loaded and unloaded by the OS simulator, any program which invokes a dynamically loaded routine multiple times, and expects it to be reinitialized each time, will probably not work as intended.

Dynamic memory allocation
See (p2,os-frs1) or (p2,os-frs2) for the extent of free storage.

Currently, the simulator gives a maximum of 1 megabyte of memory per Getmain request if variable size specified. For fixed size getmains, the requested size will be allocated if possible.

The memory allocation routines attempt to reuse storage and keep memory allocated densely. Memory is allocated from lowest to highest address.

Clock services
All of the time of day clock facilities supported under OS and MVS work under the Gnosis simulation.

The interval timing facilities are less complete. Elapsed time measurement and delay facilities are not implemented. CPU time measurement facilities are implemented, however, if the time interval specified is exceeded, the exit routine will not be called. {The code is written, but not debugged.}

Program Control (SPIE and STAE)

PL/I Checkout compiler

Attention Handling

VSAM