Link and Neighborhood Conventions

Note that in the global DSR notions we have not declared:

the layout of the packet,
byte order or turn op order,
how payload size is coded,
how SS details are coded such as:
- whether there is a return path,
- how turn ops are packed in octets.
The currency of the money field.

All these may be safely left up to per-link choice. Of course some additional widespread agreement on these will save development costs and format conversion runtime costs. Some concrete SS format proposals , but here are two plausible bit-by-bit packet layouts that includes SS information:

Format A:

M: 32 bit money field,
B: 1 bit bool (whether a return path),
P: 15 bit payload offset within packet,
L: 16 bit payload length (if packets must define their own length),
s: 8 bit length of path-pair in (bytes|nibbles),
r: 8 bit count of (bytes|nibbles) in return path,
PP: path-pair:
- R: r*(8|4) bits = reverse of return path, all zeros unless B,
- D: (s−r)*(8|4) bit destination path,
8*P − (80+(8|4)*s) bits of padding,
8*L bit payload.

In this Format

Two link ends agree on the alternative between bytes and nibbles (bytes|nibbles).
Space for return path may be allocated but all zero even if the return path is missing.
Note that the return path grows into the space freed by the destination path.

A node modifies M to deduct its toll. It increments r as it consumes D and augments R. It is free to shrink the padding if it is inclined, or stretch it if that better suits the hardware at the link ends. It may even to reformat the packet if it is moving to an alternate packet format for the next link so long as it maintains the integrity of the payload, paths, and money field.

Format B:

M: 32 bit money field,

P: 16 bit payload offset within packet,

L: 16 bit payload length (if packets must define their own length),

s: 8 bit turn op count,

c: 8 bit path cursor—count of turn ops already processed,

p: (4*s) bit sequence of XORed turn ops,
8*P − (80+4*s) bits of padding,
8*L bit payload.

It appears that the choice between formats A and B must be global. I lean towards B just now. Either can be referred to as a datagram.

Special symbols in the data stream

We explore a link format that assumes special symbols preceding the packet header, and another preceding the payload. These special symbols are generated by the hardware upon sending as the hardware transitions between fetching the header from cached memory and the payload from uncached DRAM. It likewise causes the receiving hardware to switch where the data is deposited.
Format C (datagram on the fiber):

HS: 10 baud ‘New header symbol’, not deposited in memory.
t: 8 bit packet type: 0 for datagram.
s: 8 bit turn op count,
c: 8 bit path cursor—count of turn ops already processed,
x: 8 bit pad
M: 32 bit money field,
turn ops
PM: 10 baud ‘Payload symbol’, not deposited in memory.
Payload

This is a minimal version. The special symbols might be more than 10 baud if necessary to reduce the probability of seeing them when they were not transmitted. Other redundancy might be worthwhile like the payload length in the header. Depending on hardware design, we might include error control information following each PM and perhaps HS too. In case of forward error control an error syndrome might follow the header for the software to apply to the header and/or payload.

Format C: (header deposited by hardware in CPU’s cache):

t: 8 bit packet type: 0 for datagram.
s: 8 bit turn op count,
c: 8 bit path cursor—count of turn ops already processed,
x: 8 bit pad
M: 32 bit money field,
p: (4*s) bit sequence of XORed turn ops,
y: pad
PL: 48 bit payload address, supplied by hardware, not nearby,
PS: 16 bit payload size, supplied by hardware,
HL: 16 bit Length of this header, supplied by hardware.

An interrupt announcing a new block would report the end of the headers which were deposited contiguously in the cached RAM of the CPU. Note that headers are located by the software by consulting the HL fields in a block in reverse order. They are probably processed in forward order. The software must verify that there is enough room in a header for the turn ops promised by the turn op count. The software and hardware might find it mutually agreeable to align the PL field on some power of 2 like 4 or 8. See this about allocation of DRAM for payloads.

Format C: (Prepared by software to drive transmission):

PL: 48 bit payload address, not nearby.
PS: 16 bit payload size,
HL: 16 bit Length of this header.
t: 8 bit packet type: 0 for datagram.
s: 8 bit turn op count,
c: 8 bit path cursor—count of turn ops already processed,
x: 8 bit pad
M: 32 bit money field,
p: (4*s) bit sequence of XORed turn ops,

The above is about datagrams. I presume that the hardware does not examine the packet type but deposits the header in the cached CPU memory up until the PM (payload symbol). The hardware observes some limit on header space; perhaps it refuses to write more than 64KB of headers per fiber block.

I think now that the PL field is always a multiple of 2¹⁶ in which case that PL need not be longer than 32 bits.

The hardware deposits all of the bytes between the HS (header special symbol) and PM () into cached CPU memory. Consecutive packets go into consecutive memory up until a 64KB boundary.

This feels concrete enough for some code.

See this about multiple sessions in a host.

I assume that more significant bits of an integer are transmitted before less significant bits.

I have not found a simple memory allocation scheme for storing packets in transit thru nodes but this seems suitable for intensive data flows. One scheme might want an early packet size field which could inform hardware where to put the packets. This is all tied up with error control and possible link encryption. One might want an intermediate convention of 65KB transmission blocks to simplify storage management in nodes. Such a block can accommodate 64KB packets or many small packets. It also makes latency due to block transit time for small packets about 6 μs.