Bit and Packet Costs

Dumping ground for hardware cost guesses.

I want to compute the cost of moving a bit or a packet thru the net. First I will compute the cost of its passage over a link and a node.

Bit Cost

Most bits flow in large packets with minimal overhead. They occupy the fiber for:

10⁻¹⁰ sec for a cheap 10 Gb/s fiber.
10⁻¹² sec for a very expensive fiber. (Cost is in equipment to light the fiber, and in the longer ditches.)
8∙10⁻¹⁵ sec?

I assume that the bit is deposited and retrieved, to and from SDRAM DDR3. There are two costs here: transmission time over the DRAM and DIMM pins and dwell time as it lives in memory. The latter concerns how long we buffer the packets. We buffer for link queuing plus the time to transit to the fiber, plus the time in the fiber, plus return of acknowledgment. I assume here that error control hardware computes and checks error codes. Note that payload does not make it to the local cache.

Bits in flows that are rate limited by network or destination bandwidth limits need about twice the buffering I think.

If the bit is moving between arbitrary places on Earth then we suggest that there are about 2³⁵ possible such destinations and this will involve traversing about 18 levels of logic involved in switching.

I see $0.90 for a 2Gb DDR 3 RAM chip for 2012 Feb.

Packet Cost

I count one or two cache line misses as a CPU forwards a packet. Also about 200 instructions should suffice. I implicitly assume ‘gather-mode’ DMA that takes a list of blocks of memory to be put onto the fiber. This avoids the CPU copying the data as one DMA delivers packets to DRAM and another picks it up from the same place. The CPU needs to be able to send packet headers from cache back to RAM. This involves NorthBridge architecture issues that I am not current on. This bears study.

This page about Intel’s ‘Thunderbolt’ suggests that run-of-the-mill computers are unlikely to provide more than one 10 Gb/s port.

Concrete product: DIMM 2GB. DQ0 thru DQ63 data pins. about 1Gb/s each. about 50 Vss, about 17 Vdd

Very roughly a one dollar DRAMM chip holding 2 Gb can supply data at 8Gb/s and is a good match to a fiber end.

Perhaps $10⁶ can buy a switch for 5 26Tb/s fibers. Perhaps I can get a 10⁷b/s video circuit to Akamai going thru 10 switches. Those switches can each serve 10⁷ customers like me; I am using $0.10 of capital equipment in each of those switches. $1 capital equipment all together. Most people can afford that. The good news is that hardware design scales down without losing much economy of scale.