The Mind Meld Pattern
Sometimes a data structure is:
- expensive to compute,
- infrequently enough changed,
- and frequently enough needed,
that it is strategic to compute it one day and use it on other days without having to recompute it.
Of course this situation is very familiar to any programmer and that is one of the prime reasons that files were invented.
Sometimes, however, the natural format of this data is a linked data structure in RAM and then a problem arises: writing the information that is in a linked data structure, into a file, so that it can be recovered later to reform an equivalent linked data structure.
This is commonly done using one of two patterns:
- some form of ‘marshaling’ which removes the links in the data while preserving form and content,
- copying the heap into the file bit for bit.
where the heap is the normal habitat of the linked data structure.
The Pre Compiled Header is a familiar instance of this requirement.
Mind Meld
We would write the whole heap, just after some sort of compacting garbage collection.
Unless the allocation of virtual addresses is somehow coordinated between the reader and writer, the reader will find the resurrected heap data at different addresses whereupon the linked lists are kaput.
With unnatural connivance with the infrastructure and use of mmap it might be possible to resurrect the heap in a serviceable fashion, somewhat like a mind meld.
This is all a bit ghoulish.
Alternatively the code of the reader might be savvy on this hoax with an adjustment for access to such moved data, but such is not the sate of the art for modern compilers.
PL/I has the AREA construct which supports this.
With mmap some users of the data structure will not need to fault in most of the pages that hold linked data structures that have not been recently used.
The Keykos binder provided low level support for this in a first class way.
If someone sends me e-mail I will elaborate.
Functional languages or nearly functional languages such as OCaml or Scheme are adept at using immutable data structures in support of higher level mutable objects.
A terabyte linked list structure might be fetched and cached over a communications channel on demand.
In some cases this would cost vastly less than transmitting and reanimating the entire structure via marshaling.
Several of the ideas here may require connivence with garbage collection schemes.
Marshaling
Marshaling is application dependent, slow and complex.
It also often violates abstractions or information hiding.
Also many potential readers will need only a small part of the data and will have to pay for reanimation of the entire linked data structure before using any of it.
These extra costs are liable to entirely eliminate the advantages of the precomputation.