In KeyKos the “ideal” development plan was to propose and implement very well defined and documented functionality. Small teams, usually of just one person, would:

• Document a proposed function, perhaps with all(!) usage details filled in,
• Ask for comments from other KeyKos developers,
• Iterate the above,
• Write code to provide the function,
  • normally this code would rely only on functions previously produced by this development pattern.
• Test in his own environment,
• Deliver the functionality by modifying the system so that there would be a generally available factory requestor’s key to produce objects that provide the functionality.
• Advertise the availability of the factory.
• Wait for bug reports which would normally be accompanied with a key to the broken (or misbehaving) object.
• Diagnose the problem using tools that only the particular code developer had access to (the factory builder’s key),
• Either:
  • Produce better code, produce a better factory therefrom (not deleting the old factory and thereby ruining extant objects),
  • or improve the documentation to be more accurate or less confusing.
• Replace the old factory requestor’s key in the public directory with the new key.
• Iterate the previous five steps.

Note that the above does not involve releasing the source code. Key Logic (the company) was fairly good at keeping the source code in a safe place along with magic (ad hoc) build code that seems so often necessary in all sorts of systems. This process was considered to be outside the scope of the above release procedures. It supported the model of independent software vendors better than most release procedures that bundle source with function in the release.

The above pattern implies a partial ordering, older, of object types such that the code for a type would only presume availability of older types and thus an object of the new type would be composed of only older types.

Note that this has significant implications to the problem of “library versions”. The problems that it solves arise when new shared library code replaces old code.

• With persistent objects there may never come an instant when there are no processes executing the old code.
• The new code may have function that breaks clients of the old code. Assigning blame here is not the point.

• old code exists along side new code and benefits of sharing are reduced.
• old code with known bugs may break users of the old code.

Our experience is that this leads to many fewer problems than problems of new code breaking old clients. Sometimes it may become clear that extant objects obeying the buggy code must be converted to the fixed code, or more accurately, must have their X type objects replaced by the new objects, having the same (corresponding?) state, but new behavior. KeyKos provided no set solution to this problem but there were several ad hoc solutions described. In the several years of operation none of these solutions were required but it was nonetheless important to argue that problem could be solved when required. We rehearsed some successfully, which made us feel better.

Another issue came into focus: long lived objects that process messages from potentially hostile sources. This is a fertile area for serious bugs when the buggy object maintains state shared between its clients. The source of the hostile messages may exploit a bug to gain the authority of the object and use that authority no longer limited by the benign behavior of the object. This is called “escalation of authority”. Alternatively it may alter the shared state of the compromised object in a way harmful to the other clients of the object.