The “S” in “MESI” refers to the shared state of a cache line in the on chip cache. The following rules govern when the system memory bus has been wired as the PowerPC architects intended.

Two caches on the same bus may hold the same line (data from the same physical RAM address) if they hold the same value and that value is also held in the shared RAM. In this case both cache lines are in the state “S”. This is the state that the 603 does not allow.

State “S” arises when one CPU fetches the data for a line and signals other CPUs on the bus (thru snooping) what line is being fetched. If another CPU already holds the line (in state E) then that other CPU changes its line state to S and the new CPU loads the cache line with the same data and sets the cache line state to S.

In the same situation the 603, when informed that another CPU is loading the data, empties its own cache line (puts it in state I for invalid). The result is that two 603’s on the same bus cannot hold the same line in their respective caches at the same time.

Data that is shared by the two CPUs causes much extra bus traffic on the 603. Another limitation of the 603 is that snooping “distracts” the passive CPU in the follow way: When the first CPU must go to memory (cache miss) it signals the other CPUs in order to maintain cache coherency. The other CPUs must consider the signal. On the 603 this consideration prevents the local cache from serving its CPU for one cycle.

It is possible to wire a 603’s into an MP configuration so that snooping is disabled. In this case the operating system must prevent certain combinations of mapping table entries between the CPUs. This strategy is good for some applications but not others.

I do not know what the BeOS does in this situation.