Years ago I heard a comment from some compiler person at IBM—perhaps it was Fran Allen:
This is how I recall it:
There is scarcely anything of interest to a compiler that is not potentially of great interest to the programmer as well. Indeed the discipline of independent compilation makes many things unavailable to both the compiler and programmer, that would be useful to both.
A corollary of this includes the run-time debugger as a threesome. In this case the paucity of loader semantics deprives the debugger of what would often be sufficient to debug a highly optimized program. (The compiler has optimized away some variable that it necessarily knew how to materialize, and knew in some sense that a debugger could easily understand.)
Today a large program makes its way to the RAM locations where it will be executed thru a tree of linker operations preceded by compilations and likely preceded by textual preprocessing. Each of these steps builds useful information and then discards it. I don’t propose producing textual reports to be read by people, but merely leaving this intermediate data where it can be accessed, object wise, by some of the very code that consulted it when it was new.
Persistent objects are available only in language platforms that are not typical of languages in which preprocessors, compilers and linkers are written. Such platforms currently impose various sorts of costs. Even Keykos, my favorite, could accomplish this only with substantial revamping of the tools.
The preprocessor has produced a declaration of some variable from source that did not formally look like a declaration. Programs that look at source, such as etags and cscope will not see a declaration. The compiler sees a declaration and builds a structure for it. Several applied occurrences occur, some possibly produced by the preprocessor, and the compiler responds correctly to each of these, perhaps leaving additional notes behind for debugging. Each of these notes head a trail of crumbs that allow reconstruction of the state of the preprocessing where the declaration was produced and also the applied occurrences.
Some will object that the above patterns for declarations should be avoided. I agree but sometimes the alternatives are worse.
It is not clear to me which of these two schemes is more complex. It is also not clear which is more expensive in various resources. The latter scheme requires less space but more processing.