This is about safe allocation of stack frames.
One limit to the size of a stack frame is the size of the address space.
Safe compiled code should not use the low 32 bits of a calculated frame size to ask for more stack space.
Here is an economical way to answer the more frequent question concerning availability of stack space in the prelude of a routine.
We suppose that the kernel can invalidate one page of addresses at the end of the space for each stack in an address space.
It may suffice to make such a page read-only, if that is easier.
A routine’s prelude presumably stores into a newly allocated frame.
If the stack frame size is known at compile time to be less than a the size of a page then there is no chance that a frame allocation will skip over the invalid page at the end of stack space.
When the compiler cannot be sure that the new frame is less than the size of a page then it can spend extra instructions in the prelude to ask if there is space.
This would seem to require one of:
- a register allocated conventionally during calls, devoted to locating where the stack end is stored, and perhaps additionally other “thread” parameters.
This may practically require always allocating that register.
- a fixed location, whose address is kept in the instruction stream, and which contains the address of the end of the current stack space.
This is not suitable for SMP unless the kernel can put thread unique pages at this location.
- Some operating systems can provide distinct pages, all at one address, for each distinct thread.
This costs little in the way of address mapping overhead for most hardware architectures since most of the maps are still shared among these several related spaces.
With a bit of coordination with memory trap logic, a civilized report can be made of exhaustion of stack space.
Early PL/I compilers for the 370 allocated such a register for thread specific information and used a “segmented stack”.
Every call would test if there was yet more room in the current stack segment and allocate another segment if necessary.
A stub frame was provided in the segment upon such allocation, return to which would deallocate that segment.