"So Q4/D3 will do much of the compiling on the fly while you are playing the game?"
you literally get the source to all maps with doom3/q4. when its loaded for the very first time, the engine will "precompile it", the only thing that it's doing is creating a collision map. (bsp stage, 1 out of 3 for other Id games). I'm assuming that with each release of q4 there will be the need to recompile maps as well, because collision detection will probably change.
now think of this: q3map has been through about 200+ updates for collision detection and bsp portal creation alone. Id decided to add not only the factor of the engine that usually gets updates due to bugfixes, but the world compiler as well into the mix.
Think of how many variations to world editors/compilers/tools happen with popular games, then consider the factor that d3/q4 has that embedded in the main executable.
This is one of the reasons i hated unreal, you couldn't actually tweak or modify anything that really counted.
but for performance now, all lighting is done dynamically. Maps may also be created in Maya/3dsmax, and their textures compiled in that for static lighting styles like what is used in our "skyportals", or outdoor scenes that are not oart of the interactive world, but for all intents and purposes everything is calculated by your videocard at all times.
your video card will be doing so much work that will have to be cpu dependant for online playing that by comparison q1/2/3 does nothing.
imo Id just shot themselves in the foot and they know it.