O tempora, O mores: consoles used to store the entire game in ROM. Up to about 64Mb.
The modern need for patches and DLC makes that impossible now, in addition to the other reasons. But I think it would still be possible to greatly improve load times with a bit of optimisation - it's just not commercially important. Maybe if the console QA system imposed maximum load times it would be different - but then you'd see compromises made elsewhere, such as in level size.
(I play Cities:Skylines, where one of the more popular mods is simply called "Don't Crash", which lazy-loads other mod data to improve performance and reliability)
Remember that the 3DS is still very successful, and runs just off carts (well, also downloadable titles). It seems to run very smoothly.
In practice, there have been a few games that have had patches released -- in almost all modern games a tiny fraction of the game is code, which can be easily replaced, leaving all the art and music which (hopefully) don't need as much patching.
Well, modern carts like 3DS are quite different from old cart systems in the sense that they are handled more like storage devices than extension of the memory. I think GBA was the last system that allowed for example running code directly from the cart, without loading it first to RAM.
Games on the 3DS aren't typically renowned for their over-the-top graphism quality in the overall videogame scene; if you expected the same quality with the same resolution on the desktop, then you'd be doing what pjc50 said: compromises
Sad but true. Too many beautifully rendered games that aren't terribly engaging because amazing graphics are what the mainstream market demands. (I've avoided buying the latest gen consoles partly because of this, but mostly because I just don't get the time to play games that much these days.)
That being said there are also many beautiful games that are amazingly fun, and the graphics do add to the experience, but they're more like the icing on the cake. That's not quite the metaphor I'm groping for, but you get the idea: the gameplay is a fundamental that has to be right, and then the graphics can really add something; without the gameplay the experience is always destined to be flat.
It will be interesting to see what happens with VR games. With a regular game you can always do something else - check your phone, walk away for a moment. In VR you are stuck staring at a loading screen unable to easily do anything else. We are already seeing cases where people will abandon a VR experience just because they cannot handle the load times. Content creators take note.
The modern need for patches and DLC makes that impossible now, in addition to the other reasons. But I think it would still be possible to greatly improve load times with a bit of optimisation - it's just not commercially important. Maybe if the console QA system imposed maximum load times it would be different - but then you'd see compromises made elsewhere, such as in level size.
(I play Cities:Skylines, where one of the more popular mods is simply called "Don't Crash", which lazy-loads other mod data to improve performance and reliability)