The closest thing that I am aware of was Symbolics OpenGenera.
Symbolics (who owned the first ever .com domain on the Internet) made Lisp Machines -- high-powered personal workstations whose entire OS was written in Lisp, right down to the lowest-level code. They used special proprietary processors which executed a sort of Lisp-like bytecode, enabling Lisp to be compiled down into something that ran directly on the metal -- using lists instead of the C model of bytes, words and pointers.
This is the where what is now called the "Windows key" came from: the Lisp Machine's Meta key. That's why Emacs still calls it "meta". Emacs was the native editor of these OSes.
The company devoted a lot of R&D into processor development over the years, moving from arrays of chips to single large chips to smaller chips. Their last hardware was an add-in board that fitted into 680x0 Macintosh computers and ran the Lisp machine's Genera OS in a Classic MacOS window. The code was actually executing on its own hardware in its own separate dedicated RAM and just used the Mac as a display, storage controller and network interface.
After this, the company was unable to keep up with the speed of simpler CPUs that used the C style of design. Symbolics response was to port their OS to run on the most advanced RISC chip of the time: DEC's Alpha processor, the first 64-bit RISC chip.
Symbolics wrote an interpreter for their Lispy bytecode that ran on the Alpha, enabling what was now called OpenGenera to execute as a program under DEC's OSF/1-based UNIX, later called Tru64.
This interpreter was extremely fast because it was implemented to fit entirely into the Alpha's L1 cache -- 8kB each for Instructions and Data. An 8kB runtime meant that it never got flushed while you were running Lisp code.
The typical POWER9 chip has between 80-100~ MB of L3 cache available, which I imagine is more than enough to hold Linux and your choice of userland application in cache.
Now, forcing it to only use cache would be an interesting problem...
I have fond memories of my PC's from the early 90's with RAM of 4 or 8 MB, and now am amazed that my laptop CPU has 8 MB cache.
Sometimes I wonder, if you could make an OS and useful applications that ran entirely in CPU cache.