While there are certainly plenty of people who need more storage than is practical to put in-memory, most people don't. E.g. it's rare enough to find people with single datasets in the 1TB+ range. Most databases I've come across at clients max out in the tens to hundreds of GB range. Many more could be easily split (e.g. they may be multi-TB or above, but are that way because of aggregation of data that is logically separate, such as relating to different customers, and can easily be sharded) - though whether or not that's worthwhile is a separate issue.
So while you're right, for most people it's not going to be a practical problem. Query performance tends to be a bigger problem in practice, and IO bandwidth is often the main cost driver for servers (with NVMe SSDs providing so much additional IO bandwidth that they often pay for themselves several times over by reducing number of servers needed for people who insist on "traditional" disk focused databases), pushing server costs into ranges where spending more on RAM but needing fewer servers to handle the IO load, is more and more often a good tradeoff.
The latency doesn't sound that bad. It's 200ns to access memory on another local NUMA node, and 500ns to access memory on another machine using NUMAlink.
Fascinating tech. I had no idea this could be done with Xeons.
