Perhaps if GPUDirect [1] were available on consumer devices, it would be possible to DMA training/model data in from an SSD/Optane/hypothetical DRAM-based PCIe drive and scale up to bigger problems as a result.
Alas, not even AMD allows access to their GPUDirect equivalent, and I can't imagine anyone being able to drum up enough pressure for those vendors to flip the toggle to make them available on their consumer lines.
Gpudirect is one of the last differentiators with pro version consumer cards. The 2080Ti allowed use of tensor cores on consumer cards, so I would bet they'd never enable gpudirect, since they need something different.
Alas, not even AMD allows access to their GPUDirect equivalent, and I can't imagine anyone being able to drum up enough pressure for those vendors to flip the toggle to make them available on their consumer lines.
[1]: https://developer.nvidia.com/gpudirect