> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.
Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
show comments
1matin
[delayed]
RachelF
Nice idea, but something has gone very wrong here:
>Sequential throughput: ~1.3 GB/s
[on a RTX 3070 Laptop]
This RTX 3070 chip is on PCIe 4.0 x16 which should give 64GB/s. The 8GB of GDDR6 is 448GB/s.
Swapping to an NVMe drive would be twice as fast, but with higher latency.
show comments
xfalcox
Given my dev machine has 32GB of RAM and 32GB of VRAM that sits mostly idle when I'm not running AI models, this is not that bad of an idea.
show comments
kimixa
I remember this being a thing done a while back using linux's MTD/phram drivers - https://wiki.archlinux.org/title/Swap_on_video_RAM - not sure if that's still relevant though as I don't know how it'll interact with DRM and how it handles reserving some of the vram - the suggested limit using xorg.conf is probably pretty obsolete now.
That page also has a fuse filesystem implementation on top of opencl - https://github.com/Overv/vramfs - which may be more compatible.
show comments
drdaeman
What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
molticrystal
For windows I saw something similar to this years ago. An experimental proof of concept driver that allows the creation of a ram drive from vram for NVIDIA cards. Sequential is fast as you'd expect, random has lots of room for improvement.
Remember how 16GBs used to be an enterprise level database mainframe?
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
show comments
ProllyInfamous
Why does my Apple Sillicon Mac with 32gb of RAM use (or even create?!) a swapfile, when 20gb is still unused/"free"?
Why can I not just enter a simple command to entirely-disable swapfile, like with Linux's:
>>>>swapoff -a
Seems kind of silly, unless the point is intentionally to wear-down the SSD's lifespan.
Having a GUI swapfile-disable system preference would be awesome. It would also be awesome if Apple finally abandoned this system settings/layout "phase" – it's still word-salad (compared to decades of preference panes).
#Apple #Feedback #swapfile
show comments
tlb
Building a swap device at user level used to be one of those classic unsolvable problems, because what if your daemon needs to swap in a page in order to swap in a page? Or at least it was discussed at a reason why microkernels will never work. I’m not sure what the solution is here.
show comments
NortySpock
Nice, I might try using this as I'm currently on 16 GB of RAM / 11 GB VRAM and feel like the VRAM is usually idle except for when I game or try a local LLM.
It would be nice to have dynamic scaling or even just auto-shutoff on VRAM pressure if I forget I have this enabled and then fire up a game or LLM.
willis936
I'm more interested in the opposite. Nvidia linux drivers crash when you try to address more VRAM than you have. It'd be nice if they didn't.
show comments
dlt713705
Does anyone these days really use swap for anything than S4 suspend ?
show comments
sgjohnson
>Sequential throughput: ~1.3 GB/s
sounds VERY low, also, wouldn't random read/write speed be MUCH more relevant here?
mmastrac
I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
show comments
londons_explore
It seems obvious to me that this should be a built in functionality of the kernel.
The kernels job is to manage resources - and GPU ram is one such resource, and it can be used for many of the same uses as regular ram.
There is originally https://github.com/Overv/vramfs however that has the overhead of a FUSE filesystem + loop device when using as a swap device.
The performance is rather lackluster however, it's far from a miracle "now you effectively have more ram for a 90% performance drop" - it definitely feels like traditional swapping
LouisvilleGeek
Finally a use for the expensive ram when it's not needed in workloads!
Now if it could be dynamically used and vacated on other GPU workloads?
> Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work.
Well, that does at least answer my immediate question about why I would ever swap from expensive RAM to really expensive RAM:) Feels niche, but when you want it it's a good idea.
[delayed]
Nice idea, but something has gone very wrong here:
>Sequential throughput: ~1.3 GB/s
[on a RTX 3070 Laptop]
This RTX 3070 chip is on PCIe 4.0 x16 which should give 64GB/s. The 8GB of GDDR6 is 448GB/s.
Swapping to an NVMe drive would be twice as fast, but with higher latency.
Given my dev machine has 32GB of RAM and 32GB of VRAM that sits mostly idle when I'm not running AI models, this is not that bad of an idea.
I remember this being a thing done a while back using linux's MTD/phram drivers - https://wiki.archlinux.org/title/Swap_on_video_RAM - not sure if that's still relevant though as I don't know how it'll interact with DRM and how it handles reserving some of the vram - the suggested limit using xorg.conf is probably pretty obsolete now.
That page also has a fuse filesystem implementation on top of opencl - https://github.com/Overv/vramfs - which may be more compatible.
What about backpressure, how does it handle requirements for VRAM allocation when VRAM is used for swap space?
With X11 it's not that bad (buffers are pre-allocated), but with Wayland allocations are a lot more dynamic, so running low on VRAM can easily crash the whole desktop. I just had a few of such crashes with Hyprland+llama-server+KVM switching between computers without freeing VRAM.
For windows I saw something similar to this years ago. An experimental proof of concept driver that allows the creation of a ram drive from vram for NVIDIA cards. Sequential is fast as you'd expect, random has lots of room for improvement.
>GpuRamDrive
>Create a virtual drive backed by GPU RAM.
https://github.com/prsyahmi/GpuRamDrive
Fork with AMD support:
https://github.com/brzz/GpuRamDrive/
I think you can definitely improve the throughput/iops by using BAR vs treating it like a file store/mount through cuda which adds a lot of overhead.
Similar but using OpenCL APIs, so it works on AMD (for some definition of "works" since their drivers are quite buggy): https://libguestfs.org/nbdkit-vram-plugin.1.html
Remember how 16GBs used to be an enterprise level database mainframe?
Well, GPUs also have stupid amounts of compute on them. I have to imagine that there is some kind of database format that's useful with GPU compute attached.
Since the data is already in VRAM, the GPU can sort, join, or otherwise manipulate data as needed.
Why does my Apple Sillicon Mac with 32gb of RAM use (or even create?!) a swapfile, when 20gb is still unused/"free"?
Why can I not just enter a simple command to entirely-disable swapfile, like with Linux's:
>>>>swapoff -a
Seems kind of silly, unless the point is intentionally to wear-down the SSD's lifespan.
Having a GUI swapfile-disable system preference would be awesome. It would also be awesome if Apple finally abandoned this system settings/layout "phase" – it's still word-salad (compared to decades of preference panes).
#Apple #Feedback #swapfile
Building a swap device at user level used to be one of those classic unsolvable problems, because what if your daemon needs to swap in a page in order to swap in a page? Or at least it was discussed at a reason why microkernels will never work. I’m not sure what the solution is here.
Nice, I might try using this as I'm currently on 16 GB of RAM / 11 GB VRAM and feel like the VRAM is usually idle except for when I game or try a local LLM.
It would be nice to have dynamic scaling or even just auto-shutoff on VRAM pressure if I forget I have this enabled and then fire up a game or LLM.
I'm more interested in the opposite. Nvidia linux drivers crash when you try to address more VRAM than you have. It'd be nice if they didn't.
Does anyone these days really use swap for anything than S4 suspend ?
>Sequential throughput: ~1.3 GB/s
sounds VERY low, also, wouldn't random read/write speed be MUCH more relevant here?
I seriously looked at this as a way to improve the RAM situation in a QNAP 2U unit that I was having trouble sourcing RAM for. It's somewhat annoying that legit memory-over-PCIe is gated on PCIe5 and chipset support.
In the end I just had to bite the bullet and take a gamble on finding ECC DDR4 RAM that would work with the ancient AMD chipset...
This particular implementation seems to be running over too many layers to be particularly performant. Why not a custom block driver instead?
It seems obvious to me that this should be a built in functionality of the kernel.
The kernels job is to manage resources - and GPU ram is one such resource, and it can be used for many of the same uses as regular ram.
I've implemented the same idea with OpenCL: https://github.com/theblazehen/vramblk
There is originally https://github.com/Overv/vramfs however that has the overhead of a FUSE filesystem + loop device when using as a swap device.
The performance is rather lackluster however, it's far from a miracle "now you effectively have more ram for a 90% performance drop" - it definitely feels like traditional swapping
Finally a use for the expensive ram when it's not needed in workloads!
Now if it could be dynamically used and vacated on other GPU workloads?
You want to waste VRAM, in this economy?
sidenote: it is possible to use Vulkan to map GPU memory to CPU space and even map it back to CUDA: https://x.com/steeve/status/2055042304344231978?s=20
I have long wished it was possible to do this. What a great bit of code. Thanks.
No software benchmarks? BAR for RAM is cool but I want to see how much it _actually_ beats pcie nvme
Q: Why? A: Why not?
RAM disks have always fascinated me. In a different timeline every PC has a 100gb of RAM and 50TB HDDs are the norm.
use your car for an anchor on a big boat!
Nvme ssd weights much less than GPU, and it matters for a laptop.
And now, I want HDMI as direct connection hight speed network socket.
I mean, you prompted something useful out of an AI, good job. But then use that to ask for donation? Feels weird, man.
Nice
I mean, cool, but I’d rather not?
This is why I read HN.