This is a big forwards-compatibility risk. Suppose glibc adds a new symbol, and then a GPU driver adds a dependency on that symbol. The user wants to run an old executable with the updated GPU driver (maybe the old GPU driver doesn’t support their GPU). Normally, this would work fine: the user has to use a new copy of glibc, which will be compatible with both the new GPU driver and the old executable. But with your approach, the GPU driver is forced to use the glibc reimplementation which has been statically linked into the executable. Which, since the executable is old, can’t possibly implement the new symbol.
The same issue would occur if glibc adds a new version of an existing symbol and then the GPU driver is recompiled. (Or, for that matter, if a GPU driver adds a dependency on a symbol which glibc has always supported but which isn’t in the subset that you reimplemented, though in theory that could be solved if you reimplemented 100% of the symbols.)
show comments
pjmlp
So we are re-inventing patched a.out files, back when UNIX systems started to introduce dynamic loading, before ELF was invented?
Advocates of static linking keep forgetting once upon a time UNIX only had static linking, then we had overlays, and eventually dynamic linking came to be.
show comments
eqvinox
If you can figure out your own ELF loader, you can figure out how to build a partially static executable that doesn't need this. You can mix static and dynamic linking. Build tooling around that is just shit.
show comments
torginus
I have faced a similar issue in the past, and I don't understand how static binaries from the host are supposed to solve this.
From what I remember, GPU access on Linux 'works' by accessing specific FDs under /dev, which are vendor specific - this is what these libs do under the hood.
The libraries don't have any magic powers - if the FD is inaccessible, you won't be able to do anything.
So there's some vendor specific access needed in containers anyway (or a blanket allow, which is a BAD idea).
Also not sure why dynamic linking isn't good enough for this - the issue lies with the permissions, not how you load/link libraries.
rfgplk
I've implemented the same thing for micron (more or less). One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE). That being said the way you're doing is also tricky(ish) because if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you. Doing this is safer if you control the entire runtime.
show comments
nomel
I don't know much about musl.
> GPU: Vulkan and OpenGL drivers are supplied by the host as shared objects, usually built against glibc, and a fully static musl binary cannot normally dlopen() them.
Why? Have people managed to break the ancient concept of shared libraries, and this is a fix for that?
show comments
sieve
Every couple of years, I revisit my PL dev hobby and this time I decided to create a language/runtime with pre-emptive scheduling using instruction fuel. While I always do freestanding builds, this time I decided that I also wanted to support native FFI.
That is when I realized the true horror of (g)libc. It wants to inject itself at the root of the library/program and everything from threading to dlopen/dlsym is impacted. I tried a lot of workarounds including trying to implement a loader myself, but the complexity (and fragility) grew so much that I felt it was not worth it.
Finally, I retreated into the safe world of a freestanding runtime + syscalls. FFI, if it has to happen, will occur via IPC of some kind. A second process linked against glibc that will manage calls on behalf of the clean first one.
Do people say "so", "ess-oh" or "dot-ess-oh"? The title "a .so" is clunky to the "ess-oh" gang.
show comments
setheron
If you dynamically sold an SO are you still static even if you did it "custom" ?
At that point it's a dynamic loader in another name?
show comments
simonask
It is a testament to the complete failure of the GNU/Linux userland that something like this seems at all attractive to spend time on (or, it seems, LLM tokens).
Actually, scratch that, because Windows and macOS have historically struggled with ABI compatibility as well (macOS less so, due to not caring about backward compatibility in the first place).
How did we get to the point where people feel they need to go to the length of embedding an ELF loader in their binary (!!) rather than just linking with glibc?
show comments
catlifeonmars
So not completely static, since it must link against a libc :P
show comments
nubinetwork
Why not just pass the GPU to a docker container?
show comments
jeffbee
How are we supposed to take this stuff seriously if the author (sic) isn't even willing to write the readme? Claude exists! If I want some slop I can push the button myself.
show comments
j16sdiz
> backed by its own ELF loader (x86-64 and aarch64) and a glibc ABI bridge
This is a big forwards-compatibility risk. Suppose glibc adds a new symbol, and then a GPU driver adds a dependency on that symbol. The user wants to run an old executable with the updated GPU driver (maybe the old GPU driver doesn’t support their GPU). Normally, this would work fine: the user has to use a new copy of glibc, which will be compatible with both the new GPU driver and the old executable. But with your approach, the GPU driver is forced to use the glibc reimplementation which has been statically linked into the executable. Which, since the executable is old, can’t possibly implement the new symbol.
The same issue would occur if glibc adds a new version of an existing symbol and then the GPU driver is recompiled. (Or, for that matter, if a GPU driver adds a dependency on a symbol which glibc has always supported but which isn’t in the subset that you reimplemented, though in theory that could be solved if you reimplemented 100% of the symbols.)
So we are re-inventing patched a.out files, back when UNIX systems started to introduce dynamic loading, before ELF was invented?
Advocates of static linking keep forgetting once upon a time UNIX only had static linking, then we had overlays, and eventually dynamic linking came to be.
If you can figure out your own ELF loader, you can figure out how to build a partially static executable that doesn't need this. You can mix static and dynamic linking. Build tooling around that is just shit.
I have faced a similar issue in the past, and I don't understand how static binaries from the host are supposed to solve this.
From what I remember, GPU access on Linux 'works' by accessing specific FDs under /dev, which are vendor specific - this is what these libs do under the hood.
The libraries don't have any magic powers - if the FD is inaccessible, you won't be able to do anything.
So there's some vendor specific access needed in containers anyway (or a blanket allow, which is a BAD idea).
Also not sure why dynamic linking isn't good enough for this - the issue lies with the permissions, not how you load/link libraries.
I've implemented the same thing for micron (more or less). One advice I'd give you is to _really_ take care regarding SysV/ELF ABI conventions, there's tons of undocumented stuff in there and it's really easy to mess something up or cause a security defect (see AT_SECURE). That being said the way you're doing is also tricky(ish) because if I understood your implementation correctly you're hooking this into an already running musl which could cause backwards compatibility issues if musl changes under you. Doing this is safer if you control the entire runtime.
I don't know much about musl.
> GPU: Vulkan and OpenGL drivers are supplied by the host as shared objects, usually built against glibc, and a fully static musl binary cannot normally dlopen() them.
Why? Have people managed to break the ancient concept of shared libraries, and this is a fix for that?
Every couple of years, I revisit my PL dev hobby and this time I decided to create a language/runtime with pre-emptive scheduling using instruction fuel. While I always do freestanding builds, this time I decided that I also wanted to support native FFI.
That is when I realized the true horror of (g)libc. It wants to inject itself at the root of the library/program and everything from threading to dlopen/dlsym is impacted. I tried a lot of workarounds including trying to implement a loader myself, but the complexity (and fragility) grew so much that I felt it was not worth it.
Finally, I retreated into the safe world of a freestanding runtime + syscalls. FFI, if it has to happen, will occur via IPC of some kind. A second process linked against glibc that will manage calls on behalf of the clean first one.
How this differs (is better!) from prior art - https://github.com/pg83/solo#how-this-differs-from-prior-wor...
Do people say "so", "ess-oh" or "dot-ess-oh"? The title "a .so" is clunky to the "ess-oh" gang.
If you dynamically sold an SO are you still static even if you did it "custom" ? At that point it's a dynamic loader in another name?
It is a testament to the complete failure of the GNU/Linux userland that something like this seems at all attractive to spend time on (or, it seems, LLM tokens).
Actually, scratch that, because Windows and macOS have historically struggled with ABI compatibility as well (macOS less so, due to not caring about backward compatibility in the first place).
How did we get to the point where people feel they need to go to the length of embedding an ELF loader in their binary (!!) rather than just linking with glibc?
So not completely static, since it must link against a libc :P
Why not just pass the GPU to a docker container?
How are we supposed to take this stuff seriously if the author (sic) isn't even willing to write the readme? Claude exists! If I want some slop I can push the button myself.
> backed by its own ELF loader (x86-64 and aarch64) and a glibc ABI bridge
Yacks