I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
show comments
TheJCDenton
> bit-exact against the original
What kind of sorcery is this ? Very impressive work !
show comments
flohofwoe
Almost 8ms on 1080p resolution seems extremely expensive, does the original also eat into the rendering budget as much?
show comments
tim-projects
Jensen : Nobody needs to code anymore...
Programmer: OpenDLSS...
Jensen : Wait. Not like that! (╯°□°）╯︵┻━┻
show comments
binsquare
At what point does this neural rendering take away the human touch on the art styles?
show comments
franticgecko3
How useful is this without weights?
Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
Or is my knowledge outdated here and they're just using a single generalised model?
show comments
drnick1
> A Vulkan reimplementation of NVIDIA's DLSS 5 Neural Rendering network, bit-exact against the original.
Bit-identical, I swear I heard that somewhere before.
the_voice
I think it's interesting that Nvidia is so interested in producing the hardware that fuels the future of software development, given that their primary business advantage is their software moat. This is an interesting project for sure, but turning a bunch of GPUs at Zluda[0] (an open implementation of Cuda) could be far more destructive for them, right?
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
show comments
Geee
What kind of dataset is used to train DLSS 5? Do they need to generate synthetic image pairs first?
Pantera87
So that means AMD implementation is on the horizon?
rfgplk
Something will this would historically guarantee a Senior Staff+ position at Nvidia. Wondering why Jensen doesn't put money where his mouth is ("were seeking exceptional engineers blabla") and offer him a job?
show comments
cubefox
> It takes one rendered frame (a low dynamic range proxy of it, three lanes of Gaussian noise, the previous frame's output reprojected, and five conditioning scalars) and produces four f32 channels per pixel: an RGB residual and one temporal-blend logit.
> The temporal path is implemented, but in the demo: the network's history input lanes and its per-pixel blend logit drive a reprojected feedback loop (docs/frame.md). The dlss5vk tool runs single frames with no history, which is what the reference captures were made with.
From this I assume the network uses the (via motion vectors) reprojected previous frame in order to increase temporal stability, i.e. similarity over adjacent frames. But this isn't strictly necessary, and apart from it, DLSS 5 is a pure post-process filter. So you could apply it to an old animated CGI movie like Final Fantasy (2001) [1]. Which should make it look significantly more realistic, at the cost of some flicker or other temporal instability.
One could also apply it to still images, like old renders from Tomb Raider [2], where temporal stability is not a factor. The difference to conventional text-to-image models with a "make it photorealistic" prompt would be that DLSS 5 strongly adheres to the underlying geometry.
I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
> bit-exact against the original
What kind of sorcery is this ? Very impressive work !
Almost 8ms on 1080p resolution seems extremely expensive, does the original also eat into the rendering budget as much?
Jensen : Nobody needs to code anymore...
Programmer: OpenDLSS...
Jensen : Wait. Not like that! (╯°□°）╯︵┻━┻
At what point does this neural rendering take away the human touch on the art styles?
How useful is this without weights?
Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
Or is my knowledge outdated here and they're just using a single generalised model?
> A Vulkan reimplementation of NVIDIA's DLSS 5 Neural Rendering network, bit-exact against the original.
Bit-identical, I swear I heard that somewhere before.
I think it's interesting that Nvidia is so interested in producing the hardware that fuels the future of software development, given that their primary business advantage is their software moat. This is an interesting project for sure, but turning a bunch of GPUs at Zluda[0] (an open implementation of Cuda) could be far more destructive for them, right?
[0] https://github.com/vosen/ZLUDA
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
What kind of dataset is used to train DLSS 5? Do they need to generate synthetic image pairs first?
So that means AMD implementation is on the horizon?
Something will this would historically guarantee a Senior Staff+ position at Nvidia. Wondering why Jensen doesn't put money where his mouth is ("were seeking exceptional engineers blabla") and offer him a job?
> It takes one rendered frame (a low dynamic range proxy of it, three lanes of Gaussian noise, the previous frame's output reprojected, and five conditioning scalars) and produces four f32 channels per pixel: an RGB residual and one temporal-blend logit.
> The temporal path is implemented, but in the demo: the network's history input lanes and its per-pixel blend logit drive a reprojected feedback loop (docs/frame.md). The dlss5vk tool runs single frames with no history, which is what the reference captures were made with.
From this I assume the network uses the (via motion vectors) reprojected previous frame in order to increase temporal stability, i.e. similarity over adjacent frames. But this isn't strictly necessary, and apart from it, DLSS 5 is a pure post-process filter. So you could apply it to an old animated CGI movie like Final Fantasy (2001) [1]. Which should make it look significantly more realistic, at the cost of some flicker or other temporal instability.
One could also apply it to still images, like old renders from Tomb Raider [2], where temporal stability is not a factor. The difference to conventional text-to-image models with a "make it photorealistic" prompt would be that DLSS 5 strongly adheres to the underlying geometry.
1: https://www.imdb.com/title/tt0173840/
2: https://www.tombraiderchronicles.com/images/artwork-high-res...
Seems quite similar to this repository?
https://github.com/aloshdenny/open-dlss