I’m thinking this makes fullt sense because distillation is only additive, not subtractive. So it does not remove knowledge (if we can define censorship as removal of knowledge).
show comments
data-ottawa
FYI the scrolling on iPad with trackpad is broken. A full swipe on the trackpad is about 1 inch of screen movement.
show comments
seri4l
Deepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis.
I didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.
show comments
andy99
So, there is no subliminal learning in this situation, under what conditions would we expect it. I find a transfer attack to be a bit far fetched but it’s definitely interesting.
If we trained from random initialisations on DeepSeek output (that didn’t explicitly contain the political questions) we would expect transfer? And if we fine tuned a model pretrained elsewhere on Deepseek output?
What is the line?
show comments
dluan
It'd be interesting to use this technique to create a running tally across all models of which models are censored on what topics
I know not all models can be easily abliterated or uncensored, but is there a reason to start with a model that is still censored?
ex: https://huggingface.co/huihui-ai/models
I’m thinking this makes fullt sense because distillation is only additive, not subtractive. So it does not remove knowledge (if we can define censorship as removal of knowledge).
FYI the scrolling on iPad with trackpad is broken. A full swipe on the trackpad is about 1 inch of screen movement.
Deepseek is, with difference, the most "Western" of Chinese models, so it's a bit perplexing that it was chosen to test this hypothesis.
I didn't run any benchmarks but I played around a little, and after getting around the API-level filter Deepseek V4's answers about "China-sensitive content" aren't any different from what I get from Claude and ChatGPT.
So, there is no subliminal learning in this situation, under what conditions would we expect it. I find a transfer attack to be a bit far fetched but it’s definitely interesting.
If we trained from random initialisations on DeepSeek output (that didn’t explicitly contain the political questions) we would expect transfer? And if we fine tuned a model pretrained elsewhere on Deepseek output?
What is the line?
It'd be interesting to use this technique to create a running tally across all models of which models are censored on what topics
Hijacking scroll behaviour in 2026 is wild.
so interesting!!