> The first warning was about scale itself. Bender and Gebru argued that training ever-larger models on ever-larger scrapes of the internet would produce systems that appeared fluent but had no actual understanding of language.
> The second warning was about bias amplification. The paper documented in detail that internet-scale training data contains systematic overrepresentation of dominant viewpoints and underrepresentation of marginalized ones. The models would not just absorb this bias. They would amplify it...
> The third warning was about environmental cost.
> The fourth warning was about documentation. The paper argued that the training datasets being assembled were too large for anyone to actually audit.
> The fifth warning was the one Google cared about most. Bender and Gebru argued that the deployment of these systems would centralize linguistic and cultural power in the hands of the small number of companies that could afford to train them.
Personally I'm not convinced on the first two. The third is obviously a concern. The fourth seems logical, but I'm sure what the impact is, if any. The fifth is a problem, I suppose, but one that already exists in so many other capacities.
show comments
j16sdiz
I am not sure what I should think of AI reinforced discrimination.
Some sensitive traits (e.g. Race) have high correlation with something we want to estimate (eg crime rate, credit score). The same traits can be correlated with thousands of different other attributes.
For example, to estimate the risk of loan default, (mathematically) i can use
a) race
b) zip code
c) 3 or 4 seemingly unrelated attributes, but still highly correlated to race
d) a few hundred attributes
e) a few million attributes, taking a PCA and trim down to a few hundred dimensions vector space
When does the discrimination begins or end? (a) is surely illegal, but you can argue (e) is still a proxy to the same thing.
There is no way to cut it fairly. It seems to me any kind of profiling should be illegal
stephc_int13
It seems that the main issue with AI is often not what sci-fi or EA-adjacent prophets are trying to warn us about, but the insidious dangers of the failure modes.
We are collectively not well calibrated to deal with systems that seems capable but fails in surprising ways.
Commercial planes are still under the responsibility and control of highly trained human pilots, even if I am pretty sure that full automation would be technically feasible, even without relying on modern AI, I don't think any companies would be comfortable with the liability.
show comments
simonw
> Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.
There's definitely a good, well researched article to be written about the how well the stochastic parrots paper stands up four years later. This is not that article.
WhitneyLand
This does not look good for Google.
On one hand, industrial research is different from academic research. There’s no tenure and not the same level or presumption of academic freedom. Fair enough.
The problem is they specifically wanted to bathe in the glory of an ethical research team and all the benefits that come with that.
You can’t have it both ways.
hn_throwaway_99
The first issue I have with the article is the title. I followed this whole saga very closely when it happened, and while I definitely understand the nuance of her separation, I agree with Google that Gebru wasn't fired - she quit.
I do not understand what universe you must live in to think you can come to your employer and make a large list of demands (including demands that can easily be taken as subtle or not so subtle threats to your colleagues), say "if you don't meet these demands then I'm going to quit, and quit loudly", and then when the company accepts your proposal by saying "OK, fine, we don't accept your demands so we're accepting your resignation", and then you try to backtrack with a surprised Pikachu face and then cry loudly about how Google fired you. Seriously, where I come from the response would be "get bent."
I also would highlight that the biggest complaint in the paper was how LLMs amplified bias. Google was laughed at for one of its Gemini releases from just a few years back (can't remember if it was called Gemini then) where one commenter noted "it is extremely difficult to get Google's AI to believe white people exist", as they so obviously overcorrected on the racial bias issue where image generation was creating black Nazis and Asian medieval kings of England.
show comments
epolanski
I don't want to say this has not happened, but where's the evidence of anything in this article?
According to the article she resigned, which is very different from getting fired, so what is the information the author has to substantiate this claim?
show comments
ChrisArchitect
What is/was the source of this rather than random tumblr?
This paper has not held up, like, at all. The first half of it recites Woke 1.0 principles, like a concern that LMs will thwart efforts to "decolonialize education by shifting to oral histories" in order to avoid the biases of "text". The second half of it makes predictions from axioms about LMs not truly understanding text that nobody would take seriously today.
There's philosophical grappling to be done, as with the Ted Chiang post on the front page right now, about what it is LLMs are actually doing (I'm mostly with Chiang on those core philosophical issues). But Gebru went way past that, attacking their underlying utility. The coherency of GPT 5.5 responses are not simply tricks of the mind, and frontier models (leaving aside Grok, if you want to call it a frontier model) have not in fact been engines for bias.
bethekidyouwant
“…training a single large language model produced emissions equivalent to the lifetime output of 5 cars” 5 cars?? sacrement!
neonihil
The deafening silence in the comment section says it all.
The warnings:
Personally I'm not convinced on the first two. The third is obviously a concern. The fourth seems logical, but I'm sure what the impact is, if any. The fifth is a problem, I suppose, but one that already exists in so many other capacities.I am not sure what I should think of AI reinforced discrimination.
Some sensitive traits (e.g. Race) have high correlation with something we want to estimate (eg crime rate, credit score). The same traits can be correlated with thousands of different other attributes.
For example, to estimate the risk of loan default, (mathematically) i can use
a) race
b) zip code
c) 3 or 4 seemingly unrelated attributes, but still highly correlated to race
d) a few hundred attributes
e) a few million attributes, taking a PCA and trim down to a few hundred dimensions vector space
When does the discrimination begins or end? (a) is surely illegal, but you can argue (e) is still a proxy to the same thing.
There is no way to cut it fairly. It seems to me any kind of profiling should be illegal
It seems that the main issue with AI is often not what sci-fi or EA-adjacent prophets are trying to warn us about, but the insidious dangers of the failure modes.
We are collectively not well calibrated to deal with systems that seems capable but fails in surprising ways.
Commercial planes are still under the responsibility and control of highly trained human pilots, even if I am pretty sure that full automation would be technically feasible, even without relying on modern AI, I don't think any companies would be comfortable with the liability.
> Amazon's hiring algorithm penalized resumes that contained the word "women" in any context. Healthcare risk scoring algorithms used by major US hospitals were found to systematically underestimate the medical needs of Black patients. Apple Card's credit algorithm gave wives credit lines 10x lower than their husbands for the same financial profile.
The Amazon hiring story is from 2018: https://www.reuters.com/article/world/insight-amazon-scraps-...
The "systematically underestimate the medical needs of Black patients" story seems to be this one from 2019: https://www.chicagobooth.edu/research/tolan/research/2019/di...
The Apple Card story is also from 2019: https://abcnews.com/US/york-probing-apple-card-alleged-gende...
None of those stories were about LLMs!
The stochastic parrots paper was published in 2021: https://dl.acm.org/doi/10.1145/3442188.3445922
There's definitely a good, well researched article to be written about the how well the stochastic parrots paper stands up four years later. This is not that article.
This does not look good for Google.
On one hand, industrial research is different from academic research. There’s no tenure and not the same level or presumption of academic freedom. Fair enough.
The problem is they specifically wanted to bathe in the glory of an ethical research team and all the benefits that come with that.
You can’t have it both ways.
The first issue I have with the article is the title. I followed this whole saga very closely when it happened, and while I definitely understand the nuance of her separation, I agree with Google that Gebru wasn't fired - she quit.
I do not understand what universe you must live in to think you can come to your employer and make a large list of demands (including demands that can easily be taken as subtle or not so subtle threats to your colleagues), say "if you don't meet these demands then I'm going to quit, and quit loudly", and then when the company accepts your proposal by saying "OK, fine, we don't accept your demands so we're accepting your resignation", and then you try to backtrack with a surprised Pikachu face and then cry loudly about how Google fired you. Seriously, where I come from the response would be "get bent."
I also would highlight that the biggest complaint in the paper was how LLMs amplified bias. Google was laughed at for one of its Gemini releases from just a few years back (can't remember if it was called Gemini then) where one commenter noted "it is extremely difficult to get Google's AI to believe white people exist", as they so obviously overcorrected on the racial bias issue where image generation was creating black Nazis and Asian medieval kings of England.
I don't want to say this has not happened, but where's the evidence of anything in this article?
According to the article she resigned, which is very different from getting fired, so what is the information the author has to substantiate this claim?
What is/was the source of this rather than random tumblr?
This May 26th Twitter post ...maybe? Account now suspended https://x.com/heygurisingh/status/2059251382960734593
(http://web.archive.org/web/20260526123243/https://twitter.co...)
This paper has not held up, like, at all. The first half of it recites Woke 1.0 principles, like a concern that LMs will thwart efforts to "decolonialize education by shifting to oral histories" in order to avoid the biases of "text". The second half of it makes predictions from axioms about LMs not truly understanding text that nobody would take seriously today.
There's philosophical grappling to be done, as with the Ted Chiang post on the front page right now, about what it is LLMs are actually doing (I'm mostly with Chiang on those core philosophical issues). But Gebru went way past that, attacking their underlying utility. The coherency of GPT 5.5 responses are not simply tricks of the mind, and frontier models (leaving aside Grok, if you want to call it a frontier model) have not in fact been engines for bias.
“…training a single large language model produced emissions equivalent to the lifetime output of 5 cars” 5 cars?? sacrement!
The deafening silence in the comment section says it all.