Like all ai-or-not tests this fails to keep a similar high quality threshold for both kinds so it intends to waste time not appreciation of either kind of art.
The curator was selecting human output for overlap with ai flaw/artifacts that are likely to confuse at a glance. He wasn't selecting randomly above a high quality threshold for both kinds as implied.
Typically AI is boring, takes the easy way out upon further inspection, likes lone straight lines and face front on shots and it just so happens there are many tests which he found old human examples of this, with large perspective/lighting flaws as well.
I don't care what the point of art is consesused to be, or if elephant-made art is distinguishable from a 5th grader's art.
The turing test was "obsolete" before eliza time, the solution was: it doesn't matter to me because i'm using it as if it were human.
ARandumGuy
I feel like the comment made by the author's friend captures a lot of my feelings on AI art. AI art is often extremely detailed, but that detail is often just random noise. It's art that becomes worse and makes less sense the more carefully you look at it.
Compared that to human art. When a person makes a highly detailed artwork, that detail rewards looking closely. That detail forms a cohesive, intentional vision, and incentivizes the viewer to spend the time and effort to take it all in. A person could spend hours looking at Bruegel's Tower of Babel paintings, or Bosch's "The Garden of Earthly Delights".
Overall, I've never felt the need to spend the time looking closely at AI art, even AI art that I couldn't tell was AI right away. Instead of rewarding close inspection with more detail, AI art punishes the viewer who looks closer by giving them undecipherable mush.
show comments
maeil
Even with this hand-picking I got 70% and am nowhere near an expert on either AI or human art, having dabbled in it for a day or two back when DallE and Midjourney first became popular. I'm sure someone who's into the image generation field could score 80%+ consistently even over a larger dataset just as handpicked as this one.
Telltale signs of AI:
- Cliches. "Punk Robot" is a good example, a human artist capable of creating that would've somehow been more creative.
- Obviously facial expressions and limbs are telltale signs, very uncanny valley
- If there's an accurate geomtric repeating pattern, it's extremely unlikely to be AI. Something like "Giant Ship" is afaik still practically impossible to generate.
- Weird unfinished parts of a world. See "Leafy Lane". Why is there a vantablack hole at the end of the road? It's not physically impossible but it makes little sense and a human wouldn't put it there in that painting.
maldusiecle
Fine art is a matter of nuance, so in that sense I think it does matter that a lot of the "human art" examples are aggressively cropped (the Basquiat is outright cut in half) and reproduced at very low quality. That Cecily Brown piece, for example, is 15 feet across in person. Seeing it as a tiny jpg is of course not very impressive. The AI pieces, on the other hand, are native to that format, there's no detail to lose.
But those details are part of what make the human art interesting to contemplate. I wouldn't even think of buying an art book with reproductions of such low quality--at that point you do lose what's essential about the art work, what makes it possible to enjoy.
show comments
throwawayk7h
You can't really draw many conclusions from this test, since the AI art has already been filtered by Scott to be ones that Scott himself found confusing. So What do any of the numbers at the end of this really mean? "Am I better than Scott at discerning AI art from human" is about the only thing this test says.
If you didn't filter the AI art first, people would do much better.
show comments
cicdw
This article has an implicit premise that the ultimate judge of art is “do I/people like it” but I think art is more about the possibilities of interpretation - for example, the classics/“good art” lend themselves to many reinterpretations, both by different people and by the same person over time. When humans create art "manually" all of their decisions - both conscious and unconscious - feed into this process. Interpreting AI art is more of a data exploration journey than an exploration of meaning.
show comments
jncfhnb
AI Art can be hard to identify in the wild. But it still largely sucks at helping you achieve specific deliverables. You can get an image. But it’s pretty hard to actually make specific images in specific styles. Yes we have Loras. Yes we have control nets (to varying degrees) and ipadapter (to lesser degrees) and face adapters and what not. But it’s still frustrating to get something consistent across multiple images. Especially in illustrated styles.
AI Art is good if you need something in a general ballpark and don’t care about the specifics
show comments
equestria
Eh, this is pretty unfair. That's a test of how good humans are at deceiving other humans, not a of how hard it is to distinguish run-of-the-mill AI art from run-of-the-mill human art in real life.
First, by their own admission, the author deliberately searched for generative images that don't exhibit any of the telltale defects or art choices associated with this tech. For example, they rejected the "cat on a throne" image, the baby portrait, and so on. They basically did a pre-screen to toss out anything the author recognized as AI, hugely biasing the results.
Then, they went through a similar elimination process for human images to zero in on fairly ambiguous artwork that could be confused with machine-generated. The "victorian megaship" one is a particularly good example of such chicanery. When discussing the "angel woman" image, they even express regret for not getting rid of that pic because of a single detail that pointed to human work.
Basically, the author did their best to design a quiz that humans should fail... and humans still did better than chance.
show comments
plank
Not sure I understand the article. The author specifically chose art from humans and AI that he found difficult to categorize into human or AI art.
The fact that people had a 60% success rate suggest that they are a little better in seeing the difference then he was himself?
(What am I missing? This is not like "take 50 random art objects from humans and AI", but take 50 most human like AI, and non-obvious human art from humans)
arjie
Generative AI is so cool. My wife (a creative director) used it to help design our wedding outfits. We then had them embroidered with those patterns. It would have been impossible otherwise for us to have that kind of thing expressed directly. It’s like having an artist who can sketch really fast and who you can keep correcting till your vision matches the expression. Love it!
I don’t think there have been any transformative AI works yet, but I look forward to the future.
It’s unsurprising to me that AI art is often indistinguishable from real artists’ work but famous art is so for some reason other than technical skill. Certainly there are numerous replica painters who are able to make marvelous pieces.
Anyway, I’m excited to see what new things come.
vunderba
From the article:
So maybe some people hate AI because they have an artist's eye for small inadequacies and it drives them crazy.
This is it 100%.
When somebody draws something (in an active fashion), there is a significantly higher level of concentration and thought put towards the final output.
By its very nature, GenAI is mostly using an inadequately descriptive medium (e.g. text) which a user then must WAIT until an output that roughly matches your vision "pops" out. Can you get around this? Not entirely, though you can help mitigate this through inpainting, photobashing, layering, controlnets, loras, etc.
However, want to wager a guess what 99% of the AI art slop that people throw up all over the internet doesn't use? ANY OF THAT.
A conventional artist has an internal visualization that they are constantly mentally referring to as they put brush to canvas - and it shows in the finer details.
It's the same danger that LLMs have as coding assistants. You are no longer in the driver's seat - instead you're taking a significantly more passive approach to coding. You're a reviewer with a passivity that may lead to subtle errors later down the line.
And if you need any more proof, here's a GenAI image attached to _Karpathy_'s (one of the founding members of openAI) twitter post on founding an education AI lab:
so much of the value of art, which Scott has actually endowed on these AI generated pieces, is the knowledge that other people are looking at the same thing as you.
66% here. I was pretty much scrolling through and clicking on first instinct instead of looking in any detail.
Interestingly I did a lot better in the second half than the first half - without going through and counting them up again I think somewhere around 40% in the first half and 90% in the second half. Not sure if it's because of the selection/order of images or if I started unconsciously seeing commonalities.
sieste
The only way I can explain people getting 98% accuracy on this is being familiar with the handful of AI artists submitting their work for this competition.
show comments
readyplayernull
Easy to defeat. AI can't come up with ambiguous art:
There is a strategic feature to it based on retrospection.
show comments
Barrin92
It would have been interesting to know how much time most people spent per picture because if you look at the quoted comment from the well scoring art interested person mentioned:
"The left column has a sort of door with a massive top-of-doorway-thingy over it. Why? Who knows? The right column doesn't, and you'd expect it to. Instead, the right column has 2.5 arches embossed into it that just kind of halfheartedly trail off."
You can find this in almost every AI generated picture. The picture that people liked most, AI generated with the cafe and canal, the legs on the chairs make little sense. Not as bad as in non-curated AI art, but still no human would paint like this. Same for the houses in the background. If you spend say a minute per picture with AI art you almost always find these random things, even if the image is stylized, unlike human art it has a weird uncanniness to it.
show comments
12_throw_away
Oh come on. I guess I missed the part in the "Turing test" where a human filters out 99.999% of the machine's output prior to the test.
egypturnash
It's interesting that the impressionist-styled pieces mostly fooled people. I think this is because the style requires getting rid of one of the hallmarks of AI imagery: lots and lots of mostly-parallel swooshy lines, at a fairly high frequency. Impressionism's schtick is kind of fundamentally "fuck high-frequency detail, I'm just gonna make a bunch of little individual paint blobs".
One of the other hallmarks of AI imagery was deliberately kept out of this test. There's no shitposts. There's one, as an example of "the dall-e house style". It's a queen on a throne made of a giant cat, surrounded by giant cats, and it's got a lot of that noodly high-frequency detail that looks like something, but it is also a fundamentally goofy idea. Nobody's gonna pay Michael Whelan to paint the hell out of this and yet here it is.
beepbooptheory
Duchamp rolling in his grave about this post!
show comments
d--b
I think what gave AI away the most was mixed styles. If one part of the painting is blurred, and another part is very focused, you can tell it's AI. People don't do that.
I got all of Jack Galler's pictures wrong though. The man knows how to do it.
RcouF1uZ4gsC
I don’t think that is AI art Turing test.
An AI art Turing Test would be interactive with me telling it what to draw and what changes to make and see if what is producing the art is human or AI.
Like all ai-or-not tests this fails to keep a similar high quality threshold for both kinds so it intends to waste time not appreciation of either kind of art.
The curator was selecting human output for overlap with ai flaw/artifacts that are likely to confuse at a glance. He wasn't selecting randomly above a high quality threshold for both kinds as implied.
Typically AI is boring, takes the easy way out upon further inspection, likes lone straight lines and face front on shots and it just so happens there are many tests which he found old human examples of this, with large perspective/lighting flaws as well.
I don't care what the point of art is consesused to be, or if elephant-made art is distinguishable from a 5th grader's art.
The turing test was "obsolete" before eliza time, the solution was: it doesn't matter to me because i'm using it as if it were human.
I feel like the comment made by the author's friend captures a lot of my feelings on AI art. AI art is often extremely detailed, but that detail is often just random noise. It's art that becomes worse and makes less sense the more carefully you look at it.
Compared that to human art. When a person makes a highly detailed artwork, that detail rewards looking closely. That detail forms a cohesive, intentional vision, and incentivizes the viewer to spend the time and effort to take it all in. A person could spend hours looking at Bruegel's Tower of Babel paintings, or Bosch's "The Garden of Earthly Delights".
Overall, I've never felt the need to spend the time looking closely at AI art, even AI art that I couldn't tell was AI right away. Instead of rewarding close inspection with more detail, AI art punishes the viewer who looks closer by giving them undecipherable mush.
Even with this hand-picking I got 70% and am nowhere near an expert on either AI or human art, having dabbled in it for a day or two back when DallE and Midjourney first became popular. I'm sure someone who's into the image generation field could score 80%+ consistently even over a larger dataset just as handpicked as this one.
Telltale signs of AI:
- Cliches. "Punk Robot" is a good example, a human artist capable of creating that would've somehow been more creative. - Obviously facial expressions and limbs are telltale signs, very uncanny valley - If there's an accurate geomtric repeating pattern, it's extremely unlikely to be AI. Something like "Giant Ship" is afaik still practically impossible to generate. - Weird unfinished parts of a world. See "Leafy Lane". Why is there a vantablack hole at the end of the road? It's not physically impossible but it makes little sense and a human wouldn't put it there in that painting.
Fine art is a matter of nuance, so in that sense I think it does matter that a lot of the "human art" examples are aggressively cropped (the Basquiat is outright cut in half) and reproduced at very low quality. That Cecily Brown piece, for example, is 15 feet across in person. Seeing it as a tiny jpg is of course not very impressive. The AI pieces, on the other hand, are native to that format, there's no detail to lose.
But those details are part of what make the human art interesting to contemplate. I wouldn't even think of buying an art book with reproductions of such low quality--at that point you do lose what's essential about the art work, what makes it possible to enjoy.
You can't really draw many conclusions from this test, since the AI art has already been filtered by Scott to be ones that Scott himself found confusing. So What do any of the numbers at the end of this really mean? "Am I better than Scott at discerning AI art from human" is about the only thing this test says.
If you didn't filter the AI art first, people would do much better.
This article has an implicit premise that the ultimate judge of art is “do I/people like it” but I think art is more about the possibilities of interpretation - for example, the classics/“good art” lend themselves to many reinterpretations, both by different people and by the same person over time. When humans create art "manually" all of their decisions - both conscious and unconscious - feed into this process. Interpreting AI art is more of a data exploration journey than an exploration of meaning.
AI Art can be hard to identify in the wild. But it still largely sucks at helping you achieve specific deliverables. You can get an image. But it’s pretty hard to actually make specific images in specific styles. Yes we have Loras. Yes we have control nets (to varying degrees) and ipadapter (to lesser degrees) and face adapters and what not. But it’s still frustrating to get something consistent across multiple images. Especially in illustrated styles.
AI Art is good if you need something in a general ballpark and don’t care about the specifics
Eh, this is pretty unfair. That's a test of how good humans are at deceiving other humans, not a of how hard it is to distinguish run-of-the-mill AI art from run-of-the-mill human art in real life.
First, by their own admission, the author deliberately searched for generative images that don't exhibit any of the telltale defects or art choices associated with this tech. For example, they rejected the "cat on a throne" image, the baby portrait, and so on. They basically did a pre-screen to toss out anything the author recognized as AI, hugely biasing the results.
Then, they went through a similar elimination process for human images to zero in on fairly ambiguous artwork that could be confused with machine-generated. The "victorian megaship" one is a particularly good example of such chicanery. When discussing the "angel woman" image, they even express regret for not getting rid of that pic because of a single detail that pointed to human work.
Basically, the author did their best to design a quiz that humans should fail... and humans still did better than chance.
Not sure I understand the article. The author specifically chose art from humans and AI that he found difficult to categorize into human or AI art. The fact that people had a 60% success rate suggest that they are a little better in seeing the difference then he was himself?
(What am I missing? This is not like "take 50 random art objects from humans and AI", but take 50 most human like AI, and non-obvious human art from humans)
Generative AI is so cool. My wife (a creative director) used it to help design our wedding outfits. We then had them embroidered with those patterns. It would have been impossible otherwise for us to have that kind of thing expressed directly. It’s like having an artist who can sketch really fast and who you can keep correcting till your vision matches the expression. Love it!
I don’t think there have been any transformative AI works yet, but I look forward to the future.
It’s unsurprising to me that AI art is often indistinguishable from real artists’ work but famous art is so for some reason other than technical skill. Certainly there are numerous replica painters who are able to make marvelous pieces.
Anyway, I’m excited to see what new things come.
From the article:
So maybe some people hate AI because they have an artist's eye for small inadequacies and it drives them crazy.
This is it 100%.
When somebody draws something (in an active fashion), there is a significantly higher level of concentration and thought put towards the final output.
By its very nature, GenAI is mostly using an inadequately descriptive medium (e.g. text) which a user then must WAIT until an output that roughly matches your vision "pops" out. Can you get around this? Not entirely, though you can help mitigate this through inpainting, photobashing, layering, controlnets, loras, etc.
However, want to wager a guess what 99% of the AI art slop that people throw up all over the internet doesn't use? ANY OF THAT.
A conventional artist has an internal visualization that they are constantly mentally referring to as they put brush to canvas - and it shows in the finer details.
It's the same danger that LLMs have as coding assistants. You are no longer in the driver's seat - instead you're taking a significantly more passive approach to coding. You're a reviewer with a passivity that may lead to subtle errors later down the line.
And if you need any more proof, here's a GenAI image attached to _Karpathy_'s (one of the founding members of openAI) twitter post on founding an education AI lab:
https://x.com/karpathy/status/1813263734707790301
so much of the value of art, which Scott has actually endowed on these AI generated pieces, is the knowledge that other people are looking at the same thing as you.
Previous submission: https://news.ycombinator.com/item?id=42202288 (1 comment)
66% here. I was pretty much scrolling through and clicking on first instinct instead of looking in any detail.
Interestingly I did a lot better in the second half than the first half - without going through and counting them up again I think somewhere around 40% in the first half and 90% in the second half. Not sure if it's because of the selection/order of images or if I started unconsciously seeing commonalities.
The only way I can explain people getting 98% accuracy on this is being familiar with the handful of AI artists submitting their work for this competition.
Easy to defeat. AI can't come up with ambiguous art:
https://en.wikipedia.org/wiki/Ambiguous_image
There is a strategic feature to it based on retrospection.
It would have been interesting to know how much time most people spent per picture because if you look at the quoted comment from the well scoring art interested person mentioned:
"The left column has a sort of door with a massive top-of-doorway-thingy over it. Why? Who knows? The right column doesn't, and you'd expect it to. Instead, the right column has 2.5 arches embossed into it that just kind of halfheartedly trail off."
You can find this in almost every AI generated picture. The picture that people liked most, AI generated with the cafe and canal, the legs on the chairs make little sense. Not as bad as in non-curated AI art, but still no human would paint like this. Same for the houses in the background. If you spend say a minute per picture with AI art you almost always find these random things, even if the image is stylized, unlike human art it has a weird uncanniness to it.
Oh come on. I guess I missed the part in the "Turing test" where a human filters out 99.999% of the machine's output prior to the test.
It's interesting that the impressionist-styled pieces mostly fooled people. I think this is because the style requires getting rid of one of the hallmarks of AI imagery: lots and lots of mostly-parallel swooshy lines, at a fairly high frequency. Impressionism's schtick is kind of fundamentally "fuck high-frequency detail, I'm just gonna make a bunch of little individual paint blobs".
One of the other hallmarks of AI imagery was deliberately kept out of this test. There's no shitposts. There's one, as an example of "the dall-e house style". It's a queen on a throne made of a giant cat, surrounded by giant cats, and it's got a lot of that noodly high-frequency detail that looks like something, but it is also a fundamentally goofy idea. Nobody's gonna pay Michael Whelan to paint the hell out of this and yet here it is.
Duchamp rolling in his grave about this post!
I think what gave AI away the most was mixed styles. If one part of the painting is blurred, and another part is very focused, you can tell it's AI. People don't do that.
I got all of Jack Galler's pictures wrong though. The man knows how to do it.
I don’t think that is AI art Turing test.
An AI art Turing Test would be interactive with me telling it what to draw and what changes to make and see if what is producing the art is human or AI.