It's always entertaining to try to understand these things from first principles. Color and skin color in particular are very complex things to measure and model. It's not just a physical quantity, it's also a matter of human perception based on lighting and a host of factors.
It seems like some work was done to research existing approaches, but I don't see any reference to Pantone Skin Tones [0]. Without intending to, I've worked on 3 different projects which involved measuring skin tones for makeup applications. The Pantone range is not 100% perfect, but it's pretty well done and easy to understand. There are two axes: a light/dark one, and red-yellow one [1].
This is beautiful work! First, I thought this would involve using PCA to go from 3D to 2D, which would result in an easier selector to use, but at the expense of representing every person.
Then, I thought this would stop at using the U-space vectors (the ones that form the basis of the PCA image) and their corresponding ellipse to form our color space, but no, the function fitting is a very slick idea, even if it was executed by hand.
Lastly, I love the presentation of sampling from different r values. Whether you sample from a fixed r value or a range of r values, I bet this has great applications in game design or animation.
I still don't quite get the manual data labeling process at the beginning? It seems like it would encode some bias, but the consistency of the first point cloud and the results certainly speak for themselves.
show comments
dahart
> What colors are we? The short answer is maybe something like “brown” and the long answer is very, very long.
Pete Shirley (author of Ray Tracing In One Weekend) is also a part time color scientist, and he clued me into the fact that if you crank the saturation to 100% of pictures of people of any race, the skin color comes out orange. I think I heard that some face detectors may work on this principle, but I don’t remember where I heard that.
Nice. Most of these colors look right to me, but I'm also seeing green, blue and purple? If you're curating colors from the space manually then it's not a problem.
"Only realistic colors" would be a different problem. And I'm not sure if there's a way to solve that problem without excluding some real colors too.
(Maybe if the colour selector was a different shape? I don't know much about color spaces.)
show comments
stephantul
Love it. I especially thought the introspective “aside” section on related resources was great, and I wish more people would show this kind of reflection.
show comments
stu2b50
I think it’s worthwhile to note this is under a light with a color temperature similar to sunlight.
When you’re actually painting, skin can really be any color. Blue is a perfectly valid (and fairly common) skin color - just look at someone in the moonlight. It takes two to tango, and the other dancer is the light source.
xdennis
A reporter once asked the Picard actor why his character is bald. Surely, they would find a cure by the time humanity figured out space travel. The actor countered saying that by the 24th century they would have stopped caring about such trivial things.
Similarly, you'd think that as people got less racist they would stop caring about skin color. Instead, the left is obsessed with skin color. We used to have a single emoji skin color representing all of humanity. Now we have 6. Presumably OP thinks that's too few. We used to have a single emoji for pregnant women. Now we have 18. 6 for women who identify as men, 6 for women who identify as non-women, and 6 women who don't identify.
m_w_
This is super cool! I have always loved projects that try to categorize color space. Usually my work has gone in the other direction - isGray, isCloseToWhite, isPastel - but it's the same insofar as it's defining shapes like your matplot visual.
Wish I had this on hand like this three or four years ago when I was working on VR avatar software - we had hard-coded skin tone options at the time - would have been a cool addition!
show comments
kaitlinseng
So glad to see Nyma getting recognition! I went to elementary and middle school with her, happy to see her succeeding.
Sharlin
The U and V axes seem roughly correspond to the tint/temperature (green-magenta/blue-amber) axes that you can find in a white balance tool in photo editors and also as an in-camera setting.
Skin tones are particularly difficult to pin down, besides the pigmentation there are two effects at play: 1) sub surface scattering (light goes thru and bounces under the skin a bit) and 2) some infra-red emission (the responsible for making some people appear magenta on digital photography)
So even when you get the palette right, to paint it like a believable material in a convincing way is still super hard!
Would love to see this 3-D picker ability in the many character designers out there, instead of the lame 8 swatches or whatever.
show comments
ecesena
Nice. This is relatively old work (2020) but Pinterest had/has skin tone features, for example if you're searching for cosmetic products. Worth a check if anything might be useful for your work.
https://medium.com/pinterest-engineering/powering-inclusive-...
dofm
Interesting though that the 1.5 setting (at least on my older iPhone) doesn’t really ever seem to cover the range of skin tones of indigenous Australians (“aboriginal” and Torres Strait Islanders), who apparently get their skin tones from a somewhat different expression of genes than other peoples with darker skin.
The 2.0 “outliers” cover those skin tones.
Masai and some Somali skin tones, similarly, look better represented by the outliers.
The full range of possible human skin tones is perhaps a bit of a challenge for technical colour representation, partly because diffuse reflectance seems to vary so much and that is hard to approximate on screen (e.g. as any website designer could attest, the difference between representing silver and grey is the use of highlights), though the representation of skin tones is a lot better than the early colour film era.
show comments
PullJosh
I love this! Such a beautiful use of some real nerdy math to benefit humans in a way non-nerds would appreciate.
rrradical
Very neat project. Could I use this code to generate skin tones in my own project? Or more meant to just be educational/inspirational? I guess I'm wondering if you've considered adding a license.
show comments
ptsneves
On the movie Dolemite by Eddy Murphy, they go over the fact that black people absorb light and white people reflect, and how that changes the motion picture shooting process totally. Also how one size does not fit all. Cool project! I am not much into the humanities part of the project. For me this is as natural as having a localization framework, just in this case for skin tones.
zetanor
Setting R² to various large values like 50, 500 and 5000 then moving the slider around seems to give a decent and simple visual representation of what's happening.
efitz
Never seen anyone #E6FFF8 (bottom right corner).
An albino that consumed too much colloidal silver for years?
JacobAsmuth
Me after I spend a day at the beach: #BA3A17
RIMR
For anyone confused, turn off your dark mode extension. :)
show comments
hluska
This is excellent work! You’ve got a 49 year old and a 10 year old equally into it. That’s impressive - nice work.
aaroninsf
Not my domain though I followed the data science enough to nod as if I really followed it,
but this is just lovely and inspirational work. A unique marriage of hacking in the celebratory sense, thoughtfulness, and meaningful contribution. Outstanding.
show comments
bellowsgulch
Skin is so interesting as a surface to think about how light passes through and bounces off of.
There is at least one article published by a technical artist at Pixar who describes modeling a skull within a human scan so that subsurface lighting is more accurately rendered.
I suspect you could combine this with dr_scully’s comment and GolDDranks’ to reach the nuance involved in long tail realism details.
show comments
Abh1Works
Cool work!
ralusek
If you're ##CCFFFF please go to the hospital
uhdozo
Very interesting, I'll need to read it again lol
moralestapia
Interesting.
It might be a bit biased though, as there's much more variation on fairer skin tones than on dark ones (there's only one that looks like the colloquial "black").
This contradicts all research and social media from the past decade.
It's always entertaining to try to understand these things from first principles. Color and skin color in particular are very complex things to measure and model. It's not just a physical quantity, it's also a matter of human perception based on lighting and a host of factors.
It seems like some work was done to research existing approaches, but I don't see any reference to Pantone Skin Tones [0]. Without intending to, I've worked on 3 different projects which involved measuring skin tones for makeup applications. The Pantone range is not 100% perfect, but it's pretty well done and easy to understand. There are two axes: a light/dark one, and red-yellow one [1].
[0] https://www.pantone.com/na/en-us/skintone
[1] https://cosmeticsbusiness.com/pantone-skintone-guide-80945
This is beautiful work! First, I thought this would involve using PCA to go from 3D to 2D, which would result in an easier selector to use, but at the expense of representing every person.
Then, I thought this would stop at using the U-space vectors (the ones that form the basis of the PCA image) and their corresponding ellipse to form our color space, but no, the function fitting is a very slick idea, even if it was executed by hand.
Lastly, I love the presentation of sampling from different r values. Whether you sample from a fixed r value or a range of r values, I bet this has great applications in game design or animation.
I still don't quite get the manual data labeling process at the beginning? It seems like it would encode some bias, but the consistency of the first point cloud and the results certainly speak for themselves.
> What colors are we? The short answer is maybe something like “brown” and the long answer is very, very long.
Pete Shirley (author of Ray Tracing In One Weekend) is also a part time color scientist, and he clued me into the fact that if you crank the saturation to 100% of pictures of people of any race, the skin color comes out orange. I think I heard that some face detectors may work on this principle, but I don’t remember where I heard that.
Love it!
I was looking at this a little while ago, and used some of The Pudding's data on makeup/foundation shades (https://pudding.cool/2018/06/makeup-shades/) and plotted it into the Oklab colorspace (https://observablehq.com/@55th/foundation-shades). The shades form themselves into that same crescent as seen in the article.
Nice. Most of these colors look right to me, but I'm also seeing green, blue and purple? If you're curating colors from the space manually then it's not a problem.
"Only realistic colors" would be a different problem. And I'm not sure if there's a way to solve that problem without excluding some real colors too.
(Maybe if the colour selector was a different shape? I don't know much about color spaces.)
Love it. I especially thought the introspective “aside” section on related resources was great, and I wish more people would show this kind of reflection.
I think it’s worthwhile to note this is under a light with a color temperature similar to sunlight.
When you’re actually painting, skin can really be any color. Blue is a perfectly valid (and fairly common) skin color - just look at someone in the moonlight. It takes two to tango, and the other dancer is the light source.
A reporter once asked the Picard actor why his character is bald. Surely, they would find a cure by the time humanity figured out space travel. The actor countered saying that by the 24th century they would have stopped caring about such trivial things.
Similarly, you'd think that as people got less racist they would stop caring about skin color. Instead, the left is obsessed with skin color. We used to have a single emoji skin color representing all of humanity. Now we have 6. Presumably OP thinks that's too few. We used to have a single emoji for pregnant women. Now we have 18. 6 for women who identify as men, 6 for women who identify as non-women, and 6 women who don't identify.
This is super cool! I have always loved projects that try to categorize color space. Usually my work has gone in the other direction - isGray, isCloseToWhite, isPastel - but it's the same insofar as it's defining shapes like your matplot visual.
Wish I had this on hand like this three or four years ago when I was working on VR avatar software - we had hard-coded skin tone options at the time - would have been a cool addition!
So glad to see Nyma getting recognition! I went to elementary and middle school with her, happy to see her succeeding.
The U and V axes seem roughly correspond to the tint/temperature (green-magenta/blue-amber) axes that you can find in a white balance tool in photo editors and also as an in-camera setting.
Nb. I'd advise against calling the PCA'd space XYZ because that's a well known standard color space: https://en.wikipedia.org/wiki/CIE_1931_color_space#CIE_XYZ_c...
Skin tones are particularly difficult to pin down, besides the pigmentation there are two effects at play: 1) sub surface scattering (light goes thru and bounces under the skin a bit) and 2) some infra-red emission (the responsible for making some people appear magenta on digital photography)
So even when you get the palette right, to paint it like a believable material in a convincing way is still super hard!
This part ( https://toneyalexander.github.io/inclusive-color-space/#pick... ) is especially cool.
Would love to see this 3-D picker ability in the many character designers out there, instead of the lame 8 swatches or whatever.
Nice. This is relatively old work (2020) but Pinterest had/has skin tone features, for example if you're searching for cosmetic products. Worth a check if anything might be useful for your work. https://medium.com/pinterest-engineering/powering-inclusive-...
Interesting though that the 1.5 setting (at least on my older iPhone) doesn’t really ever seem to cover the range of skin tones of indigenous Australians (“aboriginal” and Torres Strait Islanders), who apparently get their skin tones from a somewhat different expression of genes than other peoples with darker skin.
The 2.0 “outliers” cover those skin tones.
Masai and some Somali skin tones, similarly, look better represented by the outliers.
The full range of possible human skin tones is perhaps a bit of a challenge for technical colour representation, partly because diffuse reflectance seems to vary so much and that is hard to approximate on screen (e.g. as any website designer could attest, the difference between representing silver and grey is the use of highlights), though the representation of skin tones is a lot better than the early colour film era.
I love this! Such a beautiful use of some real nerdy math to benefit humans in a way non-nerds would appreciate.
Very neat project. Could I use this code to generate skin tones in my own project? Or more meant to just be educational/inspirational? I guess I'm wondering if you've considered adding a license.
On the movie Dolemite by Eddy Murphy, they go over the fact that black people absorb light and white people reflect, and how that changes the motion picture shooting process totally. Also how one size does not fit all. Cool project! I am not much into the humanities part of the project. For me this is as natural as having a localization framework, just in this case for skin tones.
Setting R² to various large values like 50, 500 and 5000 then moving the slider around seems to give a decent and simple visual representation of what's happening.
Never seen anyone #E6FFF8 (bottom right corner).
An albino that consumed too much colloidal silver for years?
Me after I spend a day at the beach: #BA3A17
For anyone confused, turn off your dark mode extension. :)
This is excellent work! You’ve got a 49 year old and a 10 year old equally into it. That’s impressive - nice work.
Not my domain though I followed the data science enough to nod as if I really followed it,
but this is just lovely and inspirational work. A unique marriage of hacking in the celebratory sense, thoughtfulness, and meaningful contribution. Outstanding.
Skin is so interesting as a surface to think about how light passes through and bounces off of.
There is at least one article published by a technical artist at Pixar who describes modeling a skull within a human scan so that subsurface lighting is more accurately rendered.
I suspect you could combine this with dr_scully’s comment and GolDDranks’ to reach the nuance involved in long tail realism details.
Cool work!
If you're ##CCFFFF please go to the hospital
Very interesting, I'll need to read it again lol
Interesting.
It might be a bit biased though, as there's much more variation on fairer skin tones than on dark ones (there's only one that looks like the colloquial "black").
This contradicts all research and social media from the past decade.
Seems to underrepresent pale skin tone variety
This hits different after a long week