Hello Claude, Matju et al,
Colour perception is probably as complicated as auditory perception and the accurate representation of colour on digital video systems is probably more difficult that the accurate representation of audio. We can perceive a much wider range of colour and brightness (chrominance and luminance) than the majority of digital video systems can represent, so this wide-gamut information has to be mapped to the narrow-gamut of the systems which we have.
To further complicate things, our response to brightness and colour is not linear and so those perceptual curves have to be taken into account during that mapping into video signals (chroma and luma). There are also many other non-linearities in how video displays work so that all has to be taken into account. Add the complexities of PAL, NTSC and SECAM to that and you're set for a party!
Most video digitisation systems also use 'chroma sub-sampling' which is a way of reducing the amount of information which needs to be stored (this is a first level of compression or coding which occurs before anything else like MPEG compression is applied). This uses that fact that we are more sensitive to differences in brightness than variations in colour. The YCbCr (corresponds the YUV colour space of our perception) video representation uses this technique which makes it more compact while giving approximately equal quality to an larger RGB representation.
Anyone working with colour video systems should know and read Charles Poynton's writing on this subject - he is the acknowledged expert. If you only have one book on your shelf it should be his "Digital Video and HDTV". He does have much useful information also on his web site: http://poynton.com/
For example, he has a guided tour of colour space here: http://poynton.com/papers/Guided_tour/abstract.html
[you can also be amazed, amused or angered by the way he vacillates between 'color' and 'colour' in his writing...]
hope that helps, Andrew
p.s. other notable datum: Poynton taught the very first ever microprocessor course given at an art school. (In the 1970s at the Ontario College of Art in Toronto)
On Wed, 12 Dec 2007, pd-list-request@iem.at wrote:
Hi,
Redirecting from GEM-dev as it's not about GEM development...
Mathieu Bouchard wrote:
if you do a polar transform on YUV, you have something easier, faster and more correct all at once. I usually just skip the polar transform: if you apply rotations directly on YUV values, you can make very believable hue shifts.
Interesting, I'm in the process of experimenting a bit with different colour spaces, got in a real headache with XYZ and CIE L*a*b and so on, but YUV's simplicity may win.
HSV is dubious in part because the apparent brightness at maximum so-called value is very variable and seems to peak high or low at secondaries or primaries: compare yellow (brightness 89%) and blue (brightness 11%). this really makes HSV suck sometimes. YUV does not have this problem.
I tried a hybrid approach:
"$1 1 1" | [hsv2rgb] | [rgb2yuv] | "0.5 $2 $3" | [yuv2rgb]
and that seems to eliminate the bad brightness mismatches, at the cost of some colours seeming a bit washed out (blue) or muddy (yellow).
Attached image demonstrates the difference.
Wondering if there's some set of "perceptual brightness curves" similar to the "isophonic curves" [1] there are for perceived loudness of different frequencies and levels led me to [2], which seems very complicated again.
On Wed, 12 Dec 2007, Andrew Brouse wrote:
To further complicate things, our response to brightness and colour is not linear and so those perceptual curves have to be taken into account during that mapping into video signals (chroma and luma).
Oh and also all those "absolutely perceptual" systems are just discarding the genetic diversity of the universe (as well as the non-genetic diversity!). One man's metamers is another one's yuck. ;)
Most video digitisation systems also use 'chroma sub-sampling'
Yes, we know about it, but I don't think that it's so relevant to colours unless using the YUV image type in GEM or PDP. (In GridFlow, YUV images are not subsampled, and also, Claude is so far only talking about individual colours, as float triplets)
This uses that fact that we are more sensitive to differences in brightness than variations in colour.
But this is not counting the fact that we could be standing close enough to the screen to actually see that much chroma resolution. No explanation of subsampling ever talks about that.
The YCbCr (corresponds the YUV colour space of our perception) video representation uses this technique which makes it more compact while giving approximately equal quality to an larger RGB representation.
In order to say that, you have to define quality to correspond to some idea of how one should look at a screen in order to justify the subsampling. But people don't watch a screen from the same distance.
Anyone working with colour video systems should know and read Charles Poynton's writing on this subject - he is the acknowledged expert. If you only have one book on your shelf it should be his "Digital Video and HDTV".
Hey. Colours are nice, but art in general is a lot more important. If I'd have only one book on my shelf it should be something else than a treatise of colorimetry.
[you can also be amazed, amused or angered by the way he vacillates between 'color' and 'colour' in his writing...]
None of the above...
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
Hello Matju et al,
You really shouldn't take my comments personally, they aren't meant to be at all. I do like you personally and respect the work that you do.
My comments were meant to provide useful information to all those who are developing and working with Digital Video in Pd. For what it's worth, the references I pointed to are standard fare for all video engineers, whether on computers or not.
I find it really sad the amount of bitterness, bickering, infighting and acrimony in parts of the Pd community. The energy that goes into this is energy that doesn't go into making Pd and associated projects better tools. I find that really sad.
That's all I have to say, Andrew
On Fri, 21 Dec 2007, Andrew Brouse wrote:
You really shouldn't take my comments personally, they aren't meant to be at all. I do like you personally and respect the work that you do. My comments were meant to provide useful information to all those who are developing and working with Digital Video in Pd. For what it's worth, the references I pointed to are standard fare for all video engineers, whether on computers or not. I find it really sad the amount of bitterness, bickering, infighting and acrimony in parts of the Pd community.
I'm not taking your comments "personally". I'm trying to reply in a mostly technical way. If you write to pd-list you can expect that the standard fare for all video engineers could eventually be questioned, and just because it's standard doesn't mean it's holy.
The energy that goes into this is energy that doesn't go into making Pd and associated projects better tools. I find that really sad. That's all I have to say,
Are you sorry that I question subsampling? It sounds a lot like it to me! But I don't even know what you are referring to... the more general a complaint is, the less it can be meaningfully addressed. No, you really sound like you deplore that I question parts of colour theory, but you group it with random unspecified other mails of pd-list in order to __________. (fill in the blank)
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
hi andrew. would love to see some patches that demonstrate what you're talking about,
it's all a bit over my head.
On Sat, 22 Dec 2007, hard off wrote:
hi andrew. would love to see some patches that demonstrate what you're talking about, it's all a bit over my head.
Try to zoom into a part of a GEM image that is in the YUV colourspace or that formerly was (e.g. digitising a TV signal or taking input from most webcams). If the image is as sharp as it can be tuned to be, you will be able to see that the colouring of the pixels is blurry compared to their intensity. The blur is either horizontal or both horizontal and vertical. For example a pure red diagonal bar over pure green background will usually show some pixels that are a different shade of red or a different shade of green. If it doesn't, it means that you picked two shades that have the same brightness, or that the bar is positioned exactly on colouring pixel boundaries. The different shades appear because the colouring pixels are twice bigger (or more) than the brightness pixels, and the boundary of the bar you are filming is being better represented by brightness than by colour.
About the non-linearity of vision... This is something else, and a good start into that, is to look at gamma correction. Gamma correction is actually correcting the monitor, which doesn't output light proportional to its electric input, and has to be compensated. I mention gamma because the gamma formula is both simple and non-linear, so it's a good starting point about non-linearity, but it doesn't actually address the non-linearity of vision. You could perhaps look at the HSV colour objects that Claude was talking about, and look at how the conversions are made (just some floatboxes and one conversion object). It is a common example of non-linear mapping from RGB, but it should also be noted that it's not linear relative to vision, no matter how superficially it may look like it's closer to one's understanding of colours. Look also at when you crossfade two colours, even if you tune your gamma correctly, how often the average of two colours doesn't feel like it's halfway between the two colours.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
It makes much more sense to see it broken down like so :
http://www.lafcpug.org/Tutorials/basic_chroma_sample.html
or http://adamwilt.com/pix-sampling.html and http://adamwilt.com/DV-FAQ-tech.html#colorSampling
These color-spaces use different sampling cadences, and as mentioned
before leverage the fact that eyes are more sensitive to luma changes
that chroma changes. Using different sampling cadences is essentially
a form of compression ( less bandwidth required for a frame), typical
Y cr'cb' or commonly known YUV video is the norm for broadcast
masters, and use what is called 4:2:2 sampling cadence (the
terminology is quite.... lacking), which means for every 4 luma
samples (or every 4 pixels, you have 4 luma samples,) but only 2 color
samples. Note that YUV does not have to have reduced sampling
cadences, 4:4:4 (one luma, one cr' one cb' sampler per pixel) is
perfectly valid, but not very common.
These reduced candences can significantly effect any attempts to do
good chromakeying or color work, why after effects and other
applications that are not strictly editors typically work at 4:4:4
RGB. Anyway, Its definitely an interesting to know, and can
significantly help make footage you are working with look better.
So while yes, it may be boring on the surface, if you plan on making
any ART with video, and it wont be in black and white, its good to
know about ;)
P.S., there was some discussion concerning the difference of human
genetics not being accounted for. The older broadcast formats for NTSC/
PAL and SECAM are not ... final, per se. There are variants of all of
these, and the YUV components can vary quite differently depending on
what version of NTSC/PAL etc you want to use in what nation/geography.
The 609 spec for SD was supposed to fix this (at least for NTSC), and
the new HDTV rec 709 actually is a combination of the various most
used components of PAL and NTSC, so conversion is simpler. So it used
to be more diverse, and complicated, not its simpler but makes more
assumptions.
Anyway, that last part was way off topic.
On Dec 21, 2007, at 11:57 AM, Mathieu Bouchard wrote:
On Sat, 22 Dec 2007, hard off wrote:
hi andrew. would love to see some patches that demonstrate what
you're talking about, it's all a bit over my head.Try to zoom into a part of a GEM image that is in the YUV
colourspace or that formerly was (e.g. digitising a TV signal or
taking input from most webcams). If the image is as sharp as it can
be tuned to be, you will be able to see that the colouring of the
pixels is blurry compared to their intensity. The blur is either
horizontal or both horizontal and vertical. For example a pure red
diagonal bar over pure green background will usually show some
pixels that are a different shade of red or a different shade of
green. If it doesn't, it means that you picked two shades that have
the same brightness, or that the bar is positioned exactly on
colouring pixel boundaries. The different shades appear because the
colouring pixels are twice bigger (or more) than the brightness
pixels, and the boundary of the bar you are filming is being better
represented by brightness than by colour.About the non-linearity of vision... This is something else, and a
good start into that, is to look at gamma correction. Gamma
correction is actually correcting the monitor, which doesn't output
light proportional to its electric input, and has to be compensated.
I mention gamma because the gamma formula is both simple and non- linear, so it's a good starting point about non-linearity, but it
doesn't actually address the non-linearity of vision. You could
perhaps look at the HSV colour objects that Claude was talking
about, and look at how the conversions are made (just some
floatboxes and one conversion object). It is a common example of non- linear mapping from RGB, but it should also be noted that it's not
linear relative to vision, no matter how superficially it may look
like it's closer to one's understanding of colours. Look also at
when you crossfade two colours, even if you tune your gamma
correctly, how often the average of two colours doesn't feel like
it's halfway between the two colours._ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC
Canada_______________________________________________ PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Fri, 21 Dec 2007, vade wrote:
These reduced candences can significantly effect any attempts to do good chromakeying or color work, why after effects and other applications that are not strictly editors typically work at 4:4:4 RGB. Anyway, Its definitely an interesting to know, and can significantly help make footage you are working with look better.
Yeah, I've encountered problems with that. GridFlow naturally treats all YUV as 4:4:4, but I was still inputting from a YUV 4:2:0 stream which was a recombination of a RGBG (double green) matrix. The RGBG thing is similar to YUV 4:2:2 as you get more resolution for one channel than the other, but because this is not the same channel, some further information is lost, and on top of that, YUV 4:2:0 discards even more information compared to YUV 4:2:2 or RGBG. This caused me trouble for an application that was very relying on the chroma... you can think of it as a chroma key, though I was not using it to make transparencies nor even any video output.
P.S., there was some discussion concerning the difference of human genetics not being accounted for.
I only mentioned it briefly and so far you're the first one to actually exchange about it. (I've always found it weird that the word "discussion" is used for something in which there are not already two persons exchanging, but quite a few textbooks use it like that, so...)
The older broadcast formats for NTSC/PAL and SECAM are not ... final, per se. There are variants of all of these, and the YUV components can vary quite differently depending on what version of NTSC/PAL etc you want to use in what nation/geography.
I don't think that most of the difference is in genetics of whole populations. It might be plain statistical error due to the sample size or the choice of the sample, or it could be the way to extract a typical measure (mean vs median vs mode), or it could be an instrumentation error (or just a choice in how it should be done, in the case where it's not clear that only one way is not a mistake), for example, filtering out high violet by using the wrong kind of glass (it happened).
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
On Dec 21, 2007, at 3:04 PM, Mathieu Bouchard wrote:
P.S., there was some discussion concerning the difference of human
genetics not being accounted for.I only mentioned it briefly and so far you're the first one to
actually exchange about it. (I've always found it weird that the
word "discussion" is used for something in which there are not
already two persons exchanging, but quite a few textbooks use it
like that, so...)
Oh Matju - you are so kind. Sometimes you remind me of Malvin from War
Games - http://youtube.com/watch?v=mNiiBrEHBWA&feature=related
You need a Jim to keep you in line :) (said with love !!)
The older broadcast formats for NTSC/PAL and SECAM are not ...
final, per se. There are variants of all of these, and the YUV
components can vary quite differently depending on what version of
NTSC/PAL etc you want to use in what nation/geography.I don't think that most of the difference is in genetics of whole
populations. It might be plain statistical error due to the sample
size or the choice of the sample, or it could be the way to extract
a typical measure (mean vs median vs mode), or it could be an
instrumentation error (or just a choice in how it should be done, in
the case where it's not clear that only one way is not a mistake),
for example, filtering out high violet by using the wrong kind of
glass (it happened).
Well, either way, the values are different, and im sure different
methodologies produced the different results, but you cant deny that
quite a large difference in genetic stock was used :) Anyway, was more
of an offhand comment for those who may be reading. Thanks,
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
On Fri, 21 Dec 2007, vade wrote:
Oh Matju - you are so kind. Sometimes you remind me of Malvin from War Games
You can't be serious.
(said with love !!)
With love like that, no need for hate! :) Fuddle duddle again. :(
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
-
Andrew Brouse -
hard off -
Mathieu Bouchard -
vade