Gamma revisited

[Simon Garrett]

ajohnw, thanks for the link to Wikipedia. That article contains a sentence that isn't quite right, and feeds a lot of the misunderstanding:

Gamma encoding of images is required to compensate for properties of human vision, hence to maximize the use of the bits or bandwidth relative to how humans perceive light and color.

(My emphasis). Some people latch on only to the first part of that sentence, which I've highlighted. It's wrong, anyway. Gamma encoding is not required, but it's useful when encoding in 8 bits or less. It's largely irrelevant in 16 bits.

Gamma encoding does not "compensate for the properties of human vision", it simply encodes images in a way that allows fewer bits to be used for a given signal to noise ratio. It's a piece of engineering expediency. A tone curve is applied before encoding, and that tone curve has to be removed (by applying the opposite tone function) before display.

A second purpose for using a tone curve (not usually exactly a gamma function) is to compensate for non-linear response in output devices (monitors).

These two unrelated purposes get confused as the tone curve applied before encoding is usually a gamma curve (though not with sRGB), and the response of CRT monitors is (quite coincidentally) often approximately a gamma curve.

Both these functions are techy, engineering functions. Ideally they would be completely hidden from photographers, and we shouldn't even need to be aware of them. In fact, with colour management, we don't need to know anything about it. Colour management deals with it, and we can forget all about it.

PS - I've just edited that sentence on the Wikipedia page, so it now reads:

Gamma encoding of images is used to optimise the usage of bits when encoding an image, or bandwidth used to transport an image, by taking advantage of the non-linear manner in which humans perceive light and color.

Let's see if that change sticks. I think it's more accurate.

[ajohnw]

I'm afraid I part agree with the original Simon. We don't see in a linear fashion and 16bit per channel displays wouldn't help as much as people might think. That's where the band width aspects come into it. The last stop would run from 65536/2 to 65535. We can't see gradations that closely spaced at this range of light levels. No mention of max brightness as that is supposed to look after itself hence the max output in plots is always 1. The brightness control is a separate entity set to suit. What it's also about is obtaining good tonal characteristics when images are viewed so I think gamma would be applied what ever bit depth was used within reason. It has also been used to "improve" images. With 16bit channels though they might be a bit more like real stops.

John
-

[mikesan]

+1 but be prepared for some very stupid questions!

I read the tutorial on heare about it and my eyes glazed over! i found myself asking why do i need to know this? all knowledge is useful but at the end of the day if i take a picture and process it on my calibrated screen and print it on my profiled printer and it looks good and i get the same results off any of my machines.... do i realy need to know???

Just being devils advocate

Mark,
I quite agree; you really don't need to know to produce excellent photographs. No more than you need to know how an internal combustion engine works to be an excellent driver. Some of us, myself included, are just incurably curious and just WANT to know.

MrB,
Thanks for that reference. I had forgotten that a free viewer is available.

[xpatUSA]

Theres always open office Mike which is free and although im not certain i assume it will read excell documents. Post on Mike!

OpenOffice is now run by Apache:

http://www.openoffice.org/download/

I can confirm that it opens Excel files.

[ajohnw]

OpenOffice is now run by Apache:

http://www.openoffice.org/download/

I can confirm that it opens Excel files.

Interesting Ted. I installed OpenOffice a while ago for my wife and by mistake installed from the Apache site. It wouldn't do a number of things that the OpenOffice that comes with my Linux distribution would do. At the time there was another version available on openoffice.org that now seems to the same Apache version. Odd what? The one from .org I installed was exactly the same as the one I use but running on windoze. This was not long after Apache took it over.

If it does look crippled in any way it might be worth looking at the previous suite that was provided with most Linux distributions. LibreOffice. When I last used it they had part added image editing facilities with more bit depth than the GIMP. Not sure where it is at now. My latest releases have shifted to Calligra but LibreOffice can be installed as an option. OpenOffice no longer there. Calligra probably isn't ready for windoze yet. Google now offer something as well. The polls I have looked at last year put LibreOffice on top in most areas - all but one. Apart from Google code none of these will allow multiple people to work on the same document. Having used synchronisation when that is going on at work I'm not sure that it's really a good thing to have.

They will all generally load and even save MS formats, perhaps some more than others. I've saved a few spread sheets on the web in microsoft format. It doesn't seem to cause any problems.

John
-

[mikesan]

First pot-hole encountered. Now that I am assured everyone is OK with Excel spreadsheets, I find that the site does not allow one to attach files of that type. Any suggestions. I think I will start with attaching a gif image of part of the spreadsheet, but that is really not very satisfying.

[mikesan]

Gamma, Part #1

For the first installment I would like to deal with some technical definitions. Here I shall defer to an expert (Charles Poynton). The referenced paper contains a wealth of excellent material, some of which I find a bit over my head. For now, please concentrate on page 3; the quite unambiguous definitions of (1) Intensity, (2) Brightness, (3) Luminance and (4) Lightness:

http://www.poynton.com/PDFs/GammaFAQ.pdf

The important points to stress here are that both Intensity and Luminance refer to physical quantities having a measurable dimension (Like the physical quantity Mass, it doesn't depend on who is doing the measuring).

Brightness and Lightness are both perceptual quantities, the response to which depends on the human eye. Neither is based on a firm objective measurement. Lightness introduces the notion of a relative (numerical) measurement of Brightness, based on comparison to an arbitrary reference value (see below).

We can now tackle the of't quoted statement: "Human vision has a non-uniform (non-linear) perceptual response to the intensity of light." More exactly, the (perceived) Lightness is an exponential function of Luminance (intensity). Stated more simply it means that when the intensity of light is changed by a factor (= X), the perceived Brightness will change by a different factor (= Y < X).

I attempt to illustrate this in the attached spreadsheet. The graph (similar to many you have seen elsewhere) is derived from the data in columns B & C. Let me explain how I conceptualize this data. I picture a dark room containing a single light bulb. That bulb is connected to a power source that allows one fine control of the power supplied to the bulb (i.e. the Luminance). As the Luminance is increased the apparent Brightness of the bulb (to a human observer) will also increase. However a point will be reached where a further increase in the Luminance will no longer result in a further increase in Brightness. We note this threshold and designate the Luminance at this point as our "white reference" and assign it a value of 100 (=Yn).

Next all values of Luminance (Yi) in the range 1 to Yn are normalized relative to the white reference: (Yi/Yn) and these are the values listed in column B (Luminance, normalized). These normalized values are then used to compute Lightness (L*) according to the equation found at: Publication CIE No 15.2, Colorimetry, Second Edition. (Vienna, Austria: Central Bureau of the Commission Internationale de L’Éclairage, 1986). The equation is found in the Poynton paper. Exactly how this august body arrived at this equation is above my pay grade.

This is really a lot of words to drive home just two important points:

1. Just looking at the graph in the spreadsheet one can immediately appreciate that a doubling of the Luminance (e.g. from 0.20 to 0.40) results in a much smaller (33%) increase in the Lightness (from 52 to 69).

2. I emphasize the same point by listing, in column D of the spreadsheet, the percent increase in Lightness at each interval, compared to the immediately preceeding interval. And in column E one finds the absolute increase in Lightness (Δ L*)over the immediately preceeding interval.

Why is this important? Many authorities have stated that the human eye can distinguish a difference in Lightness (between two adjacent areas) only when the differences is 1% or more. The CIE (as suggested in the Poynton paper) states: "a delta L-star of unity is taken to be roughly the threshold of visibility." I have highlighted both of these thresholds in the spreadsheet. You will see they do not completely agree; I have no opinion as to which is more correct. The important point is that such a threshold exists and we must be aware of this when we are dealing with data in an image file.

NO SPREADSHEET ATTACHED (verboten ). Please see attached gif.

Much more to come. Please let me know if you find any serious errors or omissions thus far.

[dje]

First pot-hole encountered. Now that I am assured everyone is OK with Excel spreadsheets, I find that the site does not allow one to attach files of that type. Any suggestions. I think I will start with attaching a gif image of part of the spreadsheet, but that is really not very satisfying.

Mike you could load your excel file onto a free file sharing site such as Dropbox or Google + and post a link to it in your CiC thread. That would allow people to access it.

Dave

[dje]

Just read the details in post 27, looks OK so far but relevance to gamma yet to be established.

[Mark von Kanel]

First pot-hole encountered. Now that I am assured everyone is OK with Excel spreadsheets, I find that the site does not allow one to attach files of that type. Any suggestions. I think I will start with attaching a gif image of part of the spreadsheet, but that is really not very satisfying.

dropbox link?

[Simon Garrett]

I'm afraid I part agree with the original Simon. We don't see in a linear fashion and 16bit per channel displays wouldn't help as much as people might think. That's where the band width aspects come into it. The last stop would run from 65536/2 to 65535. We can't see gradations that closely spaced at this range of light levels.

With more than about 14 bits of linear encoding, the human eye will not be able to see the steps even at the dark end. With logarithmic encoding, 8 or 9 bits will do - hence 8 bit jpeg is adequate for most purposes.

Interestingly, editor programs often remove any gamma encoding before arithmetic processing of image data (and then reapply it afterwards). This is because it's often easier to process the image if it's linearly encoded. For example, internally within Lightroom all processing is carried out in 16 bits linear space.

No mention of max brightness as that is supposed to look after itself hence the max output in plots is always 1. The brightness control is a separate entity set to suit. What it's also about is obtaining good tonal characteristics when images are viewed so I think gamma would be applied what ever bit depth was used within reason. It has also been used to "improve" images. With 16bit channels though they might be a bit more like real stops.

John
-

I quite agree - sometimes a gamma function or other tone curve is applied for artistic reasons. Also, because typical monitors (and printers) cannot provide as much contrast as the original scene, a slight gamma function gives a bit of a contrast boost, making the monitor image or print look more like the original.

[ajohnw]

Interestingly, editor programs often remove any gamma encoding before arithmetic processing of image data (and then reapply it afterwards). This is because it's often easier to process the image if it's linearly encoded. For example, internally within Lightroom all processing is carried out in 16 bits linear space.

.

They also generally keep data in a different colour gamut to the one that is being displayed. I believe dcraw will produce colour correct images from raw in a linear fashion. They are extremely dark. It seems from comments on it's site this is what some want as output. It also offers an auto brightening option. Chances are this one way or the other is in most pp packages.


Gamma and bit depth. It's easy to forget the complications relating to brightness. A few questions to think about

Why is studio colour work and colour matching done at relatively low screen brightness levels and in low ambient lighting levels as well.

Why is post processing generally done at 125 cd/m^2

Why do manufacturers of late set default brightnesses approaching 300 cd/m^2

The reason for the last one is that it makes the dynamic range look better and it might even match what they quote. If our eyes had pupil sizes fixed to suit either of the others they would look eye searingly bright. Also think about what happens as things brighten in respect to out ability to see tonal levels. Ambient light level specs are also available for controlled colour correction. It's very likely that these are chosen in part to suit our eyes performance wise.

My screen is set at around 110cd/m^2 and has a dynamic range of about 850:1 As supplied, much brighter that might have been a bit higher. Screens try to maintain contrast. They do this fairly well as blacks get brighter as the top end gets brighter. Using current technology an increase in back lighting leads to more leakage causing blacks to be less black than they should be at some point.

If the screens emitters could produce light things still aren't that simple even though true blacks could be produced. For one just how bright could the brightest bright be and how much of it to prevent out pupils adjusting to compensate while not loosing darker detail. Looking at an image that fits in our acute field of vision is not the same as looking around out of doors in various light levels. All sorts of things are going on when we do that largely dependent on the acute field of view but not entirely.

When we look at images our pupil size is also accounting for ambient levels. This is why some colorimeters offer calibrations based on ambient levels. Personally I have mixed feelings about that as they are measuring using 3 specific colours. Ambient light cab contain colours that they don't record. Our eyes response to brightness is also influenced by colour. Green is often used to determine it. Astronomers go a step further and use 3 colours for measuring light brightness. Screens basically increase blue levels to obtain sensible colour temperatures.

If it was possible to get "numbers" to fit in with all of this I'm pretty sure gamma will always be with us and past some point increasing bit depth doesn't help at all and never will.

John
-

[xpatUSA]

We can now tackle the of't quoted statement . . .

Please let me know if you find any serious errors or omissions thus far.

Not serious, but it doesn't look right, especially with that apostrophe. I would suggest "oft-quoted". Sorry about the hyphen (which nobody uses these days) but, in literature, "oft" is rarely found without one

time flies like a banana . . . tiktiktiktiktiktik . ..
time-flies like a banana . . . munch, munch, munch . . .

[Simon Garrett]

Very interesting points ajonhw.

To add to one point you make:

"Why is post processing generally done at 125 cd/m^2." One reason is that, as you say, the contrast ratio is higher at higher brightnesses, and the picture more punchy. However, prints have a much lower contrast range, so if the image is edited to look right at 250 cd/m^2 then it will look rather drab when printed. I print stuff, and also some of my images are shown on a local camera club projector that also has a pretty low contrast ratio, so edting at about 100 cd/m2 is about right for me.

[James G]

"Why is post processing generally done at 125 cd/m^2."

I am 'old fashioned' enough to still want to print shots on a regular basis. Respecting the above, I now always set my monitor brightness to 105 cd/m^2.
Previously when calibrating my monitor, I used to use the recommended setting from the calibration software of 120 cd/m^2. I found I always had to reduce image brightness by 20-30% to generate an acceptable print, avoiding washout.

I actually contacted Datacolor in Birmingham about this, and after an interesting discussion, was advised that a value of 100-110 cd/m^2 would be optimum for print. (In addition, I no longer use ambient light monitoring.)

Till now I have used a default gamma setting of 2.2 when calibrating the monitor,( received wisdom ) but I'm beginning to think that there could be some virtue in considering a different value to optimise for print.
Not sure how I want go about testing this yet, but will see how this thread develops.....

[mikesan]

dje & Mark-
The use of DropBox did occur to me. It's a bit cumbersome but I may have to resort to that.
Yes the first post (post #27) does not show any direct relevance to gamma. That will come (I hope). I am sneaking up on it; be patient.

xPat-
Thanks. I am usually meticulous in my use of language and any corrections in this sphere are also welcome. I think I meant to type " oft' " with the apostrophe appended to indicate the contraction from "often".

To all-
I think I need to expand on my very first post. I said that I had read extensively on the subject of gamma and still was left with many questions. I began the process of constructing this "tutorial", not because I had a sudden epiphany which made it all clear to me, and which I wished to share with others. Quite the contrary. This is a work in progress, in which I try to construct a rational explanation based on a few basic premises. I publish it here only to subject my reasoning to the critical comments of others. I hope, by this process, to arrive at a final (or at least satisfactory) answer as I proceed. I am not there yet but, with your help, I hope to achieve that goal.

Concerning critical comments and after reviewing what I have written, I discover a significant misstatement in Part #1. I stated: "the human eye can distinguish a difference in Lightness (between two adjacent areas) only when the differences is 1% or more." The correct statement (from accepted authorities) refers to a difference in Luminance (the physical intensity) of 1% as the threshold. As a result the data highlighted in the spreadsheet (gif) is incorrect. The actual point occurs in the progression of normalized Luminance not included in this data. Nevertheless the important point is that such a threshold does exist and I will revisit that when I come to the translation of sensor data to digital (file) data.

[mikesan]

Testing the use of dropbox for excel files. Feedback please.

https://dl.dropboxusercontent.com/u/...CinC_%231.xlsx

[dje]

Testing the use of dropbox for excel files. Feedback please.

https://dl.dropboxusercontent.com/u/...CinC_%231.xlsx

Works fine for me Mike

Dave

[ajohnw]

I am 'old fashioned' enough to still want to print shots on a regular basis. Respecting the above, I now always set my monitor brightness to 105 cd/m^2.
Previously when calibrating my monitor, I used to use the recommended setting from the calibration software of 120 cd/m^2. I found I always had to reduce image brightness by 20-30% to generate an acceptable print, avoiding washout.

I actually contacted Datacolor in Birmingham about this, and after an interesting discussion, was advised that a value of 100-110 cd/m^2 would be optimum for print. (In addition, I no longer use ambient light monitoring.)

Till now I have used a default gamma setting of 2.2 when calibrating the monitor,( received wisdom ) but I'm beginning to think that there could be some virtue in considering a different value to optimise for print.
Not sure how I want go about testing this yet, but will see how this thread develops.....

I went down to 110 as I found 120 or so too bright at times.

I don't print. If I did I would go for soft proofing and end to end calibration. It can be done with a scanner. A colour card is scanned to calibrate the scanner and then printed and measured with the same scanner. The same idea can be used to calibrate a camera. Argyll colour management utilities contains software for this. The site also has a link to a source of fresh cards along with their colour data.

Not sure about gamma and printing. Several "wisdoms" seem to be about. One is that dynamic range is reduced made up for by increasing saturation, an argument in favour of Adobe RGB and Prophoto but I have seen good sRGB prints. Another off luminous landscape is contrast masks. There are also loads of comments on the web about why prints come out flat, dark, muddy or what ever.

I feel that gamma is more complicated than some people may think if the whole subject to viewing the image is considered but do know that some use a different gamma as a matter of course. I have enough variables to play with so don't really want another. At one stage one of my packages showed gamma as a user alterable curve. I did find that very minor changes could improve an image on a pc screen but the adjustments were minuscule. The best changes when it worked seemed to be when the middle 1/2 span was flattened a little usually by dragging the top of that down a touch. This may be similar to using a contrast mask to flatten out the brightness range - what some feel is needed for prints. I use a package that has a slider labelled flatten brightness range. It doesn't have the effect that might be expected contrast wise and as the author of the package mentions often improves images. I'd guess by tending to move tones into the best range available on monitors. This article suggests that the same technique could be used to tame prints.

http://www.luminous-landscape.com/tu..._masking.shtml

I was glad to find that as sometimes I am not using the package with the slider and wasn't totally sure how it was done as playing with a curve wasn't easy. The slider is still a lot easier. The histogram just flattens and extends slowly as it's used.

I don't think I would play with my gamma settings for web images as I know that minor changes applied to an image and shown at 2.2 can make a lot of difference. I've never seen any suggestions about changing it for prints I'd guess because it will maximise the tonal range available hence the max output always being 1. The pp task with prints is probably getting it into 5-6 stops rather than maybe 8-9 on a screen.

John
-

[TonyW]

Mike, you say that the Lightness is an exponential function of Luminance (intensity). It is more like a logarithmic function than exponential but in any case why do you then use the power function relationship instead. Are you just bowing to convention? I know the use of the power function is the usual one but the power chosen is, from what I have seen quoted, usually the gamma of 1/2.2. Why have you chosen a power of 1/3? Do we know why 1/2.2 is usually used?

A logarithmic relationship is actually quite common. When we talk about the number of stops of aperture, that is actually a logarithmic function of aperture. If we go from f/2 to f/4, for example we are decreasing the amount of light by a factor of 4 but we count it as two stops. Then if we go 2 stops further we get an aperture of f/8 and reduce the light again by factor of 4.

As a pedant from way back, I would say that "oft quoted" is correct without the apostrophe and without the hyphen, although I realise that the hyphen is commonly used. "Oft" is simply an archaic synonym for "often".