Whites in Flowers- an Experiment

[xpatUSA]

Like Chauncey, I like to shoot flowers. An element of laziness, it's certainly easier than hockey matches, birds in flight or that passing jet. All the time in the world to set up the tripod, fiddle with the camera - a couple of snaps and back to the the computer inside. Couldn't be better!

So - why have I been fighting a flower all day, I ask myself? The whites around the edge of the flower are blown out in the review image, even though the raw data is well under the sensor saturation level. I fiddled with curves a bit, increasing the slope at high levels, and then it occurred to me that those same high levels are the ones that get scrunched up with gamma correction which you often get whether you want it or not.

So then I thought what if I develop the image without gamma correction? So I did.The only way I could do that is in a raw composite image exported out of RawDigger, so the colors below are less than perfect. That matters little - because what I am about to show you is exactly how much detail can get lost in normal highlights with a standard gamma-corrected image conversion. Voila:



At left, the raw file is developed using the manufacturer's converter, Sigma Photo Pro. The white at the petal tips has next to no contrast - as is common with white flower shots taken outside. A lot of the purple detail is washed out, too. By comparison, the contrast in the image on the right shows up lots of lost detail.

Makes ya think, dunnit?!

[chauncey]

Ted, you have shown us that the detail is present, although I don't understand your technique.
How do we take advantage of that knowledge?

[xpatUSA]

Ted, you have shown us that the detail is present, although I don't understand your technique.

The technique in producing the second image is to completely bypass the standard apps such as Capture One, PhotoShop, Aperture, etc. These applications are there to give you the prettiest possible picture, often applying lots of NR, color saturation, sharpening and . . . . gamma. Remove all that default stuff and you get to see what the sensor saw

I use an application called RawDigger - the best thing since sliced bread and an excellent diagnostic tool - not that hard to use either. One of the things it can do is make a 'raw composite image' - it takes the data straight from the red, green (averaged) and blue sensor data and exports it as a three-channel TIFF. By doing no processing to the data, no detail gets lost, IMHO. Maybe it's the wrong colour, maybe it's not sharp enough, etc., but the detail is retained, not squashed up in a gamma curve.

How do we take advantage of that knowledge?

I have had some success with the use of curves, which most editors have as options. The idea is to make the RGB curve look like the top end of the CRT gamma curve below.



That effectively 'undoes' the gamma correction that was applied during raw conversion. Of course, it's quite a game keeping mid- and low-tone parts of the curve as before (straight) and it's easy to get totally messed up and have to start over.

I'm not certain that high tones can be fully undone this way and would most certainly recommend 16-bit working, floating point if possible. RawTherapee for example.

If you're into layers (I'm not) maybe some sort of layer with the highlight detail could be merged with a regular image - others here will know.

Feel free to ask more questions . .

[davidedric]

Very interesting, Ted. I certainly don't have your depth of understanding of gamma correction, but presumably the effects depend both on whatever the camera does in producing the RAW file (which I can't do anything about) and then the RAW processor.

I use Lightroom, and I've got quite interested in what it does to the tone mapping when you pull the various "sliders" around. It has curves as well, of course, but the sliders do not change the shape of the tone curve so you have to figure it out indirectly. It's especially tricky in Lightroom because the sliders are "adaptive" - that is their exact effect depends on the image in question.

I'm going to see if I can find any images where there may be tonal compression in the highlights, and have a play.

Dave

[lukaswerth]

Feel free to ask more questions . .

Here is one question: the colors in your 2nd image look pretty nonfunctional to say the least; what are you going to do about them?

Actually, I don't think there is any difficulty to pull out all that detail you want in Photoshop through various techniques. One key among others is the luminosity blend mode, or go into lab colorspace to tackle contrast and color separately.

Lukas

[chauncey]

Lukas...can you expand on that a bit for the slower ones among us.

[Slipper]

Lukas...can you expand on that a bit for the slower ones among us.

me too please.

[xpatUSA]

Here is one question: the colors in your 2nd image look pretty nonfunctional to say the least; what are you going to do about them?

This topic was not actually proposing a workflow. The composite image was my best way to illustrate the negative effect of gamma compression on white flower highlights.

If I were to actually use a raw composite in a workflow, I would develop and save a profile in RawTherapee. I have done this for IR images, admitted a subject less concerned with color accuracy. Another route, which I have not tried, could be to convert a Macbeth card raw composite TIFF to a DNG (if that's possible) and use that to create a .dcp (camera profile) file.

Actually, I don't think there is any difficulty to pull out all that detail you want in Photoshop through various techniques. One key among others is the luminosity blend mode, or go into lab colorspace to tackle contrast and color separately.

Not everybody (myself included) has PhotoShop - so the processes you mention above are not available to some folk, sad to say. But it would be most interesting to see it done. If you would be so kind as to show us how it's done, with a result that equals or surpassed my composite image in highlight detail?

I have put a TIFF up here for you (or anybody else) to download:

http://kronometric.org/phot/post/CiC/SDIM2437.tif

Here are the adjustments to the above image that I made in Sigma Photo Pro:

Exposure Comp -0.3 EV
Contrast +0.1
Shadow -0.2

If you have a Sigma camera, I will be happy to post the X3F so that you can work ab initio.

Looking forward to your result . . .

[xpatUSA]

Another possibility is to use an image with less gamma correction - a compromise, so to speak.

Bear in mind that sRGB images have a gamma co-efficient that is actually more than 2.2 (2.4 to be precise). More scrunching at the top end, durn it!

On the other hand, ProPhoto has a gamma of 1.8. It may not be widely-known, but JPEGs can have an embedded ProPhoto ICC profile and such a JPEG might be a bit better for white flower colors.

Well, I tried it and you do get a slight improvement but not enough to warrant upsetting folks with non-color-managed browsers ;-)



The petal overlap just to the left of the histogram is more visible than in the OP. However, the histogram for the selection in the image tells it all: next to no contrast = next to no detail.

Before folks start going on about it, embedding ('saving as') a ProPhoto profile into a JPEG is no different to saving as an Adobe RGB (1998) image and comes with the same color-management caveats.

[chauncey]

Here be my version, better or worse...in the eye of the beholder.



Looks like I shoulda fixed the alignment problem, oh well.

[xpatUSA]

Most impressive, William.

Photoshop? What processing exactly (for the benefit of others)?

[davidedric]

After five minutes in Lightroom - again may be better or worse!

Dave

[chauncey]

Photoshop? What processing exactly (for the benefit of others)?

PS it was>selected the whites above 210 RGB using a temporary threshold layer and used color range
on selection>applied Shadows/Highlights to taste>duplicated image and flattened to apply HDR toning>selected flower and pasted it back into original image>used Saturation and Levels adjustment layers to taste.

It was quite close to my normal workflow so it didn't take long to do it.

Dave's method was a lot quicker though.

[davidedric]

And should anyone be interested, these are the settings I used. Basically, pulled the highlights down as far as possible with the slider, and then into the tone curve to separate the tones even more. Plus a bit of sharpening.

There is some clipping, basically because I pushed up the Whites to give an overall more pleasing image, but I am sure that could be balanced out with more care.









Dave

[xpatUSA]

After five minutes in Lightroom - again may be better or worse!

Dave

Also very impressive, David, especially if it's from the TIFF that I just posted!

My theory above recovering via a tone curve seems to work quite well, too:



That tone curve's a bit fierce though

[lukaswerth]

...
Not everybody (myself included) has PhotoShop - so the processes you mention above are not available to some folk, sad to say. ...
I have put a TIFF up here for your convenience:

http://kronometric.org/phot/post/CiC/SDIM2437.tif
...
Looking forward to your result . . .

Ted,

Oompf...got work to do? I might try my hand tomorrow at this picture - right now I am not at the machine where my CS6 is (no CC yet), and I have some other work to finish, too. The file looks a bit small, though (13MB).
I mentioned photoshop because you said in your prior email you bypassed ... "standard apps like...photoshop" - so I thought you had it.

But to answer also your questions, Chauncey and John: the point is how to get details and contrast out of highlights. If you start straight working on an image by opening a curves adjustment layer, you mess up the colors. So what you want is to create a layer which does nothing to the colors. In order to do that, you change the blending mode of the layer: among others soft light, hard light, and luminosity do nothing to the colours, and luminosity preserves them most faithfully: what this blend mode does is using the lightness, the luminosity/gradation from the layer in this mode, whereas all the colors come from the layer below. So, you can really duplicate your basic layer or introduce another layer with the move tool which you tweak for what it's worth for contrast without the slightest regard for the colors, and then set it just to luminosity blending mode.

As for LAB colorspace, you find it under image/modes/LAB: check out the channels once you converted he image: it contains one lightness channel containing all the contrast (L), and two color channels contrast: A for cyan and magenta, and B for blue and yellow. If you open in this color space a curves adjustment layer, you may separately tweak, just as you do in RGB the red, blue and green channel, the L, A, and B channel. This means: when adjusting the lightness curve, you do nothing whatsoever to the colors which can be tweaked absolutely separately. Many people also apply an unsharp mask to the lightness channel.
Just don't forget to reconvert the image again to RGB once you are done with LAB because you cannot save the file as a jpeg in this colorspace, nor print from it.

There is a chap called Dan Margulis who wrote among other Photoshop-topics also about this color space; the one I read is called "the Canyon Conundrum" -something: not bad, quite informative, even though written for those professionals working under constant time pressure for their clients, and breathing all these commercials-aesthetics which I personally loathe.

Lukas

[xpatUSA]

Ted,

Oompf...got work to do? I might try my hand tomorrow at this picture - right now I am not at the machine where my CS6 is (no CC yet), and I have some other work to finish, too. The file looks a bit small, though (13MB).

Great, looking forward to it. Sorry, can't make the file any bigger for you - it's converted from a 4.7MP Sigma SD14's X3F raw file.

I mentioned photoshop because you said in your prior email you bypassed ... "standard apps like...photoshop" - so I thought you had it.

Yes, they were all mentioned as examples.

I do have ACR 5.4/ PSE 6 but I try hard not to boast about it

Later,

[Brev00]

I do not often like the effect of pulling the highlight slider all the way down. I find it has a graying effect. I, personally, would rather have a blown highlight that is white than introduce a computer generated tone that doesn't quite match what I saw. This is an interesting discussion as it involves an issue I and probably many others have to deal with.

[davidedric]

Hi Larry,

Yes you are quite right about the greying. The problem with Ted's photo is that all the bright tones are smashed up together as we can see at the end of the histogram. To see the details you need to separate out those tones, which inevitably means the less bright tones appear grey. That's essentially what the highlights slider does with the brightest third of the tones, more or less. In Lightroom, one possibility would be to use two concentric radial filters with the Tips of the petals between them. That way you could dial down the highlights in the outer one and push them back up again in the inner one. I think I might give it a try for my own education.

Of course you could mask each individual petal, but I think Ted's intention was to illustrate the problem rather than produce a perfect image

[lukaswerth]

Okay, at long last:



Have look at it. If the picture is visible, it won't stay for long: I had to upload it on flickre, because the upload button on this side is currently not working for me.

Damn it! picture not visible on my screen, here is the link to flickr:

(Deleted again! Please see my next post.)

I must admit at least partial defeat. What I could get from this file bears no comparison from what I get from my own images.
Anyway, for those interested, I simply used an HDR toning layer, blending mode soft light. Adjusted the curve after that.

Lukas