Are pixels really blown in ACR?

[Lon Howard]

Maybe I have misinterpreted but I had always thought about pixels this way: once blown at capture, they're gone forever. Another description: "You can't get something from nothing."

But I have noticed for some time that in ACR, simply decreasing the exposure apparently retrieves much blown detail in the sky, and when I go back and review the image on the CF card, I can see that the histogram for the highlights there (not so much the shadows) agrees with the histogram when opened up in ACR. So even though I blew out those pixels at capture, I didn't really lose them ... thanks to ACR, I got something from nothing!

If I make no corrections to the raw file and simply open it in PS, then yep, those blown pixels are gone and can't be retrieved in PS.

I realize there's a limit to this but apparently I don't need to be as paranoid as I was about blowing highlights in camera, even when I only have one exposure ... there's a lot more wiggle room than I thought.

I'm just wondering though: Is there a fairly precise way to determine in camera, when you've gone too far and really HAVE lost detail forever? Thanks,

Lon

[davidedric]

I think someone more expert than I will comment. But a first thought is what do the three RGB channels look like in the histogram? Maybe you didn't blow all three?

[revi]

If you really over-expose part of the image, that is try to oversaturate the photo-sites on the sensor, there's no way to recover detail in that part of the image. Wrt the preview images, those are jpegs produced in-camera from the raw data, and are thus processed. There are several reasons why that jpeg can appear to have blown areas, where a raw developer can still recover detail.

Easiest one would be that the sensor data are captured in 12-bit precision, and a jpeg uses 8 bit/channel. Then there's a gamma correction applied, etc. In short, the preview jpeg is not a good representation of the raw data when you want to judge over- or under-exposure, unless you really know your camera (and have adjusted the jpg settings to give a better approximation). And note that the in-camera histograms are based on the preview jpeg (unfortunately).

Case in point: ETTR (expose to the right) can often give you a horrible looking, over-exposed preview jpeg (and an image that is under-exposed by several stops gives you a very reasonable looking preview)...

Remco

[Lon Howard]

I think someone more expert than I will comment. But a first thought is what do the three RGB channels look like in the histogram? Maybe you didn't blow all three?

On the CF card all three are touching the right edge with the blue spiked slightly. It's the luminosity that's spiked all the way up.

[kris]

The difference between the RAW and the JPEG format is that the first are (basically) the data camera sensor captures while the second is a processed version of the data captured by the sensor. RAW files includes a JPEG version of the data produced with the camera setting (color space, WB etc). So what you see is what you would get if you were developing the RAW data with these settings.

The developing process converts the RAW data into a different form, according the chosen settings, so it may happen that some pixel, or better color, appears saturated once converted. This is for example the case of the RED channel, try to change the color space from sRGB and RGB. When you open the file in a RAW converter, ACR or others, you can change the parameters used to develop the data and produce a JPEG version. This is why you can "recover" blown pixels.

Clearly if a pixel has saturated you cannot recover the data. Actually is a little more than this because the color is recovered using a demosaicing process that uses data from more than one pixel, so you can still recover some data even if one pixel is saturated.

Cheers
A.

[kris]

On the CF card all three are touching the right edge with the blue spiked slightly. It's the luminosity that's spiked all the way up.

The histogram is produced by using the camera settings for JPEG conversion.

[Glenn NK]

On the CF card all three are touching the right edge with the blue spiked slightly. It's the luminosity that's spiked all the way up.

I assume you mean on the camera LCD screen.

First things first - are you shooting RAW or JPEG? The results will be different.

The image on the LCD is an in-camera JPEG. It will be a JPEG whether you shoot JPEG or RAW. The camera cannot display a RAW file because a RAW file is not an image file - it is just raw data that is collected on the sensor. It must be converted to an image file.

The problem with the LCD image (being a JPEG image), is that it does not necessarily reflect what the converted RAW file will look like when you process it.

Question - what camera body are you using? If it's a Canon body (which I'm familiar with), there are methods to get the LCD screen image to more closely match a RAW image on the computer screen.

I use the RGB histogram extensively and exclusively - but in the camera settings I use a Contrast value of minus three or even minus four. By keeping the image on the LCD screen from clipping (at the right), the RAW file I process in PP is rarely clipped.

And if the image is clipped a bit, it is often possible to get a reasonable processed image - it depends to some extent on the software being used.

Hope this isn't too confusing - if so keep asking.

Glenn

[Lon Howard]

Lesson learned ... I should have posted an actual file instead of speaking in generalities. A raw file of the latest image I am working on showing this aspect can be downloaded from http://www.mediafire.com/?3xw23y20v4esdtb. Make sure you return the file to it's original state by clicking on the little icon on the right side of the bar named 'Basic', and then selecting 'Camera Raw Defaults'.

Once done, you can see that the lost detail in the sky is apparently retrieved by just reducing the exposure. It certainly LOOKS LIKE I have recovered all the the lost detail. I'm therefore assuming I did.

I understand that the LCD image in camera (Canon 5D Mk II) is an already processed jpeg image and not entirely accurate but both that and the histogram in the raw file indicate lost detail and you can clearly see that just by looking at the image.

For those not that familiar with raw converters or photoshop layers, it's very easy to correct the other parts of the image that were changed when reducing exposure.

More than anything, this seems to be proof positive that shooting in raw has powerful advantages. I'm just wondering how much wiggle room there really is when recovering lost detail. Much more than I thought, apparently.

Any additional thoughts? Thanks for everyone's input.

[Glenn NK]

Lon:

I've DL'd your CR2, and imported into LR4.2.

The blacks are slightly clipped (only very slightly), but the highlights are not blown at all.

From what I can see, it's a very well exposed image. I've increased the Exposure (which introduced some clipping), then reduced the Highlight slider a bit to bring it back and eliminate the clipping.

Are you using an older version of ACR that could be indicating clipping?

On my own 5DII, under the Picture Style menu, I have set the Contrast value to minus three - this compensates for the inaccurate JPEG clipping that is observes on the LCD without the compensation.

I've been shooting RAW exclusively since I started digital six years ago (my 30D is also set to the same values for Contrast).

Glenn

[FrankMi]

Hi Lon, in Adobe Camera Raw I was able to recover all of the highlights by setting Recovery to 30 and all of the shadow detail by setting the Fill Light to 20 without touching Exposure or any other settings with a clean histogram so all of the data is available in the RAW image.

[Lon Howard]

Thanks for your thoughts, Glenn. Maybe we're just interpreting what's blown or not blown differently, which is to say more or less: to each his own. Here is what the original raw image looks like (I just opened it and saved it without any editing, i.e., SOOC:



The histogram in ACR shows a large spike off the right side and the highlight clipping warning shows significant red in the sky on the right side and above the building too. When I eyeball the actual image without the clipping warning, I see the loss of detail.

Here's what the image looks like after only reducing the exposure in ACR to eliminate the clipping warning. I can see lots of detail now in the areas that were clipped. But that's just how these two look to me.




Although I did have to open these in PS in order to resize and save them, they look exactly the same here in CiC as they did when I viewed them in ACR; so even though I do have an older version of ACR, I don't believe that is the issue.

In fact, there really isn't an issue! Since the images look the same here as in my ACR, I'll just say that I overthunk the thing and started worrying about something that isn't there. Election overload! Thanks very much for pitching in.

[Glenn NK]

Hi Lon, in Adobe Camera Raw I was able to recover all of the highlights by setting Recovery to 30 and all of the shadow detail by setting the Fill Light to 20 without touching Exposure or any other settings with a clean histogram so all of the data is available in the RAW image.

Frank:

What did the histogram look like in ACR when you imported it?

In Lightroom, there was no highlight clipping at all, and it only required pulling the highlights down when I increased the Exposure.

On the blacks end, the clipping was very minimal. That image was as close to perfect as I've ever gotten - I usually have a wee bit of clipping in the highlights (usually specular highlights).

And as an aside, the blacks looked reasonably noise free - probably better than my 5DII would have done.

Glenn

[Glenn NK]

Thanks for your thoughts, Glenn. Maybe we're just interpreting what's blown or not blown differently, which is to say more or less: to each his own. Here is what the original raw image looks like (I just opened it and saved it without any editing, i.e., SOOC:

The histogram in ACR shows a large spike off the right side and the highlight clipping warning shows significant red in the sky on the right side and above the building too. When I eyeball the actual image without the clipping warning, I see the loss of detail.

Although I did have to open these in PS in order to resize and save them, they look exactly the same here in CiC as they did when I viewed them in ACR; so even though I do have an older version of ACR, I don't believe that is the issue.

In fact, there really isn't an issue! Since the images look the same here as in my ACR, I'll just say that I overthunk the thing and started worrying about something that isn't there. Election overload! Thanks very much for pitching in.

Lon:

How I interpret clipping is when (with all presets removed - RESET in LR) - no clipping in the RAW image when there have been no adjustments, (including presets).

I don't know how ACR works, but in LR, when the RESET button is pressed, the only thing "adjusted" is that the Camera Profile would read "Adobe Standard". When I select Landscape for your image, there is a spike at the right edge of the histo, but it's not clipping at all. However there IS a little bit of CA around the "porch" roof of the building - easily corrected.

In LR4.2 the image is "slightly" clipped at the black end, but absolutely nothing at the white end.

It's a finely exposed image from what I can see. If all mine were that clean, I'd be a happy camper.

Glenn

PS Edit: I will say what is odd is that when I apply the lens correction for the 24-105, a bit of clipping in the clouds appears - which is somewhat odd.

[Lon Howard]

Hi Lon, in Adobe Camera Raw I was able to recover all of the highlights by setting Recovery to 30 and all of the shadow detail by setting the Fill Light to 20 without touching Exposure or any other settings with a clean histogram so all of the data is available in the RAW image.

Thanks for that, Frank ... I was editing when you were posting. Yes, I just did the same thing and it worked beautifully. There are really many ways to 'get there from here' in ACR. Personally I like to pump up the blacks as far as I can and then work around that somehow (usually in PS) to get the look I want. I'm learning something new about ACR almost every day and the more I learn the more I like. Thank you again.

[FrankMi]

What did the histogram look like in ACR when you imported it?

Hi Glenn, with RAW image reset in ACR there was clipping in both the shadows and highlights and you could see the specific Blue and Red clipped areas as I have those features of ACR turned on.

[Glenn NK]

So it seems that ACR and Lightroom don't treat the RAW image exactly the same.

[FrankMi]

So it seems that ACR and Lightroom don't treat the RAW image exactly the same.

I'm using CS5 and LR 3.6 so you are one level above me in ACR. If you had also reset the ACR settings I am a bit surprised that they register the clipping levels differently but I'm sure that it is possible as they are constantly tweeking these things.

[Lon Howard]

Glenn, the only thing that I can think of is that using the reset in LR gets you to a different place than reverting back to Camera Raw Defaults in ACR. Be that as it may, the only thing that matters is that you don't lose detail and you get the output you're after, which I did. At least so far ... now I can go back to actually finishing the image.

[Glenn NK]

I'm using CS5 and LR 3.6 so you are one level above me in ACR. If you had also reset the ACR settings I am a bit surprised that they register the clipping levels differently but I'm sure that it is possible as they are constantly tweeking these things.

Frank:

There were some significant improvements in LR4 over LR3 - one of which is highlight recovery - maybe that's why I'm getting different results.

Lon:

AFAIK, the RESET in LR removes all import presets, and all subsequent edits - in effect giving a clean slate back to the original RAW file. If I'm wrong, someone will correct me.

Glenn

[dje]

Lon:


On my own 5DII, under the Picture Style menu, I have set the Contrast value to minus three - this compensates for the inaccurate JPEG clipping that is observes on the LCD without the compensation.


Glenn

Glenn I wasn't aware of this but have just tried it on my 600D and found that it does indeed improve the accuracy of the histogram in relation to blown highlights. I've actually got it set to -4.

Thanks for sharing

Dave