Results 1 to 15 of 15

Thread: Raw-Tiff-Demosaic-Gamma

  1. #1

    Raw-Tiff-Demosaic-Gamma

    I would appreciate an explanation of whether the conversion process from a RAW file to a TIFF
    includes

    1) Demosaicing
    2) Gamma Correction.

    Thanks

    Peter

  2. #2

    Join Date
    Dec 2008
    Location
    New Zealand
    Posts
    17,662
    Real Name
    Have a guess :)
    Hi Peter,

    In a word, yes (to both).
    Last edited by Colin Southern; 17th October 2012 at 01:26 AM.

  3. #3
    drjuice's Avatar
    Join Date
    Oct 2011
    Location
    Southern California
    Posts
    310
    Real Name
    Virginia

    Re: Raw-Tiff-Demosaic-Gamma

    Respectfully, I'd disagree with Colin's answer. If you use a professional package, think Photoshop or one of the vendors' packages for doing the conversion, his answer is yes. But, for some of the freely available shareware, the programs' developers have clearly decided to go for the exact representation of the source image, mosaic and all. Unfortunately, I no longer have the list of programs I looked at before I chose the one I like. But, as I recall, it was about 10 items long, so it's not uncommon.

    In the program I chose to use, dcraw (see wikipedia pages) the only color "correction" is whether you want to use the camera's setting for white balance or tediously build your own. I generally set the camera for AWB and use that setting as it gets me much closer than anything else I've tried.

    v

  4. #4

    Join Date
    Nov 2009
    Location
    Provence, France
    Posts
    906
    Real Name
    Remco

    Re: Raw-Tiff-Demosaic-Gamma

    Dcraw allows export of the raw data (converted to 8 or 16 bit, though) as a gray scale image with gamma=1 when you use the -d, -D or -E option (see
    the man page for all the details).

    I don't know of any other programs like this, but I wouldn't be too surprised if at least some use dcraw under the hood (the dcraw page has an impressive list of programs based on it )
    Last edited by revi; 19th October 2012 at 07:50 AM.

  5. #5
    ajohnw's Avatar
    Join Date
    Aug 2012
    Location
    S, B'ham UK
    Posts
    3,262
    Real Name
    John

    Re: Raw-Tiff-Demosaic-Gamma

    As I have mentioned before it is hard to find any raw based software that doesn't use some open source software called dcraw. There are a list of packages that use it on it's web site easily found on the web as revi mentions. These include Adobe. Ufraw uses it too. Dcraw does have an option for generating an image without demosaicing. I wasn't aware that ufraw could use it but as Ufraw is really a graphical interface for dcraw I have never used it with command line options. There are a couple of dcraw tutorials about on the web but if I want a pure raw image I now use ImageJ as it saves typing in the dos box or console in my case. ImageJ is a graphics processing engine for technical users with a vast array of specialised plugins. It's fairly easy to use it for this. Much of the rest is intended for image processing specialists. I some times use a pure raw image to test a lens out of curiosity. This is what they look like. The crosses are pixel boundaries. On this one contrast variations can be seen in the GB letters, blur and vertical camera shake. A shadow across the letters as well. The blur is probably CA as most lenses have at least a pixel to 1 1/2 pixels of it.

    Raw-Tiff-Demosaic-Gamma

    On the original question for the best gamma correction it's probably better to use the correct camera icc file rather than the defaults that dcraw provides. The defaults generate different results that most noticeably differ in brightness. I some times make marginal manual adjustments to gamma later as it can be useful at times.

    -

  6. #6

    Join Date
    Dec 2008
    Location
    New Zealand
    Posts
    17,662
    Real Name
    Have a guess :)

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by ajohnw View Post
    As I have mentioned before it is hard to find any raw based software that doesn't use some open source software called dcraw. There are a list of packages that use it on it's web site easily found on the web as revi mentions. These include Adobe.
    I thought it "unusual" that Adobe would need to use it, so I checked the DCRAW website. I can see a list of "Other RAW Photo Decoders", but can't see anything there to say that all of the ones listed are using DCRAW, unless I've missed something.

  7. #7
    ajohnw's Avatar
    Join Date
    Aug 2012
    Location
    S, B'ham UK
    Posts
    3,262
    Real Name
    John

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by Colin Southern View Post
    I thought it "unusual" that Adobe would need to use it, so I checked the DCRAW website. I can see a list of "Other RAW Photo Decoders", but can't see anything there to say that all of the ones listed are using DCRAW, unless I've missed something.
    Well I do know that a number of the developers in that list do use it so assume that all do. It's not usual for companies to use specialist contractors or there code in the software world.

    -

  8. #8

    Join Date
    Dec 2008
    Location
    New Zealand
    Posts
    17,662
    Real Name
    Have a guess :)

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by ajohnw View Post
    Well I do know that a number of the developers in that list do use it so assume that all do.
    Really not sure how one comes to that assumption I'm afraid, especially considering Adobe's phenomenal in-house human resources.

  9. #9
    drjuice's Avatar
    Join Date
    Oct 2011
    Location
    Southern California
    Posts
    310
    Real Name
    Virginia

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by ajohnw View Post
    Well I do know that a number of the developers in that list do use it so assume that all do. It's not usual for companies to use specialist contractors or there code in the software world.

    -
    Hi, John in Birmingham -

    First, I would not make any assumptions about people specifying the requirements for software or software designers or programmers using ANY "feature" of any sort in any other product, whether it's $old, shared, or free. I have performed all three of the preceding tasks and, depending on what the person wanting the software asks for, the program may or may not have particular feature(s). If it's something I think is necessary or appropriate or would make the program "better" (more in a moment), I'll ask the client. But, for custom software, which is basically what my team does, nothing goes in unless the client has agreed to pay for it. So, if a client wanted a custom program for reading RAW and creating a .PSD or .TIFF or ..., as bad an opinion as I would have of the client who didn't want demosaicing as part of his order, it wouldn't be included.

    This is just a fundamental part of our business process, whether anybody else likes how we do it or not.

    virginia

  10. #10

    Join Date
    Nov 2009
    Location
    Provence, France
    Posts
    906
    Real Name
    Remco

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by Colin Southern View Post
    Really not sure how one comes to that assumption I'm afraid, especially considering Adobe's phenomenal in-house human resources.
    Because it's cheaper to use already written (and tested!) code than to start from scratch? Or (as stated in the list of programs 'using' dcraw) they might have used it as
    documentation to avoid re-inventing the wheel from scratch. Or the reference to Adobe is to an old version? Or, ...

  11. #11
    ajohnw's Avatar
    Join Date
    Aug 2012
    Location
    S, B'ham UK
    Posts
    3,262
    Real Name
    John

    Re: Raw-Tiff-Demosaic-Gamma

    Hi Virginia as you work with software you are probably aware of the old and ancient proverb of if it exists use it or maybe modify it if you can. There is a lot of windows application software about that has used software written initially for unix let alone that which makes use of freely available source. There is also software about that has been literally ripped off with hardly any modification at all from open source and sold. Linux crops up in all sorts of things as it's free and works. Windows going back to very early versions prior to few people using PC's let alone windows was correctly reckoned to have sprung from an idea IBM had at the time. As the dcraw site points out people are free to use his code and I am 99% certain all of the listed software will have used it as a start at least. In that respect one of Adobe's other products Acrobat has some interesting associations Ghostscript. That in and odd sort of was has associations with Postscript and so it goes on. This area is why I stated what I did. One of the latest and most extensive examples of the same sort of thing is the Mac. Neglecting the desktop very strong rumours suggest it's running linux underneath. It would be amazing to find that the gloss on top didn't largely originate from a similar source,

    Also to add to the 99% a quote from his resume

    1997 - Present
    Author of dcraw, an essential tool for digital photography.
    Although raw digital photos provide better quality than JPEG, many photographers do not use them for fear that computers will someday be unable to read these undocumented file formats.
    Dcraw is a free ANSI C program to decipher any raw photo from any digital camera on any computer running any operating system. It is my gift to programmers, photographers, and future generations of historians.
    Thanks to dcraw source code, Adobe Photoshop and dozens of other image tools now have built-in support for raw photos, and the popularity of raw photography has grown enormously.

    You see few companies employ people full time with skills that are only needed from time to time. Adobe and others probably employ him as a contractor. Even microsoft use contratcors and consultants at certain levels in there software. Quote from some training material I read some years ago " These people tend to live in big houses and are very hard to see". Made me green with envy. I am a real time specialist - different field and much lower wages.

    As to dcraw this is the bulk of the source code. Undertaken from scratch with no knowledges would be a fair undertaking and would probably involve vast expense for your clients and many mistakes even though there isn't that much of it really. It runs incredibly quickly too.

    http://www.cybercom.net/~dcoffin/dcraw/dcraw.c

    -

  12. #12

    Join Date
    Dec 2008
    Location
    New Zealand
    Posts
    17,662
    Real Name
    Have a guess :)

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by ajohnw View Post
    You see few companies employ people full time with skills that are only needed from time to time. Adobe and others probably employ him as a contractor. Even microsoft use contratcors and consultants at certain levels in there software. Quote from some training material I read some years ago " These people tend to live in big houses and are very hard to see". Made me green with envy. I am a real time specialist - different field and much lower wages.

    As to dcraw this is the bulk of the source code. Undertaken from scratch with no knowledges would be a fair undertaking and would probably involve vast expense for your clients and many mistakes even though there isn't that much of it really. It runs incredibly quickly too.

    http://www.cybercom.net/~dcoffin/dcraw/dcraw.c

    -
    John,

    You may be interested to learn that I popped Dave Coffin an eMail to ask if in fact Adobe use dcraw in any of their products - here is a copy/paste of his reply ...

    Hi Colin,

    Other Raw Photo Decoders

    Dcraw has made it far easier for developers to support a wide range of
    digital cameras in their applications. They can call dcraw from a
    graphical interface, paste pieces of dcraw.c into their code, or just
    use dcraw.c as the documentation that camera makers refuse to provide:

    In Adobe's case, they "use dcraw.c as the documentation
    that camera makers refuse to provide". Most others use the
    dcraw source code with their own customizations.


    I'm not sure I completely understand what he means by that, but since he makes the distinction between "calling dcraw from a graphical interface", "pasting pieces into their own code" and "using it as documentation" - and then specifically states that Adobe are doing only the latter, then I would take it from that that Adobe aren't using his code in their products.

  13. #13

    Join Date
    Nov 2009
    Location
    Provence, France
    Posts
    906
    Real Name
    Remco

    Re: Raw-Tiff-Demosaic-Gamma

    Some code in dcraw uses information that the camera producer didn't document, so that had to be reverse engineered. There are for instance data blocks needed for decompression (huffman trees if I understood those bits). Then there's information on how to recognise different file formats (extension only isn't enough, apparently). Then there's some things about decryption of encrypted data as well.

    In short, enough information that is NOT provided by the camera makers to be interesting for other programmers (and taking that from a known working program is always easier than figuring it out for yourself).
    And as dcraw allows you to use a large part of the source code without any limitations (see the license in the dcraw.c file), who knows what the different users actually use (as opposed to what they say they use...)

    Remco

  14. #14
    ajohnw's Avatar
    Join Date
    Aug 2012
    Location
    S, B'ham UK
    Posts
    3,262
    Real Name
    John

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by revi View Post
    Some code in dcraw uses information that the camera producer didn't document, so that had to be reverse engineered. There are for instance data blocks needed for decompression (huffman trees if I understood those bits). Then there's information on how to recognise different file formats (extension only isn't enough, apparently). Then there's some things about decryption of encrypted data as well.

    In short, enough information that is NOT provided by the camera makers to be interesting for other programmers (and taking that from a known working program is always easier than figuring it out for yourself).
    And as dcraw allows you to use a large part of the source code without any limitations (see the license in the dcraw.c file), who knows what the different users actually use (as opposed to what they say they use...)

    Remco
    Basically it's a million to 1 that they also took a close peek at all of the source. A lot of that goes on with open source software of one sort or another. Lots of software originated from Unix. Linux came along and then software from there started playing a part. More recently lots of originally Linux software is still open source but is also available for Windows and Mac. When these applications are used and sold commercially various things happen to them. Restructuring - that basically means shifting stuff around and maybe doing the same thing in a different fashion. It's a trivial activity for a competent software engineer as the hard work has been done. Call the labels something else - software basically runs from one named section of a program to another. All this does is make compilations look look a little different at a debug level. And last but not least do nothing at all to it. Just sell it. There are obviously mixes of these as well.

    I would strongly suspect that Adobe fall weakly into the 1st category at least initially. As to skills base there are various levels of competence in the software world. I shudder to think what would happen if a C++ application programmer went to work on dcraw from scratch or as it is. People don't seem to realise that "general" software mostly makes use of functions built into the operating system and little else. Actually I take my hat off to Mr Coffin. He has provided a relatively clean easy to read piece of software that also happens to run fairly quickly.

    -
    Last edited by Colin Southern; 30th October 2012 at 05:43 AM.

  15. #15
    xpatUSA's Avatar
    Join Date
    Feb 2012
    Location
    Texas
    Posts
    2,946
    Real Name
    Ted

    Re: Raw-Tiff-Demosaic-Gamma

    Quote Originally Posted by ajohnw View Post
    Actually I take my hat off to Mr Coffin.
    Me too. Especially his support for the RAW files from my early Sigma DSLR (SD10). With his stand-alone command-line version I finally solved a puzzle with the -D option (Document Mode, no interpolation) apart from all the other arcane options available. Well done, DC!

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •