File formats and image processing - Confusion and dismay over Nikon Raw images

We make no apologies for continuing to comment on the continuing concerns over image file formats. The manner in which images are processed and saved is crucial to digital photography. We have always maintained that Jpeg is currently the best and safest file system with which to work due to it's almost universal adoption and worldwide use. It may not give the best absolute image quality with each and every image taken, but as we commented above there are problems and drawbacks with all file formats and sometimes you have to sit back and look at issues from a different perspective to fully understand all the implications involved.

No sooner had we written our comments on Jpeg v Raw than a furore erupted over comments made by Adobe Photoshop originator Thomas Knoll that indicated Nikon was trying to control how images generated in it's D2x/D2h Raw format, called NEF, were processed, by encrypting and controlling AWB. It later emerged that several companies had been doing similar things with their Raw files for some time. Adobe were concerned not that they could not decipher the encryption, so that Photoshop users could read and process the files using it's own Raw processing software, Adobe Camera Raw - ACR, rather than with Nikon's, but that by doing so they would be infringing copyright patents. To appreciate what is currently starting to happen with image file formats it's necessary to go back and take a look at film photography. 

With film you have the image taking process, where camera makers vie with each other to produce cameras that give the best results, and film companies doing likewise with film. Whilst there are obvious overlaps, the two are distinct and separate parts. And it is a question of mix and match. Find the best camera you can and that you like, and use it with the best film you can find. If a better film comes along you can change to it, no problem, it doesn't make your current camera obsolete. All 35mm cameras - the vast majority of cameras used by consumers - whether compact or SLR, use the same film format, and the different chemical formulas used to develop the different types of film, Colour, Slide or Black&White are also the same. Each film has it's own variations, that's the whole point, but there the matter ends.

With digital it's different. The process is all in one, and it's integral to the camera. It's a job lot. The are no longer common factors between different camera makers. Image sensor size and type differs and so does image processing. Even with a single camera maker different cameras use different sensors and different processing. And this is the nub of the problem. If a camera maker can produce a better sensor, or generate computer algorithms that process the information so that superior images result in comparison to another maker, they want to keep that advantage so consumers will buy their cameras in preference to others. So they patent all they can. Computer software has long been subject to patenting. If a camera maker is going to invest in research and development of image processing algorithms that are better than someone else's they want to keep it to themselves, restrict it's use and reap the benefit.

This is what Nikon and the other companies are now starting to do.  Many photographers are worried, and rightly so, that this will spread. Whilst the camera's continue to offer other file formats it's down to personal choice as to which file system to use, and which software to process images with. But if files formats become individual camera maker dependant types, unreadable with any other software but the camera makers, it will lead eventually to big problems. 

As we have said computer software is all pretty much patented. But it doesn't affect the computer user too much because they can generally choose which software to use. Digital camera users can't. The software, called in this case firmware because it's part of the equipment and allows the camera to operate, is generated and provided - installed - by the camera maker, in just the same way that other types of computer hardware, disc drives, hard drives, all manner of peripherals, have firmware installed to allow them to work.

One of the biggest worries for anyone involved in digital imaging is preserving images for future access in a readable format and preventing images being lost due to un-readable image files. A common file system is crucial. Jpeg is very common in that sense. Almost any product you buy can read Jpeg files, that's the beauty of it.

No photographer wants to save files in a format that in the future may not be readable because the software to read them no longer exists. There are often queries in computer magazines regarding saved files that are in a format that no current software can read. Sounds daft? It happens when someone upgrades to a new PC which cannot run the software used previously because it's standards are obsolete and incompatible, and new software no longer supports the file format because it's obsolete.

Our advise is to stick to Jpeg for taking and storing images - unless you fancy becoming cannon fodder is a rights war, which is what this appears to be developing into.

Extract from: Digital Camera Dynamic Range. November 2006 

   -  All our results indicate that practical DSLR dynamic range using the Jpeg file format, the range that matters most to the average users, though it might be slightly wider with more current DSLR's ( the Canon 350D has about ½ Ev stop more range than the *ist-D which is what we had concluded long before we undertook the tests), is nowhere near some of the rates that have been reported, and still hovers around the 6-7Ev range. It has also emerged that using Raw file format image capture rather than Jpeg doesn't in itself bring a wider range. Some raw converters can actually produce images with narrower ranges when used with default settings and most don't show any difference over Jpegs at all. You have to indulge in sometimes drastic individual image settings for each and every Raw shot to wring even the slightest improvement in dynamic range out of a Raw file - and it doesn't always work. And the images are noisier because the normal noise reductions algorithms applied to Jpegs aren't to Raw. This is contrary to the accepted view - that we have always argued against - that Raw files are inherently superior to Jpegs. -

March/April 2007

Raw file capture - more hype than real advantage now?

Recently we have been carrying out some trials of the latest versions of several image editors. One aspect that was of particular interest was how they managed to handle the digital camera Raw file formats. It's well known that the results that you can get with Raw conversion software seems to vary greatly, with some working much better with one camera raw format as compared with another. Adobe's ACR seems to be currently the benchmark against which all others are compared, so we thought, as Jpeg users who have always been sceptical about the supposed superiority of the Raw file formats, this would be a good chance to see what differences and benefits might be revealed, and whether shooting only in Raw file format, as many now do, actually is of real practical benefit.

For if you were to believe all you read, the comments from some digital camera users would lead you to the conclusion that only those new to digital cameras and photography in general, those with little photographic experience or knowledge, or indeed any photographic standards, use the Jpeg file format, and all real photographers, those who aspire to decent standards use the Raw formats, as nothing else is acceptable. Which is a bit of a problem for many digital camera users as most older types, digicam and DSLR, don't have this file option, it didn't exist when they were made, and nor do the vast majority of digicams sold at present. 

As well as using the Raw converters provided in Adobe Elements 5 (ACR 3.6), PhotoImpact 12 and Paint Shop Pro XI, we also used the converters provided by Pentax and Canon with their DSLR's, and just for good measure Adobe's Lightroom beta 4 Raw conversion and the VueScan scanner software, which can also read/scan and convert all Raw camera file formats. These tests have proved to be quite revealing. 

First of all we should say that to make sure that these tests really revealed what these converters could do, many of the images used were specially taken in demanding conditions, low light or very high contrast settings, with only one thing in mind, seeing what each converter could produce in terms of shadow and highlight detail as well as general image quality. Jpeg images were of course also taken at the same time for comparison. As well as using the default camera exposure settings, the one's produced in program mode using multi-point metering, images were taken both above and below the 'correct' exposure readings via exposure compensation. Usually +/- 1 & 2 stops. This applied to both Jpeg and Raw images and both test cameras. The Canon 350D was easier to use in this respect since it can take Raw+Jpeg, which the Pentax *ist-D cannot. 

All Raw conversion was initially carried out at the default settings used by each Raw converter. We know that by making manual adjustments benefits can sometimes be achieved, but when large numbers of images are involved this is just not practicable, and we wanted to judge what the practical benefits are of shooting only Raw, and then converting all the images at default settings. The time and extra work involved, the duplicate image files produced, as well as any advantages.

We felt it quite clear that Adobe currently have by far the best Raw converters of those we have tried, which produce the best results at the default settings. It's a close call whether the current ACR 3.6 or that now being developed in Lightroom are best, the results are mostly identical, although perhaps just a shade over-sharpened out of either. Some converters such as Adobe ACR are updated on a regular basis to produce the best images possible with each individual Raw file type, as camera makers are continually changing the standards used with each new camera that arrives. Which of course effectively nails the lie that has become fact by default that Raw files are just pure raw data of the sensor, which of course they are not. After Adobe the camera makers own converters seemed to work best. Paint Shop Pro conversion gives images with good detail but which are very noisy and the colours muted, whilst PhotoImpact's are lacking in detail although less noisy, but again with muted colours. Raw files processed through VueScan come out quite well. They have perhaps the lowest noise levels, but lack the detail definition of others, although they retain more visible detail in the shadows and highlights than any other image file Raw or Jpeg. With some further contrast and some sharpening they would probably be the best images of all. We should explain that they were scanned at the default settings we use for film stock, no highlight or shadow clipping, so quite low contrast, and no sharpening or colour alterations.

We found the best Raw default converted image files, those from Adobe software, either ACR or Lightroom, are in most respects, almost as good as the Jpeg files taken at the same time. And that seems the bottom line, almost, but not quite. And that's the best converted images, those from other converters being much worse on some occasions, appallingly so with some images, and none being of a level that could be described as better. It didn't matter whether images from the 350D or *ist-D were involved, the comparison was the same. Occasionally the Adobe ACR files are actually slightly better as far as image sharpness and detail definition goes, but the downside is that all the converted Raw files, from whatever source, are always much noisier, even with noise reduction enabled. Those from VueScan being the sole exception. So when printed out at normal sizes up to A4, the noise depicted offsets the extra detail and definition seen in those that manage to produce it.

Now It's true that poor image files taken in the Raw formats can sometimes be rescued by altering the processing defaults. Say when highlights have burnt out or shadow detail is not discernable. This is after all the basis under which the use of Raw file formats are promoted as better to use. When either the camera user or the camera itself have got the exposure wrong, the image involved can be altered to improve it. To recover the quality that was expected but which did not materialize, tonal quality, colour quality, detail through sharpness etc, by adjusting the WB or sensitivity or other parameter that are provided in the software. Which is why we deliberately took over and under exposed images. To see if the Raw files could actually rescue such images, and how they compared to the original or correctly exposed images.

Although we found that it's correct to say that a certain amount can be retrieved in Raw conversion software such as that produced by Adobe, in many it can't. We know because we have tried, time and again. Usually the results are considerably worse than a straight Jpeg file altered in levels to recover contrast while preserving highlights, or to reveal shadow detail, a much quicker, simpler and easier process. Most, even Adobe ACR, will try and fill in burnt out highlight detail and lost colour information with a grey colour shade when no information is available. And the results are just not worth the effort. It just looks like a selectively posterized image. Whilst revealed shadow detail just proves how noisy the image is. What is considerably worse in our opinion is the time and effort all this seems to take. 

So the basic lesson, certainly from our perspective, is not to believe all you read and hear about the superiority of the Raw file formats. There has from the start of their appearance been the assertion that they are better simply because they contain just the raw data off the sensor. Quite apart from the fact that this is not true, and cannot be so for very basic reasons, all the use of the Raw file format should enable you to do is apply post image taking, settings that would otherwise be applied as a Jpeg file is processed in-camera. And these days there are many setting changes that the individual can make to how these are applied, giving a huge range of Jpeg variation. 

Perhaps the most important lesson seems to be that if you are a competent photographer who has taken the time and trouble to get to know how to use your camera and the correct exposure modes and settings to use in various lighting conditions, then shooting Raw is an unnecessary extra burden. Takes up far more memory card space, requires post processing to use them for the simplest task, and means working with multiple files of the same image. And in most cases produces an image file that is no better, and usually worse, than a straight Jpeg. And of course unless you take all your shots on a Raw+Jpeg basis you won't know that this is the situation.

Suffice to say that we shall continue to do as we always have, shoot Jpeg. Quicker, easier, and simpler. Even bracketing Jpeg shots for insurance against disaster in awkward lighting situations is better, and still takes up less memory card space. 

----------------------------

Raw file formats - some background history and what a Raw file actually is

Much of the confusion surrounding the Raw file formats has arisen on the basis of the belief that they just contain raw sensor data. The problem is that in most cases they don't, although in some respects this is not the main issue, but rather the interpretation of what shooting in the Raw file format implies. There are those that believe that raw means literally raw in the most basic sense. That everything can be altered, which is not quite true. For example that if you take a shot at say iso1600 you can then process it at iso100 should you wish. The truth is that all Raw capture does is give you the opportunity to rescue a shot that wasn't right for some reason, providing some flexibility, by altering within a certain level the processing parameters that are applied.

Raw file formats arose for two separate and unconnected issues. They first appeared because of the need by those using the Foveon multi-layer sensors to prove that they produced superior image files to the single layer sensor 'Bayer' types. And the only way that this could be achieved was by producing the information off the sensor as the raw processed data and letting the user control the image processing parameters. So the original Sigma SD9 was the first camera to produce and use 'Raw' files. Producing Jpeg files in-camera reduced the advantage to a level that could not be seen when the Raw file format first appeared with the Foveon sensor, because cameras did not possess the processing capabilities they are now fitted with, and the simpler image processing algorithms then used could not match those which could be written for computer software conversion where hardware capabilities were not a consideration.

When other camera makers saw what Raw files converted on the computer could produce in comparison to in-camera Jpeg's they decided to follow suit, not because they were hugely superior but because they allowed users a bigger image parameter choice, and more importantly they were not Jpegs. And this last point has been a pivotal one, and the second reason Raw file formats were developed by all camera makers. To be a bargaining chip and a refuge of last resort. Because for some years now there have been a lot of lawsuits and demands for license fees over the Jpeg standard. 

Originally the Jpeg format arrived and was adopted on a large scale on the understanding that it was a 'free and open' standard. That no one owned the patents to it. Then in 2002 the Texas company Forgent Networks started claiming patent license fees from technology companies including camera makers using the Jpeg file standard, regarding the 

'672' Jpeg patent which it claims to have bought in 1997, and is part of the Jpeg standard. These legal battles have only recently now been classed as over, with Forgent dropping all remaining claims after agreeing to a final settlement reported to be around $8m. It has already won around $110m from camera makers alone. Camera makers have been pressing ahead with the development of the Raw formats as an alternative in case no final settlement could be reached. Raw files won't now disappear, but expect their further development to perhaps be scaled back. Today, with much better in-camera processing abilities the advantages of the Jpeg file standard over Raw file formats is becoming clear, not just in image quality terms but ease of use and straight to print from memory card options. What should not be forgotten is that most effort is being put into producing cameras and printers that by-pass computer use, and Raw format file use does not do that, just the opposite in fact.

Last but perhaps not least, is the knowledge of just what Raw files are, or rather what they are not. They are not a standard file format, they are what is known as 'proprietary' formats, each one being unique to each individual camera maker or software developer. So the idea that they are just the 'Raw' data coming off the sensor is not quite true, because if this was the case, then all those camera makers that use the same sensors would produce Raw image files with the same data, and they don't. Which is why each format needs Raw conversion software tailored to the information it produces. 

December 2007

The Raw file format and web forums - potential traps for the unwary 

You will have to forgive us if we make oft repeated comments about some digital camera aspects we feel are not all that they seem and hold traps for the unwary. One of the main aims of this site is to provide as much detailed information that is of help to camera users in general as possible, and particularly those that are relatively new to digital camera ownership and use, and thus have less practical experience with the medium than others.

The use of Raw files is one such subject that keeps on coming up. We keep returning to it because of issues that carry on arising with it. We last commented on this back in March/April and as with several other basic issues we intend to produce a dedicated page on it as time permits. But each time we think all the aspects likely to arise have done so a new one emerges. It hasn't this time actually, but what caught our attention was that after following countless advise from fellow users to shoot Raw, a digital camera user then found great problems with it, and particularly getting the quality they were told to expect. It was only at this stage that those that had given the advice to shoot Raw began to point out the drawbacks, all the problems and qualifications that go along with it's use. Aspects they took for granted and hadn't considered others might not be aware of.

This is the snag with web forums, as with any club or society that exists, whether it is a 'virtual' presence on the web or not. Members will offer their opinions and the benefit of their experience, but there is little or no way of knowing just how good this is. Or how complete and considered it is either. For it is one thing to give an opinion, and quite another to give advice. Generally, if you belong to any club long enough you usually find out through time and experience who the fellow members are that you can approach for impartial and honest advice. This doesn't always hold good for web forums because you don't actually know who exists as a member.

The biggest downsides the new Raw user found was that no two Raw converters, stand alone dedicated applications or those which are included with many image editing applications, produced similar results, something no one had bothered to point out. Nor the huge amount of time and effort involved just to produce a single file he could save and print, let alone a whole memory cards worth.

So lets just re-iterate a few points with regard to Raw files. No two Raw file formats are the same, even from the same camera maker, each is often tailored to a specific camera. There is no such thing as a 'standard' file type. As such no image editor/Raw file converter treats them in exactly the same way, and different results, with massive variations in some instances, can result with the same file. And that's even before you even consider moving from the default processing options used. This is totally at variance with most other 'proper' file formats which conform to a set standard, which is why there is so much confusion for the unwary. 

Another factor is not what you can do with a Raw file, which is actually very limited in the general scope of things, but what you can't do. You can't print it, crop it, re-size it, apply correction effects of any kind, say dodge and burning, or pretty much anything else. In fact all you can do with a Raw file is process it, to produce an image. Then, and only then, when you have saved it in another file format can you do anything worthwhile in respect of it

Perhaps the most interesting aspect of all this is the last posting we read that had been made. This was from a pro sports photographer, (the user involved had been trying to shoot some sports action). His opinion was simple and direct. He and his fellow sports pro's didn't bother with shooting Raw since it was found not worth the trouble, there was little benefit to be gained over Jpeg's using the lowest compression/highest quality settings, whilst shooting Jpeg was both quicker and took less card space. 

Which is the considered advice we have also given in the past. That shooting Raw is fine if you are aware of all the various aspects involved, the time and effort expended slaving over the computer, and are trying to squeeze every last drop out of your camera's image quality. But that it's a moot point as to whether any worthwhile benefit actually results over Jpeg's when a camera is used to it's full potential. Just learning about photography in general and how to use a camera to the best of it's ability is often just as beneficial

Further comments and thoughts

Okay, that's what we have written to date on the Raw file format. There is a bit of repetition of some comments for which we apologize, but in our defense they were written over a period of some two years, and in isolation from each other. Having read again all that we've written to date we can't say that there's anything that we really want to change with the benefit of hindsight. Any thoughts or comments that are now obviously incorrect. There's no new information that leads us in any way to change our minds, indeed quite the reverse. 

Just as we write this we read comments in a highly respected professional photography magazine that Raw images from the latest high resolution DSLR's, both APS-C and Full Frame, show no benefit over Jpeg capture and indeed at high ISO's are considerably worse - due mainly to noise levels - and that re-evaluation of the advice to shoot Raw needs to take place because the benefits of doing so no longer exist. And these comments come from a magazine where most contributors have considered shooting in the Raw formats as taken for granted to obtain the best image quality.