On comparisons

So the latest from Luminous Landscape is this short piece on making comparisons, in response to the minor debacle that was the comment on dynamic range comparisons. I'm kind of glad the later response went up, gives some balance and avoids me posting a rant against the original.

I agree that in print there are likely (I've not seen so can't say from experience) visible, visceral differences in output from DLSRs and Digital Medium Format (DMF). Large prints are almost certain to make that more apparent. Just as I can clearly see differences in the formats I shoot regularly once print size passes some threshold. All well and good.

Thing is, that goes no way to dispelling the myth propounded in the first piece that DMF has a larger dynamic range than DSLR. Unfortunately for LL, this is a measurable quantity and it doesn't matter what constraints you put on the range (such as the base signal to noise ratio), it is still directly measurable. And as comparisons over at DXOMark will show, there is precious little difference in modern DSLRs and DMF.

Another problem I had is that LL was using DR as a proxy for usable detail, although it is nothing of the sort. DXOMark has a separate measure for this (and the merits of that could be argued) in its tonal range measure. Again, objectively measurable. The one problem here is that when doing print comparisons, they re-base to a shrunk print - the lowest common denominator. It would be useful to see that also done for larger size (but then that would mean up-scaling lower resolutions, with attendant problems).

The biggest problem I had with the LL position is that they are using the subjective stuff, and ropey assessment of measurable values, to debunk the science as it doesn't agree with their conclusion. Unfortunately it doesn't work like that. Just because you don't agree with the measurements doesn't make them wrong. But likewise (to the chagrin of the measurebators) the numbers don't make the subjective preferences (opinions) wrong either.

Windows computing for photographers: part 3, the boot drive

This will actually be quite a lot less photography and quite a lot more general Windows computing. Non-computer nerds can probably look away now. Real experts should look away now, lest the horror of my mistakes proves too much. You have been warned...

In previous episodes I talked about memory upgrade & RAMDisk (easy) (part 1) and (part 2) a bunch of hardware purchases. This third installment will be about my adventures in boot disk land.

Life should have been so simple - install the new SSD, clone the old boot drive across, change boot sequence & then clone my application spaces. Then technology and user ignorance intervened. I made a lot of mistakes. Learn from my errors & my figuring out what I should have done.

Step 1: the failed clone

Somewhere along the way the cloning software messed up, ruining the Master Boot Record (I think). That put the computer out of action for a while as I figured out how to get things back. I forget exactly what i did but it was relatively easy after a bit of googling.

Step 2: the complete clone and failed reassignment

I tried again. this time I successfully cloned the boot drive to the new drive. Hooray! And then I got clever (ha, ha, ha) and tried to reassign the boot drive letter (C:) to the drive. Suddenly the computer wouldn't boot at all. And I managed to make te old drive "Inactive" (i.e. make it unbootable) so I couldn't even recover to the old situation. And, of course, I'd not cloned all the other logical drives off the old system before I got into this mess.

Step 3: recovering the mess

So there i was, computer a giant paperweight. Every online source effectively stopped at this point (with words along the line of "you're in real trouble if you get here") with no help on getting out of the hole. Fortunately I had my netbook to do some surfing. I also had a pile of disk drives and external enclosures for the final part of the upgrade (wait for part 4). Turns out I also had a disk copy of Norton Ghost that boots from CD. Hooray!

So I cloned all the old stuff using Ghost to one of the new drives and used the external enlosure via USB to the netbook to verify the copy.

Much searching for solutions later, I decided the only way out was to reinstall the OS on the SSD. Fortunately all my data and applications were on separate partitions and now cloned onto a different drive.

Step 4: now we're rocking

Windows install was easy, moving all the old data to the new drive was easy and I've ben rebuilding things over the past couple of weeks.

What I should have done:

Well at least started with the data transfer, then applications and finally boot drive. Instead I went in the reverse order. I should also have verified the boot clone before doing anything fancy. By working that way, I would have made myself independent of the old disks.

In conclusion:

Cloning boot drives can be quite simple, if you get a good step-by-step guide and follow it to the letter. If not, expect some (up to a lot of) fiddling. If all your stuff is on one disk in a single partition, you'll be in a world of trouble. Getting the new one to work, while the old one is in place is trickier. Finding out what to do when it fails is even harder. In the end, it might just be easier to reinstall the operating system - it's what I've done and it's going nicely.

As to the hardware: SSDs rock. Fast, quiet, low energy. Not cheap for the capacity but I'm not looking back now. Everything is faster, even with the previous improvements I'd done. Software installs take seconds, not mintues.

What would I do different?

Well, I think I should have looked to a more extensive upgrade. Maybe 2 SSDs, one small one for boot only and the larger one for the rest. The whole Windows cloning process seems designed for removing or reformating the old drive after the change.

Now I've gone through this all, my whole idea of an ideal computer set-up has changed. I can well imagine running a mirrored pair of boot disks in the future for security and speed.

In part 4, I'll cover changes to my main data storage, general comments on how it's all working and some subjective stuff on how this helps photographers (and ways to save cash while improving performance).

Why optical viewfinders won't disappear

I was going to title this along the lines of "Why the SLR will continue" but I think it's a bit wider than that. Also, this argument will throw up an anomaly that's worth noting.

In the digital world, it's all about battery life. And battery life is really important if you're away from electricity or needing to travel light. I regularly travel for 2-3 weeks when I might be 500 miles or more from electricity. Even when I'm not, I don't want to carry an array of batteries and chargers if I can help it.

Micro four-thirds is looking promising but has pretty limited battery life (in the order of 350 shots per charge). I regularly get 950 from my DLSRs. The difference on a long trip would be between 2-3 days per charge to 7-10 days. That's significant, the difference between a couple of batteries and a bag full.

As far as I can see, the big difference is the lower power required by an optical view-finder system. Without the screen/EVF to keep powered, the camera can use less power overall. The optical finder and separate metering systems of the SLRs are similar to the low power units of the film days.

There is that anomaly I mentioned: the Leica M9. Seems it only gets about 300 shots from a relatively high capacity battery. Erwin Puts even reckons as little as 100 using the 16 bit RAW. Can't understand why, as the DSLRs can achieve much more from essentially the same processing requirement.

It's an area that disappoints me in camera development - not enough focus on battery life. It seems as the capacities get bgger and cicuits more efficient that is ussed to pile in more features and ways to drain power. Would it be possible to create a digital camera with a stripped down processing system getting 2000 shots from a typical sized battery.

Media are multi

Another interesting rumination from Paul Butzi. Leads me to think - paper media and electronic media are currently poles apart, maybe at odds with one another. But yet we want convergence, or at least translation: the ability to put together electronic files easily put to paper, or paper files readily turned into 1s and 0s.

The problem seems that those writing the standards are thinking in small boxes. Most mark-up is designed for online text, images are an after-thought. Printable structure is a whole other business. Yes, I realise there are fundamental differences in paper and screen display but there can be simple way to translate between the two even to the point of double-structureed files.

Much as I enjoy learning new stuff, I'm getting fed up of having to become an expert in various technologies just to get stuff done. it's one fo those things where the answer seems tangibly close yet just out of reach. For sure technology is moving on apace but I'm never satisfied when the answer seems tantilisingly close.

Further experiments in JPEG compression

This is an expansion of the experiments on JPEG that I reported in this post. It's worth reading that before continuing here.

This time I was looking to expand the idea to the limit - just how many iterations would it take to cause the image quality to break down?

I took the same image, working with Lightroom JPEG quality 75 and continued to successively expot the full size image. At the 35th iteration I found the first artefacts - pixel blocks, about 3x3 in size, in one of the dark patches. Nothing that would show up in print. In fact, it took a while to find it at all on screen at 100%. I then continued. At 80 iterations I got bored, happy that this is well past the number of successive saves I'd ever need.

I also tried an image that had been sharpened in the first conversion from uncompressed TIFF, using Lightroom's heavy screen sharpening. After 20 iterations there was no difference, apart from the sharpening - no artefacts, halos etc. I'd expect it to go many more iterations without problem.

JPEG torture test

I also devised a torture test, creating a multi-patch image with random detail running across it. This is a graphic image better suited to GIF than JPEG. Result: noticeable degradation around the wiggly line after 10 images at quality 75, but only against pure colour (one of the white patches and the pure green bottom left). The rest was intact, including edges between patches. The file size is small, reflecting the large colour patches: 340kB for a 6MP image, much smaller than the 1.2MB for the other, 10MP image.

Torture test after 10 iterations, Lightroom quality 75
Click for 100% view

Final conclusion on using JPEG for presentation and book making: I'm sticking to lower quality levels than before, using JPEG exclusively for creating photobooks and not getting too worried about compression settings.

Experiments in JPEG compression

I wrote before about optimising jpeg compression for various output, mainly aimed at single conversion for on-screen view. Then a couple of comments on this post by Colin Jago had me thinking more about using jpeg images.

It was suggested, in reference to making photobooks, that TIFF (by inference, uncompressed images) are needed to retain image quality. I don't do that. when using Scribus I import all images as resized jpegs exported with a quality of 9 from Photoshop or 90 from Lightroom. By working with jpegs I minimise memory requirements and speed up software response. Am I losing quality in doing this?

So I've run a little experiment. I took the image above (selected for a mix of fine detail and smooth tonal areas), from its 16-bit uncopressed TIFF format and successively converted it to jpeg. that is, I converted to jpeg, took that jpeg and converted and so on. I ran 10 iterations. I tried 3 different qualities (90, 75, 50 from Lightroom), no sharpening applied in conversion. This is what I found.

The 10th quality 90 (jpeg 90-10) showed almost no discernible difference to the first jpeg (jpeg 90) which is indistinguishable from the original. That at 100% on-screen view. The one slight difference between jpeg 90-10 and jpeg 90 was slight posterization in the darkest shadows. By slight, I mean peering closely and changing my angle of view a lot. At full-screen view it is unnoticeable. It would not show in a print. Each jpeg in this series is 2.6Mb in size for a 10MP image.

jpeg 90-10 crop, click for 100%

jpeg 90 crop, click for 100%

The same story for jpeg 75-10 compared to jpeg 75. These files are 1.2MB for the same image.

For jpeg 50-10 there is definite loss of detail and a number of strange block artefacts in smooth tone areas. I had to go all the way back to jpeg 50-4 (4th in series) before it was indistinguishable from the original. The files in this series are a mere 581kB.

jpeg 50-10 crop, click for 100%. Notice the weird blocks in the smooth areas

jpeg 50 crop, click for 100%

Conclusion: it is quite feasible to use jpeg for successive output operations without losing detail. For short runs (3 or 4 successive conversions) it is feasible to use quite low quality (high compression) and not lose detail. Higher quality will support longer runs. I haven't yet run the 75 and 90 series to the point that they start to lose quality, that's a job for another day.

PDF: a layman's explanation

As Colin Jago points out, information on PDF formats and content is typically fairly obtuse. Heaven knows why.

Off I went to read the ISO standards to understand this stuff. When one digs in, there are some fairly simple explanations for this all. I shall try and elaborate.

PDF is now defined in a series of international standards and like all standards, they have to cover a wide range of uses and interpretations. Boiling standards to their essence is quite a skill, one which I get to practice frequently on a professional basis.

Essentially PDF breaks into 2 types - the generic document container and the specific print-ready file.

The general types of PDF (1.3, 1.4, 1.6, 1.7 etc) are wide-ranging descriptions of ways of publishing documents for various applications. Basically the standards define a very general container for a whole lot of data types. Some are application specific (e.g. PDF 1.6 is specifically for engineering data).

In their simplest use (and, I suppose, the one with which most people are familiar) they are just a way of formatting text documents in a way that they can be used across a variety of computer systems. In their most complex form they are multi-dimensional, multi-media applications with hyperlinks, cross-document linking, user input, and a variety of interaction capabilities all done in a way that a variety of computer systems can use them.

The standards lay down a whole lot of requirements on document structure and on the readers that present them to you.

In practice things get muddied by non-compliances, bad formatting and odd things getting embedded. Adobe help the user by providing Acrobat Reader with a pile of capability to try and sort the mess out so that you can use the content.

Then there are the PDF/X types. These are simplifications of the main PDF types specifically for printing and incorporating requirements for colour management data. There are basically 2 types, PDF/X-1 which is CMYK only and PDF/X-3 which also allows RGB colour spaces. The PDF/X standards pretty much says: text & pictures only, no fancy stuff, tell us the fonts & colourspace you're using.

The 2 big differences between the two types of PDF (normal and X-type) are

• PDF/X is a simple, flat file. No layers, hyperlinks, multimedia etc. Just text and graphics for printing on paper.
• PDF/X requires colour management data embedded so that the printer can print things properly.

There are a few niceties in PDF/X around font management and pre-determined output colourspaces that are best left to the pros with a direct working relationship with their printer.


How does this affect you, the humble photobook writer? In principle, not a great deal. If you're doing a basic photobook and writing a PDF file with embedded fonts, JPEG images with a colourspace and no fancy links, video etc anyway then there is essentially no difference between a PDF 1.3 and a PDF/X-3 file. In fact, there's little difference with PDF 1.4 either (which introduces transparency - if you don't know, you don't need to). Just write a regular PDF and it'll comply with the X-3 format. Or export as an X-3, which should take out any fancy stuff you (or your software) added inadvertently.

I hope that helps.

Movement

Fork in path, Thirlspot, Cumbria, May 2009

As Colin Griffiths comments on my last post, I enjoy the process of lugging a big camera up big hills. But it is results driven. I want movements for angle of view & focal plane control. I want fine detail and smooth tones for large prints.

For the details and tones it is (for me) prohibitively expensive to go digital.
On investigating tilt-shift lenses for 35mm it's becoming clear that the only way to properly do the movements I want is to have a large focus screen.

The more I investigate options, the more I find I have the right one.

I'm thinking of poking around with Helicon Focus, which will be cheaper than a tilt-shift lens, but it can't do some things. Subject movement is an obvious one (and I work quite a lot in windy conditions). Soft clouds is another - I like to use a front tilt to extend depth of field in the land, which helps soften up focus on the clouds. Sure I can do all that in software, and use multiple exposures.

Here's the thing, though. Using movements in the field takes no more time than running a series of careful exposures and the time I need for the final processing is a whole lot less.

Digital solutions? Here's a couple of ideas.

• Auto multi-focus. I set start and end points for focus and the camera runs a series between them.
• Digital tilt focus. Can be manual with confirm or auto. Define the points you want in focus and adjust accordingly, just like the manual process. Asymmetric tilt/swing would help a lot for manual (auto can do fancy simultaneous movements). 2 points needed for single-axis movement (tilt or swing), 3 for 2 axes (tilt and swing).

Of course, there is one problem that all this digital stuff doesn't fix: reliance on power in the field. Maybe we'll get really good solar panels that will sort that out too.

Not at the limit yet

Little Man and Skiddaw, Cumbria, May 2009
Shot on 120 roll film, 6x9 on an LF camera

Those who read the Luminous Landscape have probably seen this article by Ray Maxwell ostensibly on the applicability of Moore's Law to camera sensor development, coming to the conclusion that we're hitting the limit, especially in terms of resolution. The arguments didn't look right to me and this TOP article by Ctein neatly debunks the entire thing.

Even with my moderate grasp of the subjects of optics and digital sampling it didn't take long to confirm that the LL article was wrong and to be able to come up with my own calculations (which turned out to be very similar to Ctein's).

Maxwell omitted the Nyquist-Shannon sampling [WARNING: geeky maths link] limits which means we need pixels at most half the size of the given Airy disc or smaller to get full resolution data. Sensor arrays further reduce the pixel size required in order to sample each frequency. I reckoned we can go down to about 1/4 the given Airy disc limit. Of course Maxwell did his numbers at f/11 which suggests large pixels but optics tend to be optimised for larger apertures, with the photographer accepting the resolution/depth of field trade-off for smaller values. Even if we calculate at f/11, with the 1/4 Airy pixel size then that yields a limit of 1.5micron. For comparison a typical pocket camera 1/1.7", 12MP sensor has a pixel pitch of 2micron, so within the grasp of current technology. Compare that to the current 4-6micron for larger sensors and we could happily go to pixel counts 9-16 times those used today before hitting resolution limits.

So your 22MP 35mm full frame could stretch out to around 200-300MP and still yield noticeable resolution improvement. Another upside of extreme resolution is the ability to crop. I could see an argument for using wider angles and cropping for a lot of shots. Imagine only carrying a wide angle and mid-tele for everything. Shoot 100mm and crop the centre for a 400mm equivalent shot. Monster panoramas in one exposure.

Sufficiency is never enough anyway. The sufficiency argument has been touted since cameras hit 4MP.

And all of that ignores any future technology leaps.

[Hereby rewarding a healthy dose of scepticism and justifying my geek tag]

Li-Ion battery management

Colin Jago had a post recently in response to Panasonic's decision to restrict camera battery use to on-brand only through firmware updates. It seems all their firmware updates are getting the treatment. I realised that the opinions (mine included) seemed rather ill-informed conjecture. So off I went to investigate the whole Li-ion thing, this post is the result.

I had two mind beginning this: the cynic in me regarded the Panasonic decision as pure marketing wrapped in a dodgy safety message, the engineer in me wanted to understand the risk and whether I was actually placing myself in harm's way.

I've included some handy references at the end rather than pepper this thing with links. I read a bunch of stuff, the links provided give all the information in a handy to digest form. Of course, you will have to make up your own mind, I cannot be responsible for your actions and I'm not advising anyone to follow my lead. Caveat lector.

Side-bar: testing batteries
Scott Kirkpatrick provided this link to some testing of Olympus batteries which highlights some of the problems. What do these tests show? First that there are products out there that do not have the protections built-in that they should have. Also, that there appear to be many products using common components (partly supporting my theory of limited manufacturers).
What these tests do not show (in any way, as they didn't try) is that the OEM or high-end third party products are any better. By not dismantling the Olympus product they don't support the premise Panasonic is working under that their products are inherently better. There is also no evidence that any particular manufacturer provides a consistently reliable (or unreliable) product, these being single sample tests.

As I see it, there are 4 parts of battery care: charging, handling, storage, device design/usage

There's not a lot a user can do about the last of these, that's just the kit you use. So what about the rest?

Charging - One of the sources of risk in using Li-Ion batteries comes from the battery being over-charged. There are 2 ways this is controlled, through a charging algorithm in the charger which limits the voltage and current during the charge cycle and shuts down the charger when it's done. The second part is an over-voltage protection circuit in the battery should the charger not provide the protection or fail. Both need to fail or be absent to present a failure mechanism.
Handling - carrying, inserting etc. Most batteries have mechanical control to prevent wrong insertion (i.e. the shape of the battery and compartment must match with only one orientation allowed). In order to prevent dangerous failures of the battery out of the equipment, it should be prevented from short-circuit (which seems to present more of a risk than older types due to the internal chemistry) and protection from overheating. Again, batteries should have protection circuits for both of these problems. The US Department of Transport (DoT) rules on carrying batteries in luggage are aimed at minimising the risk of short-circuit by enforcing a carrying method that specifically stops it happening.
Storage - Similar to handling, batteries should be kept from overheating and short-circuit. There is another aspect and that is battery life. The life of Li-Ion batteries is greatly extended by storing them at less than full charge (40% seems typical advice) and at lower temperatures (e.g. refrigerated but not below 0degC).

Low-quality, no-name batteries are more likely to have poor protection circuits which means you are relying more on the charger and device to prevent problems, which increases the risk by increasing the probability of an event happening. The consequences are the same, however.

Side-bar: Panasonic and the Law of Unintended Consequences
I don't know exactly how Panasonic enforces the restriction but i presume it is some sort of electronic tag in the control circuit. Maybe they'll license it to respected manufacturers, maybe not. even if they don't, I expect a bunch of unscrupulous companies to clone their batteries. Likely the sorts of companies that don't include proper protection in their products today and spoof the exterior packaging too. If the technology isn't licensed, then the chance of poor third party products being used goes up as there aren't the reputable ones around. So the problem doesn't go away.

For Li-ion batteries they should have control circuits with over-voltage protection, over-heat protection and short-circuit protection to help minimise the risks from poor handling or usage. It does not eliminate all the risks. But then that is not unique to this particular power source (remember the old, leaky zinc-carbon batteries?).

There is an aspect that I've not talked about, and that is internal failure of the battery. The large laptop battery recall a couple of years ago highlighted this. While the chargers, devices and batteries all had the correct protection circuits all failures came from internal manufacturing defects that were not protected by the circuits. The actual number of failures was low, maybe partly determined by usage pattern as inherent risk. That's easily the biggest case of Li-ion battery dangerous failure and it had little (if anything) to do with usage.

My conclusions: It appears that Panasonic aren't guaranteeing a camera's power usage control, by implicitly requiring the protection circuits in he battery itself. They can't guarantee which charger is used, regardless of battery used, so I presume expect their batteries to provide the protection. The actual chance of failure, regardless of battery type, seems very low indeed. So the decision does not, to me, represent an appropriate response to the risk, they could just as easily have issued an indemnification as part of their warranty. Then there is Law of Unintended Consequences (see sidebar). So, to my mind, this is pure marketing wrapped in a thin safety veneer.

Risks come from: poor charging, easily controlled by using quality product. Short circuit controlled by handling regime (US DoT response is a good, risk-based approach in this regard), poor device control and there we're in the hands of the manufacturer.

My regime: I buy branded non-OEM batteries. They are cheaper but should still be good quality. I avoid the low-quality, bargain priced units. I carry and store them in a protective case and will like now store them in my refrigerator (alongside all that film).

Some links:
Good place to start is (as ever) Wikipedia.
There is a lot of information at Battery University, not just on Li-ion. Specifically they have information on Li technologies, usage and safety. There is also this nice article on Li-ion battery chargers, which explains the best how they work.

More on jpeg settings

Walcott beach, Norfolk, May 2009
From Lightroom, jpeg quality 50, 1200x800
click for full size

Ed Richards had comments on my suggestions on jpeg over at Paul Butzi's. So, as is my wont, I went off to test his points (and mine for that matter).

A few points to start: if one is creating jpeg images for displaying on a screen, then a lot of the original data is thrown away anyway. For a typical 10MP original, going to 1200x800 (I'll work with Ed's big image assumption) throws away about 90% of the original pixels. If you're an LF photog, it's going to be way more than that. So we can forget about on-screen versions ever getting to the fine detail and nuance of a really nice print from a really nice image.
The next idea is that higher quality settings really have an impact on the on-screen viewing quality. If I want to show the very best, I need the higher quality. But how true is that?

I ran a bunch of tests on a series of images with different jpeg quality settings. For this I used Lightroom but any jpeg generator would yield similar results. I ran film & digital originals. I tried detailed & mixed images (with some expanses of limited detail). I tried jpeg quality settings from 10 to 95. I visually compared the results versus quality setting and file size.

Lightroom jpeg quality 10, 1200x800

What did I find? Obviously, at low settings there were problems. Lots of artefacts around edges, detail blurring, pixelation in large, continuous areas. At quality of 30-40 most problems were gone. Some fine details started to go. At 50 most images were indistinguishable from the higher settings. The few differences needed a careful look and hopping between versions. Going from 70-95 showed no improvement on any image.
How about file sizes? At 1200x800, quality of 50 gave about 220kB per image. At 70 that was 350-400kB and at 90 all the way up to 1.5MB. Yet no visible difference between them (and I was looking carefully on a decent, calibrated monitor). 1000x667 images were proportionally (by area) smaller.

Of the hundreds of images I've posted, the average file size is around 250kB and I can't recall seeing compression artefacts or loss of detail in any of them at web sizes.

Like I say, for good, on-screen display, target a file size of around 150kB for 1000x800, up to 200kB-ish for 1200x800. The reason I've been suggesting 1000x800 is that's a 10"x8" at 100ppi, which is a decent size for a book image, working on the basis that we're talking embedded jpegs in a pdf.

Gorse, Cumbria, May 2009
Lightroom jpeg quality 50, 1200x800

Ed also made a point that he wants the highest quality to wow the guys with the huge, high quality monitors. Thus he ends up with a big pdf. Personally, I want a bunch of people to download my SoFoBoMo book, so I'd rather a smaller file size to encourage them to do so. And based on the above tests, I reckon a 5MB book file would be visually indistinguishable from a 15MB one anyway, on any screen.

A photographer's practical guide to jpeg

Stemming from the question on the last post about sizes of pdf books, I thought it was worth writing about JPEG images: sizes, resolutions, compression etc. This is based on a little technical reading and a lot of practical experimentation. Technically, I may be a little off in some areas but this will be good enough for the masses.

The aim of a JPEG image is to preserve as much detail as possible while compressing the file size as much as possible. By setting the quality/compression (these 2 are the reverse of one another - high compression gives low quality) value you are determining the trade-off between quality and file size. However,there is a general misconception as to how this works.

For highly detailed images (think lots of tree branches and small leaves) there is automatically less compression as the JPEG tries to keep the detail. If your image is highly detailed then you can actually use lower quality setting and still get an excellent image. In the other direction, large expanses of nothing compress very well. If you shoot a picture of a white wall even the highest quality setting will give a small file. JPEG is clever like that. The tricky stuff comes when there is a mixture of detail and even tones (e.g. trees against a blue sky). The even part compresses well, the detail is kept in the main but at the edges JPEG gets confused, throwing up those nasty artefacts. This is where judicious use of the quality level is needed. I'll come onto advice on values later.

The next part is the resolution or pixel size of an image. If you are printing, you'll want high quality files with lots of pixels. Typically 240-300ppi for the given print size. That means the total pixel size will be the paper size multiplied by 240 or 300 (e.g. if I print a 10"x8" at 300ppi, that'll be 3000x2400 pixels). Nice and easy.
The tricky part is on-screen display. To display on the majority of monitors, you only need an image about 1000pixels wide or 600 pixels tall. Maybe a bit more to cover larger displays. If your software works as a size & resolution then the size multiplied by resolution shouldn't be more than about 1000x600pixels (e.g. if I have a 10"x8" page for on-screen display, I need a resolution of only 100ppi to give 1000x800pixels). This is important for pdf generation, as pdf is designed to work in physical size and resolution. To cover most monitors today, you don't need more than 100ppi resolution (nor page size more than about 12"x8").

So what about recommendations for compression values? JPEG generation generally has one of three ways of setting quality: a high/medium/low scale; a JPEG compression factor (typically a number up to 100) or another numerical scle of some sort (e.g. Photoshop's 1-12).

For printing, go high. Photoshop 9 or 10 (11 and 12 really are pointless and don't practically give better results), JPEG factor 90 or 95. NOTE: JPEG compression factors are NOT percentages. the maximum value (from the JPEG standard) is 95 - any scale that goes to 100 is distracting you.

For on-screen display, you only need a medium level. Photoshop 7 or 8 (sometimes lower but go careful); JPEG factor 70-75. This will yield a files size about half that of the higher qualities and still give excellent on-screen display. Use the lower values unless there is a big mix of detail and even tones.

A final note on setting resolution versus compression, especially in pdf generation. The biggest win in file size (small being good) is a lower resolution. 300ppi has 9 times the pixels of 100ppi. To get that kind of compression with quality you need to go down to about 20-25 quality factor, which is barely recognisable. You could probably go down to 90ppi and give good display quality even on large monitors (a factor 11 smaller than 300ppi).

As for pdf books for the web (aimed squarely at the SoFoBoMo crowd) - aim for file size that gives around 50-60kB per page or 100-150kB per image overall (non-image pages generally take very little space in pdf). If you've gt 40 images in 55 pages (typical SoFoBoMo fare), that's 2.7-5.8MB total filesize.

Sometimes things are as they should be

Anyone who regularly reads my more technical posts will realise I'm constantly frustrated/annoyed by rubbish user interfaces. I like software and hardware to be easy to use, yet powerful in function. I don't like the design getting between me and results.

On this thought but unrelated to photography, I bought a Sonos music system this weekend. A much needed addition to my music system since my multi-CD player packed up and my desire to listen to decent radio. That's not the point here.

The point is, this stuff works. Just like it should. No fuss. The design, interface and set-up is super easy. It is so well designed that it makes Apple products look fussy and complicated. There are always a few minor niggles but they are just that. As a music system it's great and it is also making me rethink the way I do my home WiFi network. 3 rooms & a pair of speakers took 20 minutes to set-up. Compare that to weeks of fiddling and twiddling with my simple point-to-point WiFi to get decent performance.

Simply put, I wish more products were like this. Taking out the hassles. Getting to a usable set-up quickly and easily. A design stemming from the simple notion of "what does a regular user want this thing to do?"

On the virtual tripod

Wall art and lamp, London, March 2009

UPDATE 06/04/09: Added the missing figure

You may have seen already the new image stabilising testing at Imaging Resource (posted at TOP yesterday). Alongside the reviews is their white paper on testing methodology. Being of the geeky persuasion in these things, I read the whole thing. I've noticed some very interesting things, besides the straight results that I wanted to air.

Overall, I think it is a good effort. They've fairly well achieved the goal of repeatable, comparable results. Highlighting the statistical nature of such things is pretty important - no one will get perfectly sharp images every time when hand holding, IS or no. Being able to compensate for the human factor, in a manner that has meaning for real results is quite important in this context.

The second big thing to take away is the importance of focus in achieving good photographic results. They talk about it at length. The testing method completely eliminates focus problems but for real shooting, that will be an issue. reading through the rest of the white paper, it's pretty clear that photographer technique is (as to be expected) as important as any of the technical assistance, be that in AF or IS. We would all do well to keep that in mind.

The big thing that came to my mind, however, was rather technical and concerns type of motion. Something briefly mentioned but not addressed in the testing protocol. Let me explain.

For the purposes of a discussion on IS, there are three directions of movement that we are interested in: vertical translation, horizontal translation (left-right) and rotation about the lens axis. (I realise that vertical and horizontal motion isn't pure translation but for the purposes of this discussion, and practically, it can be approximated to such. It is also a little easier to conceptualise than rotation about the other axes.) The Imaging Resource results seem to lump the translational results and don't mention rotation at all.

Why is it important to distinguish? It goes to how stabilisation systems work, how photographers move and how one relates to the other. I reckon there are 4 major movement effects going on that cause blur: high frequency "jitter" vertically or horizontally due to holding the camera, low frequency sway due to how one stands (try standing perfectly still without any kind of movement), and last some rotation of the camera as the shutter is released due to the force applied to the button.
Mechanical stabilisation systems are really designed to only address only the first two. And with those, the relative effect of each will be determined by how steady the photographer is in general and how the camera is held. The swaying effect is unlikely to cause problems except for long exposures.
But what about rotation? How important is it and can anything be done about it? As to importance, I would rate it pretty highly given what I see in photographs. When friends ask me about blur in their photos, I reckon the biggest issue is rotation. It's fairly easy to spot: look across the frame and you'll see vertical blur that varies markedly from left to right. As stabilisation is normally designed to work vertically or horizontally, rotation won't get corrected. For lens-based systems it is impossible to correct. But for in-body stabilisers it could be possible. See the figures below. The first shows the sensor and its possible range of motion. The second and third show vertical and horizontal shifts, the fourth shows a possible rotational shift.
And in the great debate of which system is better, this isn't ever mentioned: in-body stabilisation has the potential to cure a mode of movement that in-lens cannot.

Windows computing: learning part II

So I've got a long way down the road to upgrading my computer. As ever, as I learn more, the more find I don't know and the more my original ideas have to change. I've also had a bit of assistance in understanding how some of this works.

Here's where I'm at:

The most important thing I'm learning (and this will be useful for those buying a new system) is that by buying a well specified machine first off, from a good source, I have a whole load of very good system components. The machine is over 3 years old but most stuff is suitable for quality upgrades with current components. In the past, I've found technology has passed my system components by, limiting my upgrade options. Not now. [For the nerdy: I find my motherboard is fully PCI-Express & SATA 3.0 compliant, with plenty of expansion space and will take DDR2 667 to 8GB. That's far more capability than I was expecting.]

For the first time in my PC buying life, I think standards are sufficiently stable that I can reasonably expect current upgrades to be compatible with any box I put together in 2 or 3 years' time. In the past I bought what I thought were future-proof systems, only for all the standards to move. This also falls into line with my theory that for most uses, computing power is converging on sufficiency.

I'm going to move my main working storage out of the box but not Ethernet, as I was planning. Turns out there's this external SATA (eSATA) spec. Pretty obvious in hindsight but I wasn't aware of it before. (I don't keep up to date with all technical stuff - prefer to update my knowledge as needed.) Looks easy enough to install a controller and external box. This will make future system upgrades easier - I can just move the storage over to a new configuration, or just do an intenals upgrade on the current box. [Nerdy bit: I'll be running 2 eSATA boxes each as RAID0, with the PCI controller running RAID1 between them. That way I get fast speed and redundancy against drive and controller failure.]

On top of all that, a memory upgrade and an SSD to replace my boot drive. Seems like a lot, but I reckon I can do the whole thing for about $600 (controller, 2 drive bays, 4 drives @500GB, memory & SSD), which is what I've typically paid for single drive upgrades in the past. (I remember doing a storage upgrade to 500MB for about that money years ago - at the time that storage had just dropped to under £1 per MB. Now it's more like £0.10 per GB.)

The fully programmable camera

After my latest ruminations on camera controls, I think the future will be in programmable cameras. First one to the line gets the sales (if not the girls).

What am I thinking of? Well - a camera with a bunch of buttons and a bunch of functions and the user gets to chose what button does what. And further, open firmware so the capabilities can be programmed. Like my card-mode adjustments. Maybe we'd just get access to the sequence and card write commands and get left to do our own thing. Firmware "hacks" would become the norm. A whole new industry. Great for nerdy tinkerers like me. Probably means set-up from a computer (or maybe a handheld device) but most of this stuff doesn't get changed very often, anyway.

Of course, there'd be the standard out of box set up like today, for those who don't want to get into all that.

I can see this helping camera companies, too. They'd get to see more of the ways people use their cameras, which would lead to more innovation. Development unfettered by common/current notions of the camera. Focus groups don't cut it - IME, ask people about what they'd like to see and the current product is always the reference. You only get incremental improvement ideas. Innovation comes from seeing what creative people do with what they've got, leading to more fresh ideas. Win-win.

Camera memory card modes

triggered by seeing the specs on the new Olympus E620 camera (which has 2 card slots, including the stupid xD format) I had some thoughts about how cameras use memory cards. This led me to the following ideas:

• Have all cameras take 2 cards, either 2 CF or 2 SD (not mixed - hear me Canon)
• Combined with the 2 card slots, have 2 buffers. Not sure how feasible this bit is, or where the buffer limits lie. essentially I want to get more shots in a burst.

Have some flexible card writing modes. At present we have both together, or sequentially or RAW to one and jpeg to the other. That's all fine. Here are a couple of others that I'd like:

• A-card priority. That is, if there is a card in the A slot with space, write to that first. Even if I've written some files to the B card. This means the B card is a back-up when I'm shooting lots and need to wait a bit to change cards.
• Alternate write. Specifically aimed at rapid fire mode: writing to the cards alternately. Combined with 2 buffers, and with current read/write speeds I bet even at high frame rates buffer would be almost unlimited. This would also serve a good back-up for card failure with less space taken than complete duplicates.
• I'd like to be able to combine them - that is select card write mode based on shooting mode. Use A-priority for single shot and alternate writing for rapid fire. To be really clever, the mode switch would be based on buffer status: empty buffer use A-priority, buffer (part) full, use alternate shooting.

Funnily enough, all of this comes from my experiences on safari. Often I would nicely fill a card or the buffer just at peak action. Even the 5s or so it takes to switch a card can be too long. Waiting for the buffer to clear can seem an eternity. Sports & PJ shooters would also benefit, I reckon.

Lightroom tip: selective masking

Ears up, Tanzania, January 2009

Lightroom 2.2, Windows XP
Read my tips intro if you're new.

Why is the brush tool so slow?

I've been doing quite a bit of brush work with Lightroom recently, especially playing with selective desaturation. I was getting frustrated by system slowdown and hang-ups so I went off to find out what's going on. what follows are some of the details, skip down to the Advice section for the what-to-do part.

In general, for small curves, limited regions and editing only a few images with the brush in a session, Lightroom 2.2 is very responsive. If, however, I start running large areas, many images in sessions or lots of changes, problems jump up.

How does Lightroom create the mask?

This is the central part of the problem, I think. When a mask is created, each stroke of the brush is saved as a separate multi-point curve. you can check this by saving the metadata to and .xmp file and opening it. the Mask has the form:

[crs:PaintBasedCorrections]
[rdf:Seq]
[rdf:li rdf:parseType="Resource"]
[crs:What]Correction][/crs:What]
[crs:CorrectionAmount]1.000000[/crs:CorrectionAmount]
[crs:CorrectionActive]true[/crs:CorrectionActive]
[crs:LocalExposure]0.000000[/crs:LocalExposure]
[crs:LocalSaturation]0.600000[/crs:LocalSaturation]
[crs:LocalContrast]0.000000[/crs:LocalContrast]
[crs:LocalClarity]0.200000[/crs:LocalClarity]
[crs:LocalSharpness]0.000000[/crs:LocalSharpness]
[crs:LocalBrightness]0.000000[/crs:LocalBrightness]
[crs:LocalToningHue]356.000000[/crs:LocalToningHue]
[crs:LocalToningSaturation]0.000000[/crs:LocalToningSaturation]
[crs:CorrectionMasks]
[rdf:Seq]
[rdf:li rdf:parseType="Resource"]
[crs:WhatMask/Paint][/crs:What]
[crs:MaskValue]0.900000][/crs:MaskValue]
[crs:Radius]0.023269[/crs:Radius]
[crs:Flow]1.000000[/crs:Flow]
[crs:CenterWeight]0.000000[/crs:CenterWeight]
[crs:Dabs]
[rdf:Seq]
[rdf:li]d 0.321572 0.469314[/rdf:li]
[rdf:li]d 0.326572 0.462008[/rdf:li]
[/rdf:Seq]
[/crs:Dabs]
[/rdf:li]
[rdf:li rdf:parseType="Resource"]
[crs:WhatMask/Paint][/crs:What]
[crs:MaskValue]0.900000[/crs:MaskValue]
[crs:Radius]0.023269[/crs:Radius]
[crs:Flow]1.000000[/crs:Flow]
[crs:CenterWeigh]t0.000000[/crs:CenterWeight]
[crs:Dabs]
[rdf:Seq]
[rdf:li]M 0.891740 0.296029[/rdf:li]
[rdf:li]M 0.894016 0.305266[/rdf:li]
[/rdf:Seq]
[/crs:Dabs]
[/rdf:li]
[/rdf:Seq]
[/crs:CorrectionMasks]
[/rdf:li]
[/rdf:Seq]
[/crs:PaintBasedCorrections]

the bit in bold shows two sets of points for two strokes but for a single region. Here I've replaced all the xmp <> with [] so it doesn't get screwed up by Blogger.

If you create a large mask (as I did for the whole leopard in the shot above) then you can end up with many lines, each with thousands of points. If you really want to get anal, these can be edited down with a text editor. Easily half the points can be removed (alternate points). tedious, and not recommended.

The more you edit, the more points are generated. If you have to go back over sections, you end up with lots of overlapping lines, and repeated work being done. I think, by experiment, that Lightroom is set up to interpret each line independently, which then goes to defining the area for applying the adjustments for a give brush mask. If you've got thousands of points, that's going to take time. So I suggest using some care, and good painting technique to improve things.

Advice

So here is my advice, particularly for large areas, in this order:

1. Use the mask overlay to show the mask as you paint (key O)
2. Start with a large brush and smooth continuous strokes for large areas.
3. Use the lowest flow you can get away with (cuts the number of points generated). Faster brush work needs a higher flow.
3. Avoid reworking areas
4. Zoom in and use a smaller brush for details and edges (generally with a lower flow, and slower brush strokes).

That should help cut down the number of points generated, and the number of line segments, which should maximise performance.

Lightroom tip: advanced custom curves

English Wine & Beer Shop, India, August 2008

Lightroom 2.0, Windows XP
Read my tips intro if you're new.

How do I create my own curve profiles?

I previously posted on setting a Lightroom preset to do a basic tone curve inversion. You'll need to understand that, before reading on. I then wanted to go further, introducing my own custom curves, especially for my 35mm black and white scans.

I always scan my black and white as a positive and then invert in software. For a good explanation why you'd ever want to do this, read the post Matt Alofs wrote on the subject. When a negative is scanned as positive it usually compresses all the tones into the lower half of the spectrum, so on inversion it all ends up at the top. For me, the original positive scan rarely goes to more than 128 on a 255 scale (of course I'm scanning to 16 bit output files, not 8 bit).

After inversion, the first thing I want to do is apply a levels adjustment to bring the shadows back down into the lower tones. That meant a trip to Photoshop after importing into Lightroom to get a levels adjustment. I can, in fact, combine the 2 into one Lightroom preset. Here's how. The key is to set the "black point" of the inversion to a different point, close to the maximum scan value. So, instead of my black point origin being at 255,0, I want it at 130,0 so that the maximum scan value of 128 is just above the black point (so I can tweak later).

Here's what the preset file looks like with the change.

s = {
id = "498D92B2-EB89-4054-AAA2-46D3207AB652",
internalName = "Tone test",
title = "Invert 2",
type = "Develop",
value = {
settings = {
ConvertToGrayscale = false,
EnableGrayscaleMix = true,
ParametricDarks = 0,
ParametricHighlightSplit = 75,
ParametricHighlights = 0,
ParametricLights = 0,
ParametricMidtoneSplit = 50,
ParametricShadowSplit = 25,
ParametricShadows = 0,
ToneCurve = {
0,
255,
130,
0,
},
ToneCurveName = "Linear",
},
uuid = "8422BC3B-D8AA-4981-9F69-C350D56D935E",
},
version = 0,
}

I can now import into Lightroom applying the preset to the entire batch and get pretty good results from the off. This minimises the amount of work I need to do in other software. Of course, results may vary with film, scanner and exposure technique, so you might need to tweak the values for you own set-up.

If you're not good at reading negatives (and thus scan everything to check the best ones) this should speed things up considerably for you.

But there's more. What if I want a custom conversion curve to cover standard contrast or gamma adjustments? Well that can also be done (for positive or negative), by including extra points into the ToneCurve section of the preference. Here's a (rather stupid) example:

s = { id = "498D92B2-EB89-4054-AAA2-46D3207AB652", internalName = "Tone test", title = "Curve weird", type = "Develop", value = { settings = { ConvertToGrayscale = false, EnableGrayscaleMix = true, ParametricDarks = 0, ParametricHighlightSplit = 0, ParametricHighlights = 0, ParametricLights = 0, ParametricMidtoneSplit = 0, ParametricShadowSplit = 0, ParametricShadows = 0, ToneCurve = { 0, 0, 32, 130, 65, 100, 97, 60, 255, 255, }, ToneCurveName = "Linear", }, uuid = "8422BC3B-D8AA-4981-9F69-C350D56D935E", }, version = 0, }

The same shot as above with the strange curve applied

In fact, Lightroom curves can take any number of arbitrary x,y points to define the curve, you just have to open a text editor to get at the function.

So now, if you can translate multiple curve effects into a single curve they can be applied in Lightroom. A bit fiddly, but possible. Once the technique is known, the possibilities are endless.

Rather begs the question, if this ability exists in the software, why not expose it in the GUI?

Lightroom tip: invert tone curve

Lightroom 2.0, Windows XP.
read my tips intro first.

How do I invert the tone curve?

I've seen a few utilities available online and a lot of questions on how to, but nowhere that showed you how. So here it is. I'm assuming you know how to create a development preset.

First you need to know a bit more about Lightroom presets. They work as formatted text files, which makes them very editable. In Windows XP they're stored at: C:\Documents and Settings\%username%\Application Data\Adobe\Lightroom regardless of software install directory (for example, I've got Lightroom installed on the G:\ drive). For this you'll need to get into the \Develop Presets\User Presets sub-folder.

But first, it's easiest to create a blank tone curve and save as preset, with a name you can remember (easier than editing from scratch). If, for eample, you created a preset called invert, go to the prest directory and open invert.lrtemplate. The text should look like this:

s = {
id = "498D92B2-EB89-4054-AAA2-46D3207AB652",
internalName = "Tone test",
title = "Invert",
type = "Develop",
value = {
settings = {
ConvertToGrayscale = false,
EnableGrayscaleMix = true,
ParametricDarks = 0,
ParametricHighlightSplit = 75,
ParametricHighlights = 0,
ParametricLights = 0,
ParametricMidtoneSplit = 50,
ParametricShadowSplit = 25,
ParametricShadows = 0,
ToneCurve = {
0,
255,
250,
0,
},
ToneCurveName = "Linear",
},
uuid = "8422BC3B-D8AA-4981-9F69-C350D56D935E",
},
version = 0,
}

Note the line title, which is what shows up in the presets list in Lightroom. The important bit here, however, is the bit labelled ToneCurve. This lists the end points for the curve. We're going to reverse them from {0,0:255,255} to {0,255:255,0} thus:

ToneCurve = {
0,
255,
255,
0,
},

There it is, a basic straight line inversion curve for Adobe Lightroom. I actually do something a bit different, changing the black point instead, thus:

ToneCurve = {
0,
255,
250,
0,
},

I could also change the white point by setting the first pair to 0,250.

I'm going to come on to a fancier use of this in another post.