The job of Auto Focus

With all the discussion going on in web land about Rob Galbraith's analysis of Canon 1D III AF performance, I have been missing one important thing: objective analysis of whether this is expected behaviour or not, and how it relates to the tolerance of the system. I've mentioned this before. This is the engineer in me wanting to analyse the thing and see if it's doing what I'd expect and thence, what changes would be needed to make it do what I want (if possible). This is a general discussion, applicable to any camera. It will be fairly technical but I hope even non-scientists will be able to follow it.

This got me to thinking, what exactly is an AF system doing and what results should we expect? This, then, is the result of me breaking down the job of AF so I can get a feel for how much we should expect. I figured readers might find it interesting, hence the post. Please, if you find any factual errors, leave a comment and I'll make corrections.

I will not talking about how focus is measured. I am going to assume that whatever AF is implemented, there is an objective, measurable system for determining focus that is in place. There are a two main ways of achieving this (phase detection and contrast detection) and from the perspective of AF, the actual focus measurement method is irrelevant here (as will become clear).

Defining terms

First of all, there are some important terms I'm going to use in the discussion, and some abbreviations.

Focus Tolerance (AFTol) - the range in which the subject is determined to be in focus by the AF
Focus Accuracy (AFAcc) - the degree to which the focus approaches optimal focus
Sample Frequency (FSamp) - the rate at which the AF system checks for focus

The red dot is the subject, the solid line is plane of focus, dotted lines represent focus tolerance (AFTol). Gap from solid line to red dot is the AFAcc.

Open loop - a system of control where a measurement is taken, error calculated and the controller moved to desired position. There is no feedback whether the point that the controller moved to is correct. This is how most AF works in SLRs: degree of out of focus is determined and the lens moved to where the AF thinks focus is.
Closed loop - a system of control where the degree of error is constantly checked. For AF, this involves moving the lens and then rechecking whether focus has been achieved.
Feed forward - A system of control where the measurement predicts into the future and then moves the controller where is expects the system to be needed. For AF of moving subjects, this would be the system employed for predictive AF, where the subject speed is determined and the focus moved to predicted next location. it can be applied in either open or closed loop mode.

The gamut of AF

Next up, what are the situations we need the system to cover? I reckon we've got three of them: static subjects, slow approaching subjects, fast approaching subjects (a retreating subject is just a negative approach). For the moving cases there are also steady and erratic speeds. We also need to consider AF under single shot and rapid fire. So that's a lot of operating cases to cover.

In the industrial world with a wide variety of operating modes, we would normally expect either to cover each case with a separate control scheme, or accept compromises in performance on the lesser ones, to hit the important ones. With cameras, that's going to be difficult: who's to say what's important? A landscape photographer is going to want different things from a sports shooter.

Allied to this are the AF control modes used to achieve the desired results. In general, there are 3 types: single shot, continuous and predictive. In the first case, focus is acquired once and applied to all shots in a sequence (which may have just 1 frame), the second checks focus for each frame, the third tries to get ahead of things with moving subjects by predicting where the subject will be on the next frame. In any case, open or closed loop may be applied but it is normal for current systems to use open loop control (i.e. measure once, move as desired).

Let's go through the main cases:

Static subject - fairly simple. Subject sits there, nice and steady, focus is achieved, shutter gets pressed. Think mountains, castles, cooperative portrait subjects. Focus needs only be achieved once.

Slow approach - subject moves forward slowly relative to the response of the camera. It is important to consider relative speed, compared to the camera, rather than absolute speed. More on this later.

Fast approach - subject moves relatively quickly relative to the sped of the camera. Of course high absolute speeds will be involved but the point when a subject moves from being slow to being fast will vary from camera to camera.

And now a quick look at the main AF modes

Single shot, single frame - focus is achieved once, there is one press of the shutter. Best used for static subjects. If open loop control is in place it will be almost impossible to catch moving subjects in this mode.

Continuous focus, multiple frames - With open loop control, the focus is acquired between each frame, producing a continuous shift of focus across the frames.

Predictive focus, multiple frames - The AF feeds forward the estimated speed to move the focus to where it expects the subject to be. This will require at least 2 measurements of focus (and therefore subject) position to be able to sense motion.

What are the implications?

OK, so we know what it's all doing, how does this work in practice? And how do things change as camera specs change?

First up is the notion of focus tolerance versus desired accuracy. A camera will think that a subject is in focus if it lies in the the tolerance bands. If it moves within the bands, the AF is likely to be unable to detect the motion (important later on). If, however, we can perceive, in the final picture, out of focus even within the tolerance range, then the tolerance isn't tight enough. In general, as megapixel count goes up, so this discrepancy gets bigger (manufacturers don't move their definitions of tolerance much).

Then we come to the relationship of subject speed and focus speed. If a subject moves slowly relative to the focus tolerance, then it may be hard to detect the motion (see figure below). If the first couple of points are within tolerance, it would take a third for motion to be detected. This would then lead to over-compensation 9front focus) as the subject is deemed to have moved twice as much as it actually has.

Slow movement relative to focus tolerance. Subject moves 1-2-3 but system only detects movement 1-3, shifting focus A-B for point 4 (now front focussed).

If rate of fire (fps) increases, then the speed that is "slow" in relation to the focus actually increases - the subject moves less between shots. Implication is that higher fps cameras have harder job with approaching subjects that may have been OK for slower models. Just slowing shutter speed may not help as the sample frequency won't change, unless the camera is set up to spend more time on focus for slower modes.

For relatively fast motion, things are better because the motion can be detected between frames (see below).

Fast speed - subject moves 1-2-3. AF detects motion 1-2 and moves A-B for point 3. Subject is in focus for subsequent shots.

If fps goes up, however, the fastest speed that can be focussed may go down as it is harder to move the lens between frames.

My conclusion

Faster camera, with higher resolution actually make focus worse, unless the focus tolerance is changed. Doing anything else won't help much. As fps increases, AF has less and less time to react, and will find it harder and harder to detect subject movement, unless tolerance is smaller or focus is able to detect movement within the range of focus tolerance. Resolution will make focus appear less and less precise.
Closed loop control doesn't help if the AF ha the same tolerance as open-loop. all it would do is minimise the chance of overshoot and provide a chance to correct, but that takes longer than just controlling to a more accurate position in the first place.