Sunday 8 March 2009

Windows computing for photography: learning

If your not interested in gear related stuff, look away now. These are some general ramblings on my computer gear.

I'm going through a learning loop at present on configuring my computer for best performance for photography. I'm learning (or re-learning) some things hitherto unknown to me.

In the past, my computing was mainly based around number crunching or gaming. Each iteration meant faster processor, more RAM, Better graphics card, decent amount of storage. Most of the slow-downs were up at the processing end of the chain. A computer lasted about 5 years, with a mid-life hard drive addition.

Now I'm finding things a lot different. My computer is plenty fast enough, and never seems to run out of memory unless I run huge panoramas, lots of apps or there's a software memory leak. However, I can't get enough storage. I bought a NAS box, which is really good and a great archive solution for me. Even on fast wireless, the speed is acceptable as I'm not doing lots of read-write. I just filled my primary storage, and so went off looking for something else. I'm trying to reduce reliance on in-case storage for my mass-storage, I find that a lot less portable between system upgrades. In-case for working stuff only.

Here's a big learning I've made: don't use regular network storage on a Windows system. I'm trying but it's hopeless. even with every speed tweak I can do without breaking things, I barely get more than broadband speed out of the transfers. It's actually faster transferring from the NAS box back to the network storage than from PC to storage over Ethernet. It's all in Windows network overhead. A royal pain. I can see a gigabit NAS solution in my future. at least the box I bought can be used a USB drive, which does work properly.

Then there's the actual processing end of things. I'm finding I run apps that do a lot of disk read-write, caching, database and the like. That means fast storage solutions, robust to a lot of read-write. That robustness is important: kind of rules out SSD for now. So I'm looking at RAM-based solutions, there's a couple out there. Kind of gets back to the old days of RAM cache configuration.

I've been contemplating running a Windows-over-Linux solution, something like the old Windows 3.1 with separate OS and application layers. I like that model from a conceptual pov. I will not be going Mac. No way, no how - unless prices crash and user control increases dramatically. I know I could run Windows-over-Mac but then I might as well go the whole hog to Linux. Of course, there's plenty of Mac for photographers advice around, not so much for Windows users.

And I want value for money. That's the rub. I don't want to throw cash at this problem. I want bang for the buck. I don't mind shelling out for lots of storage, that's a consequence of lots of files. But I've always disliked the idea that I must get a faster machine because the software or OS no longer supports the old stuff. While hardware seem to get more efficient, programmers seem to get lazier.

So off I go to figure this out. If I learn anything useful, I'll post more. And if you, dear readers, know something, post some comments (no Mac stuff, please).

5 comments:

  1. I have been thinking along the same lines. The only possibly useful bit of info I can pass on relates to gigabit networking. Recently I updated my network cards to gigabit without upgrading the cabling (currently CAT5E). I of course didn't expect to get true gigabit speeds (apparently very difficult to do under the best circumstances) but the actual speed increase over 100T Ethernet was very useful.
    My next computer/photo related project is to build a remote (in a separate building) backup storage unit to protect against the house coming down on my basement office. I live in a tornado/storm prone region.
    I look forward to hearing more about your Window on Linux plans.

    ReplyDelete
  2. Working with lightzone on linux, for quite some time I thought disk IO was a limiting factor, but it seems that RAM and raw processor speed can bring more improvement here. My current setup includes 2.5 GB RAM in the laptop and 4 GB in the desktop machine.

    Upgrading the network to GigaBit brings improvement, but there are 2 caveats: the cheap network cards with realtek chips rely a lot on the CPU and seem to go not always up to the maximum, whilst more expensive ones like the Intel EEpro do - and additionally their linux driver support is better.

    But Gigabit ethernets needs tremendous disk-io rates, so that one of the best german magazines (ct) recommends a (software) raid setup consisting of minimum 3 disks and a dual core processor to really make use of the network transfer capacity.

    Hardware raids with non-professional (read inexpensive) controllers were ruled out by them because after a controller failure you absolutely rely on the availability of just the identical controller that for replacement, otherwise all your data may be intact yet inaccessible on the disks. (linux) software raids do not suffer from this drawback.

    But a fileserver is still not a real backup, especially if it is in the same building. I still have no idea how to solve this problem on a budget. Amazon's S3 service could offer solutions for this, but this online backup topic needs thorough exploration regarding data security and data protection before taking a decision.

    ReplyDelete
  3. Markus, that's useful, thanks. As for my current set-up, I already run a soft raid on the main machine, have a good spec gigabit card and a decent RAID 5 4-disk NAS box (in a separate room), so think I'm pretty well covered for most of the hardware. I actually suffered a RAID failure a while back - not a pleasant experience as I couldn't recover the controller without a reformat. Trouble is, hardware RAID controllers are pricey.

    Of course it can always be higher spec. I'm going to be using a pair of external drives for rotating off-site storage.

    ReplyDelete
  4. I'm not savvy enough to contribute anything, but I am interested in following this discusssion. Since I plan to stick with Windows, anything you write on the topic interests me.

    ReplyDelete
  5. Martin, although the overhead of CIFS - windows file sharing - may be slowing things down a bit, with a 4 disk RAID 5, you are only going to get anything like gigabit wire speed for sequential reads. For small write operations, you are going to get abysmal performance. You aren't putting enough disk spindles into it to get great throughput.

    For apps that do a lot of disk hits, a two disk RAID 1 with a good controller and fast - at least 10K RPM - drives is the way to go. If your budget extends to a four disk RAID 10, even better. Controllers do fail, and if you can't get the same one, you might be in trouble, but that's what back ups are for, right?

    I do this stuff day and day out at work. Drop me an email if you've got questions.

    ReplyDelete

I like comments, especially constructive ones.
Comments get emailed directly to me before publishing , so if you want to get in touch drop a comment.
All comments moderated by me before being published, keeps the spam at bay.