Big files management (photos) -Help

edited February 2010 in General
I need suggestions from Kappers experienced in the management of big (photos) files.

The problem I am having is that at the present my photos count are around 12.000 between KAP and ground photos. Files are still growing.

The other day, I had a request for a photo of a very big fish I've got somewhere in the state of Queensland, Australia, and I didn't remember where it was. I had to open and close hundreds of folders in the Queensland file and type the keyword fish. It took me 35 minutes to locate the desired photo. The day after, my wife requested a family photo to send to her mother, and I had to open another hundreds of birthday folders trying to locate it.

This is the way my photos are organized: Well, I have a file for Australia and folders for each state of Australia. Inside each folder I have sub-folders for each city, which also have sub folders for subjects or locations inside that city. example, Australia -> Queensland -> Brisbane -> Downtown . Each photo that goes inside any of these files, folders and sub folders are given keywords for location and subject. Example, "Brisbane, Bridge, River, Kap".

Now, if I want a bridge outside Brisbane but in Queensland, I will have to check a large number of folders, opening city-by-city and typing the keyword "bridge" until I locate it. Very time consuming. If I open only Queensland, and type the keyword "bridge", ACDSee (the program I use) will list a large number of bridge thumbnails for my appreciation, and also it will take a long time till I find what I need.

Question... Is there any better way (Change the structure or organization of the files??? ) which may become faster to locate a photo by subject in thousands of photos? My problem basically, is to locate a photo which I have in my mind, but I don't recall where I've put it. (More keywords could be the solution?)

Any recommendation or consideration will be very appreciated.

Thank you guys!

Roger Australia.


  • edited February 2010
    Hello Roger,
    I have solved most of my file management issues with Photoshop Lightroom you can try it for free and later buy it if you want, if i recall correctly it was not expensive at all
    One of it main strengths is to search for photos using any number of characteristics, including keywords, ratings, camera model, ISO speed, etc
    hope this helps

  • edited February 2010
    Thanks for your suggestion Mauro, but there are two issues I don't feel good about it. The first one is to spend $300. The second is to take time to learn a new program from scratch. Anyway, note was taken about lightroom, thanks.
  • edited February 2010
    You're welcome,
    on the freeware side i know this one: aint bad at all, not as powerful as lightroom but nothing beats the cost (0)

    Or if you want to try some good shareware, there is ACDSee

  • I have been using Picasa 3 as my photo management software for about 4 years now. Others I have tried are a Kodak program that came with my first digital camera and Photoshop Elements 7 which I found really hard work. Picasa 3 is the best of the three in my opinion, plus it is freeware from Google so costs nothing.

    There is a basic windows style folder list down the left of the screen and the folders are order by date. Of course the settings are optional so there are other variations.

    I do not bother with tags as it takes too much time, especially if I'm tagging on Flickr anyway which I'm using as my offsite storage as much as a photo sharing tool.

    I do not have the problem of having many years worth of photos to sort through, but I try and name my folders as fully as possible including the date where KAP sessions are concerned, e.g. KAP Vazon Bay 15Oct09. Within each folder you can arrange photos by name, date or size.

    It really is quite basic and comes with some basic photo editing tools and creative options, but it is all I need and I can find stuff pretty quickly using even the most simple of folder naming conventions. My current laptop has over 5000 photos on it to give you some comparison, most of which is KAP work. I keep the same simple folder structure on my external hard drive. I try to avoid using sub folders as it can get out of hand really quickly. More folders = more chance of misfiling.

    Mine is only one opinion, but as there is no cost to using Picasa 3 maybe you could download it and try it out. You could import say 5% of your existing photos and see how you get on with it. The only thing you lose is a bit more time if you don't like it.

    I look forward to hearing other suggestions. It's this kind of off-topic thread that you can pick up some really useful advice and links!

    Good luck.
  • If you already have Photoshop or Elements, you may also have a copy of Bridge that would help with this tagging and organizing.

    For what it's worth, I'm using LightRoom mostly for editing my shoots. I come home, pour in hundreds of images and then scroll though them all, deleting the blurry ones, straightening the crooked, and rating the nice ones. You can do this with other apps, but Lightroom seems fast and allows me to make a first pass edit in a short amount of time. After that, I'll make quick adjustments to the image appearance using the "develop" module which is very robust and does most of what I need. Later on, it nicely automates the uploading process and automatically keeps track of where the photo was uploaded and when. To find a file I search by date taken or by a keyword. Sometimes I'll search for "photos uploaded to flickr".

    But yeah...when your collection grows to 12,000 you need a way to keep it organized. Even better if you have a tool that makes short work of that organization process and allows you to quickly do more with the image once you've found it.
  • Light Room 3 Beta is a free download and works great and its free!!
  • edited February 2010
    Aperture 3 on a mac. It's not free though.

    I have a teaching slide collection that is cross-referenced by keyword in Aperture, but I've yet to do this for most of my KAP.
  • Roger,

    I use IMATCH from, and I find it very useful though I have not used all the search facilities to any great extent. (US$64.95 at present)

    Physically I store my photos in a directory tree : Year/country/month/ *.jpg
    Backed up on two 1 TB maxtor portable hard disks (one at home and one in the office).

  • edited July 2010
    I've been struggling with this for years. I still have not found a perfect solution, so I just use folder structure and have not invested in any software. At one point, I was seriously considering one of (in no particular order): Hussam
  • edited February 2010
    I have also cast about as to the best means of dealing with organizing large archives of images. With the exception of Lightroom, which seems to be very popular, I've tried as many different products as I could get my hands on. For the moment, I landed on Adobe Bridge. I'm new to KAP, but I have an archive of terrestrial digital photos that is about 1TB and growing fast. At present, my colleagues and I are photographing archaeological artifacts and geotagging them with either GeoSetter (free) or RoboGeo (<40$). This is how we are making "digital surface collections" of diagnostic objects at archaeological sites.

    Bridge is a key part of the workflow and organization. The files come in from the cameras, they are placed into folders by location and date. Then the files are processed with the geotagging software. Bridge is used to batch apply metadata templates to the files using the "append" function. This information includes who took the photo, phone numbers, email, institution, etc. Additional name based locational tags could also be applied at this stage. After these metadata are applied, then the Bridge keyword entry form is used. I like this function very much. I have multiple lists of hierarchical keywords. Applying these is a matter of loading the list and checking the appropriate boxes, this can also be done in batch.

    The hierarchical keyword lists can be saved to a file. It is a plain text file. This file can be edited with something like notepad, and then re-loaded into Bridge. Once attribute information has been created for the images, we generate ESRI Shapefiles (GIS data). This is done using RoboGeo. Bridge is useful for recalling images based on keywords, photographer, or any other piece of metadata. Let me know if you would like further details on the workflow, and I can post something more specific.

    Thanks for raising a good question and thanks for the answers provided. I have two related questions. Does Lightroom also have the ability to apply keywords based on a checkbox form? Have others experienced problems with Lightroom reformatting or otherwise creating problems with metadata (i.e. EXIF or IPTC fields)?

    Thank you,
  • edited February 2010
    I think the major problem with large archives is that we don't grit our teeth and throw away the zillions of images that won't ever be used. I know I've got lots of them, and I'll bet you all do too.

    FWIW, National Geographic photographer Michael Melford gave a presentation at KAPiCA/06 about his use of Bridge to process the thousands of images he generates each month. Pretty much mirrors what you've said, Nathan.
  • I'm going to have to start geotagging. The potential to sort and organize and put in 3d viewers is just going to increase.

    An aside on those zillions of images. Is there generally an agreement on "similar images" when licensing? Seems like a giant loophole for KAPers as I at least often get 10 shots with a similar perspective.
  • Yes, you can add keywords to your pictures in Lightroom. So I would have KAP, the location and the subject as keywords, and maybe also "snow" if it is snowy, etc.
    and of course, you can then select photos again by (parts) of keywords. I've found this very very useful...

    So my workflow is the same as decribed above: read in RAWs into lightroom, throw away 50% or more (when autoKAPPING, I usually retain 10%), and then add keywording.
  • I'm with Brooks. Delete, delete, delete.

    My work flow is to get rid of all badly tilted images, then all the images that have missed the target, then all the dark ones, then all the blurred ones. That cuts it down quite well. I have to confess though that this is a recent development and I am having to slowly go through the last year and a half's KAP images. The only downside of getting better at KAP is that you end up with more and more images that are potentially worth keeping.

    Now all I need to find is a photo browser that makes it easier to spot the blurred images without having to open the file.

  • Thanks all for the recommendations. Keep them coming, because it has been very instructive for me (and for other too, I recon), specially having the opportunity to know how other people do in terms of managing image files, and also the programs available for the task.

    Well, I think I've got a solution for MY problem keeping my files structure untouched. When Femur suggested the program ACDSee, I said, nop, because I already have it and use it to organise my files. What I didn't know is that there is a new ACDSee program called "ACDSee Photo Manager 2009" which does exatcly what I need plus the following interesting things:

    - It can place metadata and keywords in batch and also rename images in batch.
    - Ultra fast image search (it takes the same time to search 10 or 1,000 images).
    - When you see the thumbnails, just by placing the mouse over, the image instantly enlarges (good to see details of kap photos and delete the bad ones)/

    They are having a special price for only $ 29.99 which fits my budget and because I run an older version, I don't think I'll have to learn most of the functions again.

    Cheers, and thanks again.

  • Martin, that raises a question I've been banging my head against the wall on: A qualitative measurement of sharpness.

    What I want is a program I can run a stack of images through, and get a value for the relative sharpness of each image. I can see a couple of ways to do this. One is to use a slope-based edge detection algorithm, and then bin these into a histogram and try to find a median value. Sharp photos of naturally smooth subjects will, of course, get a lower sharpness score than slightly blurry photos of a subject with a lot of detail. But when looking through images of a given subject, it would be nice to be able to say, "That's barely good enough," and then cull all the other images of that subject with lower scores than your threshold image.

    I figured in this day and age, such a filter would be commonplace and easy to find. But I really haven't seen it. Equally unfortunate is that the people who tend to be interested in sharpness at this kind of level are typically doing optical tests on lenses. So the resulting software is highly specialized, not nearly automated enough for my tastes, and never cheap.

    Maybe I'm just dreaming...

  • I keep dreaming too. I like, for example, that Facial Recognition has started to appear in applications (iLife '09 & now Aperture) - oh wait - no KAPer cares.

    How about having Google Goggles automatically recognize and tag your landmark images?

    And yes, I have zillion images too and I would love to delete the bad ones (but I don't!).

  • Mac users please skip this message.

    Roger, I was going to suggest you look further at ACDSee, and now I read that you are about to get ACDSee Photo Manager 2009. I'm not super impressed with ACDSee, but I've been using it for years now. I've also used Picasa, Fotoalbum from (better in some regards than ACDSee), and some earlier versions of Adobe Photo-something-or-other and junk from ULead. I think ACDSee has what you need, but it can get cumbersome. Then again, 12K photos is cumbersome in any universe.

    A strong warning with ACDSee PM 2009 - they screwed up EXIF and IPTC field merge. As I understand it, there are poor specs for some aspects of IPTC. If you set the "image description" field in EXIF, it is supposed to copy to IPTC "caption", and it does, but only if IPTC "caption" is empty. If you set IPTC "caption" first, it should copy to EXIF "image descritpion" and it does. But using Tools/Batch Set can screwup horribly. Once IPTC "caption" is set, attempts to set EXIF "image description" by batch will fail. You cannot erase the values either. To change EXIF "image description" you will now be forced to do it one at a time.

    There are long discussions in the ACDSee forums about the problems caused by imperfect IPTC specs and ACDSee's highly flawed attempts to implement the merge and then their subsequent fixes. ACDsee admits the responsibility for fixing it is theirs, but apparently will not fix it in this product. The previous ACDSee 10 does not have such a problem. ACDSee says that their professional upgrade product ACDSee Pro 3 does it right, but not only will this cost you more, but some users claim it DOESN'T do it right.

    I don't believe there are problems with any other EXIF or IPTC fields, and if you're careful with Batch Set, you'll be OK. Also, freeware "exiftool" is your friend.

    Sorry to ramble about a non-KAP topic, but you may have guessed that I'm pretty peeved with ACDSee - but will continue to use their software.
  • I have been using Adobe Bridge for my bagging and tagging for a few years now (lately using CS3 version) and I like it quite well. Looking at Lightroom though now so that I can make adjustments on the fly rather than having to open up ACR to make any adjustments.

    Somehting that caught my eye, and granted not KAP orientated, is that I believe Light Room 3 will come with face recognition ability that scans your archive and automatically tags the name of the person to the face and recognises them going forward. Something useful for auto-tagging family members.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

In this Discussion