I: contains 7 folders, total of those & subfolders is 700,000 files. I: incudes years of images from phones, files from PCs, etc. Every day a phone was used to take pictures, new 001.jpg images were created. The result is that there are many many 001.jpg files throughout those subfolders on I:, and they are all different images. There my be no two alike, in hundreds files all with the same exact file name.
I used Opus to list all the files in I:, flat, without folder reference. Then I used Opus to start a copy of all 700,000 files to a new flat location on F:. If a file with the same name already exists, it adds a counter. See the two pics for source and target sample.
Problem: after running for 60+ hours the copy failed. I don't know what happened, What I do know is that it is no longer running, and I have 133,000 of the 700,000 files copied. So I just start back up where I left off, right? I don't think i can.
Won't Opus incorrectly recreate all but the first file in a given group? It won't know that the second 001.jpg file on E: is already on the target drive, just named 001 (74).jpg.
So it will incorrectly create another file with that name and increment the counter, won't it? I'll end up with many duplicated files. Which is the opposite of what I was trying to do to start with.
I don't see how to fix it other than starting over and doing it one source folder at a time just to break it into more manageable chunks.
Can Opus clean up my mess? Any way to avoid starting over?
I would add the shooting time or parent folder name to the filenames, to make them unique. That can be done using the Rename dialog.
That will also fix the problem with the flattened folder interweaving different photo sets instead of grouping them together.
The Duplicate File Finder can be used to clean up any duplicate images once done, by setting it to use MD5 Checksums to find identical files with different names. (Assuming you can't delete the flattened directory entirely and start again without losing any photos, which would be easier if it's possible.)
Love the idea of adding to filenames to make them unique and to help keep sets together. Will wipe out the flat folder & start anew using Rename. Thanks!
I agree with Leo about renaming the files.
If it were me, I'd first separate out any photos without a Date/Time Original (a Picture metadata column) then rename the photos with a Date/Time Original by the Date Taken:
When the date format YYYY-MM-DD is used, the photos can be sorted chronologically when sorted by filename.
And make sure to check this box in the rename dialog:
I'm using Rename with the datetaken field, working perfectly. But it is taking forever, renaming files at a rate of one file every second. Is there a way to speed it up? Can it run multiple threads?
JPGs should be fast, but if you have other file types (especially videos) in the folder then those might be slowing things down.
The type of folder may also be a factor, if it's not a local drive or fast network drive on the same LAN, or especially if archives or FTP are involved.
It's a Buffalo Box 8TB NAS. Generally pretty reliable & speed's good.
One thing I can do while this is running is fire up Opus on another PC and run the Rename command against different folders concurrently, start from the bottom of the list. That should get me to the finish line quicker.
I'm seeing a bunch of JPG files (in total, hundreds) in various folders that are not being renamed. When i sample check them, none have a date taken although they are clearly a photograph. Maybe it's a copy of a copy of a... etc and lost that data along the way?
Is it possible to run a script that checks JPG files for the presence of a DateTaken value and if it is null then make it not null?
Photos can be stripped of their date metadata in many different ways.
In an earlier comment, I recommended you separate out photos with no EXIF DateTimeOriginal ( use the photo metadata column Date/TimeOriginal) as that is the only really reliable indication that the photo has the necessary date metadata. Photos with a Date Taken do not necessarily have an EXIF DateTimeOriginal, so Date Taken is not completely reliable.
Then when you have all the photos with an EXIF DateTimeOriginal, you can use the 'Date Taken' property to rename then because the Date Taken will be accurate.
Do NOT use the file/system Modified date.
If you want to learn more about photo date metadata, I suggest you visit the Definitions page of my Photo Organizing Stuff website.
and the Meg's Bytes blog
Stopped my journey down the wrong path & now recreating several folders to begin anew using EXIF DTO. Your page w definitions is excellent, informative reference material. Well done!
Leo,
Photos from cameras may have EXIF date metadata when they are taken, but people share photos in ways that strip the EXIF metadata.
In the digital collections I organize, between 5% and 20% of the photos do NOT have an EXIF DateTimeOriginal (Date/TimeOriginal in Opus).
Of photos without an EXIF DateTimeOriginal, up to 15% have a Date Taken, because a photo does not need an EXIF DateTimeOriginal to have a Date Taken that is 'false' and may change if the photo is modified. So Date Taken is not entirely reliable.
That is the reason I persuaded GPsoft to add EXIF DateTimeOriginal. Thank you for adding it.
Meg - you are a LOT of help! Just finished renaming all the files that have a DTO. Now a little stuck. I have a given folder with 55,000 files in it and subfolders. It contains PDFs, documents, zip files, EXEs, and media files including JPGs, BMPs, PNGs, MOVs, AVIs, MP4s.
How can I replicate only the files above that have an EXIF DTO value, flat, to a second location?
Mike, I sent you a private message. I have been happy to help, but explaining the settings and options needed requires more time than I have.
Perhaps someone else will help.