I usually use xxcopy to synchronise my files, using a batch file, but I thought it would be useful sometimes to use Dopus.
Just tried synchronising a folder and its backup folder, both on internal HDD, using Dopus, and after 20 minutes it had got a third of the way through the comparison, so I gave up. Xxopy does the whole thing in 1min 25 secs!! So what is wrong with Dopus or with what I am doing?
Another thing, when I cancelled the above sync after 20 minutes, Dopus went into a sulk and wouldn't respond, so I had to use Task Manager to send it to bed!!
For info, I'm using Vista64, and the folder being synched was a folder of Lightroom previews, so about 45000 folders and 55000 small files.
Try using Process Monitor to see what Opus is doing during that time. Is it reading the same file, or set of files, over and over or something like that?
Also try disabling all plugins and the other things suggested in the FAQ on issues that happen when reading particular folders. It's possible that the problem isn't synching but is something going wrong when the results of the comparison are displayed.
You could test that theory by viewing the two folders in Flat View mode to see if anything goes wrong. View -> Flat View in the default menus.
I reported (something like) this some years ago. The more files are in the folders to be synchronized, the more time it will take. Sadly the needed time is not proportional but exponential (from the 2nd to the 4th percent the increase is 250%).
In my case I am looking through a catalog of 200.000 files. Loading the folder tree takes only 10 minutes or so. Dopus will consume about 300 MB of RAM for this.
But when it starts comparing the files the nightmare begins. With every percent compared it becomes slower and slower. In the end the whole process takes 12 hours or so. When I had my single core machine the pc was unusable due to Dopus' high task priority. Nowadays only the noise of my cpu fan (the laptop fan starts roaring when the cpu is highly working) and the waste of time are the only annoying problems with this.
The file comparation process eats up 100% of the cycles of 1 core. Maybe the list management routine runs havoc and eats up all those cycles?
In the end this function is unusable for me. I used to make backups with this, but as you can imagine this doesn't work out.
btw. I am NOT using byte compare but only file size + date with summertime compensation and ignoring the seconds.
As all of the backup solutions I tested on Windows are totally crap, I would really appreciate having this fixed (I want my Diabolo Backup from Amiga back).
@bobf
Dopus doesn't crash when stopping of finishing the sync. It just needs ages to clean up the ram again. In the task manager you can see kb for kb being freed. With 300 MB of RAM this can take a while (even on more than 2 GHz per core)
in my case I have 650.112 files in 34675 folders (increasing from day to day and no, this is NOT my porn collection but the files I am working with )
just after having finished my above post I looked at the task manager and was shocked. Although all syncs I had done were closed, dopus was still reserving over 400 MB of RAM (instead of 15 or so). It seems as if it doesn't free memory after stopped syncs (I am not sure about finished syncs as this test would take a day).
Here is some data:
dopus after starting: 15 MB
dopus after first time when having compared 5% of the files: 292 MB
when stopped after 5%: still 292 MB
after closing the sync area of the window: 74 MB
dopus after 2nd time having compared 5% of the files: 292 MB
starting another compare and closing it after 19%: 77 MB
Most interesting these numbers are much lower than then when I looked at task manager after the last thread. The peak was over 600 MB of RAM.
If you're doing a byte comparisons on the files then it will be a lot slower than a date-only comparison, for example.[/quote]
I'm using date(different) and time, with all the boxes ticked except use filter.
Just tried again, and it takes 4 minutes to just read the folders, during which time I can't do anything because it goes into Not Responding mode if I try. That even stopped ProcMon working, and it sent off some stuff to MS about the problem.
For normal file management, Dopus works fine. I'll just keep clear of Synchronise in future.
[Try using Process Monitor to see what Opus is doing during that time. Is it reading the same file, or set of files, over and over or something like that?
Well, I just started ProcMon, but when I then started Dopus and it sat looking at the folders, I tried to look at the ProcMon window, but it had stopped and sent off error messages to MS.
Also try disabling all plugins and the other things suggested in the FAQ on issues that happen when reading particular folders. It's possible that the problem isn't synching but is something going wrong when the results of the comparison are displayed.
Ah, unchecking all the default plugins does now allow me to stop Dopus during its synchronising, and does allow ProcMon to run.
But, during the long period (up to 10 minutes) while it reads folders, ProcMon shows it is doing little or nothing other than opening and closing Registry keys.
bobf, the plugin issue is often due to video codecs choking on certain files (e.g. scanning through a huge video file looking for something which is usually near the start but isn't there at all in some files). Sometimes updating video codecs can solve the problem but if not disabling the Movie plugin should work around it.
Sounds like the sync code has some scaling issues as well, though. I've made a note to look into this in detail myself and talk with GPSoftware about what can be done to improve it. Probably won't happen for a little while though as I've got a lot in my queue right now and it's probably something that needs refactoring to make it scale better rather than a quick-fix.
For now, if you're doing a simple time/date one-day-sync, maybe the "Copy UPDATEALL" command will work better for you? That doesn't show a summary of what will happen, it just does the sync on a file-by-file basis and should be as fast as a normal file copy (or faster since it can skip things that don't seem to need updating).
If you need a summary view and haven't already tried it, Beyond Compare may be worth a look. (I've not used BC's folder syncing feature at all so I can't guarantee it'll scale better but the quality of the parts I do use -- diffing and merging -- have always impressed me and it's proven an excellent companion tool to Opus. It has a good command line interface so you can set up buttons/hotkeys in Opus to launch it on the current files/folders, almost as if it's part of Opus.)
Hopefully we can get Opus working perfectly in this situation in the future.
Many thanks for that Leo. I've rarely gone beyond basic copy'npaste stuff in Dopus, so the reference to the raw commands and the file commands opened my eyes to the wealth of stuff in Dopus.
I tried Copy UpdateAll and it worked on the same set of folders and files in about 4 minutes - a vast difference from using Synchronise. Still not as fast as xxcopy (1.5 minutes), but it has potential for some things I do.
The thing I missed though was a simple statement at the end saying what it had done, such as scanned xxx files, copied xx files, deleted x files, or similar.
Sounds like the two programs are choosing to copy different files if the time difference is that great. You may be able to make Opus do the same by adding arguments to the command.
Copy times can also be affected by copying the timestamps and attributes of files, which not everything does but Opus will do by default. That's only significant when there are lots of small files, though, and unlikely to cause such a large difference.
On some devices changing the copy buffer size in Opus can have an effect, too.
The log (Tools -> Output Window, and configured via Settings -> Preferences -> Logging) may be useful there.
[quote="leo"]Sounds like the two programs are choosing to copy different files if the time difference is that great. You may be able to make Opus do the same by adding arguments to the command.
Copy times can also be affected by copying the timestamps and attributes of files, which not everything does but Opus will do by default. That's only significant when there are lots of small files, though, and unlikely to cause such a large difference.
On some devices changing the copy buffer size in Opus can have an effect, too.
The log (Tools -> Output Window, and configured via Settings -> Preferences -> Logging) may be useful there.[/quote]
Thanks again, I can see I have a lot to learn about Dopus.
I know that this topic is very old, but I think problem still exists.
Synchro in Opus, in most cases, is good enough for me. But today I started to synchronize really big folder with many files and subdirectories. And it takes veeeery long to compare. And I'm talking about one way copy and just "date (different) or size". And it takes more than 10 minutes to compare!
So I downloaded "Synchredible" - freeware tool to check if synchro can be made faster. After I set everything, this program compare both filders under 1 minute!
Ok, you may say that it's because files are cached before (after i was using compare in Opus). That is partially true, but when I using compare again in Opus, result was the same - really slow comparsion. And after system restart that "Synchredible" tool was slower as I expect, but still much faster than Opus (whole proces takes about 2 minutes, not 15!).
It will be really worth to investigate it and speed up.
IMO, it's bigger problem that only compare function. Opus is not dedicated compare tool so it's slower - I can understand that, but it's not dedicated image viewer, so it's a little worse and has less options too. It's not dedicated FTP client, so it's slower and has no functions like FZ (even if some Opus functions are paid and FZ is free). It's not separate copy tool, so it's not as fast as Fast Copy (and other filemanagers by the way as I check). Opus is not dedicated video player, so it has no modern renderer built-in. Etc, etc...
And that may be a little annoying when someone buy your powerful manager and like it (because is really good and has nice options) but must use lot of other programs instead built-in functions, because they're made just for "average jobs". IMO - quality, not quantity. If option works bad, slow etc. don't add it for Opus just for increase number of options. And compare tool in filemanager is MUST BE option - it's about managing files, not something else. So your focus should be on this, not on (for example) some scripting functions (Opus is not programmer's tool). And, what is more important, this topic has 8 years old and nothing changed.
Copy UPDATEALL will be much quicker if you just want to do a one-way sync based on size & date, and don't need to see the interactive list of what will/won't be copied before the copying starts.
I agree the comparison stage can be slower than other tools, but Opus isn't a dedicated sync/diff tool. The sync tool in Opus is good for average sized jobs. (As in number of files to consider; size of files doesn't make much difference, unless doing a byte comparison.) For huge comparisons, dedicated tools will do better. Tools like Beyond Compare are great at what they do, and can be tightly integrated into Opus.
Of course, we might speed up Opus's sync tool comparisons in the future, but our focus is usually on areas where Opus can provide unique value. We could work for months on nothing but synching and still not offer more value than tools like Beyond Compare which have had people working on nothing but synching for years. (Similarly, Beyond Compare has some file management functionality built-in, but doesn't compete with Opus in that department. Horses for courses.)
)