If I'm correct there is no option to make DO copy junctions and other types of links "as is" or as empty folders, or is there? I would like to get asked what to do if DO encounters such links, similar to what happens if items already exist, some kind of confirmation or user request on how to handle them.
So in the middle of a copy/move operation I'd like to be able to tell DO to..
recreate the links if possible and do not copy/move what's within them
do a full copy/move of content behind those links
just create empty folders
or just skip them
It would be perfect if these options are available in "unattended mode" as well of course.
"Unattended mode" could take some boost in this regard in general, it does not seem to work for delete operations and any failed items can only be exported to csv-files, why not export to file collections to be able to re-do the operation easily? Getting offtopic here, will finish now! o)
recreate the links if possible and do not copy/move what's within them
do a full copy/move of content behind those links
just create empty folders
or just skip them
It wouldn't be hard for DO to provide an interactive mode for the above steps. But based on my experience, prompts requesting how a symlink should be processed becomes mentally exhausting very quickly. Each prompt requires you understand the context of the target link and its purpose.
But, your idea makes perfect sense for unattended mode. I've implemented some of that logic. But it's virtually impossible to process all four of those decisions steps in one pass. Even with something as sophisticated as XXCopy that allows dozens of argument combinations, unattended mode requires at least two passes.
I finally ended up using 7-Zip with the WIM container format to copy symlinks/junctions "as-is" between machines.
Finally, what you're requesting has tremendous value, because copying reparse points "as-is" enables DO + NTFS to be a poor man's WinFS. Once I figured out how to copy symbolic links "as is", I was able to hack together a virtual path system that is now the foundation of my DO customization sync engine. ( It's just folder naming system using junctions and symlinks that abstract physical paths.)
Once you go down the virtual path route for DO, impossible sync scenarios, like labels and status icons, become trivial. For example, my customcolors.ocx uses the virtual path system. Thus, I can sync alternate data streams btw machines because there is an indirection between an icon's GUID and its physical PNG file.
Of course and I agree that it could end up being exhausting for a vast number of links and very heavy usage of them.
Unless you are within your windows profile folder, this is probably not the regular case!? o)
Most of the time I deal with a handful of links within a folder structure, asking what to do with them is totally manageble I think and would be much less exhausting than trying to copy the structure piece by piece and leave out/recreate the links by hand while doing so.
I agree with you that using these junctions and links is a great way to handle advanced sync scenarios or to bring data from multiple sources into one convenient place. "Poor mans WinFS", kind of - yes. o)
Thanks, that's a start and something I will try. It raises new questions though. Were there any links? Should I recreate them? But I see, I can at least copy some folders without the gigabytes of data that are linked somewhere down in the regarding folder structure.
This is not particularly complicated at all, and does not need two passes.
However, it might require "elevated" permission to support creating the reparse-points at the destination. It might also require that the elevated/user account also have seBackupPriv to read all the xattr/stream info properly.
But basically when copying reparse-points (symlinks, junctions, etc) we might care when the link-path target is absolute (not relative), because we might have to rewrite the link-target based on current-dir versus new-dir location. In practice, though, if a path was absolute, it should stay absolute. Relative paths should remain unchanged and work as-is because presumably they were already correct as relative paths.
I have command line tools that do this all the time because it is a such a fundamental operation for moving files. I do not know if DOpus has integrated this as a default copy option at any time recently. I have discussed or raised this topic in the past when DOpus first started supporting symlink operations.
If DOpus does support it, it would be great to have a simple example that clearly illustrates how to drag-copy with reparse-point copying (rather than reparse-point target content copying).
A related, and smart copy feature I also support (which I think this form of DOpus copy should support) is that when a "hardlink" file is copied on the same volume, it SHOULD NOT copy the file, but rather recreate another hardlink. As the intention here is to COPY links, NOT their target-content.
Iterating hardlink peers, true-path (collapse intermediate symlinks [reparse-points] etc) are supported in the WinAPIs even if the documentation is "obscure". If assistance or pointers to the documentation on how to do the all the above is useful, I'd be happy to provide assistance/information.
So DOpus does NOT have a way to author a drag-copy action that will copy and replicate files/folders including reparse-points (links) RATHER THAN (NOT) their targets. I.e., copy literally?
You could make a button which runs RoboCopy or similar to do that. That's what I've used (although for something slightly different, not junctions). You could also have a script which uses the Opus commands, if you wanted to.
I'm still not sure it's as simple as you say it is, though. If the junctions/links point to things outside the folder(s) being copied, you might want the destination to point back to the original drive/folder, or to point to a new copy that would have to be made in addition to the actual selected folder(s) that were copied, or you might even want an error message because you didn't expect that to even happen, or you might want everything converted into a normal file/folder. (And where to make those copies would be unknown, especially if the links point to a different drive or even a different machine.)
It still seems to me, like someone said above, like the kind of thing that would require either user input on each junction or a script that each user wrote/customized to their individual expectations, since there's no inherently right answer with a lot of the situations that can arise.
Disk-image backup tools may also be a better option in a lot of cases, although it of course depends on what you're copying (and not copying!) and why, and how (and on which machine) the destination is intended to be used. I don't think it's simple at all, nor that the answers to all the unknowns will be the same for every user or even for every copy made by the same user.
I can share a binary copy my product/tool with you so you can try it "edgescript" toolset.
It really is pretty much as simple as I described. This pattern is used extensively in NodeJs work, and vscode fully supports and understands these principles in its folder trees, search mode, workspaces because it is common practice to use links to manage sharing and configuration within and across volumes.
Hardlinks are the common practice to share busy-box like multi-purpose binaries. Almost all the git toolset on windows uses that model.
We can talk about this offline or wherever is convenient. DOpus is such a fantastic tool that I really would love to see it seamlessly integrated and supporting these patterns. I don't know one person whom I have shown DOPus to who has not only purchased it, but also fallen in love with and become dependent on its capabilities.
Not everything / everyone will use that pattern, though. We'd need something more general than that if it was going to be built in to our Copy command.
If you have a tool that does what you need already, making an Opus button which runs the tool and passes it the relevant arguments would be what I'd do.
I already have DOpus doing what I need using my tools. But that is different from DOpus out-of-the-box supporting an alt-copy that copied reparse-points literally. While it MIGHT NOT be the default people expect. Having the link-targets copied and links disappear is often much worse and breaks the copy. [and is a total fail on recursion cases that become huge since all de-duping provided by the links is lost].
My thought would be to provide an (opt-in) alt-copy mode built-in via a right-click drag option. Also possibly having a DOpus option to make that the default copy-on-drag-action.
In my experience, you rarely want the link targets copied, you almost always want the links copied. But understanding links is a power-user concept [noting that for many people, DOpus is a power-user shell].
To be able to copy junctions "as is", I started to use FastCopy. Find it here: https://fastcopy.jp/en/
It has a commandline interface as well, which allows to create buttons/drag-menus.
For syncing folders via button I use robocopy.exe (which can exclude junctions by switch).
For syncing ADS data (like "ratings" and "tags" set from DO) I use.. aehm.. nothing.
I do not know of any tool which can sync and backup the things you can set with DO (fence here o).
It would be nice if a future DO comes up with some copy-prep dialog, which allows to setup the copy/sync operation beforehand (ACL yes/no, ADS yes/no, Junction yes/no/ask etc.).
The FastCopy way is the right direction, more options need to be accessible directly, not via settings/prefs which need to be remembered/re-set.
A decent unattended mode on top, which allows to set more options and export as file collection would be nice as well. File and folder structures getting more and more complex, the example with vscode, and nodejs was a good one, copying samba linux shares is often challenging as well.
The current situation where several tools are required to copy/sync everyday folder structures does not feel right. All these are workarounds, since we lose big on the "having" side. Unattended mode, progress, filters, flatview etc. - any DO feature is obviously not at hand with the workaround tools.
An industry leading tool is required to handle all these tasks, I can only think of one! o)
Opus can copy them at the same time the files themselves are copied, but I'm guessing you mean if the files haven't changed but the metadata has, and you want to sync just the new metadata over? Yeah, I'm not aware of any tools that can sync that on its own either.
Even the dedicated comparison tool I use ignores ADS entirely, as far as I can tell. (Despite checking things like the R/H attributes, much to my confusion when it flags two files as different but comparing them shows they're the same.)
Yes, exactly.
Because of that I wrote a powershell script which only copies the ADS, it kinda works, but it surely is a very low-fi approach on this and only makes sense in specific circumstances.
are you using the CLI version of this awesome tool in DOpus via buttons/menus?
If so, I'd like to ask if you could kindly share these so that other users could do the same. Thank you!
(by the way, the newer FastCopy version can also copy hardlinks in addition to junction/symlinks!)
Although I agree with you that it'd be great if DO had a native support of more advanced copying scenarios...
