Order of items in XML config files changes non deterministically

Help & Support
1

I use a Git repository to sync DOpus settings across two Windows 11 computers. [1]

I found that in some config files the order of XML items changes regularly. Thus, Git indicates there was a change, while all what happened was that DOpus reordered the XML entries.

For example, in /dopusdata/Formats/wild.off, the structure is:

<folder_formats>
    <path type="wild">
    ... path 1 ...
    </path>

    <path type="wild">
    ... path 2 ...
    </path>
<folder_formats>

It’s frequent that DOpus switches the order of path 1 and path2, while the content of the path items remains unchanged. That’s a problem as I regularly have to commit “DOpus reordered entries”. Another example is /dopusdata/ConfigFiles/evalfilegroups.xml, see screenshot below.

Do you have any ideas why this happens? Different library versions? Data structures with non-deterministic/undefined ordering?

I’d be nice of XML content could be sorted in some way before being written to the config file.

dopusdataConfigFilesevalfilegroups.xml
dopusdataConfigFilesevalfilegroups.xml679×949 89.5 KB

  1. (See Sync Dopus Configuration - #7 by dpuser441 for details, but it shouldn’t matter for the issue I’m having.) ↩︎

2

My wild.off hasn't changed in over a month (since I last edited something within it), so it doesn't seem to modify itself here at least.

Details with the full config files/paths and what changed may reveal something.

3

Interesting. I’m using DOpus on two devices and I’m not sure yet whether it’s an issue that only occurs across devices or also on a single device - that’s why I asked about possibly different libraries (I think DOpus doesn’t use .net, does it?). I’m using the same DOpus version on both devices.

Here’s another example where 2 lines were re-ordered to another line by DOpus in prefs.oxc:

image
image837×242 38.5 KB

Obviously this is a non-issue when not using version control, but having configs in Git creates unintentional file changes.

I’ll update this post when I know more. Thanks.

4

Another example from today: /dopusdata/ConfigFiles/foldertoolbars.xml. The text behind the blurred bars are identical respectively.

image
image874×226 27.1 KB

5

Did you make other changes affecting the files?
Or maybe done things like copy/paste a button to then change one of the two to something else?

I also have my whole dopusdata folder under git but I can't remember seeing things like this.
Most of the files (e.g. other than recent files, state, ...) do not change by themsleves, and even when changing because of a user action (e.g. adding a button to a toolbar), the change is only affecting the part that changed.

6

In many applications, XML is basically a database. In databases, if there’s no key meant to imply sort order, one can expect this to happen.

7

No to both. It seems to happen without making any other changes that could explain the behavior.

That’s exactly my assumption as well. Data structures might be involved that come with an undefined/non-deterministic sort order, e.g. hash tables or key:value maps. I also assume it only happens when using DOpus on different devices. Btw, one has an Intel CPU and one an AMD, in case that might matter.

That’s why I suggested to apply some kind of unique “sorting” before saving the XML file.

8

I think I found a way to reproduce this. It happens on a single machine as well, not just when syncing to another.

I have the following two evaluator groups:

Evaluator groups 
<?xml version="1.0"?>
<evalgroupscheme reverse="yes" scheme_name="Date: YYYY-MM" sort="yes">
	<eval>return DatePart(value, &quot;yyyy-MM&quot;)</eval>
	<groups enable="no" />
	<columns>
		<col default="no" id="11" />
		<col default="no" id="3" />
		<col default="no" id="6" />
		<col default="no" id="10" />
	</columns>
</evalgroupscheme>

<?xml version="1.0"?>
<evalgroupscheme reverse="yes" scheme_name="Date: YYYY-MM-DD" sort="yes">
	<eval>return DatePart(value, &quot;yyyy-MM-dd&quot;)</eval>
	<groups enable="no" />
	<columns>
		<col default="no" id="11" />
		<col default="no" id="3" />
		<col default="no" id="6" />
		<col default="no" id="10" />
	</columns>
</evalgroupscheme>

They’re shown as such in my config file /dopusdata/ConfigFiles/evalfilegroups.xml:

image
image567×568 47.1 KB

Now, when I open DOpus Preferences, navigate to Preferences / File Display Columns / Evaluator Groups, make no changes at all, and click “Ok” to close preferences, those lines toggle:

image
image563×629 52.8 KB

Repeating going to Prefs + clicking Ok reverts the toggle, and vice versa.

9

Thanks for the examples. All of those should be in a fixed order once re-saved by the next beta. (You may still see the order change the first time it re-saves a file.)

10

Amazing, thanks @Leo. :heart:

11

Small detail: based on @Leo’s explanation, I suppose Git may still indicate there was a change, as it seems like Opus will still re-save - IF Git looks at last change date rather than comparing content. But that’s probably okay for you.

12

Opus won't usually write out a config file if the contents haven't changed. There's a check for that. So the date shouldn't change unless something in the file does.

AFAIK Git doesn't care about dates on local files other than to know if they might have been modified. It checks the content to see if they really were modified, and the server stores the commit dates rather than the original file dates. (At least, that's how every other source control I've used worked.)

13

That’s right, Git should only consider file contents (and attributes/permissions), but not modification times. Thanks for fixing this issue. :+1: