I'd really love to see LZX archive support in Opus.
I understand that you're currently re-using code from 7-zip, which doesn't officially support LZX. However, I found that they do have an LZXDecoder in the source files.
Check the CPP\7zip\Compress folder of the 7-zip sources archive, and you'll find:
Lzx.h
LzxDecoder.cpp
LzxDecoder.h
I'm not sure why 7-zip doesn't seem to use that on the front-end, or indeed open LZX archives, but perhaps the code can be re-used to implement LZX support in Opus anyway?
If that doesn't help for some reason, I can offer an alternative also: I'm using this piece of code to support LZX archives in Amiberry, an Amiga emulator for ARM devices:
I could try to implement it in Opus myself, but since I haven't written anything for Opus before, it will take me some time to understand how plugins work. I was hoping that the information above would be enough to get it officially added, but if all else fails, I'll have to take the time to develop it myself...
That was already linked to two replies earlier, in 2018.
If 7-Zip can open this type of archive, adding support to Opus via our existing Archives plugin should be easy enough. I am not sure if it can open them or not, though, and don't have any to hand to test.
I have looked at the sources of the Total Commander's LZX plugin and the implementation that @MiDWaN proposed from amiberry project.
It seems that the best bet is to use the implementation from amiberry, but someone will need to decouple it from the UAE first.
It is interesting that there is no actual library to work with LZX. For example, the LZX official site only contains a UNIX console unpacker, and its code can't be used as a library, because it mixes the unpacking logic with console output.
The version included in Amiberry is somewhat minimal, as it’s adapted to what the emulator uses only. But the full unlzx sources in portable C are available on Github in various repos, for example here: GitHub - svein83/unlzx
If you look at the Total Commander UnLzx plugin, they took the code you proposed, but they had to apply numerous patches on top of it to make it work properly on Windows. Also, the original code that you mentioned is not a library. It is a console application (the unpacking code is full of printf)
There is a lot of work needed to turn it into a DOpus Plugin.
To be honest, I don't know how much functionality is needed for the DOpus VFS plugins. Thus, it is hard to estimate the effort. But I can assume it should be somewhat similar to the Total Commander plugin.
I think it is possible to write a JScirpt plugin that will unpack the LZX archive using unlzx.exe on double click to a temp folder and open it. That will simulate the VFS plugin.
I've taken a stab at the matter -- please take a look here and let me know what you think.
This plugin explicitly assumes iso8859-1 character encoding for now; it seems to work with all the archives i have so.. hopefully does for you, too.
PS. I've also taken a stab at adf/hdf support, refreshing the original implementation and addressing a handful of bugs, while likely introducing some other. Would appreciate the feedback on that one, too.
Wow, amazing work. Just in case, did you check what workarounds were done in Total Commander UnLZX plugin? As far as I remember, it was mostly related to file names and timestamp conversion. Their source code archive also contains test lzx files. Do they work?
i haven't looked at TC, and the VFS plugin doesn't exactly support datestamps, comments, or protection bits at this point - just plain filename (with proper character conversion - only from Latin1) and folder structure.
with all honesty i'm super curious in getting compression aspect covered, but don't have a good source and reverse engineering huffman coding for lzx is quite difficult.
if you guys need comments, date stamps, protection bits - do let me know. i can look into this.
as for test unlzx impl -- i overhauled the original implementation - to the point it's cleanly broken down and destructured. this would be trivially parallelizable if that ever made sense, taking full advantage of any modern multicore/multithreaded cpu. I'll take a look at their test files and update this thread in a bit.
Alright - thanks for prompting me to try these. I discovered that - to my surprise - std::filesystem::relative() started throwing exceptions with paths that are clearly relative so ended up rewriting this until i figure that out (pretty weird given the code is -fno-exceptions - or perhaps that's exactly why...)
the test files mostly pass - except one that seems to have something funny about its directory structure. I'll have that patched up shortly.
ahh nice i see what i missed - lzx typically persists only file metadata, but these test archives capture empty directories, too. brilliant - i'll have that addressed in a minute.
working with C++ for 20 years I am pretty confident you can. but yeah, crash is a real concern here.
The uses of standard library are all within what should never throw an exception but it looks like std::filesystem has some quirks such as "yes i can canonicallize these paths" + "yes these paths are relative" = "no i cannot canonicalize these paths to extract relative path component". that's pretty surprising.
as for the remaining broken archive -- i'll get the datetime parsing done and will upload a version that honors protection bits, comments, and date stamps soon. there's 3 nice datetime variants there to cover, these test files are great.
