Speed up single file extraction with DLL

[Pique7]

Hello! I have noticed that the 7-Zip File Manager is able to extract single files from a large archive much faster than the DLL.

The 7-Zip File Manager seems to skip many parts of the archive while the DLL is extracting the whole archive although I only need a single item.

Is there a way to speed up extracting a single file with the DLL?

[Pique7]

It is even faster to uncompress the whole archive instead of a single file ...

[ip7z]

check it again with command line:

7z t a.7z file -bt

[Pique7]

check it again with command line:

7z t a.7z file -bt

Thanks for your reply, but it seems I was mistaken. I think the command line executable is okay and also quite fast. Only the DLL is so slow, when I want to extract just a single file. I use it from a Delphi program by means of the mORMot2 interface unit:
https://github.com/synopse/mORMot2/blob/master/src/lib/mormot.lib.win7zip.pas

A minimal code example:

procedure ExtractSingleFileFromArchive1;
var
  rd: I7zReader;
begin
  rd := New7zReader("MyArchive.7z", fh7z);
  rd.Extract(345, ".\MyDestPath1\", False);
end;

[ip7z]

So you have some intermediate code.
Try to measure exact time of extraction for comparison of two cases.
Also compare CPU usage and the number of threads in Task Manager.
Also check the number of files in archive.

[Pique7]

For example, I have a solid LZMA2 archive (approx. 2.5 GB) with about 8200 files. I have chosen a file from the middle of the archive (index 4100), file size about 30 KB:

• 7-Zip File Manager: approx. 9 seconds total extraction time with almost constantly 100% CPU usage.
• Delphi/Java + DLL: approx. 50 seconds total extraction time with varying CPU usage between 20% and 100% (but most time around 25%).

It is difficult to see how the number of threads changes, but I think the 7-Zip File Manager definitely uses more threads (number increases noticeably).

I have tried both Delphi + DLL and Java + DLL. The result is very similar. I have noticed that both (Java + Delphi) use IArchiveExtractCallback.GetStream(), but the 7-Zip File Manager obviously uses another interface.

With non-solid archives, the extraction performance is notably better, but of course the size of the archive is higher.

[ip7z]

Check that 7z.dll files are identical in both cases.

[ip7z]

Check that you use 64-bit versions of 7z.dll in both cases.

Create one big a.tar archive containing text files or executable files like big chrome.
Compress a.tar to a.7z with lzma2 / Fast compression level.
a.7z must be smaller than a.tar.
Decompress a.tar with both programs. And compare extraction time.

[Pique7]

Check that you use 64-bit versions of 7z.dll in both cases.

I am currently not able to test the 64-bit-DLL with Delphi, because I have only 32-bit Delphi, but obviously also the 32-bit version of 7-Zip File Manager extracts single files much slower than the 64-bit version. I didn't expect that there would be such a big difference whether you use the 32-bit or 64-bit version (you have to use a 64-bit version of 7-Zip v19.00 or higher).

I have also tested two Java libraries and the extraction speed of single files is very similar to the 32-bit version speed, so I initially assumed the same cause (I think one library has its own implementation and the other library uses an older version of 7-Zip).

[ip7z]

7-zip needs pretty big memory buffers for multithreaded decompression. And 32-bit version has no more than 2 GB.
Also 64-bit version uses asm version of LZMA decoder, that is 30-50% faster for single thread.