Additional explanations in "Cannot add zim X to the library" error

[bwenrich]

The kiwix-manage add command is difficult to debug when failures happen. Multiple different scenarios produce the same error message, so it is unclear what is the problem.

I understand that there are multiple reasons why the add command might fail, ex:

• The Library XML does not exist
• The Library XML is not writable due to incorrect permissions
• The Library XML is not writable due to read-only filesystem
• The ZIM file does not exist
• The ZIM file is not readable due to incorrect permissions
• The ZIM file is corrupt
• (Other reasons I have not considered here)

Steps to reproduce

# Step 1: Create the ZIM file which is not readable to the kiwix user (UID=808)
/data $ ls -l /data/content/ifixit_en_all_2022-06.zim*
-rw-------    1 root     root     2717836715 Jun 18 10:24 /data/content/ifixit_en_all_2022-06.zim
-rw-------    1 root     root            92 Aug 19 03:19 /data/content/ifixit_en_all_2022-06.zim.sha256
/data $ id
uid=808 gid=808
# Step 2a: Attempt to add the unreadable ZIM
/data $ kiwix-manage /data/library/multi.zim.xml add /data/content/ifixit_en_all_2022-06.zim
Cannot add zim /data/content/ifixit_en_all_2022-06.zim to the library.
# Step 2b: Attempt to add a non-existent ZIM
/data $ kiwix-manage /data/library/multi.zim.xml add /data/content/not_existing.zim
Cannot add zim /data/content/not_existing.zim to the library.
# Step 2c: Attempt to add a corrupt or invalid format ZIM
/data $ dd if=/dev/urandom of=/data/content/corrupt.zim bs=1M count=10
/data $ kiwix-manage /data/library/multi.zim.xml add /data/content/corrupt.zim
Cannot add zim /data/content/corrupt.zim to the library.
# Step 3: Observe that the failures in 2a, 2b, and 2c produce the same error message, but have different causes.

Actual Behavior

The error message "Cannot add zim to the library." appears with no additional detail.

Expected Behavior

The error message "Cannot add zim to the library." appears with additional detail about why the failure happened. If it is not possible to detect the reasons for the failure, perhaps guidance about common issues could be displayed.

Environment

kiwix-serve:latest in Docker/k8s. /data volume is an NFS mount

kiwix-tools 3.3.0
libkiwix 11.0.0
libzim 8.0.0

[kelson42]

Sounds a good idea indeed.

[SauravChittal]

Hello,
I wanted to work on this specific issue and while I understand what the issue itself is, I'm a bit more confused about the method of solving the issue. I understand that we need to check whether the ZIM file or the XML file has certain permissions.

However, understanding what is causing the specific issue requires manager.addBookFromPathAndGetId function to return the empty string, which because the class is abstracted away from this file, I'm unable to check which specific if statement has failed.

It will be possible to check for all of the pointers presented above on why the function might fail, by trying to call the file using FILE pointers, however, I was wondering if that is the recommended method, since it doesn't truly account for other potential reasons why that might occur.

Thank you so much :)

[kelson42]

@veloman-yunkan maybe you can help here? I would do a function which checks all what should be checked given a file path and call it only if the addBookFromPath() fails so the user gets the appropriate hint.

[SauravChittal]

Would it be fine if I made such a function, and the made a pull request based on how I think you would check for the issue?

[mgautierfr]

I would make addBookFromPathAndGetId throwing a exception if it cannot add it and put in the exception (as a message string or as a specific subclass) the cause of the error.

Then, the client kiwix-manage would have to catch the exception and print the information to the user.

[veloman-yunkan]

@veloman-yunkan maybe you can help here? I would do a function which checks all what should be checked given a file path and call it only if the addBookFromPath() fails so the user gets the appropriate hint.

@kelson42 I can only concur with @mgautierfr's comment above.

[kelson42]

@veloman-yunkan @mgautierfr OK!