HubApi snapshot doesn't check existing files list in offline mode

[nezhyborets]

Hi guys.

It looks like in HubApi method

func snapshot(from repo: Repo, revision: String = "main", matching globs: [String] = [], progressHandler: @escaping (Progress) -> Void = { _ in })
        async throws -> URL { ... }

specifically in it's "offline mode" part - the code does check integrity of existing files, but doesn't check if the list of existing files is "complete".

This leads to the issue when if for some reason a file went missing from a model directory - offline load of the model would still finish successfully, even though I believe it should return error in such case, because a file could be some very important file, like *.safetensors.

I could make a PR to fix this. The method could, after successful online download, remember filenames returned by try await getFilenames(from: repo, revision: revision, matching: globs) by putting them to some ./.another_metadata file. Then during offline load, if actual files in folder don't have any of remembered filenames - such offline load would be considered failed, so that the user could fallback to online load or at least understand that there is an issue.

[pcuenca]

Thanks for the report @nezhyborets! This looks a bit like an edge case to me, where preventing it might introduce considerable complexity. May I ask if you have experienced this problem in the wild, and what were the circumstances that led to it? I think offline mode is mainly used by iPhone apps, where access to the Hub downloads folder is essentially off-limits to third-party apps. Of course, I could be missing important use cases, so any details you can share would be helpful.

If we decide to go forward with this, I would also recommend we take a look at the strategies used by the Hugging Face Hub API. We attempt to model our APIs after them, when we can, to leverage their additional experience on the topic and provide a somewhat consistent experience to users.

[nezhyborets]

Hi @pcuenca , thanks for the quick reply!

Looks like python lib also doesn't check if all the required files are there

But we do experience the issues in our app 😔

For the context, we've implemented downloading of a model like this pseudo code:

func getModel() -> URL {
    if availableOffline {
        Task {
            updateOnlineInBackground() // doesn't block the app from moving forward
        }

        return modelDir
    } else {
        return try await downloadOnline()
    }
}

We did this because relying on online mode didn't work good for us, it would re-download some files from hub even though they haven't been changed, because Hub API backend would return a wrong etag.

But even if it'd work - online integrity checks still take time, which we wanted to avoid.

Eventually - we just want to app to initialize as quick as possible if there are model files available locally on device, even if they're outdated, but not if they're partially missing.

[nezhyborets]

By the way the easiest way to reproduce the issue seems to be to terminate the app during online load. On the next run the model dir would be incomplete and offline-mode load would "incorrectly" (by our means) be successful.