File cache without kernel cache

[natbprice]

It seems like a common use case for blobfuse2 would be to simply have files downloaded from blob storage when first accessed, cached for a fixed period for future access, and re-downloaded again if accessed after the cache time has expired. During this cache period the file may or may not have been updated externally, but this is acceptable for this hypothetical use case so long as file is not older than the specified cache time. This appears to be straightforward to configure using --file-cache-timeout option.

However, in my testing it seems like I also need to use --disable-kernel-cache or I will keep accessing the old file even after the specified file cache timeout. The default behavior includes kernel caching and this can lead to non-intuitive results if you are not aware of multiple caching layers and interactions. The option to disable kernel caching seems important, yet it doesn't appear on the landing page and is not included in the nice flowcharts on caching.

Somewhat related is the --direct-io option and I found some discussion of this related to resolving caching issues. However, this seems like an extreme option because it not only disables kernel caching, but also the blobfuse2 file caching. This seems like a less common use case, but it has more discussion than the option of simply disabling kernel cache.

Is my understanding correct that we must use --disable-kernel-cache to ensure blobfuse2 can manage the cache in accordance with the specified --file-cache-timeout?

Would it make sense to update the documentation to better explain the --disable-kernel-cache option?

By the way, I am using blobfuse with Azure Batch as configured using Azure Batch SDK (https://learn.microsoft.com/en-us/azure/batch/virtual-file-mount?tabs=windows). I am not sure if this might be causing any atypical behavior, but it does add another layer of abstraction making it harder to configure and debug issues.

[vibhansa-msft]

There are two levels of caching when you are using Blobfuse2 to mount your container.

• Linux kernel caches both the data and metadata of a file being operated on.
• Blobfuse level data and metadata caching

Kernel level metadata caching can be controlled using timeouts configured while mounting, but kernel data cache can only be turned on or off and not controlled with timeouts. Once kernel caches the data, it will continue serving it from its cache as long as the page cache is valid. Only way to clear it on demand is to wipe out entire kernel cache.
Blobfuse level cache can be controlled with timeouts (file-cache and attr-cache timeouts).

When 'direct-io' is used both kernel and blobfuse level data and metadata cache is disabled. This means you get refreshed contents of files you are using but at a cost of higher number of calls to storage container as every call may results in REST call to container.

'disable-kernel-cache' is another option which allows you to just disable the kernel level data and metadata caches while blobfuse level cache remains and user can configure the respective timeouts for them. This allows you to fine-tune refresh intervals based on your application expectations. If you wish to retrieve the refreshed contents of a file within some given interval, then set file-cache-timeout and attr-cache-timeout to that number (say 5 second for example) and disable the kernel cache using this CLI.

[vibhansa-msft]

this wiki page explains this.