Support http range requests for static assets like pmtiles (Protomaps)

[do-me]

Hey,
I wanted to build an app with Protomaps background map tiles. It's a single large pmtiles file that the client accesses via http range requests to get only the data of the map extent and for the exact zoom level needed. More detailed info here: https://protomaps.com/intro

When setting up a minimal example like this one https://github.com/do-me/protomaps-example with tauri (e.g. running through the wizard with plain JS, no frontend framework, simply using the index.html and zurich.pmtiles file) I am getting this error:

Error: Server returned no content-length header or content-length exceeding request. Check that your storage backend supports HTTP Byte Serving.

Are range requests already supported or is there some kind of setting I could use to activate them?

This would also benefit httpfs with DuckDB or other methods that rely on this function.

I found #7870 pointing me to this reference in the docs: https://v1.tauri.app/v1/api/js/tauri/#functions. Would that be sufficient?

[do-me]

@bpevs had the same idea and got it working! https://github.com/inro-digital/tauri-offline-maps
Will analyze when I find the time.

[curry-battle]

I'm currently working on this!
But it doesn't seem to work properly on Android.

You can observe this behavior in the repository by @bpevs. (This Repository helped me a lot! )
https://github.com/inro-digital/tauri-offline-maps

1. cargo install tauri-cli
2. cargo run android init
3. cargo run android dev

After launching the Android Emulator and checking the logs via DevTools, I found that Range requests to pmtiles cause an error (this error does not occur on iOS).

'Range': 'bytes=0-16383' // this is OK (StatusCode is 206)
'Range': 'bytes=1697743-1998593' // this is error (No StatusCode returns. net::ERR_FAILED)

The code that handles the Content-Range header in asset protocol is here, but at least from the feature flags, there doesn’t seem to be an OS-dependent implementation, so it shouldn't be an issue:

tauri/crates/tauri/src/protocol/asset.rs

Lines 29 to 33 in ddc4693

	fn get_response(
	request: Request<Vec<u8>>,
	scope: &scope::fs::Scope,
	window_origin: &str,
	) -> Result<Response<Cow<'static, [u8]>>, Box<dyn std::error::Error>> {

Does anyone know why Range requests fail on Android?

[bpevs]

nooooooo I never ended up checking it on Android, because I have to complete a few other things before I try to implement this into my app properly (even getting as far as I did was more a "I prooobably shouldn't be working on this yet, but I can't help it"). I was intending to integrate this functionality into my Android tauri app later, but I was hoping it'd just translate cleanly. I guess not 😭.

Off the top of my head, not sure why this error would happen, but in the past with any cross-platform file storage issues, I've generally been frustrated by what parts of the filesystem are available, how they are accessed, and those inconsistencies creating weird issues (You're sure the files are completely downloaded, and BaseDirectory.AppData is usable?).

I'm still not quite at the point where I should be working on this, but if it's not solved by mid-march, that's probably around when I will start looking at this more deeply.

[curry-battle]

(You're sure the files are completely downloaded, and BaseDirectory.AppData is usable?).

I understand what you mean!
Putting the Content-Range issue aside, I have some reference code on hand that might be helpful, so I'll submit a PR soon.
(And I hope you’ll join me in facing the same issue😂)

Note: Just to share in advance, before the PR...

• It seems best to use the download method from tauri-apps/plugin-upload for saving files.
  • https://github.com/tauri-apps/tauri-plugin-upload
• Instead of BaseDirectory.AppData, you should use await appDataDir() to resolve the destination path.
  BaseDirectory.AppData just returns 14, while appDataDir() internally resolves the path.

  • tauri/packages/api/src/path.ts

    Line 33 in ddc4693

    AppData = 14,

  • tauri/packages/api/src/path.ts

    Lines 73 to 76 in ddc4693

    	async function appDataDir(): Promise<string> {
    	return invoke('plugin:path|resolve_directory', {
    	directory: BaseDirectory.AppData
    	})

[do-me]

By the way, I was able to get range requests running on iOS when dumping a pmtiles file in the public webapp directory (with svelte). I suppose it should work just as well with appdir too. Will share some minimal example the next days.

[curry-battle]

I've submitted a PR!
As mentioned above, since I uses tauri-plugin-upload in PR, it avoids the filesystem issue originally raised in the issue.

[bpevs]

OH right btw just to clarify (since I don't think anyone mentioned this specifically), but I AM currently using protomaps on Android with remotely-served files. So I doubt the issue is any strange browser compatibility things, and is more specifically related to the local files.

[curry-battle]

So I doubt the issue is any strange browser compatibility things, and is more specifically related to the local files.

Yes! When browsing PMTiles on the internet with Android, I can send requests without any problem too.

To check its behavior (again borrowing from bpevs repository 🙏 ), you can change the PMTiles references as follows.

https://github.com/inro-digital/tauri-offline-maps/blob/main/src-www/index.ts#L25-L37

 // Download and add Basemap
 map.on('load', async function () {
-  const url = await downloadAndSavePMTiles(
-    'https://static.bpev.me/maps/world.pmtiles',
-    'world.pmtiles',
-  )
   map.addSource('world', {
     type: 'vector',
-    url: `pmtiles://${url}`,
+    url: `pmtiles://https://static.bpev.me/maps/world.pmtiles`,
     attribution: "Natural Earth"
   })
   worldLayers.forEach(layer => map.addLayer(layer))
 })

[kartonrad]

@curry-battle i think the behaviour you're seeing on android is a result of a chromium bug,
i have written up a few things in the (unfortunately still closed) issue page: https://issues.chromium.org/issues/40739128#comment30

[kartonrad]

Hey, did you get the Range Requests working?

Or did you avoid the issue?

[kartonrad]

In the meantime, i think you must host on localhost and avoid the custom protocol.

[curry-battle]

To be honest… I had already given up on this use case and developing it with Tauri.
I truly hope this issue gets resolved for everyone.
(Sorry for the unhelpful comment.)

[do-me]

What are you using instead? Did you go native?

[curry-battle]

What are you using instead? Did you go native?

I ended up choosing native development (Swift for iOS and Kotlin for Android) instead.
That said, I have not actually started the implementation yet.

This may be slightly off-topic from the current issue, but in my use case, I simply needed offline map capability. In other words, I was trying to bundle pmtiles as Tauri assets and implement Range Requests to support that functionality.
With native development, I can straightforwardly use the offline features provided by Mapbox’s Mobile Maps SDKs, so I plan to explore that approach.

To digress further, my use case also required performing GPS operations in the background.
Implementing this in Tauri would have required building my own plugin, which is quite challenging.
Since iOS and Android frequently change permissions and platform-specific constraints, using those native features directly seems much simpler when working with Swift and Kotlin.😭

[kartonrad]

My android code now falls back to a localhost server written using hyper, which calls the same protocol handler.

Stillthat damn Webview issue needs to be fixed, so i really need to make another chromium issue