[v2] Can sparse checkout be used for syncing with a repository

[jarviliam]

Hey Flipt team 👋
Thanks for your work on v2!
Regarding managing feature flags via gitops, in our case we would want to host the feature flags in the same repository as the code but the codebase is quite large. Is it possible to configure flipt to do only a sparse checkout on the flags path itself instead of cloning the whole repository?

[dosubot[bot]]

Hi @jarviliam! I'm Dosu and I’m helping the flipt team.

Flipt v2 does not currently support Git sparse checkout for syncing only a subdirectory or path from your repository. When configured for GitOps, Flipt will clone the entire repository and then scan for feature flag files within the specified directory after the clone. There’s no built-in option to limit the clone or sync operation to just a subdirectory using native sparse checkout features. You can, however, configure Flipt to only look for flags in a specific path after cloning the full repo, but this won’t reduce disk usage or clone time for large repositories source.

If you’re interested in this capability, you might want to track feature requests or discussions for future updates, but as of now, sparse checkout isn’t supported in Flipt’s Git integration source.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[erka]

Hey @jarviliam

It’s an interesting idea. Thank you for bringing it up. We’ll need to do some research to see if it’s possible with go-git.

[GeorgeMac]

Hey @jarviliam (and @erka) 👋 (old contributor crawling out of cave)

So the subtle bit here is there difference between sparse checkout and partial clone.
The sparse checkout feature on its own is supported by go-git, however, checkout is about what gets exported from git object storage into your worktree (when talking about git CLI for example). However, on it's own, it has nothing to do with optimizing the fetch calls which get the repo git objects.

In fact, we don't even use the go-git worktree in Flipt, we skip that and the index and we go direct to git object storage.
So sadly sparse checkout wouldn't even be leverage-able or give any benefits.

Partial clone support, or rather pushing down --filter to the fetch (see: filter-spec) would allow us to e.g. do an initial clone without blobs and then fetch blobs by oid to download the minimal flag only data. That would be amazing.

I believe with git cli when you configured sparse checkout (and maybe --cone on the init) I think git figures out what filters to apply when fetching for you.

Flipt could do this, but so far, filter-spec support has only been implemented at the plumbing level of go-git and has not yet been exposed on git.FetchOptions (see PR for fetch at protocol level). There is more work to do in the client there before we can take advantage of it for this.