Turn module into generic VCS plugin

[palkerecsenyi]

Closes #188

Closes #189

Closes inveniosoftware/invenio-oauthclient#353

---

Description

Changes this module into a new generic plugin for any Version Control System (VCS).

• Renamed module to invenio_vcs. This is open to discussion and I'm happy to choose another name or even keep the existing one if we decide the migration would be too difficult.

• Full description coming soon

This is a WIP and not all features are implemented yet. In particular:

• GitLab contrib. This is a priority as it's needed to test a lot of the other features (e.g. auth). It's very difficult to test e.g. OAuth without it.
• OAuth user ID correlation
  • i.e. if the VCS provider uses the same OAuth server to authenticate the user as the Invenio instance, we should check the user IDs to make sure they match. This is useful for CDS-RDM where users will be able to link CERN GitLab, which uses the same CERN SSO.
  • We could express this through a more versatile hook function that returns whether/not we should accept the authenticated user.
  • Update: This can be done relatively easily by configuring a custom info_serializer handler in invenio.cfg. See the example for CDS: feat(vcs): support for new VCS integration CERNDocumentServer/cds-rdm#554
• Sync VCS repositories straight into the vcs_repositories table instead of the OAuth remote user extra_data field.
  • This will make querying a lot easier so we can paginate/search on the repository list page, which is currently very slow for users on e.g. GitLab instances where they have membership of thousands of repos due to group membership.
• Check duplication for organisational/team repos if multiple people activate them
  • What happens if a user is deleted? How can we transfer the repos?
• Repo name should not be unique individually. It is unique as a tuple of (provider_id,provider,name)
• UI bug with menu not being able to differentiate between multiple dynamically-registered entries
  • For example:
• Unit tests
• Documentation
• Migration script and guide
• Careful testing of DB migration for existing GitHub repos/releases
• Some UI pages have not been adapted and continue to throw errors
• JSONB extra_data in oauthclient
  • See perf(models): change extra_data to JSONB invenio-oauthclient#360

Upcoming features

We will merge this PR when:

• it provides an end-user experience equivalent to the current GitHub integration, preferably as similar as possible.
  • This has some scalability issues but for now we will avoid changing too much
• a clear migration script and guide are available and have been thoroughly tested with existing Zenodo data
• unit tests have been updated
• as many bugs fixed as possible

However, the following features will only be implemented in a future PR:

• Refresh token support
  • In the existing GitHub impl we use access tokens which are non-expiring by default. This is a security issue in case of a database leak and is recommended against by RFC 6749.
  • A PR exists (OAuth2 Token refresh implemented invenio-oauthclient#328) but needs some more work (last commit May 2024)
• Support for private repositories
• "Link-only" OAuth without an option to "sign in with" a remote
• React + API-based UI for pagination/search of repos, using OpenSearch
  • See GitLab integration product-rdm#235 (reply in thread) for details\
• Notifications on successful archive GitLab integration product-rdm#226 (comment)
• Selecting community to directly publish the repo to (especially on community-mandatory instances): GitLab integration product-rdm#235 (comment)
• Propagate permissions so users who have access to a repo also have access to records created from releases
  • GitLab integration product-rdm#235 (comment)

[palkerecsenyi]

This PR has been split into multiple smaller and easier-to-review chunks. Please see #188 for details.