Skip to content

Maybe support use within ephemeral environments #38

@PeterJCLaw

Description

@PeterJCLaw

Currently the summary of usage for this package looks like:

(prod)$ python manage.py devdata_export devdata
(prod)$ tar -czf devdata.tar devdata/
(local)$ scp prod:~/devdata.tar devdata.tar.gz
(local)$ tar -xzf devdata.tar.gz
(local)$ python manage.py devdata_import devdata/

This requires that the user can manage the data shuffling themselves and in turn implicitly requires that both sides of the export & import exist for longer than the django command. For environments such as Kubernetes, where a pod may only exist while a command is running, this pattern is a little harder to use. It's definitely still possible though can require a bit more juggling, especially if trying to script the usage rather than use interactively.

I was wondering whether it would make sense for devdata to provide some more direct support for this sort of use-case.

Approaches I can think of which might help:

  • let devdata export to/import from a (compressed) tar file directly, removing the need to have tar in the target environment (as an implementation, this might e.g: use a local temporary directory rather than reading the archive for each file)
  • maybe allow for that archive to come from a Django storage backed location
  • and/or allow for the import/export folders to be universal paths

Universal paths could be used on their own, though given the number of files which devdata tends to create that's likely to be quite inefficient -- I suspect that upaths would probably be more useful as a path to an archive file rather than to a directory.

Open to other ideas too, including if there's an easy way to use devdata in kubernetes which I've missed (if so, perhaps we could document this too?)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions