update data sources for country-level electricity demand #1828
Conversation
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Pull Request Overview
This PR updates the data sources for country-level electricity demand by expanding from a single OPSD source to multiple sources including ENTSOE and NESO. The changes implement a hierarchical data retrieval strategy where ENTSOE data takes priority, followed by OPSD, then NESO for UK-specific data.
- Splits electricity demand retrieval into three separate rules for different data sources (OPSD, ENTSOE, NESO)
- Updates the build process to combine data from multiple sources with priority handling
- Adds extensive manual adjustments and data quality fixes for various countries and time periods
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/retrieve_electricity_demand_opsd.py | Updates mock_snakemake call to match new rule name |
| scripts/retrieve_electricity_demand_neso.py | New script to retrieve UK electricity demand from NESO API |
| scripts/retrieve_electricity_demand_entsoe.py | New script to retrieve European electricity demand from ENTSOE |
| scripts/build_electricity_demand.py | Major refactor to handle multiple data sources and extensive manual adjustments |
| rules/retrieve.smk | Splits single retrieval rule into three separate rules for different sources |
| rules/build_electricity.smk | Updates input parameters to reference new data source files |
| doc/retrieve.rst | Updates documentation to reflect new retrieval rules |
| doc/data-retrieval.rst | Adds documentation for new data sources |
| doc/configtables/toplevel.csv | Adds ENTSOE token configuration option |
| config/config.default.yaml | Adds secrets section and increases interpolate_limit |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: Copilot <[email protected]>
|
|
||
| - **Source:** ENTSO-E Transparency Platform | ||
| - **Link:** https://transparency.entsoe.eu/load-domain/r2/totalLoadR2/show | ||
| - **License:** CC-BY 4.0 (cf. https://transparency.entsoe.eu/content/static_content/download?path=/Static%20content/terms%20and%20conditions/220218_List_of_Data_available_for_reuse.pdf) |
There was a problem hiding this comment.
Note that the actual total load is not listed under the data available on a CC-BY 4.0 license.
Updated link to ENTSO-E terms and conditions: https://transparencyplatform.zendesk.com/hc/en-us/articles/40921911218961-Legal-Terms-and-Conditions
Issue originally raised here.
Uh oh!
There was an error while loading. Please reload this page.