Skip to content

Update reporting.R#2320

Open
johannah-pik wants to merge 9 commits intoremindmodel:developfrom
johannah-pik:cleanVariableHarmonization
Open

Update reporting.R#2320
johannah-pik wants to merge 9 commits intoremindmodel:developfrom
johannah-pik:cleanVariableHarmonization

Conversation

@johannah-pik
Copy link
Copy Markdown
Contributor

@johannah-pik johannah-pik commented Mar 25, 2026

Purpose of this PR

Include REMIND/EDGE-T variable harmonization, which is described in detail here:
pik-piam/reporttransport#21
Please explain your PR here.
Change edgeTransport reporting calls and include ratio of "FE|Transport" (with bunkers) EDGE-T to REMIND before harmonization to the REMIND.mif as an indicator

Type of change

Indicate the items relevant for your PR by replacing ◻️ with ☑️.
Do not delete any lines. This makes it easier to understand which areas are affected by your changes and which are not.

Parts concerned

  • ◻️ GAMS Code
  • ☑️.\ R-scripts
  • ◻️ Documentation (GAMS incode documentation, comments, tutorials)
  • ◻️ Input data / CES parameters
  • ◻️ Tests, CI/CD (continuous integration/deployment)
  • ◻️ Configuration (switches in main.gms, default.cfg, and scenario_config*.csv files)
  • ◻️ Other (please give a description)

Impact

  • ◻️ Bug fix
  • ◻️ Refactoring
  • ☑️.\ New feature
  • ◻️ Change of parameter values or input data (including CES parameters)
  • ◻️ Minor change (default scenarios show only small differences)
  • ◻️ Fundamental change of results of default scenarios

Checklist

Do not delete any line. Leave unfinished elements unchecked so others know how far along you are.
In the end all checkboxes must be ticked before you can merge
.

  • I executed the automated model tests (make test) after my final commit and all tests pass (FAIL 0)
  • I adjusted the reporting in remind2 if and where it was needed
  • I adjusted the madrat packages (mrremind and other packages involved) for input data generation if and where it was needed
  • My code follows the coding etiquette
  • I explained my changes within the PR, particularly in hard-to-understand areas
  • I checked that the in-code documentation is up-to-date
  • I adjusted forbiddenColumnNames in readCheckScenarioConfig.R in case the PR leads to deprecated switches
  • I updated the CHANGELOG.md correctly (added, changed, fixed, removed, input data/calibration)

Further information (optional)

  • Runs with these changes are here:
  • Comparison of results (what changes by this PR?):

@johannah-pik johannah-pik requested a review from tonnrueter March 30, 2026 17:52
@johannah-pik
Copy link
Copy Markdown
Contributor Author

@LaviniaBaumstark @tonnrueter do you consider it as necessary that I "executed the automated model tests (make test) after my final commit" for this change?
I did not use that for a while - how does that work/ where can I find more information about it?

# generate transport extended mif
reporttransport::reportEdgeTransport(edgetOutputDir,
EDGEToutputPriorHarmonization <-reporttransport::reportEdgeTransport(edgetOutputDir,
isTransportExtendedReported = TRUE,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @johannah-pik - I find the "double"-reporting pragmatic, and I don't see a more runtime-efficient way to do this. One thing that would improve this a little bit would be not to do the extended reporting, but only isTransportReported.

Copy link
Copy Markdown
Contributor Author

@johannah-pik johannah-pik Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jmuessel, thanks for your feedback :) I am also in favor of keeping the double reporting and the detailed transport data to render our transport compScen.
The reporting itself is not particularly computationally intensive (for what you get).
I meant the double loading of the giant REMIND.mif file for comparison (I am marking that in the code)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Then I think we can keep it as is. Thx again!

mifs <- list.files(".", recursive = FALSE, full.names = TRUE)
mif <- mifs[grepl(".*withoutPlus\\.mif", mifs)]
#Select matching variables
REMINDvars <- as.data.table(read.quitte(mif))
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are loading the REMIND.mif just to calculate one additional variable

scenarioName = scenario,
gdxPath = file.path(outputdir, "fulldata.gdx"),
isStored = FALSE,
isHarmonized = TRUE)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we are loading it as well to check the harmonization.
I did that before the request of the additional variable.
Maybe putting the test into a seperate function and calling it in reporting.R would be more elegant.
Anyway - that can be refined later on :)

@robertpietzcker
Copy link
Copy Markdown
Contributor

@LaviniaBaumstark @tonnrueter do you consider it as necessary that I "executed the automated model tests (make test) after my final commit" for this change?
I did not use that for a while - how does that work/ where can I find more information about it?

you clone your REMIND branch with all your changes into a folder on the cluster, and then simply type make test into the shell. and then wait for 30-40 minutes until you get a "fail 0" output :-)

Copy link
Copy Markdown
Contributor

@robertpietzcker robertpietzcker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't comment on the code, but if it works, thanks a lot for implementing this!

@LaviniaBaumstark
Copy link
Copy Markdown
Member

yes, make test is mandatrory for all PRs in the develop branch and afterewards you have to tick the box

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants