Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

Stress test failing, giving start year warning #326

Closed
Clare2D opened this issue Nov 9, 2020 · 11 comments
Closed

Stress test failing, giving start year warning #326

Clare2D opened this issue Nov 9, 2020 · 11 comments
Assignees
Labels
bug Something isn't working

Comments

@Clare2D
Copy link
Contributor

Clare2D commented Nov 9, 2020

Unknown error occurs with user_id 77 which seems to cause an alternative failure type

`-- running for: 77 (test #77) --------------------------------------------------

-- web_tool_script_1.R ---------------------------------------------------------

-- web_tool_script_2.R ---------------------------------------------------------
Loading PACTA.analysis
[1] "Calculate Stress Test for Equity Portfolio"
Error in check_portfolio_consistency(.) :
Start year of the analysis and first year in the input portfolio do not match.`

@Clare2D Clare2D added the bug Something isn't working label Nov 9, 2020
@jacobvjk
Copy link
Member

jacobvjk commented Nov 9, 2020

This error means that the first year in the portfolio passed to the stress test does not match the start year of the analysis. Without having seen the data, my best guess is that only data from some scenario that I am filtering out is given..

@jacobvjk
Copy link
Member

jacobvjk commented Nov 9, 2020

The issue comes from an almost empty portfolio file being passed to the stress test

Equity_results_portfolio.rda looks like this:
Screenshot 2020-11-09 at 09 08 44

If this is expected/desired behaviour on the PACTA_analysis side, I will add stricter checks when reading in the data and skip if necessary. However it could be good to think about whether this is a result file we want to produce.
Note: the Equity_result_map.rda is not created at all, which seems like the logical thing to do.

@Clare2D

@Clare2D
Copy link
Contributor Author

Clare2D commented Nov 9, 2020

Ah right this is the only output? Yes I guess we should see whether there's an alternative solution here.

@jacobvjk
Copy link
Member

jacobvjk commented Nov 9, 2020

yea, the output was a one liner in this case. Regardless, we should add checks both to the PACTA part and the stress test part that look at some minimum expected content in these files. We should think about what that is, might not be so straight forward. But I think that would be really helpful because it will lead to less errors seemingly coming from other parts of the scripts

@Clare2D
Copy link
Contributor Author

Clare2D commented Nov 18, 2020

Did we resolve this? I looked for a PR that closed this but didn't find one!

@jacobvjk
Copy link
Member

@Clare2D not explicitly..

(1) I think this requires a decision however on what would make sense to still pass to the stress test, i.e. a minimum required Pacta result as ST input. Which is not yet defined.

(2) The problem lined out here was still that an empty result was being written for Equity, which is a PACTA side issue. Especially the fact that it was only written for one of the two allocation methods. So I would suggest we rename the issue and/or make follow up issues.

@Clare2D
Copy link
Contributor Author

Clare2D commented Nov 19, 2020

I suggest we split this into its follow up issues :)

@jacobvjk
Copy link
Member

follow up for the stress test: https://github.com/2DegreesInvesting/StressTestingModelDev/issues/101

@jacobvjk
Copy link
Member

follow up for PACTA output checks: #337

@jacobvjk jacobvjk reopened this Dec 2, 2020
@jacobvjk
Copy link
Member

jacobvjk commented Dec 2, 2020

Due to the urgency, I will add an intermediate fix

@jacobvjk
Copy link
Member

jacobvjk commented Dec 2, 2020

I can't replicate this issue anymore. It seems like the missing PACTA data were an idiosyncrasy of an earlier, less stable version. When I rerun this user's analysis, I get expected PACTA results and stress test results without failure. Unless someone else can replicate the problem, I would close this issue again. @Clare2D

@cjyetman cjyetman closed this as completed Dec 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants