Adjustments to bug reports, enable collecting bug reports for all proofs #2047
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rv-jenkins
added a commit
to runtimeverification/pyk
that referenced
this pull request
Sep 6, 2023
In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
ehildenb
requested review from
nwatson22 and
tothtamas28
and removed request for
nwatson22
|
Note that this works as-is, but until runtimeverification/pyk#637 is merged into KEVM, this race condition will exist. Rerunning the tests usually fixes it, because on second run the directories already exist. |
tothtamas28
approved these changes
Sep 7, 2023
Baltoli
pushed a commit
to runtimeverification/k
that referenced
this pull request
Apr 9, 2024
In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
Baltoli
pushed a commit
to runtimeverification/k
that referenced
this pull request
Apr 9, 2024
…tion/pyk#637) In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
Baltoli
pushed a commit
to runtimeverification/k
that referenced
this pull request
Apr 9, 2024
…tion/pyk#637) In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
Baltoli
pushed a commit
to runtimeverification/k
that referenced
this pull request
Apr 10, 2024
…tion/pyk#637) In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
Baltoli
pushed a commit
to runtimeverification/k
that referenced
this pull request
Apr 10, 2024
…tion/pyk#637) In runtimeverification/evm-semantics#2047, we have a minor race condition when creating bug reports where one thread will check if the directory to create the report in exists, see that it doesn't and then create it. Meanwhile, another thread will already have created the directory, so it the first thread will fail because now the directory does exist. This allows for that case. --------- Co-authored-by: devops <[email protected]> Co-authored-by: rv-jenkins <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Blocked on: #2034Blocked on: runtimeverification/pyk#623Blocked on: #2041This makes some adjustments to how bug report generation in handled in KEVM:
--bug-report PATHoption is used from upstream, where you actually specify the path to dump the bug report. The path can contain directory structure, which will be created.exec_*functions in__main__.pynow accept the fully createdBugReport, which is produced by the--bug-report PATHoption, rather than just abool.BugReportfrom the testing fixtures when calling these functions, so that you can do--bug-report --bug-report-dir SOME_DIRargument topytest, and it will generate bug reports for all tests.To generate bug reports for ALL proof tests, you can run:
This will produce a folder in KEVM at
kevm-pyk/brs, with subdirectory structure:kevm-pyk/brs/prove/test_pyk_prove: Bug reports of proofs run with RPC prover.kevm-pyk/brs/prove/test_legacy_prove: Ignore.kevm-pyk/brs/prove-booster/test_pyk_prove: Bug reports of proofs run with the RPC prover with the booster turned on.kevm-pyk/brs/prove-booster/test_legacy_prove: Ignore.kevm-pyk/brs/foundry/: Bug reports of Foundry proofs run with the RPC prover.kevm-pyk/brs/foundry-booster/: Bug reports of Foundry proofs run with the RPC prover with the booster turned on.