Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too many discards is now also a problem in conformance #4856

Open
lehins opened this issue Jan 24, 2025 · 3 comments
Open

Too many discards is now also a problem in conformance #4856

lehins opened this issue Jan 24, 2025 · 3 comments

Comments

@lehins
Copy link
Collaborator

lehins commented Jan 24, 2025

Today on CI I ran into this failure:

Failures:

  test/Test/Cardano/Ledger/Conformance/Spec/Conway.hs:27:7: 
  1) Conway.Conformance, Ticks transition graph, RATIFY
       *** Gave up! Passed only 98 tests; 1000 discarded tests:
        0% None
       
       83% CC yes votes ratio  	=0%
        8% CC yes votes ratio  	0%-20%
        1% CC yes votes ratio  	=100%
       
       85% DRep yes votes ratio	=0%
        4% DRep yes votes ratio	=100%
        3% DRep yes votes ratio	0%-20%
       
       53% SPO yes votes ratio 	=0%
       38% SPO yes votes ratio 	0%-20%
        1% SPO yes votes ratio 	40%-60%
       
        3% Accepted at least one NoConfidence

  To rerun use: --match "/Conway/Conformance/Ticks transition graph/RATIFY/" --seed 457525973

This needs fixing otherwise it is gonna nag us forever as the old tracing testing system still does occasionally

@TimSheard
Copy link
Contributor

Does this men we are not generating enough input with enough votes that actually pass?

@lehins
Copy link
Collaborator Author

lehins commented Jan 24, 2025

From the looks of it is that during generation of random data we are doing too many discards and looks like a 1000 discards is a limit.

Maybe @Soupstraw has a better insight into what is actually going on.

@Soupstraw
Copy link
Contributor

Soupstraw commented Jan 27, 2025

I don't know for sure, I haven't changed anything in RATIFY generators. I suspect it might have something to do with the exists constraints that I used to generate more valid votes.

Do we know when these kinds of failures started to appear?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants