My Test for Stateful Fuzzing isn't catching the invariant

[BlackAnon22]

StatefulFuzzCatches.sol

// SPDX-License-Identifier: MIT
pragma solidity 0.8.20;

// INVARIANT: doMoreMathAgain should never return 0
contract StatefulFuzzCatches {
    uint256 public myValue = 1;
    uint256 public storedValue = 100;

    /*
     * @dev Should never return 0
     */
    function doMoreMathAgain(uint128 myNumber) public returns (uint256) {
        uint256 response = (uint256(myNumber) / 1) + myValue;
        storedValue = response;
        return response;
    }

    function changeValue(uint256 newValue) public {
        myValue = newValue;
    }
}

StatefulFuzzCatches.t.sol

// SPDX-License-Identifier: MIT

pragma solidity 0.8.20;

import {Test} from "forge-std/Test.sol";
import {StdInvariant} from "forge-std/StdInvariant.sol";
import {StatefulFuzzCatches} from "../../src/invariant-break/StatefulFuzzCatches.sol";

contract StatefulFuzzCatchTest is StdInvariant, Test {
    StatefulFuzzCatches sfc;

    function setUp() public {
        sfc = new StatefulFuzzCatches();
        targetContract(address(sfc));
    }

    function statefulFuzz_catchesInvariant() public view {
        assert(sfc.storedValue() != 0);
    }

}

The output I get after compiling the test

┌──(bl4ck4non💀bl4ck4non-sec)-[~/Documents/web3/sec_audit/sc-exploits-minimized]
└─$ forge test --mt statefulFuzz_catchesInvariant -vvv
[⠊] Compiling...
No files changed, compilation skipped

Ran 1 test for test/invariant-break/StatefulFuzzCatches.t.sol:StatefulFuzzCatchTest
[PASS] statefulFuzz_catchesInvariant() (runs: 64, calls: 2048, reverts: 30)
Suite result: ok. 1 passed; 0 failed; 0 skipped; finished in 170.04ms (168.93ms CPU time)

Ran 1 test suite in 170.92ms (170.04ms CPU time): 1 tests passed, 0 failed, 0 skipped (1 total tests)

I don't know where the issue is from

[EngrPips]

Hello @BlackAnon22, what is the issue?

[BlackAnon22]

In the TSwap Section of the course, Patrick wrote the StatefulFuzzCatches.sol file which contains an invariant, but then when he wrote his test script StatefulFuzzCatches.t.sol which was similar to what I wrote (sent the test script in the previous discussion) he was able to break the invariant while I wasn't.

So I want to know why I couldn't break the invariant even when I had a similar script and even set fail_on_revert to false

[EngrPips]

Have you tried increasing your number of runs and confirmed that you are using the same seed as Patrick for the runs?

[BlackAnon22]

Yes I have

This is the link to his test script

[EngrPips]

If you are running the exact set-up as him then I expect that you get the same result

[BlackAnon22]

I'm not actually, that's why I sent the script and also my output earlier

[EngrPips]

One suggestion I will give is that you should run forge clean then delete your cache folder if you have any and then run foundryup and then retry the test with the new seed I suggested. Let's see if that helps.

[BlackAnon22]

Alright, I'll try that

[EngrPips]

Yeah, Try that. Let's see.

[BlackAnon22]

I tried doing this on 2 different PCs, it didn't work still

[EngrPips]

Thats tough. I am unsure what could be wrong. I won't let that block me anyway I will just continue with the course and revisit it later.

[djz-kagen]

I ran into this today where the stateful fuzz test was passing. I know it was mentioned above with increasing runs but that is the answer. Either updates to foundry or my platform (WSL) causes the randomness to "change" even with the same seed as Patrick.

Update number of runs in foundry.toml for invariant.

[invariant]
- runs = 64
+ runs = 1000
depth = 32
fail_on_revert = false

It catches it on run 367 so 64 runs would miss it.

Encountered 1 failing test in test/invariant-break/StatefulFuzzCatchesTest.t.sol:StatefulFuzzCatchesTest
[FAIL: panic: assertion failed (0x01)]
        [Sequence]
                sender=0x0000000000000000000000000000000000000768 addr=[src/invariant-break/StatefulFuzzCatches.sol:StatefulFuzzCatches]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=changeValue(uint256) args=[0]
                sender=0xE7FFbC990cBd9C9d22c89fd5Ab74f34D8828eDd5 addr=[src/invariant-break/StatefulFuzzCatches.sol:StatefulFuzzCatches]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=doMoreMathAgain(uint128) args=[0]
 statefulFuzz_catchesInvariant() (runs: 367, calls: 11744, reverts: 159)

[EngrPips]

Interesting, Thanks for this heads-up @djz-kagen

[BlackAnon22]

Ran 1 test suite in 972.71ms (967.39ms CPU time): 0 tests passed, 1 failed, 0 skipped (1 total tests)

Failing tests:
Encountered 1 failing test in test/invariant-break/StatefulFuzzCatches.t.sol:StatefulFuzzCatchesTest
[FAIL: panic: assertion failed (0x01)]
        [Sequence]
                sender=0x000000000000000000000000000000000000085f addr=[src/invariant-break/StatefulFuzzCatches.sol:StatefulFuzzCatches]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=changeValue(uint256) args=[0]
                sender=0x000000000000000000000000000000000000162d addr=[src/invariant-break/StatefulFuzzCatches.sol:StatefulFuzzCatches]0x5615dEB798BB3E4dFa0139dFa1b3D433Cc23b72f calldata=doMoreMathAgain(uint128) args=[0]
 statefulFuzz_testMathDoesntReturnZero() (runs: 378, calls: 12096, reverts: 172)

Encountered a total of 1 failing tests, 0 tests succeeded

This worked for me, thank you @EngrPips @djz-kagen for your help

[EngrPips]

Awesome, keep crushing it buddy.