Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvements to instruction testing #128

Open
mininny opened this issue Jan 17, 2025 · 0 comments
Open

Improvements to instruction testing #128

mininny opened this issue Jan 17, 2025 · 0 comments

Comments

@mininny
Copy link
Collaborator

mininny commented Jan 17, 2025

Currently, Asterisc has variety of fuzzing tests and instruction tests.
However, they fail to capture all of the instruction encoding which might arise in production environment. While the whole
combination of encoding parameters are too large, we should at least aim to validate the instruction encoding used in Kona.

For example, the following test executes each instruction in Kona against a VM with zero values in all registers and memory:

func TestAllInstructions(t *testing.T) {

	testSuiteELF, err := elf.Open("../../../../kona/target/riscv64gc-unknown-none-elf/release-client-lto/kona")
	require.NoError(t, err)
	defer testSuiteELF.Close()

	originalVmState, err := fast.LoadELF(testSuiteELF)
	require.NoError(t, err, "must load test suite ELF binary")

	contracts := testContracts(t)
	addrs := testAddrs
	env := newEVMEnv(t, contracts, addrs)

	for _, prog := range testSuiteELF.Progs {
		
		// ignore non-executable sections
		if (prog.Flags & 1) == 0 {
			continue
		}
		
		for i := uint64(0); i < uint64(prog.Memsz); i+=4 {

			// start with fresh VM and copy 1 instruction from program to memory
			// this way we avoid instruction being written over by one another
			vmState := fast.VMState {
				PC:        0,
				Memory:    fast.NewMemory(),
				Registers: [32]uint64{},
				ExitCode:  0,
				Exited:    false,
				Heap: 0x7f_00_00_00_00_00,
			}
			instr := make([]byte, 4)
			
			originalVmState.Memory.GetUnaligned(prog.Vaddr + i, instr)
			vmState.Memory.SetUnaligned(0, instr)

			instState := fast.NewInstrumentedState(&vmState, nil, nil, nil)
	
			wit, err := instState.Step(true)
			require.NoError(t, err)
			
			fastPostState := vmState.EncodeWitness()
			fastRoot, err := fastPostState.StateHash()
			require.NoError(t, err)
		
			// Now run the same in slow mode
			input, err := wit.EncodeStepInput(fast.LocalContext{})
			require.NoError(t, err)
			slowRoot, err := slow.Step(input, nil)
			require.NoErrorf(t, err, "slow VM err at PC %08x: %v", vmState.PC, err)
			if slowRoot != fastRoot {
				t.Fatalf("slow state %x must match fast state %x", slowRoot, fastRoot)
			}

			// Now run the same in EVM implementation
			_, evmPostHash, _ := stepEVM(t, env, wit, addrs, i, nil)

			if evmPostHash != fastRoot {
				t.Fatalf("evm state must match fast state \nfast:%x\nevm: %x\n", evmPostHash, fastRoot)
			}
		}
	} 
}

While this can't guarantee correctness or equivalency for all initial states, it can guarantee there is at least one "happy path" that is equivalent in the three implementations. Which also means at very least all three implementations decode the instruction in the same way, without raising exceptions.

Describe the solution you'd like
We should consider adding tests that validate all instruction encoding contained in the Kona binary in differential tests by applying them to each of the three (slow/fast/sol) implementations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant