Test Vectors Wanted

[colinoflynn]

Would be nice to have some (real) examples of this working on protocols. If you've got some simple test vectors please post them to this issue. You can see the demo.py example for how it should work.

NB - I'm very lazy, please don't post wireshark captures or something, only post either Python or "raw" hex (which can convert easily, and when I fixed #2 will "just work").

The tool requires at least two messages of the same length, it's not brute-forcing the per-message CRC. This is required for the differential to work.

[mliberty1]

I tried using Fitterbap comm frames, which use CRC-32 by default. The examples below omit the SOF1, SOF2, and EOF bytes, which are not included in the checksum.

Here's the code:

# Fitterbap example 1 - CRC32
# https://github.com/jetperch/fitterbap/blob/main/include/fitterbap/comm/framer.h

crcb.search(
    [[0x00, 0x01, 0x07, 0x5c, 0x02, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08],
     [0x00, 0x01, 0x04, 0xf2, 0x02, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05],
     [0x00, 0x01, 0x07, 0x5c, 0x02, 0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88],
     [0x00, 0x01, 0x0f, 0x6f, 0x02, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f, 0x10]],
    
    [[0xbf, 0x81, 0x0c, 0x54], 
     [0x29, 0xe7, 0x51, 0x72], 
     [0xb8, 0xe8, 0xde, 0xfa], 
     [0x8a, 0xee, 0x5c, 0x83]])

And here is the output from c7026ed:

Input parameters:
    32-bit CRC size
    4 total messages, with:
       2 messages with 14 byte payload
       1 messages with 11 byte payload
       1 messages with 22 byte payload

Working on messages of 14 length:
Traceback (most recent call last):
  File "C:\repos\third-party\crcbeagle\demo.py", line 9, in <module>
    crcb.search(
  File "C:\repos\third-party\crcbeagle\crcbeagle\crcbeagle.py", line 313, in search
    intersect = set.intersection(*diffsets)
TypeError: unbound method set.intersection() needs an argument

[colinoflynn]

Thanks - CRC32 wasn't tested yet, so was most likely to break ;-) I'll try it out.

[colinoflynn]

Alright a real bug that I failed to parse messages of different length not side-by-side (oops), second issue was around CRC init now fixed. This will pop out for fitterbap:

import struct
from crccheck.crc import Crc32Base
crc = Crc32Base
def my_crc(message):
  crc._poly = 0x4C11DB7
  crc._reflect_input = True
  crc._reflect_output = True
  crc._initvalue = 0x0
  crc._xor_output = 0xD1BB79C7
  output_int = crc.calc(message)
  output_bytes = struct.pack("<I", output_int)
  output_list = list(output_bytes)
  return (output_int, output_bytes, output_list)

Which looks like it's giving the 'reversed' representation of the constant you are using (but then doing reflect-in/out). The incorrect init value is currently just an artifact of the code, it doesn't look for generic solutions. Will probably update it at some point to try the 'normal' init values once it knows the polynomial, thanks for the great vectors!

[mliberty1]

Great! I confirm that it is working now. For what it's worth, the Fitterbap CRC-32 code was partially generated using pycrc:

python pycrc.py --model=crc-32 --algorithm=tbl --generate=c

You may also want to consider adding a link to Philip Koopman's CRC page in your README for anyone trying to select a "good" CRC (spoiler use CRC-32C by default):
https://users.ece.cmu.edu/~koopman/crc/index.html

All the variations of computing CRC make creating compatible software very annoying, so thanks for putting together this tool!