Stop using fork for erasure coded reads

[Kimahriman]

If rust-rse/reed-solomon-erasure#108 gets merged, we can use the library directly instead of relying on a fork for custom matrices

[Xuanwo]

Hi, it seems like https://github.com/rust-rse/reed-solomon-erasure has been inactive for a year. Is there any chance we can find a replacement or alternative? We are interested in using hdfs-native, but this issue is causing a problem for us.

[Kimahriman]

I briefly looked at https://gitlab.com/kschibli/isa-l-rs (since Hadoop already integrates with isa-l) but that was more complex to implement and hasn't been touched in over three years. Also considered just trying to follow the Java implementation and roll my own in Rust, but once I figured out the reed-solomon-erasure worked as long as I could use a custom matrix I didn't spend much more time looking for alternatives. Especially since you can use a patch to use the fork even in downstream crates:

[patch.crates-io]
reed-solomon-erasure = { git = "https://github.com/Kimahriman/reed-solomon-erasure.git", branch = "SNB/23C24_external_matrix" }

[Xuanwo]

Using patch is not ideal because it will prevent us from uploading to crates.io.

[Kimahriman]

Yeah you need to put it behind a feature which is unfortunate. I'm open to suggestions or contributions if you have other ideas!

[Kimahriman]

I started working on a custom implementation now that I've started to wrap my head around how reed-solomon works. Hoping I might have something working in a few weeks 🤞