vs noodles and bedrs and rust-bio

[Kentalot]

Hi,

I was looking into bed support in my rust code and saw that this repo, noodles, rust-bio, and bedrs had bed support. I think the noodles and bedrs are aiming for bedtools-like abilities like this one. Have you looked at the other repos and found any big differences? What advantages and disadvantages do you think your repo has? Thanks a lot and I'll keep an eye on the project.

[brentp]

Hi,
noodles is quite different as it parses BED (and VCF and BAM/CRAM). bedder-rs uses noodles extensively. I think the same is true for rust-bio.

bedder-rs and bedrs are more similar, but as I understand it, bedrs uses in memory data structures and bedder uses streaming to do the intersections. I am very interested in bedrs and will follow as it develops.

Now, the disadvantage of both libs is that they lack functionality to operate on a range of formats. To now, bedder only supports bed and VCF (though BAM/CRAM are easily added). Instead, I am focusing on how everything will look for the final user, including python bindings to the lib.

[brentp]

oops. wrong branch: https://github.com/brentp/pyo3_ref/blob/noodles/src/lib.rs

[brentp]

out of curiosity, did you benchmark noodles recently? I thought performance had improved lately.

[Kentalot]

I don't know how recent is recently, but we did some tests that ran a vcf (read and then modify and write) a few times through code using noodles and parallel code using rust-htslib maybe a month ago. It's not a very thorough test but we just wanted to compare on a superficial level and we were leaning towards rust-htslib since htslib is well used and tested by the community. So if it's similar or even slightly slower, we'd go with that. Since our superficial results came out that rust-htslib was much faster, we decided to go with that in general. We haven't did a test since though. Maybe it's worth it to do it eventually, but for now, we will stick with one to finish the job first and we can always optimize later.

[brentp]

ok. that's recent. Maybe I'll re-consider as well.

[Kentalot]

you might want to do your own testing. I was looking over our code and the noodles code and htslib code is actually quite different so it is not necessarily exactly equivalent. I don't want to spend the time to make them more equivalent, but I'll assume that even if we optimized the noodles code more, the performance is at least similar, which means rust-htslib for us still for now.

[Kentalot]

Actually after some more thought, I think we do require the support for intervals and intersecting and all that, which as you said, rust-bio doesn't provide. Is bedder-rs ready to be used overall or it needs a lot more work before it's ready? The main README page seems to say that downstream APIs to perform operations on the intersections is not ready, but does that mean the intersections are ready to use? If so, I'll try using this as a first pass. Thanks!

[brentp]

I think it's ready. the main algo is very simple as it requires the inputs to be sorted, it's doing streaming intersections.
You could even write your own impl of Positioned for rust-htslib VCF
and then write the PositionedIterator to iterate over the file and yield your Positioned structs and be on your way.

Here's everything you need to know about the Intersections:

bedder-rs/src/intersection.rs

Lines 38 to 51 in 2a995f2

	#[derive(Debug)]
	pub struct Intersection {
	/// the Positioned that was intersected
	pub interval: Rc<Position>,
	/// a unique identifier indicating the source of this interval.
	pub id: u32,
	}
	/// An Intersections wraps the base interval and a vector of overlapping intervals.
	#[derive(Debug)]
	pub struct Intersections {
	pub base_interval: Rc<Position>,
	pub overlapping: Vec<Intersection>,
	}

Those are public so you simply access them and match on the Position enum to get out your concrete type:

bedder-rs/src/position.rs

Lines 59 to 68 in 2a995f2

	#[derive(Debug)]
	pub enum Position {
	Bed(crate::bedder_bed::Record<3>),
	// Note: we use a Box here because a vcf Record is large.
	Vcf(Box<crate::bedder_vcf::Record>),
	Interval(crate::interval::Interval),
	// catch-all in case we have another interval type.
	#[cfg(feature = "dyn_positioned")]
	Other(Box<dyn Positioned>),
	}

In your case, you may need to use Other(Box<dyn Positioned>), but to get feedback, I'd be willing to implement a feature that adds built-in support for rust-htslib VCF.

You'll hit some bumps, I'm sure, but I'll respond quickly. No worries if it's too much of a PITA, indeed it's early days for the library.