Adding optional arg to BF.INSERT to allow users to check if their bloom filter can reach the desired size

[zackcam]

Overview

This PR is proposing to add an extra optional arg to the BF.INSERT command or create a new command in order to allow users to know if their creation will be able to scale to what they desire. For option 1, we would add an optional arg called “ATLEASTCAPACITY” and validate whether it is possible to achieve X capacity given the bloom object memory usage limit (by default 128MB), fp rate, Tightening, expansion, and scale outs. For option 2 we will add a new command that would output the capacity that can be reached before having an error thrown from scaling out given the bloom object memory usage limit (by default 128MB), fp rate, Tightening, expansion.

Option 1: Amend the BF.INSERT command

Adding ATLEASTCAPCITY as an optional arg of BF.INSERT. This argument would allow the user to specify what they think their filter will scale up to. If the capacity that they think it will scale to isn't viable given the specifications provided then we will throw an error. New bf.insert command will look like:

BF.INSERT <key> [CAPACITY capacity] [ERROR fp_error] [EXPANSION expansion] [NOCREATE] [NONSCALING] [ATLEASTCAPACITY wantedcapacity] [ITEMS item]

Other name proposals for ATLEASTCAPACITY name:

• MINPOTENTIALCAPACITY
• PLANNEDSIZE
• VALIDATECAPACITY
• GROWTHCAPCITY
• TARGETCAPACITY

There are three new error messages associated with this:

pub const WANTED_CAPACITY_EXCEEDS_MAX_SIZE: &str =
    "ERR Wanted capacity would go beyond bloom object memory limit";
pub const WANTED_CAPACITY_FALSE_POSITIVE_INVALID: &str =
    "ERR False positive degrades too much to reach wanted capacity";
pub const NON_SCALING_AND_WANTED_CAPACITY_IS_INVALID: &str =
    "ERR Specifying NONSCALING and ATLEASTCAPCITY is not allowed";

Note: the user specifying NONSCALING and ATLEASTCAPACITY isn't allowed as this new argument is only useful for seeing if a filter can scale to the desired capacity.

Option 2: Create a new command BF.GETMAXCAPACITY

This would check what the max capacity of a bloom filter is given a capacity, error rate and expansion. This command will look like:

BF.GETMAXCAPACITY [CAPACITY capacity] [ERROR fp_error] [EXPANSION expansion]

This command would then output the capacity that could be reached before having an error thrown from scaling out. This would not create a filter given the specified arguments.

Testing

Added checks for these three error messages in test_bloom_command.py
If we choose to implement option two will create new tests for this.

[madolson]

All things considered, I think the current naming and API arguments make sense. The only alternative I thought of is we can add it BF.INFO, which might make sense either way. We don't need a new command, and diverging on info should be OK since it's written to be extensible.

[zackcam]

All things considered, I think the current naming and API arguments make sense. The only alternative I thought of is we can add it BF.INFO, which might make sense either way. We don't need a new command, and diverging on info should be OK since it's written to be extensible.

Would we want to add to both BF.INFO and BF.INSERT?
I think that adding to both would make the most sense as adding to BF.INSERT can allow start up confirmation of the size they want is possible. While adding to BF.INFO can provide the max capacity information at any point while also showing how big it could get not just if it can get bigger than what the user wanted.