Feature request - multiple module conflict algorithm

[dineshm72]

There is a type of issue, a multiple module conflict, which the current FTC process will not be able to resolve correctly.

Sometimes an issue only arises if 2 or more modules are enabled simultaneously. Using the current process, unless you get very lucky, you will fail to uncover either module, and FTC will report a random ending module as the culprit.

To wit: somewhere in the tree search one of the two (or more) conflicting modules gets disabled and the problem disappears. The process then considers the ones that were not disabled to be non-culprits (even though they are required to have the issue), and they get disabled for the rest of the search, and you basically waste the rest of the tree search time randomly eliminating half of the remaining modules at each step, never seeing the issue appear again until finally a random one of them remains, is identified, and then is not confirmed.

I'm not quite sure what the ideal search algorithm for this would be, but it can't be the same reducing binary search being used by FTC currently. I think maybe a search that works in the opposite direction might work. e.g.

1. Split the modules into two groups, call them Y and Z. Turn off half the modules (say Y) and keep Z on. See if the issue remains.
   -1a. If not, do not consider Z to be safe, but keep them all enabled, and then split Y in half and re-enable half the Y ones that were disabled. See if it comes back.
   --1a1. If not, re-enable half of the remaining ones again, and so on.
   --1a2. If so, disable half the ones which you just re-enabled
   ---You should eventually get to one module that is contributing to the problem
   -1b. If so, disable another half of the ones that were still enabled, and loop back to 1
   ...

You may have to iteratively do this process until each branch finally identifies 1 or 0 modules which cause the issue. Any time you cut out half of a group and the issue remains, you can throw those modules out of the search, but any time the issue goes away you have to keep them all and just narrow the search again.

I.e. say after step 1 you go to the 1a and then 1a1 branches. You will eventually find a single module which is contributing (hopefully), but the whole time you still have all the Z modules enabled. You eventually need to prune down the Z tree too to see if any of the modules in that group are contributory.

I think this process would work for a 2 module conflict, but I'm not sure how to generalize it for N module conflicts (or maybe it might work for those already, I'm having trouble getting my brain wrapped around it). It also seems very inefficient and time consuming. Perhaps there is a better algorithm out there for finding multiple object conflicts like this.

[esheyw]

This would be a nice-to-have, but it's going to remain out of scope for the foreseeable future. To some extent this is what pinning is for; if it's a module-module conflict, generally you should be able to intelligently pin likely culprits IMO. I'd take a PR, but I don't plan on doing major work on FtC for a while.

[dineshm72]

Yep understood.

Pinning might work if you have some ideas what is causing it in the first place, though obviously less so if you don't.