Ability to use intra.broker.goals in anomaly detection / self healing

[amuraru]

Currently the intra.broker.goals goals can only be used in rebalance API when rebalance_disk=true
In environments where goal-violation based anomaly detection and self healing is enabled it would be useful to be able to define/configure these intra.broker.goals as well.

ref: https://gitter.im/kafka-cruise-control/Lobby?at=5eb9c02903af9f38ff6790df

[efeg]

@amuraru Thanks for submitting the feature request. There is no strong reason for not having this feature. One design option is to add another dedicated anomaly detector similar to goal_violation detector to check for intra broker goal violations and fix them when needed.

[amuraru]

That could work but was wondering if the regular interbroker goals can be unified with these intra brokers goals. One edge case in your proposal would be when say DiskCapacityGoal proposal is executed but as it is not aware of individual broker disk may reassign partitions to already at-capcity individual disk

[ecojan]

@efeg can you please provide some insight on the above question?

[ev-hines]

@efeg I'd be happy to look into submitting a PR to add this functionality. Same design question as amuraru though. Would unifying these goals be better so as to not cause race conditions between the other non-inter broker disk anomaly detectors.

If not, what would be the expected way to synchronize these components?

[imans777]

related: #2234,

I'd be very happy this PR gets reviewed and merged.