NaN in Sparse Training

[william-silversmith]

When training on labels that may not have any positive classes in the field of view, normalization causes an NaN to propagate through the network. Would it be possible to add an epsilon to the denominator to prevent this gift that keeps on giving?

Sachin and Nick get around this (amongst other reasons) by limiting their samples to those containing a positive class.