You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Since the dataset on each host is unbalanced, should find out a better way to do sync since on host3 with the smallest dataset always finish the epochs faster and will loop in the local small dataset. Causing overfitting.
Approaches could be:
Try different sync intervals according to different size of ds
try larger epochs for hosts with smaller ds
try different weight learning for hosts. Host with smaller ds should weigh less(gain less weight) during local training
put on_batch_end to on_epoch_end
it would be better that host would be able to wait for others to reach epoch end then do sync(currently I think it would initiate the sync when it reaches the sync frequency with
The text was updated successfully, but these errors were encountered:
Since the dataset on each host is unbalanced, should find out a better way to do sync since on host3 with the smallest dataset always finish the epochs faster and will loop in the local small dataset. Causing overfitting.
Approaches could be:
The text was updated successfully, but these errors were encountered: