Skip to content

Commit

Permalink
add suggestion to disable autocast and inference mode
Browse files Browse the repository at this point in the history
  • Loading branch information
kacper-kleczewski committed Oct 10, 2024
1 parent 2796d61 commit e415c0a
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion model_navigator/core/dataloader.py
Original file line number Diff line number Diff line change
Expand Up @@ -369,8 +369,9 @@ def _sample_filename(idx: int, num_samples: int) -> str:

def _validate_tensor(tensor: np.ndarray, *, raise_on_error: bool = True):
if any(np.isnan(tensor.flatten())):
message = "Tensor data contains `NaN` value. Please verify the dataloader and model."
message = "Tensor data contains `NaN` value. Please verify the dataloader and model. Consider disabling autocast or inference mode in TorchConfig custom configuration."
if raise_on_error:
LOGGER.warning(message)
raise ModelNavigatorUserInputError(message)
else:
LOGGER.warning(message)
Expand Down

0 comments on commit e415c0a

Please sign in to comment.