You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This assumes that we preprocessed the the input data file and wrote the result to data.jsonl
However, this isn't the case if we've set process_data=False in TrainArgs. If this is the case, we should be honoring the data_path argument as the input .jsonl data file.
The text was updated successfully, but these errors were encountered:
When setting up the arguments to subprocess
torchrun
inmain_ds.py::run_training
, we set the data path to the following:This assumes that we preprocessed the the input data file and wrote the result to
data.jsonl
However, this isn't the case if we've set
process_data=False
in TrainArgs. If this is the case, we should be honoring thedata_path
argument as the input .jsonl data file.The text was updated successfully, but these errors were encountered: