Is it possible to write all the console logs to a file?

[Galaxy-Husky]

Now the default logger of lightning writes all the logs to the console, which is like:

/home/ping/mambaforge/envs/lightning/lib/python3.10/site-packages/pydantic/_migration.py:282: UserWarning: `pydantic.utils:Representation` has been removed. We are importing from `pydantic.v1.utils:Representation` instead.See the migration guide for more details: https://docs.pydantic.dev/latest/migration/
  warnings.warn(
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1,2,3]
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1,2,3]

  | Name          | Type                          | Params
----------------------------------------------------------------
0 | bert          | BertForSequenceClassification | 109 M
1 | valid_metrics | MetricCollection              | 0
----------------------------------------------------------------
109 M     Trainable params
0         Non-trainable params
109 M     Total params
437.935   Total estimated model params size (MB)

Is it possible to write all the console logs to a file?

[justusschock]

Hey, you can just redirect stderr and stdout with tools like

https://docs.python.org/3/library/contextlib.html#contextlib.redirect_stdout

Cheers,
Justus

[Galaxy-Husky]

Thank you for your advice. I tried as

with redirect_stdout(f):
    cli.trainer.fit(cli.model, datamodule=cli.datamodule)

Everything else is fine, except that the progress bar takes up too many lines.
I'd like to control the logger of lightning, which is 'logging.getLogger("lightning.pytorch")', but only can capture a few lines, especially in ddp mode.

[justusschock]

heu, we don't use the logging module for tgis. You could just play with the arguments if the progressbar callback, but tqdm printing to many lines in files is a common issue with non-tty output streams.

[Galaxy-Husky]

I'm sorry, I don't understand what tgis is. Could you explain what it means?