Kafka logging

[saloni-2404]

how are the log files maintained in case of Kafka operator?
Is there any cleanup or does it push the logs anywhere else?
Why I am asking is to understand the threshold of the pods. The pods shouldn't come down cause of filling logs.

[scholzj]

The logs are written to standard output where you can collect them thorough Docker with the usual Kubernetes tools such as Fluentd etc. Unless you changed the logging configuration to work that way, the logs are not written to any files inside the container.

[saloni-2404]

Okay, and what about these configs to put in kafka.yaml are they for kafka logging as well:
spec.kafka.config:
log.cleaner.delete.retention.ms: "3600000"
log.retention.ms: "3600000"

and similar parameters?
Would you suggest these if yes?

[scholzj]

Ok ... so I understood your question was about logging such as the log messages printed by the application and secribing what is happening. The options you point to above, they are about Kafka logs => s in the commit logs storing the messages sent to Kafka by producers.

If I got it wrong and you meant the commit logs with the messages from producers, then the answer is different => you need to take care that the disk does not get full by properly configuring the topics. For example using the retention settings in bytes or in time. Apache Kafka does not do this for you.