Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core] Use spdlog fd sink within pipe logger #50173

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

dentiny
Copy link
Contributor

@dentiny dentiny commented Feb 1, 2025

This PR is the followup for #50144, which integrates FD sink to the pipe logger.
The benefit is we have less file descriptor related code within pipe logger, and all write, flush and close feature handled in spdlog logger.

One behavior change, we use default formatter for all possible sinks, which appends newliner to each message.

@dentiny dentiny added the go add ONLY when ready to merge, run all tests label Feb 1, 2025
@dentiny dentiny requested a review from jjyao February 1, 2025 11:12
@dentiny dentiny force-pushed the hjiang/spdlog-fd-sink-pipe branch from d6699e1 to 942fd8f Compare February 1, 2025 11:14
@dentiny dentiny force-pushed the hjiang/spdlog-fd-sink-pipe branch from 942fd8f to 46bf843 Compare February 1, 2025 11:26
src/ray/util/pipe_logger.cc Outdated Show resolved Hide resolved
src/ray/util/pipe_logger.cc Outdated Show resolved Hide resolved
Comment on lines +197 to +202
auto logger_sink = std::make_shared<non_owned_fd_sink_st>(handle);
auto logger = std::make_shared<spdlog::logger>(
/*name=*/absl::StrFormat("pipe-logger-%s", file_path), std::move(logger_sink));
logger->set_level(spdlog::level::info);
logger->set_pattern("%v"); // Only message string is logged.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the case for no tee no rotation? In this case, we don't write to the logger?

Copy link
Contributor Author

@dentiny dentiny Feb 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes you're right, logger is not necessity here, but otherwise we will have two implementation for StreamRedirectionHandle,

  • one using file descriptor, and take flush_fn
  • another using logger, no need for flush_fn, which beats the purpose for this PR to simplify code structure

@@ -224,7 +224,7 @@ TEST(PipeLoggerCompatTest, CompatibilityTest) {
stream_redirection_handle.Close();

const std::string stdout_content = testing::internal::GetCapturedStdout();
EXPECT_EQ(stdout_content, kContent);
EXPECT_EQ(stdout_content, "hello\nworld\n");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why changing this? it should just match kContent right

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, in the pr we use spdlog for all sinks, which means newliner at the end for everything.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but for this test, kContent already has the newliner at the end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thats for code style consistency, we're using raw string to match elsewhere

@dentiny dentiny requested a review from jjyao February 1, 2025 19:50
Signed-off-by: dentiny <[email protected]>
Signed-off-by: dentiny <[email protected]>
@dentiny dentiny force-pushed the hjiang/spdlog-fd-sink-pipe branch from fb006b7 to 3f4dc0b Compare February 1, 2025 23:07
Signed-off-by: dentiny <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants