Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[823] add detailed disk metrics #827

Open
wants to merge 21 commits into
base: master
Choose a base branch
from

Conversation

archeoss
Copy link
Contributor

@archeoss archeoss commented Aug 2, 2023

Closes #823

@archeoss archeoss requested a review from ikopylov August 2, 2023 20:43
@archeoss archeoss marked this pull request as draft August 2, 2023 21:01
@archeoss archeoss marked this pull request as ready for review August 2, 2023 22:30
@archeoss archeoss changed the title 823 add detailed disk metrics [823] add detailed disk metrics Aug 3, 2023
pub(crate) struct SpaceInfo {
#[serde(flatten)]
total_space: Space,
disk_space_by_disk: HashMap<DiskName, Space>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, this is a breaking change.

@idruzhitskiy can we support both old and this new formats in OldPartitionsRemover?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is another problem here: you try to combine logical and physical structure in this map. Probably, it is better to split them. occupied_disk_space_by_disk is created to represent space info within logical structure. I suggest to keep it without changes (occupied_disk_space_by_disk: HashMap<DiskName, u64>) and add additional disk_space_by_disk for physical structure (its key will be a mount point of the disk, not its name)

pub(crate) struct SpaceInfo {
    #[serde(flatten)]
    total_space: Space,
    occupied_disk_space_by_disk: HashMap<DiskName, u64>,  // key is a Bob disk name
    disk_space_by_disk: HashMap<String, Space>  // Key is mount point
}

@@ -145,12 +145,18 @@ pub(crate) struct NodeConfiguration {
}

#[derive(Debug, Serialize)]
pub(crate) struct SpaceInfo {
struct Space {
total_disk_space_bytes: u64,
free_disk_space_bytes: u64,
used_disk_space_bytes: u64,
occupied_disk_space_bytes: u64,
Copy link
Member

@ikopylov ikopylov Aug 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we will decide to allow breaking changes, then this field should be renamed to occupied_by_bob_disk_space_bytes. Such a name will better reflect the meaning of the field

bob/src/api/mod.rs Outdated Show resolved Hide resolved
bob/src/api/mod.rs Outdated Show resolved Hide resolved
bob/src/api/mod.rs Outdated Show resolved Hide resolved
bob/src/api/mod.rs Outdated Show resolved Hide resolved
@archeoss archeoss marked this pull request as draft August 20, 2023 10:02
@archeoss archeoss marked this pull request as ready for review August 21, 2023 11:56
occupied_disk_space_by_disk: HashMap<DiskName, u64>,

/// Key - Mount point or Bob disk name if no mount point was found
disk_space_by_disk: HashMap<String, Space>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just thought that it would be better to use disk name as the key (on Linux it is sda1, sdb1, on Windows it is C:, D:), because this is the common way of identification of disks in OS

Copy link
Member

@ikopylov ikopylov Jan 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think, it would be better to do it later in another request. Right now, mount point would be enough here

bob/src/api/mod.rs Outdated Show resolved Hide resolved
for mount_point in disks.keys() {
/// Maps [`DiskSpaceMetrics`] to the corresponding mount point.
///
/// `f_fsid` field of [`statvfs`] is used to ensure that disks are unique.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the code of the function, you return mount point as the key. Please, fix the comment

pub(crate) fn update_space_metrics(&self) -> DiskSpaceMetrics {
/// Returns the updated space metrics of this [`HWMetricsCollector`].
///
/// Key -- `f_fsid` of underlying [`statvfs`] function, represents file system ID
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Comment is incorrect

bob/src/api/mod.rs Outdated Show resolved Hide resolved
let bsize = stat.f_bsize;
let blocks = stat.f_blocks;
let bavail = stat.f_bavail;
let bfree = stat.f_bfree;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, return back explicit type declaration, because it is required for proper calculations. On some targets the type of all this values can be 'u32' instead of 'u64', so it is better to explicitly convert it to u64 here to avoid overflow on multiplication operation later

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nerd mode on
Theoretically, if that's the case, it shouldn't even build at this point. Multiplication of two u32 numbers produces another u32 number, but DiskSpaceMetrics has u64 field, and rust prohibits any implicit type conversions, thus the compiler should produce the error
In order to convert the type we must use either as <type> syntax or .into() method
nerd mode off
But maybe I'm missing something

In any case, the type annotations are placed back, they wouldn't hurt

}

#[derive(Debug, Serialize)]
pub(crate) struct SpaceInfo {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update openapi in config-examples

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to integrate utoipa for openapi auto-gen at some point

Copy link
Contributor

@idruzhitskiy idruzhitskiy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Metrics: add detailed information about total, used and free space on every disk
3 participants