Skip to content

Commit

Permalink
Ensure that the orchestrator drops the container bundle after unpacking
Browse files Browse the repository at this point in the history
Once we unpack the archive, there is no need to keep the buffer around
in memory; this can waste a sginificant chunk of memory (we've now got
containers that are 10+ GB in size, that's not a trivial amount).

The logic is that first we convert the `Vec<u8>` into a `bytes::Bytes`
in `lib.rs`, and pass the ownership of that buffer to `run`. Second, in
`run`, the buffer is converted into a `Reader`, and ownership passes to
`archive`; thus, after unpacking the archive, we should ensure it goes
out of scope sooner rather than later.

Bug: 396664122
Change-Id: I791499d35684d87bf10bc38b3ff5111cf511c0d2
  • Loading branch information
andrisaar committed Feb 18, 2025
1 parent c5b9cee commit 28edc4a
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 4 deletions.
9 changes: 7 additions & 2 deletions oak_containers/orchestrator/src/container_runtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,13 @@ pub async fn run<B: Buf>(
) -> Result<(), anyhow::Error> {
tokio::fs::create_dir_all(container_dir).await?;
log::info!("Unpacking container bundle");
let mut archive = tar::Archive::new(container_bundle.reader());
archive.unpack(container_dir)?;

// Ensure that the archive, with the reader and bundle, go out of scope so that
// we don't accidentally keep a copy of the container bundle around in memory.
{
let mut archive = tar::Archive::new(container_bundle.reader());
archive.unpack(container_dir)?;
}

for entry in walkdir::WalkDir::new(container_dir) {
let entry = entry?;
Expand Down
5 changes: 3 additions & 2 deletions oak_containers/orchestrator/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -187,8 +187,9 @@ pub async fn main<A: Attester + ApplicationKeysAttester + Serializable>() -> any
group_keys.context("group keys were not provisioned")?,
cancellation_token.clone(),
),
crate::container_runtime::run(
&container_bundle[..],
// Explicitly convert the Vec into `bytes::Bytes` to pass ownership into `run`.
crate::container_runtime::run::<bytes::Bytes>(
container_bundle.into(),
&args.container_dir,
user.uid,
user.gid,
Expand Down

0 comments on commit 28edc4a

Please sign in to comment.