Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support remote-building to macOS hosts #714

Merged
merged 12 commits into from
Nov 10, 2023
18 changes: 3 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,8 @@ While `nix-installer` tries to provide a comprehensive and unquirky experience,

### Using MacOS remote SSH builders, Nix binaries are not on `$PATH`

*For Nix installations before Determinate Nix Installer version 0.14.1.*
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part can be removed, we do not need to document old niche problems in the READMEs of new ones. I suspect we could look into cutting a new release quite soon, even.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might should keep a list of problems and quirks of old releases. Like the below note about nix-darwin should be solved now, now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old releases have their own READMEs with quirk lists! :)

In the nix-darwin case, users may still try to uninstall Nix on their own before trying to use this tool, and we might get it wrong. We should be able to remove it if we implement a correct fix though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!


When connecting to a Mac remote SSH builder users may sometimes see this error:

```bash
Expand All @@ -288,25 +290,11 @@ There are two possible workarounds for this:
* Update `/etc/zshenv` on the remote so that `zsh` populates the Nix path for every shell, even those that are neither *interactive* or *login*:
```bash
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ] && [ -n "${SSH_CONNECTION}" ] && [ "${SHLVL}" -eq 1 ]; then
. '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix
```
<details>
<summary>This strategy has some behavioral caveats, namely, <code>$PATH</code> may have unexpected contents</summary>

For example, if `$PATH` gets unset then a script invoked, `$PATH` may not be as empty as expected:
```bash
$ cat example.sh
#! /bin/zsh
echo $PATH
$ PATH= ./example.sh
/Users/ephemeraladmin/.nix-profile/bin:/nix/var/nix/profiles/default/bin:
```
This strategy results in Nix's paths being present on `$PATH` twice and may have a minor impact on performance.

</details>

### Using MacOS after removing `nix` while `nix-darwin` was still installed, network requests fail

Expand Down
166 changes: 166 additions & 0 deletions src/action/macos/configure_remote_building.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
use crate::action::base::{create_or_insert_into_file, CreateOrInsertIntoFile};
use crate::action::{
Action, ActionDescription, ActionError, ActionErrorKind, ActionTag, StatefulAction,
};

use std::path::Path;
use tokio::task::JoinSet;
use tracing::{span, Instrument, Span};

const PROFILE_NIX_FILE_SHELL: &str = "/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh";

/**
Configure macOS's zshenv to load the Nix environment when ForceCommand is used.
This enables remote building, which requires `ssh host nix` to work.
*/
#[derive(Debug, serde::Deserialize, serde::Serialize, Clone)]
pub struct ConfigureRemoteBuilding {
create_or_insert_into_files: Vec<StatefulAction<CreateOrInsertIntoFile>>,
grahamc marked this conversation as resolved.
Show resolved Hide resolved
}

impl ConfigureRemoteBuilding {
#[tracing::instrument(level = "debug", skip_all)]
pub async fn plan() -> Result<StatefulAction<Self>, ActionError> {
let mut create_or_insert_into_files = Vec::default();

let shell_buf = format!(
r#"
# Nix, only on remote SSH connections -- for remote building.
grahamc marked this conversation as resolved.
Show resolved Hide resolved
# See: https://github.com/DeterminateSystems/nix-installer/pull/714
if [ -e '{PROFILE_NIX_FILE_SHELL}' ] && [ -n "${{SSH_CONNECTION}}" ] && [ "${{SHLVL}}" -eq 1 ]; then
. '{PROFILE_NIX_FILE_SHELL}'
fi
# End Nix
"#
);

let profile_target_path = Path::new("/etc/zshenv");
create_or_insert_into_files.push(
CreateOrInsertIntoFile::plan(
profile_target_path,
None,
None,
0o644,
shell_buf.to_string(),
create_or_insert_into_file::Position::Beginning,
)
.await
.map_err(Self::error)?,
);

Ok(Self {
create_or_insert_into_files,
}
.into())
}
}

#[async_trait::async_trait]
#[typetag::serde(name = "configure_remote_building")]
impl Action for ConfigureRemoteBuilding {
fn action_tag() -> ActionTag {
ActionTag("configure_remote_building")
}
fn tracing_synopsis(&self) -> String {
"Configuring zsh to support using Nix in non-interactive shells".to_string()
}

fn tracing_span(&self) -> Span {
span!(tracing::Level::DEBUG, "configure_remote_building",)
}

fn execute_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
self.tracing_synopsis(),
vec!["Update zshenv to import Nix".to_string()],
grahamc marked this conversation as resolved.
Show resolved Hide resolved
)]
}

#[tracing::instrument(level = "debug", skip_all)]
async fn execute(&mut self) -> Result<(), ActionError> {
let mut set = JoinSet::new();
let mut errors = vec![];

for (idx, create_or_insert_into_file) in
self.create_or_insert_into_files.iter_mut().enumerate()
{
let span = tracing::Span::current().clone();
let mut create_or_insert_into_file_clone = create_or_insert_into_file.clone();
let _abort_handle = set.spawn(async move {
create_or_insert_into_file_clone
.try_execute()
.instrument(span)
.await
.map_err(Self::error)?;
Result::<_, ActionError>::Ok((idx, create_or_insert_into_file_clone))
});
}

while let Some(result) = set.join_next().await {
match result {
Ok(Ok((idx, create_or_insert_into_file))) => {
self.create_or_insert_into_files[idx] = create_or_insert_into_file
},
Ok(Err(e)) => errors.push(e),
Err(e) => return Err(Self::error(e))?,
};
}

if !errors.is_empty() {
if errors.len() == 1 {
return Err(Self::error(errors.into_iter().next().unwrap()))?;
} else {
return Err(Self::error(ActionErrorKind::MultipleChildren(
errors.into_iter().collect(),
)));
}
}

Ok(())
}

fn revert_description(&self) -> Vec<ActionDescription> {
vec![ActionDescription::new(
"Remove the Nix configuration from zsh's non-login shells".to_string(),
vec!["Update zshenv to no longer import Nix".to_string()],
grahamc marked this conversation as resolved.
Show resolved Hide resolved
)]
}

#[tracing::instrument(level = "debug", skip_all)]
async fn revert(&mut self) -> Result<(), ActionError> {
let mut set = JoinSet::new();
let mut errors = vec![];

for (idx, create_or_insert_into_file) in
self.create_or_insert_into_files.iter_mut().enumerate()
{
let mut create_or_insert_file_clone = create_or_insert_into_file.clone();
let _abort_handle = set.spawn(async move {
create_or_insert_file_clone.try_revert().await?;
Result::<_, _>::Ok((idx, create_or_insert_file_clone))
});
}

while let Some(result) = set.join_next().await {
match result {
Ok(Ok((idx, create_or_insert_into_file))) => {
self.create_or_insert_into_files[idx] = create_or_insert_into_file
},
Ok(Err(e)) => errors.push(e),
// This is quite rare and generally a very bad sign.
Err(e) => return Err(e).map_err(|e| Self::error(ActionErrorKind::from(e)))?,
};
}

if errors.is_empty() {
Ok(())
} else if errors.len() == 1 {
Err(errors
.into_iter()
.next()
.expect("Expected 1 len Vec to have at least 1 item"))
} else {
Err(Self::error(ActionErrorKind::MultipleChildren(errors)))
}
}
}
2 changes: 2 additions & 0 deletions src/action/macos/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
*/

pub(crate) mod bootstrap_launchctl_service;
pub(crate) mod configure_remote_building;
pub(crate) mod create_apfs_volume;
pub(crate) mod create_fstab_entry;
pub(crate) mod create_nix_hook_service;
Expand All @@ -16,6 +17,7 @@ pub(crate) mod set_tmutil_exclusions;
pub(crate) mod unmount_apfs_volume;

pub use bootstrap_launchctl_service::BootstrapLaunchctlService;
pub use configure_remote_building::ConfigureRemoteBuilding;
pub use create_apfs_volume::CreateApfsVolume;
pub use create_nix_hook_service::CreateNixHookService;
pub use create_nix_volume::{CreateNixVolume, NIX_VOLUME_MOUNTD_DEST};
Expand Down
19 changes: 16 additions & 3 deletions src/cli/subcommand/repair.rs
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,22 @@ impl CommandExecute for Repair {

if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
Ok(ExitCode::FAILURE)
} else {
Ok(ExitCode::SUCCESS)
return Ok(ExitCode::FAILURE);
}

grahamc marked this conversation as resolved.
Show resolved Hide resolved
#[cfg(target_os = "macos")]
Hoverbear marked this conversation as resolved.
Show resolved Hide resolved
{
let mut reconfigure = crate::action::macos::ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed();

if let Err(err) = reconfigure.try_execute().await {
println!("{:#?}", err);
return Ok(ExitCode::FAILURE);
}
}

Ok(ExitCode::SUCCESS)
}
}
10 changes: 9 additions & 1 deletion src/planner/macos.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,9 @@ use crate::{
action::{
base::RemoveDirectory,
common::{ConfigureInitService, ConfigureNix, CreateUsersAndGroups, ProvisionNix},
macos::{CreateNixHookService, CreateNixVolume, SetTmutilExclusions},
macos::{
ConfigureRemoteBuilding, CreateNixHookService, CreateNixVolume, SetTmutilExclusions,
},
StatefulAction,
},
execute_command,
Expand Down Expand Up @@ -166,6 +168,12 @@ impl Planner for Macos {
.map_err(PlannerError::Action)?
.boxed(),
);
plan.push(
ConfigureRemoteBuilding::plan()
.await
.map_err(PlannerError::Action)?
.boxed(),
);

if self.settings.modify_profile {
plan.push(
Expand Down
Loading