I think it'd just require making a debian package for it which installs to that path and adds an entry.conf for it. Shouldn't require much or any change to the installer.

Is there a way to make it so a Debian package won't install on non-UEFI systems? I know there's a lot of of people (myself included) who are still running Pop!OS on BIOS hardware. That was part of why I thought that doing it as part of the installer might be a better idea.

Also, that wouldn't allow us to add it as a boot option to the installer ISO, right?

The installer would only make it available for new installs. A package could make it available to all systems in an update. The debian package could use a postinst script to copy the EFI binary to the EFI directory and set up the entry for it (if it even requires a boot conf), and a postrm script could remove it. The script could do nothing on a system without an EFI partition mounted.

If you think that's the better solution, I'll look into how I can create a Debian package for this instead of adding it to the Pop!OS installer setup.

I think it would be better to have the deb install the files in the ESP and create the laoder entry--and on remove do nothing unless purged. This would allow the package to be included on install (creating the entry) and not be installed in the resulting system.

Support would be able to request install of the deb package again if the memory test is not found in the systemd-boot menu.

With this design, would the Pop!OS ISO have the entry as well?

I don't think that it would, but something like that could be added separately.

I feel like having the package remain as part of the resulting install would be important because then it could be updated with a new EFI binary as part of your normal apt update+upgrade process, or through the Pop!Shop. Support would still be able to do apt reinstall --purge pop-memtest or whatever if necessary, but it would be kept up-to-date continually by default. The same package could also theoretically be expanded later to include other tools, so maybe naming it something like pop-diagnostics to start would be better? These are decisions that can made later since they won't affect the basic plan.

The Pop!OS ISO would need to have it added to the opening menu, I think? On UEFI systems it shows the "Try or Install Pop_OS" screen, which only has the one menu option at the moment. Adding a second one ("Test System Memory" perhaps) would be a matter of adding the menu option to /boot/grub/grub.cfg and have it point to the executable, I think. And yeah, this would have to be separate from a Debian package.

I'm working on a proof-of-concept for this (with a package name named pop-diagnostics) and while it turned out to be fairly trivial to have the package tools copy the appropriate files to /boot/efi, I've run into a snag. Attempting to upgrade the package fails, as dpkg uses symlinks as part of the upgrade process, so it can rollback to the original files easily if the upgrade fails, and that doesn't work on an EFI partition since it's FAT32.

ckw@flatline:~/Documents/projects/memtest$ sudo dpkg -i pop-diagnostics_0.1.1_amd64.deb 
[sudo] password for ckw: 
(Reading database ... 258533 files and directories currently installed.)
Preparing to unpack pop-diagnostics_0.1.1_amd64.deb ...
systemd-boot present, continuing install
Unpacking pop-diagnostics (0.1.1) over (0.1.0) ...
dpkg: error processing archive pop-diagnostics_0.1.1_amd64.deb (--install):
 unable to make backup link of './boot/efi/EFI/memtest/memtest.efi' before installing new version: Operation not permitted
Errors were encountered while processing:
 pop-diagnostics_0.1.1_amd64.deb

This is not a unique problem. as there's a Raspbian hack to get around the same problem and a Ubuntu issue about this is almost ten years old, etc. If anyone here has any ideas about the proper way to work around this problem? I'm looking into options but they're all a bit hacky. Or would this be something better handled by the same tools that maintain the recovery partition?

Perhaps the package shouldn't be placing files in the EFI partition directly, but through a postinst / postrm script.

That was in fact my thought. Right now they're being placed using a .install file:

memtest86plus/build64/memtest.efi /boot/efi/EFI/memtest/
src/memtest.conf /boot/efi/loader/entries/

If they're instead placed via a postinst script, and removed on uninstall/purge with a postrm script, dpkg won't try to do the symlink thing, right? My concern is that this does feel a bit hacky, but if this is the proper method, I'll look into doing it that way instead.

Yes there wouldn't be a symlink issue to worry about. All the packages that add files to the EFI partition (kernelstub, grub) make the EFI installation a separate part of the package installation. kernelstub gets run with initramfs updates which copies kernels over and generates configs. There could be an initramfs hook that installs the files if they're missing, if not done directly by the postinst.

The package would install the files to somewhere in /usr/lib or the like as their official package locations, and then the scripts would copy them to the ESP? I think having them be part of the package scripts might be better than using initramfs hooks in this case, since errors in the hook could cause the process of creating the boot image to fail entirely. It would also allow users to install the package from the recovery partition; even though the recovery partition resets itself on restart, the files copied to the ESP would still be present.

This seems like a good way to move forward, and I'll try to find the least hacky way to get the postinst and postrm scripts to do this.

I've got a prototype in place that installs, uninstalls, upgrades, etc: https://github.com/cwsystem76/pop-diagnostics

Rather than moving this repo to the pop-os github org, I think using my personal repo to experiment and then doing a new, clean repo with those experiments as a foundation would be better.

Summarizing some Slack discussion here:

• Disagreement about whether the staging should occur in /usr/lib or /usr/share. I think that the FHS guidelines suggest it should be /usr/lib (or possible /usr/libexec), others think it should be /usr/share. This is a trivial thing to change at this point in the packaging; do you have any opinions on the subject, @mmstick ?
• It was suggested that we have a user-accessible script in the staging area, something like memtest-utils.sh, that could be run by users to reinstall, remove, or otherwise adjust the files in the ESP. This script could then be called by the postinst and prerm package scripts to do the work instead. I think that this is a good idea and unless there are objections I'm going to add that to the next test version. This has been completed.
• It was also suggested that the files be left in the EFI on a normal package uninstall, and only remove them when the package is purged. This seems reasonable, any objections?
• There was talk about using kernelstub to install the files into the ESP. I don't think this is a good idea, as this would increase interdependence between packages, require adding new functionality to kernelstub, make it harder to add the files to the ESP when booted to recovery, etc. I believe we can pass on that idea.
• Other folks in Support asked if it would be possible to add this as an option when booting from a USB stick prior to install. I'm not sure how difficult this would be, although right now the ISO boots on a UEFI system to a menu that only has one option; how do we add additional options?
• There may be a way to make this work with GRUB systems but it would be significantly more complicated and I'm not sure if it's worth the effort, given that we haven't shipped a BIOS system in a while and it's a very small percentage of Pop!OS users on non-System76 hardware. I'm keeping my notes on this but not actively pursuing it.

Name of the update script would be memtest-update.sh

I was thinking memtest-utils.sh because it can also be used to remove it, or possibly even add launch parameters if we add that functionality.

The latest commit now has memtest-utils.sh, which is called by the package postinst and prerm scripts. An example command that a user can run in a terminal:

sudo /usr/lib/pop-diagnostics/memtest/memtest-util.sh reinstall

A future version of this script will add an "options" verb to let systemd-boot pass launch parameters to the EFI executable, like keyboard=legacy or nosm when necessary for troublesome hardware.

The "options" feature has been added to memtest-utils.sh and the readme and comments generally improved. For example:

sudo /usr/lib/pop-diagnostics/memtest/memtest-util.sh options keyboard=legacy

will write a custom configuration file that will not use the new USB keyboard drivers, and will instead rely on the UEFI firmware to initialize the keyboard in legacy mode.

The current Beta 2 build doesn't work with the latest Coreboot firmware on my oryp6, but the dev build from nightly source does. One of a few reasons to hold off on any public release until it's out of beta and we've done a lot more testing.

[edit] This problem has been resolved.