Feature Request: Container start/stop priority

[Riewest]

If this feature already Exists/Requested/Denied I didn't see it, sorry in advance. Also this is a cool project!

General Idea

It would be nice if we could tie in some custom bash scripts at certain points in the backup lifecycle in order to facilitate some custom backup logic. I think if it were just some Environment Variables/labels that default to empty it would allow users to map a scripts volume that then gets ran if the vars are mapped to files.

Concept Use

The main use I see for this is kicking off a post script which could Compress/Remote Backup/Trigger Webhook/Etc...

Proposed Vars

1. PRE_BACKUP_SCRIPT
   • Ran before any containers start backing up
   • Pass in the Source/Dest Directories as args
2. PRE_CONTAINER_BACKUP_SCRIPT
   • Ran before each individual container backup
   • Pass in the Container Name/Source/Dest as args
3. POST_CONTAINER_BACKUP_SCRIPT
   • Ran after an individual container backup
   • Pass in the Container Name/Source/Dest as args
4. POST_BACKUP_SCRIPT
   • Ran after all containers have been backed up
   • Pass in the Source/Dest Directories as args

Whether or not the nautical process should exit based on the script exit codes could also be a flag: SCRIPT_PASS_REQUIRED

[Minituff]

I like this idea.

I do already have Lifecycle Hooks added which seem to do most of what you are looking for

However, it is missing the following features:

• it doesn't pass any arguments
• there is no logic based on if the script fails or not.
  • maybe we could have the container not be backed up if the script fails
• Also I am missing the PRE_BACKUP_SCRIPT and POST_BACKUP_SCRIPT

I can also expand out the documentation of Lifecycle Hooks because I know I tested it out by calling a mounted bash script file, but the docs don't actually show how to mount the script.

Do you think adding those missing features would give you the capabilities you are looking for? Anything else I missed?

[Riewest]

I somehow missed the Lifecycle Hooks.

Those are close but not quite what I'm looking for since they run in the container that is being backed up. The hooks I proposed would be ran inside the nautical container and allow processing on the backed up files which wouldn't be available in the running container.

The main hook I want is the POST_BACKUP_SCRIPT after all containers have been backed up so that I can have a script which compresses and the sends it to a remote using any of the various remote backup tools.

The next most useful one would be POST_CONTAINER_BACKUP_SCRIPT so that if I wanted I could compress each containers data individually so if I ever have to restore I could just download that specific data but I could handle that in the POST_BACKUP_SCRIPT with some extra logic on my end.

Whatever you think is the best way forward I think I can make work. Even just a call to a webhook that the backup is done would probably work for me since I could trigger a pipeline to do the compress/remote stuff

[Minituff]

Got it. I see what you mean.

You may be able to get away with these webhooks since they run before or after all the containers are backed up.

In addition I could add another global variable for a script that could be called in addition to the webhooks that would run inside nautical and not the other containers.

[Riewest]

Those would definitely work in my use case, I swear I looked at the Docs 😅!

I still think the custom scripts would be a QOL feature but for how I plan to use nautical (Tied in with a gitlab pipeline) these should be sufficient for me.

Thanks for the help!

[Minituff]

I swear I looked at the Docs 😅!

All good. There's a lot going on in there so its easy to miss :)

I still think the custom scripts would be a QOL feature but for how I plan to use nautical

Yeah I think this would be helpful as well. I'll keep the ticket open as a reminder to myself to add it when I get some free time.

---

Also, how did you discover this project? Just curious as to how most people found Nautical 😅

[Riewest]

I came across it on the selfhosted subreddit while looking for a good way to stop/backup/restart containers.

If I didn't find this I was gearing up to write something with much less functionality and way more hacky than this to suit my needs. So this was a great find.

I'm coming over from unraid where I used the Appdata Backup Plugin which works similarly to this and is where I stole the script idea from.

The only other thing that tool does that would possibly be nice addition here (unless I missed it in the docs) would be some way to order the container stop/start.

I think the most natural way to do that would be to have a way to tie a docker compose project (Stack) to a nautical group that way nautical could just call docker compose -p GROUP stop backup all the containers that need it in the stack and then docker compose -p GROUP start. That would allow nautical to take advantage of the compose "depends_on" and stop/start containers safely.

But maybe just a simple PRIORITY label that nautical then sorts could accomplish that as well.

[Minituff]

I came across it on the selfhosted subreddit while looking for a good way to stop/backup/restart containers.

Love that subreddit. I learned so much from it.

---

The only other thing that tool does that would possibly be nice addition here (unless I missed it in the docs) would be some way to order the container stop/start.

I think the most natural way to do that would be to have a way to tie a docker compose project (Stack) to a nautical group that way nautical could just call docker compose -p GROUP stop backup all the containers that need it in the stack and then docker compose -p GROUP start. That would allow nautical to take advantage of the compose "depends_on" and stop/start containers safely.

But maybe just a simple PRIORITY label that nautical then sorts could accomplish that as well.

Hmm, I may need a little more detail here. I could pretty easily add another endpoint to the Nautical API that would let you start/stop containers & groups -- and your script/curl would be able to hit that API.

I could also add any other endpoints pretty easily; For example: an endpoint to backup a countainer/group.

[Riewest]

Haha yea that subreddit is good.

The start/stop order feature would be completely separate feature from the script one that started this thread.

Some services will fail if they can't find their database (or other dependencies) running so it would be nice to have containers in groups be able to startup in a certain order.

Its really not a huge issue since most containers should be set to restart unless-stopped (or always) so would eventually get back up and running.

Option 1

Lets say I had this compose file:

services:
  databaseA:
    # ...
  serviceA:
    # ...
    depends_on:
      - databaseA

When I deploy it with docker compose it will wait to start serviceA until databaseA is running and it will shut them down in the reverse order.

It would be nice to somehow have nautical respect that start/stop order of the compose project.

Option 2

If you didn't do it that way I think a simple integer based priority label could do the trick. (Not sure how nautical works on its backend so maybe this wouldn't work)

Add a label to containers and then at start nautical would start them by priority.

In this example nautical would stop all of them but then start the highest prioity first(databaseA) and then serviceA second.

services:
  databaseA:
    # ...
  labels:
    - "nautical-backup.priority=100"
  serviceA:
    # ...
    depends_on:
      - databaseA
  labels:
    - "nautical-backup.priority=0"

[Minituff]

Got it! I'm following now.

I like option 2. I think we could do something like this.

services:
  databaseA:
    # ...
  labels:
    - "nautical-backup.group=myGroup"
    - "nautical-backup.group.myGroup.priority=101" # This would first since priority is higher
  serviceA:
    # ...
    depends_on:
      - databaseA
  labels:
    - "nautical-backup.group=myGroup"
    - "nautical-backup.group.myGroup.priority=100" # This would next

Because some containers can be a part of multiple groups. This way we could break up the priorities on a per group basis.

[Riewest]

Yea that makes sense to me!

[Minituff]

Hey, I released a new version that should have this feature working! Let me know what you think.

[Riewest]

Looks good!

The only thing I noticed when running it is that it seemed to start all the containers at once but that just could be because they are immediately reporting as up so it just starts them all faster than my vs code plugin updates.

[Minituff]

Awesome! Yeah that would be my guess. The loop starts the containers sequentially so it must be that they are turning on really quick.

[Minituff]

Actually, now that I think about it more, one thing I just thought of-- So the priority applies to start and stop.

So if you had a priority of 90,100,105.
They would stop in this order: 105 --> 100 --> 90
And they would start in the same order: 105 --> 100 --> 90

Do you think it would be necessary to separate them? Maybe something like this?

 - "nautical-backup.group.myGroup.stop_priority=105"
 - "nautical-backup.group.myGroup.start_priority=100"

Or does this not have a usecase for you?

[Riewest]

I think one priority number is enough. But it should just reverse the order for starting them.
(Stop highest priority first descending)
(Start lowest priority first ascending)

[Riewest]

It might make sense to reverse that logic a little bit actually for new users.

Stop lowest priority first ascending
Start highest priority first descending.

Then users could mark there Databases as "highest priority" and they wouldn't be shutdown until the app using it was already offline