Moving Atuin to a daemon

General
1

If you’re a package maintainer, I’m especially interested in your input here. Open to all comments from anyone, but I’d like to avoid breaking systems and setups

Atuin does not run a daemon to sync. There are two kinds of sync that happen

  1. In the foreground, when someone runs atuin sync
  2. In the background. This happens after a command runs, and is forked so as to not block the shell. It only happens once a certain duration has elapsed since the last sync

This was very simple to implement, and didn’t require that users setup or maintain any sort of service. At the time, I felt this best - Atuin’s install could be super simple, and not need any maintenance.

However… as time has gone on, this approach has had a whole bunch of downsides.

  • If the user wishes to sync after EVERY command, and there’s some sort of delay, we can end up with multiple syncs happening concurrently. This can even happen if the sync interval is low, there are issues with the filesystem, IO pressure, whatever. The workarounds for this kind of problem, realistically, are a hack.

  • Even if the user isn’t syncing, writing to SQLite before and after every command can cause problems on systems that are a bit unusual, or under high load.

  • Atuin runs some tasks on a schedule (check for updates, sync if needed, etc). It would also be good to be able to run periodic maintenance of the database, potentially pruning it, that kind of thing. Doing so at the moment would require adding more to the “post command” tasks, and writing it in a way that accounts for potentially multiple instances running at once. Or somehow modifying cron, which we cannot rely on systems having.

  • does this smell racey to anyone? it’s starting to.

I won’t carry on listing issues, but we do semi often have problems that come up that just would not exist if Atuin was running as a persistent background process, and most expensive compute was not tied to a shell-driven lifecycle.

The daemon

I’d like to move Atuin to be a daemon. We can eliminate a whole class of issues this way, and I think it’s the right direction for the future.

We could use crates like daemonize, which will handle forking/PID file management/etc. We probably wouldn’t need to do much in the way of system setup, and could just have something like atuin daemon start the background process.

I’d rather not though, as afaik all systems we run on have some sort of system for managing background services. Unless this turns out to be too much of a nightmare.

The lifecycle would then be much simpler. The background process can handle actually writing to sqlite, syncing when required, all that. The foreground becomes basic - just writing over some socket to the background process. We can eliminate the forking after each command.

The TUI can be unaffected, as SQLite handles multiple readers without issue

2 Likes
2

runit on void

runit is a pretty simple system, at least to set up - I add a service file to the package, e.g.:

#!/bin/sh

exec atuind

and then once a user enables the service the daemon runs on system start with basic supervision. If I also redirect stderr to stdout in that script, any logs printed will be automatically sent to the void logging daemon.

For configuration usually daemons have their own system-owned file like /etc/atuin/config.toml.

1 Like
3

On windows there is a manager for background tasks (called services iirc). The only problem i would see is windows defender (antivirus) not liking it.

1 Like
4

systemd user service will work great for this use case. atuin doesn’t need to daemonize itself, but only to ship a .service file (and probably a .socket / dbus service file to activate automatically).

BTW for Linux distro packages, it’s expected that the automatic update check is shipped disabled (the system package manager handles updates).

2 Likes
5

Excellent! That looks pretty easy, thanks. How does it handle user services? It would need to be owned by the user and not system.

I’m not sure there’s much we can do there, but will check

Discussed this a few times before, but essentially that can be left up to package maintainers. My priority is ensuring that users keep their install up to date, others can decide how that should be done for their distro

Given that there’s still a bunch of out-of-date packages, I’d rather default to letting users know they’re running behind

image

1 Like
6

Just requires some minimal user effort: Per-User Services - Void Linux Handbook

I think we can safely assume anyone using void should be prepared to do that, but it wouldn’t hurt to add a void-specific user guide.

1 Like
7

Sounds good to me!

The more I think about it, it seems like Atuin would need to support both.

  1. A daemon ran by systemd/runit/brew services/etc. This is preferred, but may require some level of user interaction
  2. Totally automatic, no changes needed, but handling forking/PIDfile ourselves (well, with a library). This ensures there’s some level of support for almost any system at all, with minimal effort.
1 Like
8

In my not-quite-ancient history (it was this millennium) as a de facto windows sysadmin both windows defender and enterprize-y malware protection would look significantly more suspiciously at a service than a shell script.

As noted not a lot you can do (code signing might help a bit) but it may be a bigger barrier for folks at BigCo.s

1 Like
9

Totally makes sense - will see if there’s some sort of fallback we can sort for Windows then :thinking:

(also - welcome to the forum!)

1 Like
10

We could also run it in a shell then minimize to system tray (idk if thats different than starting it as a service)

11

I see two approaches. Both should have a socket listener, for the reason that even with the same user, I might want to run multiple instances (release and dev, personal and work, whatever). Currently, I can give them different db/config paths, and the socket path should follow the same pattern.

  1. the socket is bound by the service manager (systemd, or equivalent). When atuin starts in a shell, it connects to the socket, and the service manager starts the daemon.

  2. the socket is not bound. When atuin starts up, it attempts to connect to the socket, fails, and as the first client is therefore responsible for forking the daemon. It should probably bind the socket immediately, and pass that to the daemon, just to minimise the race window here.

In either case, the daemon can exit after a while being idle, and get restarted when shell commands are entered again.

Assuming we have #2 working, what is the benefit of doing #1? Possible answers include:

  • making packagers and distros happy, by fitting more closely with their machinery, reporting, logging, etc
  • the service manager (in particular systemd) can do some sandboxing of the daemon
  • shutdown may be cleaner, because a pure atuin daemon doesn’t easily know when the user logs out or the last client has disconnected (noting that clients will connect and exit per shell command).

I don’t know if those are compelling reasons, particularly compelling enough to do several times for several different system service managers, but that’s the basis for a decision, I think.

The nice thing about this model is that the atuin code can be the same, and the difference is mostly whether the service manager preempts the first atuin client. So that’s an optional enhancement if there’s sufficient merit.

12

I don’t think it should exit on its own, especially when started as a service.

Logging will be easy, thus issues can be easier to locate.

Also, atuin can do both. The needed work is minimal, just a flag to

  • not daemonize
  • use stdin instead of creating and binding a socket itself
  • do error logs without timestamps

atuin can just try to connect, and if failed, start a daemon itself. So if the service is properly installed it will connect to it, else it starts its own.

13

Good point, or to rephrase: the policy for idle exit should be delegated to the service manager, if used.

Yeah, exactly.

14

I don’t think you’d trip up Defender. You do need Administrator permissions to install a service, but you can run in the background without being a service.

1 Like
15

You don’t need to minimise to system tray if you don’t spawn a console at all.

16

Well yeah, you’re shipping data off to an offsite service. You’d need a way for administrators to disable this automatically for every instance of Atuin installed, and to set the company-wide Atuin sync server.

17

I’m not against this, but for where we are now I’d like to avoid planning for + thinking about enterprise use cases.

Fwiw, Windows isn’t properly/officially supported. It’s why we don’t build releases for Windows, don’t list packaging/install steps for it, and don’t run tests for it.

A few contributors (ty @YummyOreo) do a great job of patching things up when we break windows support though :raised_hands:

2 Likes
18

100%, I think this is the way ahead! Basically what I meant earlier, but worded far better

Actually not the case - it would be good for the daemon to be able to handling syncing regardless of shell activity. This means that it can update the search index with activity from other machines, ensuring history/etc is properly up-to-date with no prompting

But yeah, once the bulk of the work for shifting it into a daemon is done, there won’t be a big difference between starting it with daemonize/similar or some init system. Pretty sure that crate supports Windows too, so maybe that’s enough there.

19

Sadly it doesn’t :frowning:

20

Awh, that’s a shame.

I haven’t looked deeply, but there’s this too: GitHub - mullvad/windows-service-rs: Windows services in Rust

1 Like