I have only one permanent VM that I keep around (with my slowly being worked on NixOS configuration, so Atuin on Nix, but built manually from the latest master). I have used Atuin in the past with about 3 other VMs when I was testing stuff.
I have almost 150k entries in my history and the conversion with atuin history init-store is estimated to take almost 10 minutes. Can this be speedup with some parallism?
It should be sped up in the latest weekly release - Weekly release 2024.08
Mostly a SQL transaction batching issue, it was a single transaction per record beforehand
Ellie, I’m super impressed. Atuin is one of my absolute must-have tools.
Did the migration today on 3 of my regular work machines - flawless. Love the atuin store status
confirmation.
A quick question - what happens to the existing “old” blob on your end now that I’ve moved over to the new method?
I’d imagine you purge these after a time?
Glad to hear it, thank you!
The old blobs are still hanging around, and will do for a while - until sync v2 is the default I imagine. In the next release it will be the default for new users only, then I’ll hit the switch for everyone.
Also need to figure out a neat way of slowing the rollout so I don’t end up with a lot of upload at the same time
Hi everyone
I noticed that although everything was getting stored remotely correctly, the up arrow wasn’t showing the command from both computers as it used to.
It used to work fine, I found this thread and made sure I followed the steps.
Here are the outputs
❯ atuin store status
host: 018b2c2b-7664-7a06-8b36-3bcf9a1c84c9
store: history
idx: 891
first: 018e6798-c1f7-7693-b7d0-f6a6aa22c12d
created: 2024-03-22 19:17:28.183391732 +00:00:00
last: 018fa121-442f-7c06-8d6d-a2db466b6393
created: 2024-05-22 16:27:43.023387245 +00:00:00
host: 018b2c28-45bc-7c6d-9b71-b28588746193 <- CURRENT HOST
store: history
idx: 14195
first: 018e2ea9-bab2-7e02-87ab-2c3d94e4f847
created: 2024-03-11 17:57:39.122259192 +00:00:00
last: 018fa121-defe-71f8-9615-010b479c495d
created: 2024-05-22 16:28:22.654603473 +00:00:00
atuin store status
host: 018b2c2b-7664-7a06-8b36-3bcf9a1c84c9 <- CURRENT HOST
store: history
idx: 893
first: 018e6798-c1f7-7693-b7d0-f6a6aa22c12d
created: 2024-03-22 19:17:28.183391732 +00:00:00
last: 018fa121-9038-75e2-a308-eae85e24b64a
created: 2024-05-22 16:28:02.48868615 +00:00:00
host: 018b2c28-45bc-7c6d-9b71-b28588746193
store: history
idx: 14187
first: 018e2ea9-bab2-7e02-87ab-2c3d94e4f847
created: 2024-03-11 17:57:39.122259192 +00:00:00
last: 018fa11d-6227-7ab9-a630-f313114437b0
created: 2024-05-22 16:23:28.55117354 +00:00:00
Is there anything else I have to do? Maybe another parameter?
Edit
I think this fixed it, but I just had to wait for a few minutes. It seems to be working now.
Glad it’s sorted! Lmk if you have any other issues
Was the delay after switching to sync v2?
I am in a strange situation. I have some records I cannot decrypt (~1500) and when I run atuin store purge, they are removed but as soon as I run atuin sync, they get downloaded again.
What to do now?
Also unrelated to that: How can I remove the history store for a specific host?
So, first I’d disable auto_sync to make sure no hosts push bad history
atuin store purge
- this will delete all records in the store that cannot be decrypted with the current keyatuin store verify
- verify that the previous operation was successfulatuin store push --force
- this will delete all records stored remotely, and then push up local data. Run this on the machine that has been purgedatuin store pull --force
- this does the opposite to (3). Delete all local data in the store, and pull from the remoteatuin store rebuild history
- ensure your history.db is up to date after all these operations
Currently not possible, though with a similar set of steps to the above it could be. Due to the nature of sync, ensuring that those steps propagate properly is difficult
I have made several edits and figured it out. When I was logging into each host I was generating a new key instead of using the same key for the same user. The result was that each host could only decrypt their own entries and it would spit out the error message about encryption. This wasn’t actually related to the new sync v2 it was just me making a newbie mistake. Ooops!
Have a great day!
Thanks for letting us know what it was! I’ll see if we can make it harder to make this mistake in the future
I have three nodes and get different amount of history when I use this:
atuin sync --force ; atuin history list --format "{host}" | sort --ignore-case | uniq -c --ignore-case | sort -n
Output examples from two nodes:
# on FV:
5425 sa
23942 ar
75240 FV
# on sa:
5470 sa
23942 ar
75229 fv
I’d expect a little skew of < 10 commands due to syncing/running stuff on one machine after another, but this is a little much?
Storage status:
# sa
host: 018b1f22-f498-7bd1-9573-ddf4c61970dd
store: history
idx: 27138
first: 018d9ce7-2feb-7c7b-acd9-fef5d519e6a8
created: 2024-02-12 10:40:13.291778 +00:00:00
last: 01907d6f-f0e6-7449-8816-2e1fedb97f62
created: 2024-07-04 11:10:06.566287 +00:00:00
host: 018a975e-e082-7198-a370-7362d00f20f9
store: history
idx: 87289
first: 018eed8c-00ba-7838-9bdf-848888a9569a
created: 2024-04-17 19:32:39.226825384 +00:00:00
last: 01907a3c-7705-7809-8251-2e73c252489f
created: 2024-07-03 20:15:01.38110538 +00:00:00
host: 018a98ec-8d19-7c04-8751-a3781bc6ce3e <- CURRENT HOST
store: history
idx: 1046
first: 018d9f6a-6e9a-7587-aabb-8faae3c63b58
created: 2024-02-12 22:22:48.986073096 +00:00:00
last: 01907d76-3780-72fb-ba90-afde09db9eef
created: 2024-07-04 11:16:57.856668815 +00:00:00
# FV
host: 018a975e-e082-7198-a370-7362d00f20f9
store: history
idx: 87289
first: 018eed8c-00ba-7838-9bdf-848888a9569a
created: 2024-04-17 19:32:39.226825384 +00:00:00
last: 01907a3c-7705-7809-8251-2e73c252489f
created: 2024-07-03 20:15:01.38110538 +00:00:00
host: 018a98ec-8d19-7c04-8751-a3781bc6ce3e
store: history
idx: 1045
first: 018d9f6a-6e9a-7587-aabb-8faae3c63b58
created: 2024-02-12 22:22:48.986073096 +00:00:00
last: 01907d75-5c5d-7a2b-9a40-718aaef6972e
created: 2024-07-04 11:16:01.757814519 +00:00:00
host: 018b1f22-f498-7bd1-9573-ddf4c61970dd <- CURRENT HOST
store: history
idx: 27151
first: 018d9ce7-2feb-7c7b-acd9-fef5d519e6a8
created: 2024-02-12 10:40:13.291778 +00:00:00
last: 01907d7c-6711-7ee1-8cc9-0a97399246c6
created: 2024-07-04 11:23:43.249684 +00:00:00
I’m not dependent on this, but it would be interesting to see why there are these differences…
Either way: thanks for your work
Looks like a skew of around 20 records? Not super implausible, depending on when sync has been ran. --force
doesn’t actually do anything on sync v2, but what’s the output of atuin sync
on each machine?
Otherwise, how about atuin doctor
?
Either way: thanks for your work
<3
# FV, mac with gnu utils:
atuin sync ; atuin history list --format "{host}" | gsort --ignore-case | guniq -c --ignore-case | gsort -n
Uploading 3 records to 018b1f22f4987bd19573ddf4c61970dd/history
[00:00:00] [###################################################################################################################################] 3/3 (0.0s)Downloading 1 records from 018a98ec8d197c048751a3781bc6ce3e/history
[00:00:00] [###################################################################################################################################] 1/1 (0.0s)4/2 up/down to record store
Sync complete! 104729 items in history database, force: false
5432 sa
23942 ar
75355 FV
# sa:
atuin sync ; atuin history list --format "{host}" | sort --ignore-case | uniq -c --ignore-case | sort -n
Uploading 2 records to 018a98ec8d197c048751a3781bc6ce3e/history
[00:00:00] [###################################################################################################################################] 2/2 (0.0s)Downloading 3 records from 018b1f22f4987bd19573ddf4c61970dd/history
[00:00:00] [###################################################################################################################################] 3/3 (0.0s)3/4 up/down to record store
Sync complete! 104775 items in history database, force: false
5479 sa
23942 ar
75354 fv
doctor on sa
:
[Filesystem] ZFS is known to have some issues with SQLite. Atuin uses SQLite heavily. If you are having poor performance, there are some workarounds here: Pool timed out while waiting for an open connection, ZFS · Issue #952 · atuinsh/atuin · GitHub
{
"atuin": {
"version": "18.3.0",
"sync": {
"cloud": false,
"records": true,
"auto_sync": true,
"last_sync": "2024-07-04 20:00:20.878997476 +00:00:00"
},
"sqlite_version": "3.44.0"
},
"shell": {
"name": "zsh",
"default": "zsh",
"plugins": [
"atuin"
],
"preexec": "built-in"
},
"system": {
"os": "Arch Linux",
"arch": "x86_64",
"version": "rolling",
"disks": [
[... much zfs and docker stuff ...]
FV:
{
"atuin": {
"version": "18.3.0",
"sync": {
"cloud": false,
"records": true,
"auto_sync": true,
"last_sync": "2024-07-04 19:58:24.02087 +00:00:00"
},
"sqlite_version": "3.44.0"
},
"shell": {
"name": "zsh",
"default": "zsh",
"plugins": [
"atuin"
],
"preexec": "built-in"
},
"system": {
"os": "Darwin",
"arch": "arm64",
"version": "14.5",
"disks": [
{
"name": "Macintosh HD",
"filesystem": "apfs"
For the sa
host ~50 records are missing