Search output order is not what I expected

buro · January 30, 2024, 8:47pm

Hello all,

I just started using atuin, before I used a fzf zsh plugin.

What make me uncomfortable is the order of the result: I expected the order to be by timestamp, but with the search_mode=‘fuzzy’ the result is re-ordered in some other way.

I saw the code that do that:
The sql with “order by timestamp”

github.com

atuinsh/atuin/blob/f6b541dbedcc3ae719dd3d1d0f889f2ca8e0f7d6/atuin-client/src/database.rs#L408


      
              sql.limit(limit);
          }
          
          if let Some(offset) = filter_options.offset {
              sql.offset(offset);
          }
          
          if filter_options.reverse {
              sql.order_asc("timestamp");
          } else {
              sql.order_desc("timestamp");
          }
          
          let git_root = if let Some(git_root) = context.git_root.clone() {
              git_root.to_str().unwrap_or("/").to_string()
          } else {
              context.cwd.clone()
          };
          
          match filter {
              FilterMode::Global => &mut sql,

Then a re-ordering here:

github.com

atuinsh/atuin/blob/f6b541dbedcc3ae719dd3d1d0f889f2ca8e0f7d6/atuin-client/src/database.rs#L522


      
          
              sql.and_where_is_null("deleted_at");
          
              let query = sql.sql().expect("bug in search query. please report");
          
              let res = sqlx::query(&query)
                  .map(Self::query_history)
                  .fetch_all(&self.pool)
                  .await?;
          
              Ok(ordering::reorder_fuzzy(search_mode, orig_query, res))
          }
          
          async fn query_history(&self, query: &str) -> Result<Vec<History>> {
              let res = sqlx::query(query)
                  .map(Self::query_history)
                  .fetch_all(&self.pool)
                  .await?;
          
              Ok(res)
          }

And here the “real” reordering (if fuzzy):

github.com

atuinsh/atuin/blob/f6b541dbedcc3ae719dd3d1d0f889f2ca8e0f7d6/atuin-client/src/ordering.rs#L7


      
          use minspan::minspan;
          
          use super::{history::History, settings::SearchMode};
          
          pub fn reorder_fuzzy(mode: SearchMode, query: &str, res: Vec<History>) -> Vec<History> {
              match mode {
                  SearchMode::Fuzzy => reorder(query, |x| &x.command, res),
                  _ => res,
              }
          }
          
          fn reorder<F, A>(query: &str, f: F, res: Vec<A>) -> Vec<A>
          where
              F: Fn(&A) -> &String,
              A: Clone,
          {
              let mut r = res.clone();

I tried with ‘fulltext’ search mode and results are ordered as expected, by timestamp, but I don’t like the searching algorithm.

I would like to have the choice to order the result by timestamp and searching by fuzzy algorithm.

What do you think?
Thanks!

ellie · January 31, 2024, 11:21am

Fuzzy tries to sort by the best match, not by time. Unfortunately we won’t be able to make large changes to it - what some people dislike, other people like, and vice-versa. I would be open to making it configurable though, but I’m not sure the fuzzy algorithm would be best for that.

What didn’t you like about the fulltext algorithm? Did you try skim?

We’ve got this issue opened for enhanced querying

github.com/atuinsh/atuin

Add enhanced search querying

opened 08:24AM - 19 Dec 23 UTC

ellie

enhancement

While we have filter modes, Atuin does not do an amazing job of exposing the con…textual information we store in the TUI search It would be great if searches could work like this in the TUI: `cargo build cwd:~/src exit:0` This would do a normal search for `cargo build`, but specify that it should only apply to commands that exited successfully in the `~/src` dir We'd probably want to implement the following | name | function | -------| --------- after | filtering history after a date before | filtering history before a date cwd | filtering by directory exit | filter exit codes duration | filter by command duratio host | filter by hostname (maybe hold off until host aliasing)

I’d like to implement this with something like Tantivy, which will give us much better control over which fields from history are most important

buro · January 31, 2024, 12:44pm

fulltext is too “strict” to me. I give a try to skim, it seems really close to fuzzy syntax.

If I can give a feedback the documentation doesn’t mention about the fuzzy reorder (is the only search_mode that changes the order) and I suppose that the fuzzy mode makes the --reverse option useless because ignored by reordering.

ellie · January 31, 2024, 12:58pm

I think the documentation assumes the user is aware that the search tries to find the best match, which is not necessarily the most recent one. Fuzzy will give truly horrible matches if it strictly orders by time

Would this work better for you?

github.com/atuinsh/atuin

feat: add an exact search mode

atuinsh:main ← lilydjwg:exact

opened 11:03AM - 31 Jan 24 UTC

lilydjwg

+8 -1

This pr adds a new search mode called "exact" as skim calls it, in which the que…ry are matched as substrings of the target. In this way, there are much fewer results than the so-called fuzzy search mode, thus provide more relevant results as long as the query is correct. I usually can't relate fuzzy results to my queries, so I always use this kind of substring match. There is also a full text search mode, which I find less useful as only one substring can be provided. ## Checks - [x] I am happy for maintainers to push small adjustments to this PR, to speed up the review cycle - [x] I have checked that there are no existing pull requests for the same thing

buro · January 31, 2024, 1:30pm

I was not aware, but now I am

Use of “exact” instead of “fulltext” seems better in my opinion. But I will go with skim that could be a “super set” of “exact”.

I notice that the main (maybe the only) difference between skim and fuzzy is the use of AND instead of OR for separated words. I like it.

Because of that I think my best option is skim.

Thank you Ellie for your work and fast response!