Search backend refactor with typesense impl (#5528)

* initial elasticsearch impl

* working elastic cluster

* replace SearchError with ApiError for preparation of search backend

* start factoring meili out to trait

* move meili to backend

* update routes to use search backend trait

* wip

* Update projects.rs

* search backend is only init'd once in config

* wip

* wip: backend agnostic

* change search internal routes to delegate to backend

* initial elasticsearch impl

* fix filtering

* elastic impl

* refactor indexing into its own module

* clean up elastic code

* fix ci

* fix tests

* fix elastic health check

* fix up env rebase

* fix compile

* dummy commit to update github pr

* Fix rebase

* Elastic basic https auth

* Fix duplicate projects showing up

* Fix up tests

* Replace search `ApiErrors` with `eyre::Reports`, propagate background task errors

* clean up agents files

* make index chunk size configurable

* make `match_phrase` in elastic case-insensitive

* use current/next indices and swap between them

* test case for error body

* Fix failing case

* da merge

* factor out common stuff from search backends

* allow fetching hit metadata from search results

* allow customising elasticsearch search config

* bit of docs

* add mappings to indices for elastic

* Implement Typesense

* wip

* fix up some sort fields stuff

* use different approach to filterable field sets

* remove a bunch of search fields which weren't used for filtering

* bucket text matches

* Bucketing by text_match for typesense

* fix tombi lint

* fix some sentry errors and dont prioritise 2+ term matches

* tweak ts query settings

* expose some more search settings

* query sort changes

* small fixes

* should fix pagination stuff

* fix healthcheck maybe

* ragebait ci

* tests

* tests

* revert environment

apps/labrinth/src/models/v2/search.rs

@@ -159,15 +159,20 @@ impl LegacySearchResults {
     pub fn from(search_results: crate::search::SearchResults) -> Self {
         let limit = search_results.hits_per_page;
         let offset = (search_results.page - 1) * limit;
+        let crate::search::SearchResults {
+            hits,
+            page: _,
+            hits_per_page: _,
+            total_hits,
+        } = search_results;
         Self {
-            hits: search_results
-                .hits
+            hits: hits
                 .into_iter()
                 .map(LegacyResultSearchProject::from)
                 .collect(),
             offset,
             limit,
-            total_hits: search_results.total_hits,
+            total_hits,
         }
     }
 }

apps/labrinth/src/models/v3/projects.rs

@@ -1020,20 +1020,3 @@ impl FileType {
 )]
 #[serde(transparent)]
 pub struct Loader(pub String);
-
-// These fields must always succeed parsing; deserialize errors aren't
-// processed correctly (don't return JSON errors)
-#[derive(Serialize, Deserialize, Debug)]
-pub struct SearchRequest {
-    pub query: Option<String>,
-    pub offset: Option<String>,
-    pub index: Option<String>,
-    pub limit: Option<String>,
-
-    pub new_filters: Option<String>,
-
-    // TODO: Deprecated values below. WILL BE REMOVED V3!
-    pub facets: Option<String>,
-    pub filters: Option<String>,
-    pub version: Option<String>,
-}