Add new article on how to choose embedder type #3058
Conversation
How to see the preview of this PR?Go to this URL: https://website-git-deploy-preview-mei-16-meili.vercel.app/docs/branch:3040-choose-embedder-guide Credentials to access the page are in the company's password manager as "Docs deploy preview". |
|
|
||
| Meilisearch does not support searching images, audio, or any other content not presented as text. This limitation applies to both queries and documents. For example, Meilisearch's built-in embedder sources cannot search using an image instead of text. They also cannot use text to search for images without attached textual metadata. | ||
|
|
||
| In these cases, you will have to supply your own embedder. |
There was a problem hiding this comment.
| In these cases, you will have to supply your own embedder. | |
| In these cases, you will have to supply your own embeddings. |
|
|
||
| In these cases, you will have to supply your own embedder. | ||
|
|
||
| ## Only choose Hugging Face when self-hosting small static datasets |
There was a problem hiding this comment.
This was initially advice for Cloud users using HF embedders because we were generating the embeddings locally (running on our Cloud infra). This is no longer the case, we removed the option on the Cloud and we've replaced it with the Hugging Face Inference points using the REST embedder option.
Self-hosted users can still use HuggingFace as an embedder option, as they can tweak their infrastructure to fit their specific needs.
There was a problem hiding this comment.
We can either remove this section, or point users in the direction of how to set a HF embedder using the REST option (for Cloud) and the API reference (for self hosted)
|
|
||
| OpenAI returns relevant search results across different subjects and datasets. It is suited for the majority of applications and Meilisearch actively supports and improves OpenAI functionality with every new release. | ||
|
|
||
| In the majority of cases, and especially if this is your first time working with LLMs and AI-powered search, choose OpenAI. |
There was a problem hiding this comment.
(Feel free to ignore this one as it might be just me) I wonder if this doesn't sound a bit too biased towards OpenAI, making it sound like it's our provider of choice over others. Maybe we can phrase it around "ease of config" or "easiest for beginners" as it only requires pasting the OpenAI key.
|
|
||
| If you are already using a specific model from a compatible embedder, choose Meilisearch's REST embedder. This ensures you continue building upon tooling and workflows already in place with minimal configuration necessary. | ||
|
|
||
| ## If dealing with non-textual content, choose the user-provided embedder |
There was a problem hiding this comment.
| ## If dealing with non-textual content, choose the user-provided embedder | |
| ## If dealing with non-textual content, choose the Custom (user-provided) embedder |
|
|
||
| ## If dealing with non-textual content, choose the user-provided embedder | ||
|
|
||
| Meilisearch does not support searching images, audio, or any other content not presented as text. This limitation applies to both queries and documents. For example, Meilisearch's built-in embedder sources cannot search using an image instead of text. They also cannot use text to search for images without attached textual metadata. |
There was a problem hiding this comment.
pinging @dureuill as I'm not 100% sure about this one
There was a problem hiding this comment.
yes that is correct. We may want to specify that, by supplying the embeddings generated using their own embedder, the user can indeed achieve these use cases.
Closes #3040