Is there somewhere to see what LLMs other than OpenAI are supported? I saw another thread that said mistral is not. I am using an open source LLM (openChat) and trying to see how I can connect it to Marqo.
Hi @megler,
Marqo provides the support for a variety of embedding models to convert your images and text into vectors. A full list can be found in the dense retrieval page in our docs.
Marqo is a good choice of vector database to pair with LLMs for use cases like Retrieval Augmented Generation (RAG) however Marqo is not a platform for hosting the generative LLMs themselves, instead it provides the embeddings generation, storage, and retrieval for search. The mistral model referred to in the other post was one designed for embeddings and not one designed for chat.
Typically we recommend hf/e5-base-v2 for text and open_clip/ViT-B-32/laion2b_s34b_b79k for images and text, these are good medium sized models. For larger models the hf/multilingual-e5-large and open_clip/ViT-L-14/laion2b_s32b_b82k are good choices.
I haven’t used OpenChat but looking at their API it should be able to support a RAG implementation with Marqo as the knowledge store if that is what you are wanting to build?
Hello Owen!
Thanks for the reply! Sorry if my question was not clearly phrased. You are correct, I am looking to use Marqo to pair with a LLM for RAG. I’ve had decent success with OpenChat out of the box and am now investigating what open source I can use to augment its knowledge via a knowledge store.
I believe I had a fundamental misunderstanding of how RAG worked. It just clicked. Thank you again for your time!
1 Like