Solon Embeddings 0.1

We’re releasing Solon Embeddings 0.1, a family of open models for semantic search in French. Both versions, Base and Large, are available on Hugging Face under the MIT license.

With Solon, we’re working on a fundamental part of document search: finding the right passage when a question and its answer use different words. Here’s how the models work, how we evaluated them, and what to consider when using them with your own documents.

Finding meaning beyond matching words

Imagine searching a collection of contracts:

Question: “How can we end this agreement?” Passage to find: a clause headed “Termination.”

This example illustrates the problem semantic search addresses. An exact keyword match won’t always connect a question to the passage that answers it.

An embedding model turns each text into a vector: a sequence of numbers that captures aspects of its meaning. We can then compare the question’s representation with those of the passages to select likely matches.

In a retrieval-augmented generation system, or RAG, this search happens before an answer is written. The retrieved passages give the language model documentary context. If search misses an essential document, fluent writing won’t make up for it.

That’s also why we distinguish similarity from interpretation. Two clauses can both concern termination while setting out opposing conditions. Embeddings help find them; comparing their meaning still requires careful reading.

Solon Base and Large compared

We offer two model sizes so teams can weigh retrieval quality against the resources required. These are the specifications of the published versions:

Configurations and results are available in the Solon Base and Solon Large repositories. The mean score summarizes our initial evaluation; it is not a percentage of correct answers.

Large performs better in this evaluation. Base produces more compact vectors. To choose between them, we recommend measuring retrieval quality on your own questions, alongside latency and memory use under the same conditions.

How we evaluated the models

We brought together nine French-language benchmarks: six tasks from MTEB, a passage reranking task derived from MIRACL, and two datasets developed by our team. Together, they cover classification, text similarity, and ranking passages by their relevance to a query.

Solon Large ranks first among the models in the comparison published with version 0.1. That result belongs to a specific evaluation and set of models. It doesn’t establish a lead across every use case or over models released since.

The research behind MTEB highlights this difficulty: strong performance on one task does not necessarily carry over to others. An average provides a reference point. On its own, it won’t tell you how a model will search a law firm’s documents.

To make our evaluation open to scrutiny, we’ve published Ordalie FR-STS, which measures text similarity, and Ordalie FR-Reranking, which evaluates passage ranking. The evaluation code is available too. Since our team created two of the nine datasets, testing on an independent collection is a useful complement to these results.

Moving from benchmarks to your documents

The 512-token limit means long documents need to be split into passages before indexing. A token is a unit of text, not necessarily a whole word. That split deserves attention: separating a clause from its definitions can remove context the model needs to retrieve it.

There’s another integration detail to preserve. Our model cards specify the prefix query : for search queries, with no equivalent prefix for passages. Text preparation and encoding settings should remain consistent between indexing and search.

For an initial test, start with questions people actually ask of your documents and identify the passages you expect to find. Then inspect the top results. Which answer the question? Which merely concern a similar topic? What is missing? This review helps you compare models and adjust passage boundaries using examples your team can judge.

The Solon weights are available to run these tests on your own infrastructure. If you use the models, we’d like to hear which documents they handle well and where they struggle. Write to us at [email protected].