Professional services
Semantic matching and deduplication at archive scale
Models that recognise when two documents say the same thing — the quiet capability behind clean knowledge bases and find-the-precedent search.
This page describes a problem class and the architecture we deploy for it, written against a named public benchmark — the open dataset or competition where the world's data scientists have tested approaches to this exact problem against a hard metric. It is not a client engagement, and the benchmark's results, prizes and rankings belong to its host and participants, not to us. Client work is confidential and is only ever published with written permission.
The problem
What it costs when this goes unsolved.
Every mature archive accumulates near-duplicates: the same question answered four ways, the same clause redrafted across a hundred templates, the same matter described differently by two teams. The costs are quiet but real — professionals re-answer solved questions, and search returns four versions with no signal about which is current. Matching by meaning rather than by string is what fixes it.
Paraphrase detection looks easy until the pairs get hard: same words with different meaning, different words with the same meaning, and — the trap the benchmark for this problem made famous — structural signals that leak the label without the model learning any semantics. An evaluation that ignores that produces a system that demos well and fails on the archive.
How we build it
The architecture, stage by stage.
01
Embedding models tuned to your domain
Sentence encoders fine-tuned on your document pairs, because general-purpose embeddings blur exactly the distinctions professionals care about.
02
Cross-encoder verification on candidates
Fast similarity search proposes; an expensive pairwise model confirms — precision where it counts, at a compute cost that scales.
03
Graph structure over the pair level
Duplicate relations are transitive and cluster-shaped; resolving them over the co-occurrence graph catches what pairwise scoring misses.
04
Leakage-aware evaluation
Test sets constructed so the model is measured on semantics, not on structural artifacts — the methodological lesson this benchmark taught the field.
Model families on this problem class: Fine-tuned sentence encoders · Cross-encoder re-ranking · Similarity graphs · ANN search.
What you get
What an engagement hands over.
Everything below goes in the scope document before you sign it, with a fixed price or a rate with a ceiling — the same terms as every other engagement in the catalogue.
- A semantic-matching service over your document or ticket corpus
- Deduplication passes with human-review queues for the marginal cases
- 'More like this' retrieval integrated into your existing search
- Precision/recall reporting at the thresholds your workflow actually uses
Provenance
The benchmark behind this page.
Quora Question Pairs, run by Quora, is the public proving ground for this problem class. The figures below are the host's, cited as context for how seriously this problem is tested in the open — they are not our results and we do not claim them.
How to buy this
The services this build draws on.
More proof
Next step
Thirty minutes on whether this fits your problem.
Bring the constraint — the regulator, the data boundary, the latency budget. If your data cannot support this build yet, the call will conclude with what to fix first, not with a proposal.