Full search page →

Why does this document rank first?

AdvancedNLP 09 · Language Modeling with Transformers (Natural Language Generation)Interactive plot

Description

Inspect BM25 term saturation and document-length normalization on a fully visible local corpus.

Try this: Search “red”, then “robots objects”. Change length normalization and saturation; explain each rank swap from document terms.

Lab · Why does this document rank first?

Top document
D2
Top score
1.86
Matching documents
4

IDF rewards terms found in fewer documents. k₁ controls how repeated matches saturate; b controls the penalty for length relative to this corpus mean.

Try this: Search “red”, then “robots objects”. Change length normalization and saturation; explain each rank swap from document terms.

Six synthetic documents, lowercase English word tokenization, distinct query terms, positive smoothed IDF. A lexical relevance score is not evidence that a document answers a question.

Method sources

Open in the Playground →