Full search page →

Alignment, hard negatives and collapsed features

BeginnerNLP 04 · Lexical Semantics and Word EmbeddingsInteractive plot

Description

Move paired and unpaired unit vectors, then inspect the normalized matching loss used by contrastive learning.

Try this: Bring a negative closer to the anchor than the positive. Does a lower temperature help the correct pair? Then collapse all vectors.

Lab · Alignment, hard negatives and collapsed features

Positive probability
0.896
Matching loss, nats
0.109
Positive cosine
0.94

Loss = −log(exp(sim⁺/τ) / Σ exp(sim/τ)). Temperature sharpens whichever candidate already has the highest similarity. Collapsed unit vectors give equal probabilities and loss log(4).

Try this: Bring a negative closer to the anchor than the positive. Does a lower temperature help the correct pair? Then collapse all vectors.

One anchor, one positive and three negative candidates; vectors are controlled directly rather than trained. The same matching geometry explains image–text retrieval, but these are not CLIP features or semantic predictions.

Method sources

Open in the Playground →