Datasets and benchmark suites

These are the topics in this section, each on its own page with a stable path you can bookmark or share.

What topics are covered in this section?

• What is the SIFT1M dataset?
• What is the GIST1M dataset?
• What is the GloVe embedding dataset?
• What is the BIGANN dataset?
• What is the DEEP1B dataset?
• What is the ANN-Benchmarks methodology?
• What is the VIBE benchmark?
• What is Big ANN Benchmarks?
• What is a synthetic intrinsic-dimension dataset?
• What is a filtered benchmark dataset?

Datasets and benchmark suites name the standard corpora and methodologies used to compare ANN systems – SIFT, GIST, GloVe, BIGANN, DEEP1B, ANN-Benchmarks, and related filtered or synthetic sets.

Why do shared datasets matter?

Without common vectors and ground truth, recall numbers are incomparable. SIFT1M and GIST1M are classic vision-derived sets; GloVe brings text embeddings; BIGANN and DEEP1B stress billion-scale regimes. ANN-Benchmarks, VIBE, and Big ANN Benchmarks define protocols, not just files. Synthetic intrinsic-dimension sets and filtered benchmark datasets probe geometry and predicates that vanilla sets miss.

Pick a suite that resembles your embedding dimensionality and filter behavior.

How should Weaviate users use them?

Weaviate publishes ANN benchmark results across parameter sweeps on public sets – use the closest match as a prior, then re-run on your data. Public sets rarely capture your filter correlation or update churn, so production golden sets remain mandatory. Dataset pages here explain what each corpus is good for, not which vendor wins.

Part VIII’s suites chapter narrates selection criteria.

Where next?

Skim SIFT1M, GloVe, and ANN-Benchmarks pages, then Part VIII. Add filtered and synthetic sets when predicates or intrinsic dimension are your unknowns.

This section catalogs the yardsticks of ANN research. Next, read “What is the ANN-Benchmarks methodology?” or Part VIII’s suites chapter, then build a small internal golden set that mirrors production.