What is text-to-image retrieval?
Text-to-image retrieval is cross-modal search that embeds a natural-language query into a joint multimodal vector space and returns the nearest stored image (or image-bearing object) vectors – typically with HNSW – so a phrase like "red trail runners on gravel" can surface matching photos without relying on exact tag keywords.
How can words find pictures without shared keywords?
Classic image search depends on filenames, folders, or human tags. If nobody labeled a photo "ceramic mug with blue glaze," keyword lookup fails even when the pixels clearly show one. Text-to-image retrieval instead uses encoders trained so captions and matching images land near each other in one embedding space: contrastive training pulls paired text-image examples together and pushes unrelated pairs apart. At query time the text tower (or multimodal text path) turns the user’s sentence into a vector in that same space; nearest-neighbor search over image embeddings returns conceptual neighbors. The index does not need to understand English grammar – it only needs consistent geometry. Exact scan works for tiny galleries; production media libraries use HNSW so descriptive search stays interactive as asset counts grow into millions.
That pattern is a directed slice of broader cross-modal retrieval.
How does text-to-image differ from image similarity and reverse lookup?
Image-to-image similarity starts from a photo and finds look-alike or meaning-alike pictures in a vision (or multimodal) space – no text required. Reverse image lookup often aims at near-duplicates or known originals. Text-to-image starts from language: shoppers type intent, designers describe a mood board, analysts ask for "charts comparing quarterly revenue." The reverse direction – image-to-text – retrieves captions or documents for a seed image; both directions share the joint space when the model was trained for them. Product UIs sometimes combine lanes: text-to-image for discovery, then image-to-image for "more like this result." Quality still tracks domain fit. Models trained on web photo-caption pairs may struggle on industrial parts, medical scans, or diagram-heavy PDFs unless you use embeddings tuned for those visuals. Conceptual nearness is not identity match or facial recognition; those need stricter pipelines and thresholds.
Weaviate turns the joint space into a filterable HNSW catalog.
How do you run text-to-image retrieval in Weaviate?
Configure a collection with a multimodal vectorizer so image blob fields (and optional captions) project into one index, or bring your own jointly trained vectors. Ingest catalog or DAM assets as objects with image properties; Weaviate stores the embeddings in HNSW. Query with nearText – Weaviate embeds the query string with the same multimodal path and returns the nearest image objects. Attach filters for rights, collection, stock, or tenant so legal and inventory rules gate the shortlist. Named vectors can keep a dedicated multimodal space beside a text-only tower when you need both. For scanned documents and slide decks, page images embedded for text queries support visual RAG without forcing OCR first. Tune ef, limit, and distance on labeled text→image pairs; a graph that looks healthy on text-to-text recall can still under-serve descriptive visual queries if the geometry is harder. Optionally rerank or diversify when near-duplicate product shots crowd the top.
Operations fail most often when ingest and query leave the joint space.
What breaks text-to-image HNSW quality in production?
Mixing a text-only encoder at query time with a vision-only corpus (or two multimodal models that were never aligned) invents nonsense neighbors. Changing the multimodal model without re-embedding every image splits the space. Inconsistent preprocessing – different resize, color space, or language normalization between ingest and query – shifts points enough to tank recall. Over-trusting top-1 without human or labeled evaluation hides systematic misses on rare attributes (materials, viewpoints, fine-grained styles). Strict property filters with low selectivity empty shortlists the same way as in text ANN; measure filled-k under real predicates. Latency includes embedding the text query plus the HNSW walk – cache repeated phrases where traffic allows. After catalog refreshes, confirm new images are indexed and searchable by descriptive probes, not only by id lookup.
Text-to-image retrieval is HNSW over a joint text-image space – language is the query, pixels are the hits. Next, read "What is a joint embedding space?" for how that geometry is learned, "What is cross-modal retrieval?" for other modality pairs, or "What is update freshness in retrieval systems?" when captions and images change after first index.