Skip to content
audiotale.net
Sign in
NewsNews

EmbeddingGemma 2 technical report: 740M open multimodal embeddings, how to run it and the float16 trap

Google's Apache 2.0 embedding model maps text, code, images, audio and video into one 768-dimensional space; the model card has the numbers and the pitfalls.

audiotale.net team

· 6 min read

Written by AI from the sources linked below. How we use AI

On this page
  1. Key points
  2. What to do
  3. What happened
  4. Background
  5. The numbers
  6. What it means
  7. What we don't know yet
  8. What to watch next
  9. Sources

Google DeepMind's EmbeddingGemma 2, an open model that turns text, code, images, audio and video into vectors in one shared space, now has a technical report: Google linked the paper from its blog on 17 November 2026, and the arXiv listing (2610.37364) shows it submitted on 27 October and revised on 9 November. The model itself has been downloadable since 6 October under the Apache 2.0 licence. The report is the reason to look again, because the model card holds the detail that decides whether it works in your search or retrieval system: vector sizes, memory costs, a numerical-precision trap and a quality cliff at 128 dimensions.

Key points

  • EmbeddingGemma 2 has 740 million parameters in total: a 270M text model plus optional 170M vision and 300M audio encoders, per the model card.
  • One 768-dimensional vector space covers text (including code), images, video and audio; the context window is 8,192 tokens, shared by all inputs.
  • Google reports an MTEB code score of 78.68, up from 68.76 for EmbeddingGemma 1, a gain of 9.92 points.
  • Vectors can be cut to 512, 256 or 128 dimensions (up to 6x less storage), but 128 dimensions costs a lot of multimodal quality, and float16 inference returns NaN or degraded embeddings.
  • Weights are on Hugging Face (google/embeddinggemma-2) and Kaggle under Apache 2.0, and Google lists support in transformers, sentence-transformers, MLX, vLLM, SGLang, Ollama, LM Studio and llama.cpp.

What to do

If you want to run it, this is the short, checked path from the model card.

  1. Install: pip install -U sentence-transformers transformers, then SentenceTransformer("google/embeddinggemma-2").
  2. Use the task prefixes. Queries use prompt names such as SearchQuery; documents use Document. A titled document must be formatted by hand as title: {title} | text: {content}, because the Document prompt applies title: none. Prefixes apply to text only; pass images, video and audio without any.
  3. Do not use float16. The model card says the activation range exceeds float16, so you get NaN or silently degraded embeddings instead of an error. Use bfloat16 where the hardware supports it and float32 otherwise, including on most CPUs.
  4. If you truncate vectors, re-normalise them. Pass truncate_dim and normalize_embeddings=True to encode(). Skipping the normalisation "degrades ranking quality silently", says the card. Queries and documents must use the same dimension.
  5. Load only what you need. Text only is about 270M parameters, text plus image 440M, text plus audio 570M, everything 740M, by turning off the vision or audio config.
  6. Test on your data. Google itself says the model has had no post-training alignment, safety tuning or output moderation, so filtering and fairness checks are the deployer's job.

What happened

On 17 November Google's developer blog pointed readers to the EmbeddingGemma 2 technical report, describing the model as expanding its text-only open embedding model "with native multimodal understanding." The launch post is dated 6 October, and the model has been public since then: the Hugging Face repository is open without gating and the listing shows hundreds of thousands of downloads. The paper's abstract says the authors, 77 contributors led by Henrique Schechter Vera, use simplified per-layer embeddings and a multi-stage training curriculum, and claim a new Pareto frontier across modalities, outperforming specialist models and models two to twelve times larger. Those comparisons are the authors' own claims; we have not reproduced them.

Background

An embedding model converts content into a list of numbers so that similar things end up close together. Search, retrieval-augmented generation (RAG), classification and clustering all rely on one. EmbeddingGemma 1 was a text-only open model; its multilingual MTEB score of 61.15 is the baseline in Google's table. Version 2 builds on the Gemma 4 architecture, adds vision and audio encoders, and keeps the aim of running on consumer hardware such as phones and laptops.

A single space for several media matters because it allows cross-modal search: a text query can retrieve a video moment or an audio clip, and a product listing with photos and a demo video can be indexed as one vector. The model card gives exactly that example, with <|image|> and <|video|> placeholders marking where media sits in the text.

The numbers

Google's model card reports results for the full-precision checkpoint at 768 dimensions. The headline gain is in code retrieval; the text score is close to flat.

EmbeddingGemma 2 vs 1, text benchmarks (768 dimensions)score
EmbeddingGemma 2 vs 1, text benchmarks (768 dimensions) (score)
score
MTEB code, EmbeddingGemma 278.68
MTEB code, EmbeddingGemma 168.76
MTEB multilingual, EmbeddingGemma 261.36
MTEB multilingual, EmbeddingGemma 161.15

Source: Google model card, google/embeddinggemma-2 on Hugging Face

For the other modalities the card lists only EmbeddingGemma 2 scores: 65.67 on MIEB lite (images), 57.28 on MMEB v2 image, 67.84 on MMEB v2 visual documents, 50.67 on MMEB v2 video, 69.54 on MSEB retrieval and 49.87 on MAEB (audio).

The more practical table is the truncation one. Shortening vectors saves storage and speeds up search, and the card shows what it costs.

Overall multimodal score (MMEB v2) by vector sizescore
Overall multimodal score (MMEB v2) by vector size (score)
score
768d59.01
512d58.38
256d56.24
128d45.65

Source: Google model card, google/embeddinggemma-2 on Hugging Face

Text-only workloads fare better: the multilingual MTEB score is 61.36 at 768 dimensions, 61.17 at 512, 60.41 at 256 and 57.89 at 128, and the code score falls from 78.68 to 71.41 at 128. The card calls quality "close to lossless down to 256 dimensions" and says 128 suits text-only work, to be validated on your own workload.

Capacity is set by the shared 8,192-token window. Defaults are 280 tokens per image, 140 per video frame and 25 per second of audio, which works out to about 29 images, 58 frames or 327 seconds (the launch post says 5.5 minutes) in a single modality with no text. Lowering the vision budget can fit about 114 images or frames. Google also says that, quantised on a Pixel 11 Pro, the model needs about 191 MB of active RAM for text-only weights and about 567 MB for the full multimodal model.

What it means

For developers building search or RAG. A permissively licensed, laptop-sized model that handles code and several media in one index removes the need for separate pipelines. The 6x storage saving from truncation is real, but the 128-dimension multimodal result (45.65 against 59.01) is a warning to test before shrinking.

For on-device and privacy-sensitive apps. Local media search without sending files to a remote service is the use case Google and the press coverage emphasise. The selective loading of encoders helps keep memory down.

For the open-model field. Google claims the lead among multimodal embedding models under one billion parameters on code and audio benchmarks. Because these are the company's own figures on public benchmarks, treat them as a starting point for your own evaluation.

What we don't know yet

  • Independent results. We found no third-party benchmark reproducing Google's claims.
  • Real-world cost. The Pixel memory figures are Google's, measured on one device with quantisation; the card does not give latency per query.
  • Safety behaviour. Google states the model has had no output moderation, but not how embeddings behave on harmful or sensitive content.
  • Training data detail. The card says the pre-training data runs to January 2025 and covers more than 140 languages, while the model is described as understanding "100+ languages"; the paper may clarify the difference.

What to watch next

Watch for the promised availability in Google's Model Garden, independent leaderboards on MTEB and MMEB, and vector-database vendors adding multimodal support. Also watch for quantised community builds, since the float16 trap affects anyone converting the model.

Sources

Written by

audiotale.net team

The official account of the people who run audiotale.net: announcements, conversation starters and the newsroom's drafts. Posts written with AI say so. Every other account here is a real person.