EmbeddingGemma 2 technical report: 740M open multimodal embeddings, how to run it and the float16 trap
Google's Apache 2.0 embedding model maps text, code, images, audio and video into one 768-dimensional space; the model card has the numbers and the pitfalls.
· 6 min read
Written by AI from the sources linked below. How we use AI
On this page
Google DeepMind's EmbeddingGemma 2, an open model that turns text, code, images, audio and video into vectors in one shared space, now has a technical report: Google linked the paper from its blog on 17 November 2026, and the arXiv listing (2610.37364) shows it submitted on 27 October and revised on 9 November. The model itself has been downloadable since 6 October under the Apache 2.0 licence. The report is the reason to look again, because the model card holds the detail that decides whether it works in your search or retrieval system: vector sizes, memory costs, a numerical-precision trap and a quality cliff at 128 dimensions.
Key points
- EmbeddingGemma 2 has 740 million parameters in total: a 270M text model plus optional 170M vision and 300M audio encoders, per the model card.
- One 768-dimensional vector space covers text (including code), images, video and audio; the context window is 8,192 tokens, shared by all inputs.
- Google reports an MTEB code score of 78.68, up from 68.76 for EmbeddingGemma 1, a gain of 9.92 points.
- Vectors can be cut to 512, 256 or 128 dimensions (up to 6x less storage), but 128 dimensions costs a lot of multimodal quality, and
float16inference returns NaN or degraded embeddings. - Weights are on Hugging Face (
google/embeddinggemma-2) and Kaggle under Apache 2.0, and Google lists support in transformers, sentence-transformers, MLX, vLLM, SGLang, Ollama, LM Studio and llama.cpp.
What to do
If you want to run it, this is the short, checked path from the model card.
- Install:
pip install -U sentence-transformers transformers, thenSentenceTransformer("google/embeddinggemma-2"). - Use the task prefixes. Queries use prompt names such as
SearchQuery; documents useDocument. A titled document must be formatted by hand astitle: {title} | text: {content}, because theDocumentprompt appliestitle: none. Prefixes apply to text only; pass images, video and audio without any. - Do not use
float16. The model card says the activation range exceedsfloat16, so you get NaN or silently degraded embeddings instead of an error. Usebfloat16where the hardware supports it andfloat32otherwise, including on most CPUs. - If you truncate vectors, re-normalise them. Pass
truncate_dimandnormalize_embeddings=Truetoencode(). Skipping the normalisation "degrades ranking quality silently", says the card. Queries and documents must use the same dimension. - Load only what you need. Text only is about 270M parameters, text plus image 440M, text plus audio 570M, everything 740M, by turning off the vision or audio config.
- Test on your data. Google itself says the model has had no post-training alignment, safety tuning or output moderation, so filtering and fairness checks are the deployer's job.
What happened
On 17 November Google's developer blog pointed readers to the EmbeddingGemma 2 technical report, describing the model as expanding its text-only open embedding model "with native multimodal understanding." The launch post is dated 6 October, and the model has been public since then: the Hugging Face repository is open without gating and the listing shows hundreds of thousands of downloads. The paper's abstract says the authors, 77 contributors led by Henrique Schechter Vera, use simplified per-layer embeddings and a multi-stage training curriculum, and claim a new Pareto frontier across modalities, outperforming specialist models and models two to twelve times larger. Those comparisons are the authors' own claims; we have not reproduced them.
Background
An embedding model converts content into a list of numbers so that similar things end up close together. Search, retrieval-augmented generation (RAG), classification and clustering all rely on one. EmbeddingGemma 1 was a text-only open model; its multilingual MTEB score of 61.15 is the baseline in Google's table. Version 2 builds on the Gemma 4 architecture, adds vision and audio encoders, and keeps the aim of running on consumer hardware such as phones and laptops.
A single space for several media matters because it allows cross-modal search: a text query can retrieve a video moment or an audio clip, and a product listing with photos and a demo video can be indexed as one vector. The model card gives exactly that example, with <|image|> and <|video|> placeholders marking where media sits in the text.
The numbers
Google's model card reports results for the full-precision checkpoint at 768 dimensions. The headline gain is in code retrieval; the text score is close to flat.
For the other modalities the card lists only EmbeddingGemma 2 scores: 65.67 on MIEB lite (images), 57.28 on MMEB v2 image, 67.84 on MMEB v2 visual documents, 50.67 on MMEB v2 video, 69.54 on MSEB retrieval and 49.87 on MAEB (audio).
The more practical table is the truncation one. Shortening vectors saves storage and speeds up search, and the card shows what it costs.
| score | |
|---|---|
| 768d | 59.01 |
| 512d | 58.38 |
| 256d | 56.24 |
| 128d | 45.65 |
Source: Google model card, google/embeddinggemma-2 on Hugging Face
Text-only workloads fare better: the multilingual MTEB score is 61.36 at 768 dimensions, 61.17 at 512, 60.41 at 256 and 57.89 at 128, and the code score falls from 78.68 to 71.41 at 128. The card calls quality "close to lossless down to 256 dimensions" and says 128 suits text-only work, to be validated on your own workload.
Capacity is set by the shared 8,192-token window. Defaults are 280 tokens per image, 140 per video frame and 25 per second of audio, which works out to about 29 images, 58 frames or 327 seconds (the launch post says 5.5 minutes) in a single modality with no text. Lowering the vision budget can fit about 114 images or frames. Google also says that, quantised on a Pixel 11 Pro, the model needs about 191 MB of active RAM for text-only weights and about 567 MB for the full multimodal model.
What it means
For developers building search or RAG. A permissively licensed, laptop-sized model that handles code and several media in one index removes the need for separate pipelines. The 6x storage saving from truncation is real, but the 128-dimension multimodal result (45.65 against 59.01) is a warning to test before shrinking.
For on-device and privacy-sensitive apps. Local media search without sending files to a remote service is the use case Google and the press coverage emphasise. The selective loading of encoders helps keep memory down.
For the open-model field. Google claims the lead among multimodal embedding models under one billion parameters on code and audio benchmarks. Because these are the company's own figures on public benchmarks, treat them as a starting point for your own evaluation.
What we don't know yet
- Independent results. We found no third-party benchmark reproducing Google's claims.
- Real-world cost. The Pixel memory figures are Google's, measured on one device with quantisation; the card does not give latency per query.
- Safety behaviour. Google states the model has had no output moderation, but not how embeddings behave on harmful or sensitive content.
- Training data detail. The card says the pre-training data runs to January 2025 and covers more than 140 languages, while the model is described as understanding "100+ languages"; the paper may clarify the difference.
What to watch next
Watch for the promised availability in Google's Model Garden, independent leaderboards on MTEB and MMEB, and vector-database vendors adding multimodal support. Also watch for quantised community builds, since the float16 trap affects anyone converting the model.
Sources
- Google DeepMind model card: google/embeddinggemma-2 on Hugging Face
- Google: EmbeddingGemma 2: an open, lightweight multimodal embedding model (6 October 2026)
- Google: Read the EmbeddingGemma 2 technical report (17 November 2026)
- arXiv: EmbeddingGemma 2 technical report, 2610.37364
- Let's Data Science: DeepMind releases EmbeddingGemma 2 for edge search
Written by
The official account of the people who run audiotale.net: announcements, conversation starters and the newsroom's drafts. Posts written with AI say so. Every other account here is a real person.
Read next
Arete-27B: Nous retunes Qwen3.8-27B, Apache-2.0, flagged propaganda 22% to 1% on its own test
Nous Research released a supervised fine-tune of Qwen3.8-27B aimed at politically sensitive prompts; all results come from Nous's own judge and prompts, with no independent test yet.
audiotale.net team7 min read
Inkling 2 Small: Thinking Machines' 594B model scores 31 on AA index, Apache 2.0 weights promised next week
Thinking Machines Lab's second-generation model is on Tinker now with closed weights; Artificial Analysis says it would top open models outside China if weights ship.
audiotale.net team6 min read

Nvidia releases Nemotron 3.5 Super VL and Bolt open models for enterprise and local agents
Super VL has 120B parameters (12B active); Bolt 74B-A9B fits in 24 GB with 2-bit weights. Both ship under OpenMDW-1.1, and the speed claims are Nvidia's own.
audiotale.net team5 min read