A team led by Amr Abdelkhalik examined the 305‑million‑parameter audio encoder used in Gemma 4 and EmbeddingGemma 2. They kept the encoder weights fixed and trained either a 22.6 M‑parameter CTC reader (frozen system) or added 3.1 M adapter parameters plus a 41.5 M hybrid CTC/attention reader (adapted system). Compared to Cohere Transcribe Arabic (2.07 B parameters), the frozen system matched character error rates on most tests and ran 3.4 × faster on CPU. The adapted system achieved statistically similar word error rates on Saudi customer‑service calls (19.20 % vs 19.27 %) and beat a 0.6 B‑parameter recogniser on several conditions, while decoding 139 × faster on one GPU.
Why it matters
The results show that a frozen or lightly adapted Gemma 4 encoder can deliver near‑commercial Arabic speech‑recognition accuracy with far lower compute, useful for companies that need cost‑effective, self‑hosted solutions.
