Oossa

Exa cuts document-embedding costs 47% on AWS chips

Exa says software and kernel changes lifted embedding throughput on AWS Inferentia2 and Trainium2, reducing its cost per million documents by 47%.

NoteBy Published by Oossa: 1 min read

Exa says it improved its document-embedding system on AWS Inferentia2 and Trainium2 chips, which it uses to prepare web pages for semantic search. The company reports a 47% improvement in both performance and cost per million documents compared with its initial setup.

Its work included custom attention and neural-network kernels, plus changes to memory use. Exa says it validated outputs against its reference model using a 0.999 cosine-similarity threshold; it did not report a change to its search product or customer pricing.

Why it matters

For Exa, lower embedding cost per million documents can make maintaining an index of billions of pages less expensive; the company did not say whether customers will see a price or service change.

Was this article useful?
Share

Read next

Oossa · Newsletter

The week in AI, explained

Every Monday: the stories worth knowing, in plain language. Free, no spam.