Exa says it improved its document-embedding system on AWS Inferentia2 and Trainium2 chips, which it uses to prepare web pages for semantic search. The company reports a 47% improvement in both performance and cost per million documents compared with its initial setup.
Its work included custom attention and neural-network kernels, plus changes to memory use. Exa says it validated outputs against its reference model using a 0.999 cosine-similarity threshold; it did not report a change to its search product or customer pricing.
Why it matters
For Exa, lower embedding cost per million documents can make maintaining an index of billions of pages less expensive; the company did not say whether customers will see a price or service change.