- AWS AI Factory product will let enterprise and public sector customers deploy AWS chips, networking and AI services and Nvidia’s compute platform
- Womp womp. AI factories were one answer to the question of how telcos could escape falling into a repeat of the dumb pipe trap
- The cloud giant announced general availability of its Trainium 3 chip and teased Trainium 4
AWS RE:INVENT, LAS VEGAS – AWS came to the table with a full house this year, with CEO Matt Garman talking up a pair of AI infrastructure announcements alongside a trio of model updates on the keynote stage.
Here’s what you need to know from the big event.
AI Factories
AWS debuted its AI Factories product, which will let customers deploy AWS chips, networking and AI services and Nvidia’s compute platform in their existing data center space to meet both AI compute and data sovereignty requirements.
On one hand, it’s really a shame to see AWS make this announcement. Why? Because AI factories were one answer to the question of how telcos could escape falling into a repeat of the dumb pipe trap.
Operators from DT and Orange to Swisscom and Telenor started taking baby steps in the right direction. But while they were faffing about and moving at the speed of, well, telcos, AWS was busy building the exact package enterprises and public sector clients are looking for.
On the other hand, we told you it was going to end this way. There could still be some role for telcos to play on the sovereignty front, but AWS is already explicitly targeting those with regulatory and sovereignty requirements by touting its ability to offer dedicated infrastructure that can be deployed in a company’s existing facilities.
AvidThink Founder and Principal Roy Chua thinks this release could cut both ways. While AWS AI Factories “could be viewed as competitive” with some telco initiatives, he told Fierce operators could “also view AI Factories as a way to have AWS help with their own private AI capabilities — separate, private AI infrastructure that telcos can leverage for their own businesses.”
They have no choice but to make lemonade from these lemons, we suppose.
Big bag o’ chips
There were also two big pieces of chip news at re:Invent. The first was around the general availability of Trainium 3, the AI training chip that AWS announced at re:Invent last year. Trainium 3 offers 4.4 times more compute, 3.9 more memory bandwidth and five times more AI tokens per mw of power than the previous generation chip.
AWS also teased Trainium 4, which remains under development. Garman said that compared to Trainium 3, the next generation chip will deliver six times the FP4 performance, 4 times the memory bandwidth and two times the memory capacity.
But this added muscle doesn’t necessarily mean AWS is making a run at Nvidia.
“I don’t know who needs to hear this, but every new AI chip that gets launched isn’t ‘Built to compete with or take down Nvidia.’ We are at the beginning of a massive tech supercycle and every chip that can be produced for AI is being sold," Futurum Group CEO Daniel Newman told Fierce.
That is, there’s enough demand to go around for everyone and it’s not a zero-sum game.
Still, it could be a huge boon for Amazon if Trainium “can break out of the 'overflow' positioning into broad Fortune 500 adoption,” Futurum Group VP and AI Platforms Practice Lead Nick Patience told Fierce. So far, though, that has yet to happen.
Models on models
AWS introduced a series of new options in its Nova foundation model family. These include the Nova Lite workhorse model for various workloads, Nova 2 Pro for intelligent reasoning and Nova 2 Omni with advanced multi-modal input and output capabilities.
It also came out with Nova Forge, a new service that Garman said introduces the concept of open training models. That is, Forge allows enterprises to inject their proprietary data at various Nova training checkpoints to create custom “Novellas” that can be used in Amazon Bedrock.
Sony is among those who are using Nova Forge, tapping the tool to create a Nova 2-based internal model to improve its compliance practices.
Think of this as a step beyond retrieval augmented generation (RAG). AWS isn’t just letting enterprises run inference queries on their data, but actually integrating said data into the model training process to make custom models.
We see the potential for this to further boost uptake of Trainium, which is designed to handle these kinds of fine tuning workloads.
The model updates came in addition to Amazon’s introduction of new Frontier Agents. You can read more about those – and the controversy surrounding the company’s rapid AI advancements – here.
Tackling technical debt
AWS is also tapping into the power of agentic AI to tackle technical debt.
AWS Transform is getting a new agentic AI toolset (AWS Transform Custom) that is designed to speed modernization of old enterprise code and applications, and work across any API, framework, architecture or language.
This is obviously a huge deal given staggering amounts of technical debt have been hindering AI adoption. As Garman noted onstage, Accenture has estimated that technical debt costs U.S. companies a combined $2.4 trillion and Gartner has found that 70% of IT budgets are consumed by legacy systems.
Indeed, AWS isn’t the only one working on this problem. Microsoft rolled out AI-powered app modernization capabilities through GitHub CoPilot in May.
AWS is notably taking aim at Windows modernization, billing its agents as a quick and easy way to upgrade .NET apps, SQL Server and older user interfaces to open source alternatives that aren’t attached to licensing agreements. There are also new agents for mainframe and VMware migrations.
Chua told Fierce the new tools could help accelerate cloud migrations (yes, those are still happening), ultimately “freeing resources for AI/investment.”
He added that telcos might even benefit, potentially being able to use the tools to “transform decades-old core systems” like billing and OSS/BSS.