Amazon Web Services released a tool called TorchNeuron on Monday that lets AI startups move their AI applications onto Amazon's Trainium AI chip with minimal effort. The software works with PyTorch, the free, open source toolkit developers use to build, train, and run AI models. According to Amazon, applications written with PyTorch can run on Trainium after a change to just one line of configuration, with no code rewrite required.
The release puts Amazon in more direct competition with Nvidia, whose software advantage has helped make its hardware the default choice for AI applications and fueled its rise as the leading AI hardware provider. Nvidia has spent years making its CUDA software, the infrastructure that lets applications run on its GPUs and other hardware, work seamlessly with PyTorch. That effort has made Nvidia's hardware deeply embedded in the toolkit, essentially by default, according to the report.
Amazon says about 85% of AI applications are written with PyTorch. Facebook originally developed the toolkit, and since 2022 it has been part of the Linux Foundation, which maintains open source projects. TorchNeuron is adapted from PyTorch, so developers already familiar with PyTorch know how to use it. Amazon is not the first company to make it easy to bring other chips to the platform; PyTorch users can also port their applications to AMD GPUs or Apple's chips with a simple configuration change.
Amazon argues that porting to Trainium is not only easy but also gives AI startups access to its entire cloud. AWS claims this makes it a better choice for startups than neoclouds, many of which are tied to Nvidia and its hardware, such as CoreWeave, Nebius, and Firmus. Colin Brace, VP of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips, said building on AWS means a company is not betting on a single-purpose compute provider and then scrambling to build everything else around it. He said the infrastructure to go from prototype to production is already in place on AWS, allowing a founder to move fast and scale when needed.
AWS says that because the same chip handles both training, which teaches an AI model, and inference, which puts a finished model to work, and is tightly integrated with the cloud infrastructure startups need, its cloud will save them money even if AWS does not offer the cheapest per-token price. Brace described Trainium's value as price-performance plus being one platform for both training and inference, which he said improves the economics across a full workload. He pointed to Deepgram, an AI voice app provider and a competitor to ElevenLabs, which he said moved to Trainium using native PyTorch support and is processing 70% more audio per dollar at production-grade accuracy.
Amazon also says it is pushing PyTorch to be less automatically centered on Nvidia and will contribute the code to do that to the main PyTorch project. In a blog post announcing the tool, the company said it is working with the PyTorch community to remove code that ties the framework to specific chips, with the goal of making PyTorch more hardware-agnostic for the entire developer community, not just Trainium users. Brace said the real story is removing the lock-in that kept developers from having a real choice at all.
More software news from TechManNews.
