Fractile Ltd., a British developer of artificial intelligence inference chips, has raised $720 million in a Series C funding round, according to Reuters. Thrive Capital, Founders Fund, Lightspeed and Redpoint led the round, with roughly a dozen additional investors participating, including the U.K. Sovereign AI Fund, the British Business Bank and CapitalG, Alphabet Inc.'s investment arm. The company is now valued at $7 billion.
Fractile was founded in 2022 by AI researcher Walter Goodwin, who serves as chief executive. The company says its chips are expected to deliver up to 100 times better performance than current silicon while cutting energy use by 90%. Fractile attributes those gains to eliminating the need for AI models to shuttle data to off-chip HBM memory.
Large language models complete prompt responses through a workflow of numerous steps. After each processing step, an LLM sends results to an off-chip HBM memory chip and then retrieves them to set up the next step. Those repeated round-trips consume significant time and slow inference.
Fractile is not alone in targeting that bottleneck. Cerebras Systems Inc., which went public in May, sells wafer-size accelerators that cut data movement by keeping information in a large onboard cache that is faster than off-chip HBM modules. Cerebras' newest processor, the WSE-3T, can hold 44 gigabytes of data. That cache is built on SRAM, a memory type historically added to processors in only small quantities; its cells use six transistors apiece, which read and write data faster than the single transistor in HBM cells, though SRAM is expensive and has limited capacity.
Fractile's approach differs: rather than integrating SRAM, it stacks standard DRAM memory directly on top of its inference chips. Placing memory immediately above a processor's logic circuits shortens the distance data must travel, speeding traffic and reducing the electricity needed to move it. Samsung Electronics Co. is developing similar technology; in August it debuted an HBM variant called zHBM that can be stacked atop AI accelerators and moves data to and from the host chip over tiny copper wires. Samsung says zHBM will be eight times faster than HBM5.
Fractile estimates its chips will let LLMs process several thousand tokens per second. It plans to ship the processors as part of rack-scale systems that include custom software, a category that includes the compilers many AI chipmakers offer to convert customers' LLMs into a format compatible with their silicon.
More company and startup news from TechManNews.
