As computing demands for AI grow, graphics processing units (GPUs) are doing the heavy lifting required to train and run AI models that require large-scale, high-performance computing. Where hyperscalers such as Amazon Web Services, Microsoft Azure and Google Cloud offer cloud computing for any type of application, a new generation of cloud infrastructure providers known as neoscalers are harnessing GPUs to deliver fast, scalable and powerful computing optimized for AI.
ABI Research recently reported that GPU-as-a-Service (GPUaaS) will generate more than $65 billion in revenues by 2030[1]. To monetize these high-performance offerings, neoscalers will require ultra-scalable, high-performance network infrastructure, with low-latency, low-jitter, and low-loss attributes required for GPUaaS and other AI infrastructure and platform services.
Neoscalers – the new wave of cloud and AI builders
Rather than bundling GPU power with proprietary services, neoscalers are focused on delivering direct access to high-performance GPU computing power at a price intended to make large-scale AI viable for more businesses. Neoscalers are designed from the ground up to maximize GPU efficiency and meet the performance demands of AI-first applications.
Some neoscalers have repurposed GPUs from cryptocurrency mining offerings, while others are building AI-native infrastructure from the ground up and relying on optimized business models and low overhead to improve price-performance. In any case, neoscalers are expanding the GPU-as-a-service market and targeting AI suppliers and users.
For AI workloads, neoscalers offer:
- Dedicated access that reduces contention and ensures reliable performance. Some neoscalers offer bare-metal access to GPUs.
- Flexible capacity management with the aim to deliver required capacity faster than hyperscalers. Delay is deadly for generative AI applications.
- Lower cost per GPU hour. Neoscalers have repurposed and optimized hardware configurations to reduce costs including GPU usage or customer egress charges and offer flexible pricing.
The one thing that neoscalers need to prioritize as they establish service offerings and grow the market is network connectivity. If traditional cloud infrastructure isn’t optimized for AI, traditional network infrastructure isn’t either.
Why optical connectivity is now mission-critical
AI applications require network infrastructure that can efficiently and reliably handle the large amounts of data generated by machine learning models and deep neural networks. Traditional networks struggle with upload-heavy workloads, latency challenges, and the sheer scale of data movement generated by AI applications. Nearly 51% of users report “biases, errors and limitations” as their main complaint with GenAI.[2] While there are multiple causes, latency is a major contributor to AI performance issues. As AI adoption continues to grow, purpose-built AI-Optical networks are increasingly mission-critical for neoscalers. Specifically:
- High bandwidth and low latency are table stakes for AI networks. Optical networks must keep pace by maximizing capacity per wavelength, spectral efficiency, and low-latency performance while augmenting the photonic layer for massive scale.
- AI applications create unpredictable traffic behaviors. AI traffic bursts happen randomly and, without warning, can disrupt the network. As a result, optical networks need to be more programmable so performance is not compromised and network resources are utilized intelligently.
- Neoscalers are moving fast. AI-optical network builds need to be deployed on an accelerated timeline and need to be future-proof to support rapid business growth. As a result, neoscalers require solutions that can be tailored from a broad and proven technology portfolio, and they need a level of operational expertise that ensures successful deployment.
The emergence of neoscalers is driving innovation in optical networking resulting in AI-optical networks that are optimized for AI processing. Scalable and high-performance AI-optical networks are the foundation for reliable AI data transfer and processing by neoscalers. As optical networks become more complex, neoscalers will rely on vendors with proven experience designing and delivering AI-optical networks.
The challenges ahead
Optical networks are best suited to deliver the capacity required, over the distances needed to connect neoscalers to businesses. The performance and scalability of AI solutions relies on fast, reliable and available network infrastructure. As neoscalers build out purpose-built AI computing infrastructure, the network cannot be neglected. Yet, all optical networks aren’t created equal. While neoscalers don’t have to build the network, the optical network foundation must be optimized. Neoscalers cannot monetize AI-focused GPU access without the right optical network.
There are many challenges when deploying an AI-ready optical network. Each becomes more pronounced given the increasing scale and performance demands expected of AI applications and GPU-as-a-service offerings. Strengthening underlying optical infrastructure requires the right combination of performance optics, pluggable transceivers, and open line systems, built on a network design that ensures service level objectives are met.
Power and space, and their associated costs, are a constant concern. For neoscalers, which are often constrained by fiber resources between data centers and physical space inside them, optical networks must be designed to scale in increasingly smaller footprints. The level of design work required to minimize space and power is no easy feat; accordingly, deep expertise in network design is required to ensure successful deployment and operations.
Looking forward, optical networking will see an acceleration of technological advancement to meet the needs of AI solutions providers, neoscalers included. For this emerging class of solution provider, there is increasing priority on eliminating the bottlenecks to GPU performance. Building an ultra-scalable, high-performance AI-optical network is instrumental to this goal – and along with essential compute infrastructure – a foundational technology for driving rapid business growth.