Posts Tagged 'AI accelerator processing units'
-
January 21, 2025
AI at Scale: A Special Report
By Kirt Zimmer, Head of Social Media Marketing, Marvell
Marvell’s business is accelerated infrastructure for the AI era, which is a fast-evolving space that can occasionally confuse even the most earnest student. To help you keep up, we’ve partnered with VentureBeat to explore a multitude of content about that subject:
- Build or buy? Scaling your enterprise GenAI pipeline in 2025
Enterprise leaders are debating whether to buy AI tools, build their own, or some combination of the two. Companies like Wayfair and Expedia offer valuable insights for organizations looking to scale LLMs effectively. - Purpose-built AI hardware: Smart strategies for scaling infrastructure
Custom AI hardware is the unsung hero of scalable AI infrastructure, helping to tackle a range of issues including performance, cost, and security. For enterprises looking to transition in this rapidly evolving landscape, there’s some great advice here. - AI factories are factories: Overcoming industrial challenges to commoditize AI
Sixty years ago, Alabama was home to a 1.6GW coal fired power plant with the world's tallest chimney. That same site today houses a Google data center. The operations are obviously very different, but some of the infrastructure challenges are somewhat familiar. Read what AI 'factory' really means. - 4 bold AI predictions for 2025
We’ve seen plenty of “predictions for the coming year” pieces in other publications that are honestly pure fluff, but this ain’t that. If your brain is activated by inference costs, reasoning models, transformer alternatives and LLM scaling laws, you’ll appreciate that even annual predictions can be smart and thought-provoking.
- Build or buy? Scaling your enterprise GenAI pipeline in 2025
-
January 15, 2025
Next Up for Custom AI Accelerators: Co-Packaged Optics
By Michael Kanellos, Head of Influencer Relations, Marvell
Computer architects have touted the performance and efficiency gains that can be achieved by replacing copper interconnects with optical technology in servers and processors for decades1.
With AI, it’s finally happening.
Marvell earlier this month announced that it will integrate co-packaged optics (CPO) technology into custom AI accelerators to improve the bandwidth, performance and efficiency of the chips powering AI training clusters and inference servers and opening the door to higher-performing scale-up servers.
The foundation of the offering is the Marvell 6.4Tbps 3D SiPho Engine announced in December 2023 and first demonstrated at OFC in March 2024. The 3D SiPho Engine effectively combines hundreds of components—drivers, transimpedance amplifiers, modulators, etc.—into a chiplet that itself becomes part of the XPU.
With CPO, XPUs will connect directly into an optical scale-up network, transmitting data further, faster, and with less energy per bit. LightCounting estimates that shipments of CPO-enabled ports in servers and other equipment will rise from a nominal number of shipments per year today to over 18 million by 20292.
Additionally, the bandwidth provided by CPO lets system architects think big. Instead of populating data centers with conventional servers containing four or eight XPUs, clouds can shift to systems sporting hundreds or even thousands of CPO-enhanced XPUs spread over multiple racks based around novel architectures—innovative meshes, torus networks—that can slash cost, latency and power. If supercomputers became clusters of standard servers in the 2000s, AI is shifting the pendulum back and turning servers into supercomputers again.
“It enables a huge diversity of parallelism schemes that were not possible with a smaller scale-up network domain,” wrote Dylan Patel of SemiAnalysis in a December article.
-
December 19, 2024
Custom, Copper and Cross-Country Connectivity: Eight Big Trends for Marvell in 2024
By Michael Kanellos, Head of Influencer Relations, Marvell
What happened in semis and accelerated infrastructure in 2024? Here is the recap:
1. Custom Controls the Future
Until relatively recently, computing performance was achieved by increasing transistor density à la Moore’s Law. In the future, it will be achieved through innovative design, and many of those innovative design ideas will come to market first—and mostly— through custom processors tailored to use cases, software environments and performance goals thanks to a convergence of unusual and unstoppable forces1 that quietly began years ago.
- At Marvell AI Day, we announced that custom processors would represent 25% of the AI accelerator market by 2028 and exceed $42 billion. We also announced we had secured four designs with three of the major hyperscalers.
- In July, we unveiled StructeraTM A and Structera X,2 CXL controllers for near memory compute and increasing memory capacity, respectively. Merchant and custom versions of the chips were released at the same time, a reverse of the traditional pattern where custom chips might follow merchant ones by a lengthy lag.
- At OCP, Meta showed off FBNIC, a custom network interface developed in collaboration with Marvell to reduce the time needed to resolve network issues.3 Make that four out of four hyperscalers. See the image below.
FB NIC on display at OFC- In December, we announced a five-year collaboration that includes supplying custom AI products and other devices for AWS.4 The collaboration also encompasses performing semiconductor design tasks in the cloud, which promises to reduce the time and cost of creating new chips.
- Finally, we ended the year with an alliance for custom high bandwidth memory with Micron, Samsung Electronics, and SK hynix.5 Custom HBM can reduce chip real estate by up to 25% and increase memory capacity by up to 33%.6 Custom HBM runs against industry orthodoxy,7 writes analyst Joseph Byrne, but it could pay dividends.
-
June 18, 2024
Custom Compute in the AI Era
This article is the final installment in a series of talks delivered Accelerated Infrastructure for the AI Era, a one-day symposium held by Marvell in April 2024.
AI demands are pushing the limits of semiconductor technology, and hyperscale operators are at the forefront of adoption—they develop and deploy leading-edge technology that increases compute capacity. These large operators seek to optimize performance while simultaneously lowering total cost of ownership (TCO). With billions of dollars on the line, many have turned to custom silicon to meet their TCO and compute performance objectives.
But building a custom compute solution is no small matter. Doing so requires a large IP portfolio, significant R&D scale and decades of experience to create the mix of ingredients that make up custom AI silicon. Today, Marvell is partnering with hyperscale operators to deliver custom compute silicon that’s enabling their AI growth trajectories.Why are hyperscale operators turning to custom compute?
Hyperscale operators have always been focused on maximizing both performance and efficiency, but new demands from AI applications have amplified the pressure. According to Raghib Hussain, president of products and technologies at Marvell, “Every hyperscaler is focused on optimizing every aspect of their platform because the order of magnitude of impact is much, much higher than before. They are not only achieving the highest performance, but also saving billions of dollars.”
With multiple business models in the cloud, including internal apps, infrastructure-as-a-service (IaaS), and software-as-a-service (SaaS)—the latter of which is the fastest-growing market thanks to generative AI—hyperscale operators are constantly seeking ways to improve their total cost of ownership. Custom compute allows them to do just that. Operators are first adopting custom compute platforms for their mass-scale internal applications, such as search and their own SaaS applications. Next up for greater custom adoption will be third-party SaaS and IaaS, where the operator offers their own custom compute as an alternative to merchant options.
Progression of custom silicon adoption in hyperscale data centers.
-
June 27, 2023
Scaling AI Infrastructure with High-Speed Optical Connectivity
By Suhas Nayak, Senior Director of Solutions Marketing, Marvell
In the world of artificial intelligence (AI), where compute performance often steals the spotlight, there's an unsung hero working tirelessly behind the scenes. It's something that connects the dots and propels AI platforms to new frontiers. Welcome to the realm of optical connectivity, where data transfer becomes lightning-fast and AI's true potential is unleashed. But wait, before you dismiss the idea of optical connectivity as just another technical detail, let's pause and reflect. Think about it: every breakthrough in AI, every mind-bending innovation, is built on the shoulders of data—massive amounts of it. And to keep up with the insatiable appetite of AI workloads, we need more than just raw compute power. We need a seamless, high-speed highway that allows data to flow freely, powering AI platforms to conquer new challenges.
In this post, I’ll explain the importance of optical connectivity, particularly the role of DSP-based optical connectivity, in driving scalable AI platforms in the cloud. So, buckle up, get ready to embark on a journey where we unlock the true power of AI together.
Recent Posts
- In AI, The Voyage from Bigger to Better Is Underway
- The evolution of AI interconnects
- Lightwave+BTR Innovation Reviews Ranks Marvell Products as Among the Industry’s Best
- Marvell and IIT Hyderabad Partner to Support India’s Next Generation of Technology Innovators
- Radha Nagarajan named to the National Academy of Engineering
