Ep. 18 – WTF is NVIDIA?
|
Curious Reader! Welcome to this week’s Curious Companion newsletter. What you came for is below, and you can CLICK HERE to listen to the episode if you decide you prefer earbuds to eyeballs. Happy reading! In this episode I break down what NVIDIA is, why its GPUs (hint: these are the “chips” you hear so much about) became the backbone of the modern AI boom, and how CUDA locked in its dominance. The conversation moves from gaming history to trillion-dollar valuations and data centers stacked with H100s, offering a clear picture of why nearly every major AI system runs on NVIDIA hardware. No AssumptionsI’m not going to assume that you know what NVIDIA is, and not because I’m worried about making an ass out of you and me, but because I have already made an ass out of me while chatting with Lex last week. While doing research for this episode I told her how stoked it makes me that between ChatGPT and YouTube we can learn so much for free. She asked what I was learning and I mentioned CUDA, which we’ll chat about in a bit, and then I said that next week’s episode was going to be about NVIDIA, to which she responded, “…?”. Ass made. So, I won’t assume you know what NVIDIA is, buuuuut I’m gonna guess that you’re curious, which is why you’re subscribed to this newsletter. That in mind, Imma learn you some shit because it’s nice to be in the know about things, and also, NVIDIA has become a very integral company in this AI madness (and in the US economy as a whole), and I think you should know about them. What is NVIDIA?NVIDIA Corporation (“NVIDIA”) is an American technology company famous for designing “chips.” They outsource the manufacturing, but these chips—more formally known as graphics processing units (GPUs)—are integral for running AI models. Circling back to NVIDIA being a big economic and stock-market player, they are part of what’s referred to as the Magnificent 7: Microsoft, Apple, Amazon, Alphabet (Google), Meta, NVIDIA, Tesla. The stock performance of these 7 companies has disproportionately moved the entire market, especially during the AI boom. Why the Name NVIDIA?NVIDIA was founded in 1993 in Santa Clara, CA by Jensen Huang, Chris Malachowsky, and Curtis Priem. The company made graphics cards for gaming PCs. When the three founders were starting the company they used the placeholder NV, short for “next version” or “next vision.” As they got closer to launch they wanted a name that incorporated “NV” since it was already on a lot of early internal files. Someone on the team landed on the Latin word invidia, which means envy (insert eye roll), dropped the “i,” and the rest was history. A Little NVIDIA HistoryNVIDIA started out by making graphics cards for gaming PCs, with a goal of creating chips capable of real-time 3D graphics, not just flat 2D images. In 1999, six years after the company took on this mission, they succeeded and released the GeForce 256, officially coining the term GPU (graphics processing unit). The GeForce 256 is the direct ancestor of the GPUs used today to run ChatGPT. How GPUs Became AI GoldAround 2005–2006, researchers outside the gaming world noticed something: the math used in 3D graphics—matrix multiplications and linear algebra—was identical to the math used in early neural networks and physics simulations. If you could run those same operations on a GPU (graphics processing unit) instead of a CPU (central processing unit), you could do these computations WAY faster. Here’s where NVIDIA came in and solidified their future with AI. In 2006, NVIDIA released CUDA (Compute Unified Device Architecture), a proprietary programming framework that made it possible to run massively complex computations, including AI math, on GPUs. For my historians in the audience, this approach of using GPUs—more specifically NVIDIA’s GPUs—for AI math didn’t actually take off until 2012 when it was used to win an image-classifying competition. Feel free to ChatGPT the term “AlexNet” if you’re interested in learning more. The moment AlexNet succeeded, NVIDIA became the default for the chips used for AI. They were the only company already building hardware—and a software ecosystem (CUDA)—that could scale it. From that point forward, every major breakthrough in AI math—from GPT-1 to ChatGPT—has run on NVIDIA GPUs. Translation: NVIDIA designs the chips that all the big players (excluding Google) use to run their AI models.
So How Big of a Deal Is NVIDIA?NVIDIA is valued at $4.7 trillion dollars. They hold 92% of the discrete GPU market, meaning the kind of GPUs (read: chips) used for data centers and high-performance computing. For the tech nerds in the audience: NVIDIA’s main chip (and one of the most advanced chips ever made) is the H100. Each chip can cost $25,000–$40,000. NVIDIA packages these into DGX H100 systems—racks that contain eight GPUs linked together. That’s $320k on the high end for one DGX system. Companies like OpenAI and Meta buy and use thousands to tens of thousands of these DGX units. If your brain works like mine and you find it helpful to see visuals instead of just reading names, here are two YouTube videos you can check out: Can Anyone Catch Up?It’s not the hardware but rather the software—CUDA—that really protects NVIDIA’s foothold. CUDA is what the global AI ecosystem runs on. Switching from NVIDIA’s CUDA ecosystem to Google’s TPUs wouldn’t just be swapping one chip for another. It would be like trying to turn every gasoline car into an electric vehicle overnight. Even if you had the cars, the entire infrastructure—the fueling system, the mechanics, the parts, the roads—is built for gasoline. The same is true in AI. The world’s tooling, code, and workflows are built around NVIDIA GPUs, so moving to TPUs would require rebuilding the whole ecosystem. So…likely not happening any time soon. Da SummaryNVIDIA is a company that is absolutely integral to AI. They make the chips (GPUs) that nearly all of these AI companies use to run their models. They’re worth a lot of money, they’re making a lot of money, and they likely won’t be replaced any time soon because of both the hardware (GPUs) and the software (CUDA) that they designed. How I Used ChatGPT This WeekEach episode I include a section where I briefly discuss how I used ChatGPT that week. This week I want to share a use case that my friend Corin shared with me by way of an IG Reel. Chris McCuasland is a British stand-up comedian, actor, and TV personality. He has a hereditary eye condition called retinitis pigmentosa which gradually led to the loss of his vision. The Reel is a clip from the Graham Norton Show where Chris explains how he uses AI to describe pictures, drawings his daughter did, photographs—anything—and it will describe it to him, and as he says, “with so much more patience than any human.” SUPER SUPER cool use case. I think a lot of the shit that AI is being used for these days is exactly that, shit. But I also think it’s a remarkable technology with so much promise and value, and this is a perfect example of that. Da Wrap-upHopefully now if and when you hear the name NVIDIA, you’ve got a pretty solid understanding of who that is, what they do, and why they’re important. As always, endlessly appreciative for you and your curiosity. Catch you next Thursday. Maestro out. AI Disclaimer: In the spirit of transparency (if only we could get that from these tech companies), this email was generated with a very solid alley-oop from ChatGPT. I write super detailed outlines for every podcast episode (proof here), and then use ChatGPT to turn those into succinct, readable recaps that I lightly edit to produce these Curious Companions. Could I “write” it all by hand? Sure. Do I want to? Absolutely not. So instead, I let the robot do the work, so I can focus on the stuff that I actually enjoy doing and you get the content delivered to your digital doorstep, no AirPods required. High fives all around. Did someone forward you this email? Stay curious. |