One of our favorite pieces from the past 12 months. Originally published July 22, 2022.
It takes a lot of GPU power to run today’s most powerful AI models. Even most extreme gaming pc built (opens in new tab) wouldn’t be able to handle the kind of oomph it takes to get the best AI dungeon (opens in new tab) models in progress. In order for Latitude to power this less numerical, more free and creative online D&D campaign, you need something truly beastly.
According to Nick Walton, the CEO of Latitude and the software engineer who helped design the initial code for the D&D inspired story generator (opens in new tab)“If you want to run the biggest AI for AI dungeon, you don’t just need a powerful GPU, you need one TROS of eight or 16 massively powerful GPUs.” Even those should be more powerful than the best graphics cards (opens in new tab) on the consumer market.
Language models such as AI Dungeon’s Dragon experience, based on the 178 billion parameter Jurrasic-1 Jumbo language model developed by AI21 (opens in new tab)require enormous computing power to recognize and recall important story elements, and use complex writing styles.
Nick pointed me to the DGX station A100 (opens in new tab) as a reference for the kind of machines Latitude uses to make the game more intense AI models (opens in new tab). It’s a computing monster, even with just four of Nvidia’s $10,000 A100 GPUs – the first and essentially the most powerful Ampere graphics cards to surface.
AI Dungeon … is the most demanding game in terms of graphics card requirements
Nick Walton, CEO of Latitude
“AI Dungeon, especially at launch and probably still is, is the most demanding game in terms of graphics card requirements,” says Nick, “and for a long time it had no graphics.”
For machines like the DGX Station with, say, eight top-of-the-line A100 GPUs jammed in, you’re looking at about $150,000 in upfront tech costs, and that’s just to run the most advanced AI – the company has several other AI models who must run it at the same time for your dungeon pleasure. That’s one of the reasons the company uses AI service providers, and even those have struggled to get GPUs lately.
Obviously there are also huge energy costs associated with running an AI company like this; Nick tells me that at one point the company was spending almost as much on running the machines as it did on the payroll of the 16-person team (including the higher educated).
Adding to that energy cost is of the game AI image tool (opens in new tab), which made the decision to roll out Latitude to all players for free at the beginning of this week. Nick was able to clarify how it worked for us.
He says a catalog of tens of thousands of images is pre-generated, each is captioned and then linked to stories as they play, rather than being generated on the fly. He makes it clear that generating graphics on the fly isn’t really viable for an operation like this, but that the AI models behind the process are smart enough to know it can get away with matching an image to the caption “Mountain forest with castle”, for a story in which a medieval kingdom is mentioned.
Nick also laments that working with AI can be a huge challenge. Check what the AI says to stop it go rogue (opens in new tab) is difficult, since “AI can be like an unruly toddler sometimes.” You can kind of steer it in the right direction, but sometimes if you “instruct it not to do something, you’ve prepped its brain and it’s probably going to do that thing.”
Fortunately, the team is working on optimizations to not only make their AI more efficient, but also keep it under control. All of that becomes more important as the game progresses Steam (opens in new tab) next week July 28.
Our Fraser believes that video games need more Dungeon Masters (opens in new tab), and Nick tends to agree. In fact, it was his fascination with the freedom and non-linearity of Dungeons and Dragons that led him to create AI Dungeon.
Nick truly exudes a passion for AI storytelling, believing that this kind of technology will change the storytelling landscape in two major ways. First, “AI is going to enable experiences with freedom and dynamism never before possible.” He talks about how many people have completed those same quests in World of Warcraft, for example, and how boring it is to know that other people are doing the same thing as you.
“I want to play an experience where I go to this small town and go through all these unique events that no one has experienced before, and end up being the hero of the town,” says Nick. “Maybe no other player in the whole game cares about that city, but I DO! Here I’ve had all these cool experiences that were unique and where I had the freedom to decide where the story goes.”
And it’s not just a unique experience that AI storytelling offers. “The second thing that I think is really powerful,” says Nick, “is the ability of AI to enable an entirely new, massive generation of creators…allowing everyone to be a creative director of their own experience.” Nick is convinced that creators will one day be able to orchestrate and bring their creative visions to life through the power of AI.
Frankly, that’s a much nicer view of the future of AI than most people, so I appreciate it. It’s exciting to see companies like Latitude expand and evolve with something I’m so passionate about: emerging stories. And I’m sure the Steam launch is going to be a success so put AI Dungeon on your wish list because boy are you in for a to drive.