AI as a video game designer: How it may soon lend a helping hand

Matthew Guzdial and Mike Cook – research pioneers in the field of using AI to develop video games – provide their expertise on letting a machine design games

It’s a thrilling concept – allowing any idea from any individual, no matter their skill level, to be borne out in a video game.

Matthew Guzdial will be the first to admit that such a vision is still a long way off, but already some games exist that were created by AI systems.

Mike Cook is one of the foremost researchers in this field, responsible for ‘Angelina,’ which relies on genetic algorithms to create various small web games – some of which can be surprisingly fun.

“There’s a stumbling block, though – the amount of knowledge that you needed to create one of these systems,” Guzdial said.

“Angelina grew as a game designer, as Mike did. And that’s not a coincidence, right? It’s because Mike learned more about games that he could then represent in computation.

“So there’s a chicken and egg problem here, where you have to know about game design before you can have games made, and the kind of games you can make with this approach are limited by your knowledge.”

Super Mario meets Brick Breaker

In 2018, Guzdial and Mark Riedl realized that they could instead try to learn that necessary knowledge from a secondary source. “In our case, we used gameplay videos,” he said.

“Instead of having this big knowledge requirement bottleneck, what if we could just feed the system gameplay videos and see what it could learn?”

Guzdial fed the system two-minute gameplay videos of Super Mario Brothers, Megaman, and Kirby, and let it build a game based on what it had learned.

The result was a 2D platformer that combined the Super Mario’s Goomba – the enemy that you can squish by jumping on it – with the ground tiles, so every time the player lands on an object, it gets destroyed, forcing constant movement, and carefully planned jumping sequences.

“I actually showed that to Mike Cook, and he said that it looked like a combined Super Mario Brothers and Brick Breaker.”

Again, the game is remarkably good for what it is – something built by a machine based on watching six minutes of gameplay.

Of course, it’s not fair to overhype the ‘Killer Bounce,’ which is clearly limited in scope and lacking many of the elements of a modern game.

To pull off something like this in a 3D world would require a better way to translate level geometry from video, as well as a way to handle significant mechanic variation (such as large amounts of menus, hidden information, and so on). Still, it offers a glimpse of the future.

The future of gaming

Guzdial’s work leads to questions about the nature of creativity – if the system is trained on games designed by humans, can it truly create something new, and is that a creative endeavor?

He pointed to the work of researcher Anurag Sarkar, who is studying systems that can learn from level design in as many as eight different titles to produce a new output.

“And that means you can do all kinds of cool stuff like say, ‘what’s the level design that’s halfway in between these two level designs from these two different games?’ Or ‘what’s the level design that looks the least like this game?’”

It means relying on prior work, but Guzdial notes that according to cognitive creativity studies, humans often follow the same combinatorial pattern.

“We find existing content to make new things,” he said. “It’s one of many ways that humans make new things, but a pretty common one.”

Even though he is immersed in the cutting edge of what’s possible with AI, and hopes to open up game creation to many more people, Guzdial doesn’t believe human game-makers should worry about losing their jobs any time soon.

But, he added, “I totally see a world, maybe 10 or more years out, where we’re going to have all of these new AI tools that you as a designer are going to have at your disposal in order to, for example, quickly prototype a new idea.”

The result “is not going to be perfect, it’s gonna be a little janky. But it will give you a better sense of whether the concept is fun at all, whether it’s worth spending more time making this mechanic in the actual game.”

Mass Effect: Andromeda - a cautionary tale

Another potential application of AI is in co-creation, working in tandem with game developers.

This already happens, albeit in a very limited way, in open world games, where simplistic terrain generators are used to create the basic world that the richer game is built on top of.

No Man’s Sky is famous for leaning heavily into procedural generation, while this year’s Microsoft Flight Simulator combines real-world data with generated interpretations of the visual input.

Doing something like this requires the game’s vision to line up with what procedural content generation can deliver, lest the studio finds itself in a serious bind.

For three years, the developers of Mass Effect: Andromeda tried to generate a galaxy similar to No Man’s Sky.

“But this totally didn’t work for their design process,” Guzdial said.

“They were trying to do this thing where all the planets would be procedurally generated, but all the characters and their arcs would be hand-made. How can you possibly fit these two things together?

“It just sort of broke their design process. And they ended up largely remaking the game from scratch in 18 months, which is why it turned out the way it did.”

Andromeda is a cautionary tale many will bear in mind when integrating more advanced elements into level design. But perhaps that design process, too, will be guided by AI.

“Mario Maker 2 had just come out,” Guzdial recalled of a 2019 project, “and so there was this natural understanding that people had in terms of doing level design as a task in a game. We made an AI system with a similar interface, called Morai Maker.”

Morai presented users with a Marioesque level creator where they could start building a level and then hit ‘turn.’

At this point, it would hand the proceedings over to one of three machine learning models (Bayes network, Markov chain, and long short-term memory), which would add to the level design.

“We ran an initial study where we had around 100 people interact with this tool,” Guzdial said. They then had to rank which of the models they thought was the best.

“There was no strong preference among the users for any of these tools, if you looked at their data in aggregate. But we found this interesting thing: if you looked at their data individually, people had very strong preferences.”

Game design personalization

Different users preferred different versions of AI-based level design assistants, depending on what they expecting to achieve.

“An LSTM, which is a deep neural network, does very well on the sort of things users have already seen – it would do a nice job when you were creating something like the original Super Mario Brothers levels,” Guzdial explained.

“The Markov chain was the most out there, it produced the most varied results because it was hyper local in terms of its reasoning.”

“And so, if you were producing something highly unique, you tended to prefer it, and the Bayes net sort of fell in between those two.”

It is perhaps not too surprising to find out that people have different goals, and different opinions on level design, leading to different algorithmic experiences.

This, Guzdial realized, opens up a whole other avenue for the AI-based designer – personalization.

“The idea being that we could have the [AI designer] start in a really general state, in a sort of extremely average level design partner,” Guzdial said.

“And then from there, we would have it actively be retraining during the interaction with the user.”

AI generating code and ‘learning the rules’

In a follow up study with professional level designers, some said they wanted their AI-based partner to be like a ‘friend’ that was just a fun experience to play with, others wanted a ‘collaborator’ at an equal level, a ‘student’ that repeated what they did, or a ‘master’ that told them what to do.

“It’s not generally the case that the software you’re using is adapting to you,” Guzdial said.

“That’s a very new experience for people, and so figuring out how to structure this, not just in terms of the way that the machine learning part works, but also in terms of the way the editor looks, and the way that the UI communicates what is happening – these are all totally unknown.”

The work is ongoing, as are plans to expand the system’s capabilities, and those of the fully autonomous designer.

“Fingers crossed, in the next week or two we will be starting a study where, for the first time, we will add a rule learning portion of our tool to our automated game design [platform],” Guzdial said.

“So you can provide examples of rules that you want to exist, and the AI will learn the rules and generate the code, and then you can play the game with those rules.

“And we don’t know what the heck is going to happen.”

To find out more about the emerging applications of artificial intelligence in video game development and publishing, download our EBook – ‘How AI will revolutionize video games’