Black & White: A Game of Reinforcement Learning
The video game Black & White is a very interesting example of a video game AI. (Disclaimer: I have not played this game; the citations for this post’s information are below). I recently came upon an article by James Wexler known as “Artificial Intelligence in Games,” and I wanted to present the information in it in a simpler manner to show a powerful example of AI in games for laypeople. Credit for this information goes to Wexler.
The game Black & White is a god-game where the player must use their god powers in order to progress the civilization and defeat an enemy god. The player-god has the ability to move things with a disembodied hand, influence the minds of their worshipers, and use miracles, though only within the boundaries of their influence, which is expanded by causing other villages to worship them.
The groundbreaking aspect of this game is the Creature. The Creature is a companion to the player, an anthropomorphic animal which exhibits a personality and is the agent that the invisible player uses to interact with locations outside of their influence. The creature is not directly controllable; rather, it is controllable using a machine learning AI.
Every type of object in the game’s world can be represented as a piece of data in the creature’s mind, and each piece of data is mapped to a numeric value to represent the creature’s “feelings” about the object. The creature uses a situational calculus system to interact with objects on a basic level.
Situational calculus is a form of AI, where an agent recognizes the “world” it is interacting with and keeps data in a “situation”, a history of other actions and events, to form cause-effect pairs in order to be able to determine what it should and should not do. The creature uses a Belief-Desire-Intention model, meaning that it assesses its desires, which are information about what it needs to accomplish (i.e. a Creature may have a desire for food); it assess its beliefs, the stored information it has already accumulated about how the world works (i.e. a Creature may have figured out that humans are a good source of food); and finally, it formulates its intentions, which are the sequence of actions which it takes to fulfill a desire and are influenced by beliefs (i.e. a Creature has the desire to eat, it has the belief that humans are a good source of food, so it forms the intention to eat a nearby human).
The creature learns things through small experiments, and by copying the player’s actions, i.e. if the player uproots a tree the Creature might start doing that as well. This allows for the Creature to almost imprint on the player as if the player were a parent!
The Creature also implements a reinforcement learning system — when it does something the player dislikes, the player has the option to slap it or pet it to let it know if the player is pleased with its action or not. In this way, the Creature can learn what to do more and what to do less. If the Creature picks up a bad behavior, slap it a few times to tell it not to do that. If the Creature does something you like, pet it to express your appreciation. The Creature then stores those appreciation values in association with its actions so it can know what to do to please its god-parent. In this way, the Creature can win over villages that worship enemy gods because the player taught them how to do so.
The Creature has limitations in its AI abilities, however. Its AI is somewhat simplistic, and does not make full use of situational calculus to allow the Creature to form plans of what to do based on the player-god’s requests. The game itself was released in 2001 and to program a complex AI would have required a lot of system resources, and still could require a lot of system resources in the modern day depending on how sophisticated the AI gets.
Now, let’s step away from Black & White and James Wexler’s article. Let’s talk about the way AI can be used in existing video games. Note that we won’t be talking about games like AI Dungeon because AI Dungeon doesn’t use reinforcement learning in the way the Creature does.
AI’s which learn and develop would be very frustrating to players who are looking for predictability. The inherent problem with AI enemies in a video game is that a video game NPC can already easily overpower a player even without a machine learning AI to assist it. So an enemy NPC would not have much good use for an AI.
However, one thing that an AI such as this could help create is a dynamic story. Often, RPGs (role-playing games) have a preset storyline with non-player characters who have a set and defined behavior. Using learning such as that of the Creature in Black & White, an RPG game could have a dynamic story wherein different NPC’s use their belief-desire-intention models to perform actions that can influence the direction the game plays out. This could be especially interesting in a more open, sandbox game such as Minecraft, where the world has a hard set of existing rules and predefined events that can be exploited to create creations and stories, or one such as Legends of Zelda: Breath of the Wild where an RPG element already exists alongside the open-world quality of the game.
The idea of more models like this — BDI’s and reinforcement learning — really excites me, and I’d love to see this idea more commonly. I hope it excites you, too!
Rishab Borah is a Student Ambassador in the Inspirit AI Student Ambassadors Program. Inspirit AI is a pre-collegiate enrichment program that exposes curious high school students globally to AI through live online classes. Learn more at https://www.inspiritai.com/.
Bibliography
Wexler, James. “Artificial Intelligence in Games.” University of Rochester Department of Computer Science, 7 May 2002, www.cs.rochester.edu/~brown/242/assts/termprojs/games.pdf.
