Playing Doom Using Llm

Doomed to Code: How we Teamed Up with Fireworks AI at MistralAI Hackathon to Conquer the Shores of Hell

GLM 5.2 is live! Opus-level intelligence at open-source rates. Pay per token on serverless. Try it today.

Blog

Playing Doom Using LLM Doomed to Code: How we Teamed Up with Fireworks AI at MistralAI Hackathon to Conquer the Shores of Hell

PUBLISHED 5/6/2024

Table of Contents Teaching the LLM to “see”

Game buffers

ASCII representation

Grid generation It's playtime

Chain of thought

Reinforcement learning You are the Demon Slayer

LLM Guidance

Multiple-choice tokens Killed by Signal Crunch time The Aftermath Building with Fireworks AI

Table of Contents

Guest Post by: Bhav Ashok, Umut, Paul - winners of Mistral AI SF Hackathon It was March 23rd 2024, the day of the hackathon we had been waiting for. The beautiful Shack15 space, located in the Ferry Building of San Francisco, was buzzing with excitement as sponsors set up their booths and eager hackers streamed in. I (Bhav Ashok), an ML researcher & founder from San Francisco, arrived at the location as I searched for my teammate Umut in the sea of hackers. Umut, a digital nomad and ML engineer from Turkey, whom I had met a week ago, mentioned he was going to be at the hackathon and was looking for a team to join. Since I hadn’t formed a team either, we decided to team up. As we settled into our workspace for the next 24 hours and started discussing ideas, Paul, a visiting MSc Student at Stanford originally from Paris, France, introduced himself and explained he was also looking for a team. We bounced around a few ideas before eventually agreeing upon one encompassing all our interests. The original idea was to build a “Game Copilot” which advised and demonstrated how to do certain things in the game. To illustrate this, we decided to use the video game Doom. Doom is a video game created in 1993 and has since gained cult status among hackers who have sought to run it on various hardware such as calculators, ATMs, fridges, and even a toaster . At the Mistral AI hackathon, it was obvious that we just HAD to see if Mistral-7B could play Doom. Fig 1. Overview of our approach We decided to use Fireworks AI for our hackathon project due its ease of use and developer experience. It enabled us to rapidly iterate during the short 24 hour timeline of the hackathon and quickly deploy our model minutes before the demo. Teaching the LLM to “see”

The model had to “see” the screen to play the game but the Mistral models only operated on text. We briefly considered using vision models but soon decided not to go down that path because it had been done before and was beyond the scope of the hackathon. We had to get clever. Representing them with bounding boxes was the simplest option but it required the model to do math. Representing each RGB pixel in ASCII would encode a sense of “spatial awareness” but not the semantics of the objects. Representing each pixel as a character corresponding to the object seen on screen would give us the best of both worlds - a compact representation and “spatial awareness”. Game buffers

In order to generate this representation, we had to leverage the information provided by the game engine. When the player moves around the map, the game engine renders several buffers. One such buffer that was relevant to our purposes was the label buffer. In the label buffer, each pixel contains metadata about the object that was rendered at that pixel such as its object type, 3D pose and bounding box.

Screen Buffer

Label buffer Fig 2. Visualization of buffers available from VizDoom ASCII representation

In order to encode each object, we did the simplest thing and used the first letter of the object name. For example the enemy character ‘Zombieman’ would be represented as the letter ‘Z’. A similar process was also used to generate ASCII values for the floor and wall buffers.

Fig 3. ASCII representation of Doom as seen by the LLM There were several challenges that we explicitly chose not to address due to time constraints. • The ceiling and floor were both encoded with the same object ID so while we could represent them separately, we decided to encode both of them with ‘F’ • Dead characters had the prefix ‘Dead’ before the object name. We decided not to handle this case for the purposes of the demo since the game would end once the enemies were dead.

Grid generation

To generate a more compact grid from the filled buffers, we mapped the larger screen space to the grid space we defined. We chose a grid size of 32x64 because it more closely matched the screen dimensions and fit within the context length. The context length of mistral-7B is 32K and a grid of 32x64 characters roughly corresponds to 2048 tokens. Together with our prompt, our total query length would be less than 3000 tokens. We could get away with a much larger grid size but it would be a tradeoff between latency and performance. To create a compelling demo, we wanted it to be fairly low latency and opted for a smaller grid size. Each pixel in grid-space corresponded to a block in screen-space. To simplify this step, we used the label in the center of the block. Several other adaptive approaches are possible such as assigning higher weights to the more important objects in the game. For example in this simplified grid, the enemy in the distance may not be represented if we just take the center pixel of each grid.

Fig 4. Center pixel representation Instead if we ensure enemies take precedence over other game objects, we can represent that an enemy exists in that block. This enables the agent to make more informed decisions about how to engage or hide from the enemy.

Fig 5. Adaptive weighted representation Although we did not get time to implement this adaptive approach during our hackathon, we are excited to see what other encodings are possible. It's playtime

To create training data, we had to play the game. Trust us, we had no other option, really.

Fig 6. Collecting training data After more than 20 playthroughs, we finally generated enough data to teach the model basic commands to play the game autonomously.

1 2 *Fig 7. JSON file of our training dataset*

The training data we generated only included an ‘ideal’ playthrough where the player would navigate directly to the enemies and engage them. However, one challenge with autonomous game playing is that the data the agent sees at each time step is dependent upon the previous action...