Pacbot 2023 Semester Recap
This season we are making major changes to Pacbot, including bringing the competition to UIUC.
Last season we won 1st place at the 2023 Harvard Undergraduate Robotics Pacbot Competition. During the competition, we saw how much potential the Pacbot competition has and felt that the competition should happen more than once per year, and in different locations. Therefore on February 10th, 2024, we will be hosting the first UIUC Pacbot Competition.
For this competition, we needed to create our own competition environment: a physical arena, integrate the game server and work on bot detection using computer vision (CV). Over the 2023 semester, we worked mainly on creating our own physical maze for the competition.
As for the robot, we started upgrading it. We are working on creating a slightly smaller form factor so our bot can traverse the maze faster: incorporating reinforcement learning into the high level, incorporating localization methods, and improve low-level controls.
SOFTWARE – HIGH LEVEL
For our bot, we’re creating a model for the bot to act on, as well as a simulation to test with.
Model: The PacBot model applies recent advances in the field of reinforcement learning to the tournament Pac-Man game. More specifically, the PacBot model applies the advanced Monte Carlo Tree Search and model-based reinforcement learning methods presented in the paper on MuZero – the model that achieved superhuman and state of the art performance on a number of game tasks, including chess, shogi, and Atari. At a high level, this model functions as a black box that, when given an input consisting of the Pac-Man positions, ghost positions, cherry positions, and all other relevant information, returns the best optimal next move for the Pac-Man from that state. It does this by, at each state, performing a number of simulations of the game into the future and working backwards to find out which move would lead it to the most optimal outcomes. The model is trained in such a manner that the values associated with certain states or actions during its simulations (whether those states/actions are good or bad for the Pac-Man score) reflect the reality of the game, but there is no inherent need for the data in those simulations to bear any resemblance to the actual game board. Intuitively, this means that the PacBot model can learn unique and creative patterns when simulating next game states, yielding better results than if this information were simply hardcoded in.
Simulation: We first recreated our past code from C++ to Python in order to make the code complement a Reinforced Learning environment, which we planned to do. We used the Open AI Gym module to create our Pacbot environment and the MuZero architecture for our Reinforcement Learning model. While changing our base code to Python we found that the implementation of the ghosts had to be recreated. This meant producing code for four different types of ghosts from Pacman. This involved implementing each ghost’s frightened, scatter, and movement patterns. This includes implementing how the ghosts exit the entrance box at the start of the game and when they have to enter it while Pacman eats the cherry power-up. Next, we recreated the code that revolved around the cherry power-up. We also had to redo the grid of Pacman to match Harvard’s playing field.
SOFTWARE – LOW LEVEL
For low level software, we are working to improve controls, incorporate localization methods, and use the Real-Time Operating System (RTOS).
RTOS is an Operating System designed to handle real-time tasks and applications, especially under crucial time constraints. For Pacbot, RTOS proves extremely important for deterministic behavior, resource management, task scheduling, and improving the low-level controls in our bot.
We started learning and trying to develop ideas with regards to how to implement our own scheduling system. Some challenges have been the learning curve with regards to systems programming as well as understanding some of the fundamental concepts. Recently there have been attempts made in creating a testing environment for our own scheduling system using FreeRTOS.
To prepare for the 2024 Pacbot competition, the low-level software team investigated a latency-related issue for the ToF laser distance sensors. We identified several areas for improvement and are evaluating an alternative sensor. Additionally, to improve autonomous navigation, we have prototyped and simulated a Monte-Carlo Localization system using 4 cardinally mounted laser ToF sensors and the motor encoders. To increase velocity control responsiveness, we have also prototyped a Phase-Locked Loop estimator based on the ODrive project. In simulation, the estimator can outperform naive secant differentiation in both latency and precision. Going forward, we hope to apply these learnings to our new bot platform and improve ToF latency on the legacy platform for our competition in February.
ELECTRICAL
During the 2023 Harvard Pacbot competition, we noticed at certain times the computer vision system unable to track the robot on the arena. We decided to use retroflective material on a small target that will be placed on every robot during the competition.
For the full setup, we created a CV lighting PCB which will be mounted over the arena to track the robots. The CV lighting PCB is a custom printed circuit board that is an integral part of the computer vision system for the UIUC Pacbot competition. It contains 8 high-powered LEDS that flash on and off, causing light to reflect from the pacbots and be picked up by the camera. This allows for much faster and more accurate tracking of the robot’s position.
MECHANICAL
One of the main projects the mechanical team worked on was the pacbot arena. The arena provides a way for the team to test the functionality of the robot, and create a higher success rate for our pacbot to clear the board. To create an 8ft by 9ft arena that could be easily stored, we used 8 boards of 2ft by 4ft plywood as the floor, and 3d printed parts to connect the floor to the walls in order to assemble an arena. We created notches in the floor to allow for the connectors to be placed and not moved when inserted into the floor.
NEXT SEMESTER
The UIUC Pacbot competition will take place on February 10th, and the Harvard Competition will take place on April 27th. For the UIUC Competition, we are focusing mainly on finishing pacbot competition environment (painting the physical maze and creating an overhead setup for the projector and lighting pcb) and testing within it. After that, we will shift gears back to the robot. We will incorporate all software that we are working on, as well as create a new pcb and chassis for our bot.
