Tetris

We use a Baxter robot to solve a puzzle inspired by the classic computer game of tetris. While using the same pieces, the robot's goal is to solve a puzzle in which all tetris pieces fill the frame without spaces.

This project demonstrates Baxter's use of computer vision, path planning, sensing, and actuation to complete the task of moving components between two places in a two dimensional plane. We believe that this task is particularly interesting because of the different shaped pieces adding a level of complexity as well as the increasing difficulty to place each piece on the board as the margin of allowed error becomes smaller. Some applications that require a similarly high level of precision include box-packing for deliveries or surgical robots.

Some of the interesting problems we had to solve:

• The board is rather large relative to Baxter's range of motion. This means that there is very little margin for how everything needs to be placed. One direct consequence of this is: since we're using both hands (one for gripping and the other for camera vision), we need to alternate moving the arms over the scene.
• As more pieces get placed on the board, it becomes a tighter fit for the other pieces. Accurate placement of pieces becomes more crucial as the board fills up. This complexity is magnified by the odd shapes of some of the pieces.
• We wanted to be able to do this project without the use of AR markers. However, we found that getting vision to detect the pieces was significantly more challenging than expected. Particularly, the glare of lighting against the table made it difficult to detect edges. We also needed a way to detect the shape of each piece, perhaps using some kind of convolutional neural network for classification. In the end, we believed the complexity to be prohibitive given time constraints and fell back on using AR markers.