Matthew Gonzalez, Dr. Jason Isaacs, & Dr. Abbasi Bahareh
The Sawyer robot represents a pioneering leap in robotics, poised to redefine household tasks through the fusion of virtual reality (VR) and computer vision. Its six degrees of freedom allow it a wide range of motion, and with future versions of other humanoid robots currently being developed this proof of concept will help set the groundwork for future software development.
In its current stage Sawyer cannot currently use its on-board camera system for anything more than basic object recognition. This leaves room for situations where a new unseen and unexpected object shows up in Sawyers range of motion. This is where the Optitrack motion capture camera system comes into play, which allows you to place cameras around an environment and allow you to track objects. Integrating this system with Sawyer will give Sawyer the ability to see a 360-degree field of view along with accurate depth detection which would exceed that of 2D videos.
Our goal in this project would be to combine Sawyer, Optitrack and Artificial Intelligence to allow Sawyer to see and react to its environment and be able to operate autonomy to tasks that it is given. We have primarily found information on these topics though forums dedicated to software systems, and other robotics related forums on computer vision and its integration with robots. To that end we will also be working off Summer Surf Research done last year in 2023 by Sameeh Olipani, Alejandro Antonio and myself where we had success in allowing Sawyer to perform a pick and place action and had it place wooden planks to form a box. To expand upon this research, our next step, will be streaming the Opitrack information from a windows virtual machine to a Linux system that will then integrate the data into ROS so that Sawyer and other programs can utilize the information properly. We are currently in the process of transforming this data into something Sawyer can utilize by writing custom scripts in Python that helps to transform the data into a format readable by Sawyer through the SDK (software development kit) supplied with the robot.