For Entering the Story click here: Story Mode Activate!
Tutorial by Michigan Robotics
The Story: Once upon a time, there was a robot named PR2, tasked with helping its human companions in their daily routines. PR2 was highly intelligent, able to perform complex tasks like cooking breakfast or tidying up the living room.
Problem: But one particular day, PR2 encountered a problem that made it question its understanding of the world around it. It was a regular morning, and PR2’s mission was simple: fetch the milk for breakfast. The milk, as always, was kept in the fridge. With its sensors and advanced object detection algorithms, PR2 was certain that it could locate the milk without any trouble. Confident in its abilities, the robot approached the kitchen and activated its detection systems, scanning the room for the familiar white carton. But after several moments of scanning, PR2 grew puzzled. There was no milk in sight. The robot rechecked its belief database, confirming that the milk should be in the fridge. Yet, when it looked at the fridge from the outside, all it could perceive was the fridge door, standing silently in front of it. PR2 couldn’t see through the fridge, and no matter how advanced its sensors were, the milk remained hidden behind that solid barrier. Then came a sudden realization. Of course! The fridge door was closed, and until it opened the door, the milk would remain out of reach, not just physically, but also perceptually.
Understanding: PR2 had encountered a fundamental truth about the world that even the most advanced technology couldn’t escape: sometimes, things are hidden from view until the environment changes. The fridge door acted as a shield, blocking PR2’s sensors from detecting the objects inside. The realization was profound. PR2 understood that some actions are prerequisites for others. The act of opening the fridge wasn’t just a step in retrieving the milk—it was necessary for perceiving the milk at all. Without opening the fridge, PR2 couldn’t gather the data it needed to find the milk. This taught PR2 an important lesson about tasks and perception. Just as humans understand intuitively that opening doors, drawers, or cabinets reveals hidden objects, robots like PR2 needed to learn these causal relationships. The door was a barrier, and barriers needed to be removed before certain actions—like detecting the milk—could be performed.
The Solution: PR2 opened the fridge door, and like magic, the milk appeared, perfectly placed on the middle shelf. With its sensors now able to detect the milk’s familiar shape and color, the robot completed its task. From that day forward, PR2 never forgot that sometimes the world is more than just what it can see. To truly interact with its environment, it had to consider not only the objects it needed to manipulate, but the steps required to even perceive those objects in the first place. And so, PR2 became a better robot, one that understood the subtle, yet vital, importance of knowing when to act before it could perceive. Because in a world full of hidden objects, perception wasn’t just about what was visible—it was about what could become visible, with the right actions.
Software Components
CRAM
is a software toolbox for the implementation of autonomous robots.
source codeKnowRob
is a knowledge processing system for robots.
source codeOpenEASE
is a web-based knowledge service providing robot and human activity data.
source codeGISKARD
is a framework for constraint- and optimization-based robot motion and planning control.
source codeROBOKUDO
is a perception framework targeted for robot manipulation tasks.
source codePyCRAM
is the Python 3 re-implementation of CRAM. PyCRAM is a toolbox for designing, implementing and deploying software on autonomous robots.
source code
Courses and/or Turorial
Courses:
Integrated Intelligent Systems
This lecture covers contemporary techniques of AI in cognitive robotics.
Robot Programming with ROS
Getting into the Robot Operating System ROS is a steptone for future roboticists.
SUTURO - sudo tidy-up-my-room
Students in this project design their own application to run on a real robot.