}}

Dynamic Retail Robotics Laboratory

This laboratory focuses on addressing the complex challenges robots face within retail settings. Robots in this lab can autonomously deploy themselves in retail stores and constantly adapt to changing retail environments, including shelf layouts and product placements. They are trained to manage inventory, guide customers, and integrate real-time product information from various sources into actionable knowledge. Our goal is to develop robots that not only support shopping and inventory tasks but also seamlessly adjust to new products and store layouts, enhancing customer service and operational efficiency in the retail ecosystem.

In this laboratory, you are provided with two versatile robot action plans tailored for retail environments. The first plan focuses on creating semantic digital twins of shelf systems in retail stores, while the second is designed for restocking shelves. You have the flexibility to choose the specific task, robot, and environment. Once selected, you can execute the action plan through a software container, streamlining the process of implementing these robotic solutions in real-world retail settings.

For Detailed information click
here!
Michaela Kümpel (Knowledge Graph) and Simon Stelter (Robot Stocktaking and Manipulation)
Tel: +49 421 218 64021, +49 421 218 64014
Mail: michaela.kuempel@cs.uni-bremen.de,
stelter@cs.uni-bremen.de
Profile Michaela Kümpel
Profile Simon Stelter
Interactive Actions and/or Examples
ProductKG: A Product Knowledge Graph for User Assistance in Daily Activities
Abstract
The Web offers plenty of product information that is valuable for supporting decision processes. Research
on Web knowledge acquisition and the Semantic Web has led to the creation of many domain ontologies
and Web applications. What still is lacking is a connection of such knowledge to the real world. If object
information is linked to environment information, users can get better, more personalised support in
their daily activities like shopping or cooking since this enables them to link information about leftover
products in the fridge to recipe information or a health profile to products the user is looking at in
the store. It has been shown that semantic Digital Twins can successfully link object to environment
information that can be used by agents like smartphone or service robot. Such semantic Digital Twins
can offer even more services to users if they are connected to product information from the Web.
This work introduces ProductKG, an open-source product knowledge graph integrating modular
product information from the Web as well as accurate environment information from a semantic Digital
Twin that can be customised for different applications and used devices as an example knowledge graph
for assisting users in daily activities. We describe the design process and modularity of the knowledge
graph as well as example applications of it, including an Augmented Reality shopping assistant, a dietary
recommender and a hands-free recipe application. The modular ontologies enable personalisation of
applications as well as accessing object information in relation to the current environment. We evaluate
the acceptance of one example application through a user study. ProductKG is publicly available and
will be maintained and extended over time in order to facilitate various applications such as in the retail
and household domain.
Resource Website: https://michaelakuempel.github.io/ProductKG/
Application Website: http://productkg.informatik.uni-bremen.de/
and https://ai.uni-bremen.de/productkg
Publication
Robots Collecting Data: Modelling Stores
Abstract
Retail stores are a promising application domain for autonomous robotics.
Unlike other domains, such as households, the environments are more structured, products are designed to be easily recognizable, and items are consciously placed to facilitate their detection and manipulation.
In this book chapter we exploit these properties and propose a mobile robot systems that can be deployed in drugstores and autonomously acquire a semantic digital twin model of the store.
This facilitates autonomous robot fetch and place and shopping in a virtual replica of the store.
The potential commercial impact is substantial because in the retail business stores are an information blackbox and being able to automate inventory on a regular basis could improve the knowledge of retailers about their business drastically.
In this video the Kuka KMR IIWA performs a stock tacking action in the retail lab.
Publication
Manipulation planning and control for shelf replenishment
Abstract
Manipulation planning and control are relevant building blocks of a robotic system and their tight integration is a key factor to improve robot autonomy and allows robots to perform manipulation tasks of increasing complexity, such as those needed in the in-store logistics domain.
Supermarkets contain a large variety of objects to be placed on the shelf layers with specific constraints, doing this with a robot is a challenge and requires a high dexterity.
However, an integration of reactive grasping control and motion planning can allow robots to perform such tasks even with grippers with limited dexterity.
The main contribution of the letter is a novel method for planning manipulation tasks to be executed using a reactive control layer that provides more control modalities, i.e., slipping avoidance and controlled sliding.
Experiments with a new force/tactile sensor equipping the gripper of a mobile manipulator show that the approach allows the robot to successfully perform manipulation tasks unfeasible with a standard fixed grasp.
Publication
Robotic Clerks: Autonomous Shelf Refilling
Abstract
Nowadays, robots are used in the retail market mostly for warehousing, while they could be of great help in different in-store logistics processes as discussed in previous chapters.
The present chapter deals with the shelf replenishment task;
its execution by a robot requires overcoming of technological and methodological barriers in the handling of single products rather than the boxes containing them.
The challenges a robot has to face to replenish a supermarket shelf are all related to manipulation in narrow spaces of products with a large variety of size, shape, weight, and fragility.
The solution proposed by REFILLS is based on a robotic system where perception is used at all hierarchical levels of the control architecture, from high-level task planning algorithms and motion planning to reactive control layers based on physics models, where tactile and visual perception are combined to achieve highly reliable manipulation of items.
Experiments in an emulated supermarket shelf are carried out to demonstrate the effectiveness of the approach.
Publication
Shelf replenishment in Simulation
Software Components
Research 
See also