University of Bamberg
open search form
You are here

Information Systems and Applied Computer Sciences

Multimodal Intelligent Interaction

  1.  University of bamberg
  2. Faculties
  3. Information Systems and Applied Computer Sciences
  4. Subject groups
  5. Applied Computer Science
  6. Multimodal Intelligent Interaction
  7. Research
  8. Topics
  9. Human-Robot Interaction

Shared Understanding and Joint Action

When humans and robots work together, they need to communicate and coordinate their actions. Speech, gestures, gaze and movement derive their meaning from the shared situation. A central question is how robotic systems can interpret human input and behavior while making their own actions understandable.

Shared tasks combine communication with spatial and temporal coordination. Expectations may differ, and input may be ambiguous. How can misunderstandings be detected and resolved? How can a robot’s behavior help people anticipate its next actions and adjust their own actions accordingly?

Technical experiments and user studies guide the development of interaction methods in our group. We examine how people experience the interaction and how the design of robot behavior affects the way people and robots carry out tasks together.

Selected Research Projects

JAMES: Joint Action for Multimodal Embodied Social Systems
EU FP7 · 2011–2014
Multimodal interaction with multiple participants, task and motion planning, and evaluation of different interaction approaches.

BAJA: Basic Aspects of Joint Action
DFG EXC CoTeSys · 2009–2010
Multimodal signals, object handovers, and the effects of different motion profiles in human-robot interaction.

JAHIR: Joint Action for Humans and Industrial Robots
DFG EXC CoTeSys · 2006–2009
Nonverbal communication, dialog-based assembly, and physical collaboration with industrial robots in shared workspaces.

JAST: Joint-Action Science and Technology
EU FP6 · 2004–2008
Multimodal interaction in a shared assembly task, combining dialog, goal inference, task planning, and physical execution.

Selected Publications

Wildgrube, Fabian, Perzylo, Alexander, Rickert, Markus, & Knoll, Alois. (2020). Semantic Mates: intuitive Geometric Constraints for Efficient Assembly Specifications. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 6180–6187. https://doi.org/10.1109/IROS40897.2019.8968041

Perzylo, Alexander, Rickert, Markus, Kahl, Bjorn, Somani, Nikhil, Lehmann, Christian, Kuss, Alexander, Profanter, Stefan, Beck, Anders Billeso, Haage, Mathias, Hansen, Mikkel Rath, Nibe, Malene Tofveson, Roa, Maximo A., Sornmo, Olof, Robertz, Sven Gestegard, Thomas, Ulrike, Veiga, Germano, Topp, Elin Anna, Kessler, Ingmar, & Danzer, Marinus. (2019). SMErobotics: Smart Robots for Flexible Manufacturing. IEEE Robotics & Automation Magazine, 26(1), 78–90. https://doi.org/10.1109/MRA.2018.2879747

Rickert, Markus, Gaschler, Andre, & Knoll, Alois. (2019). Applications in HHI: Physical Cooperation. In Ambarish Goswami & Prahlad Vadakkepat (Eds.), Humanoid Robotics: a Reference (pp. 2221–2259). Springer. https://doi.org/10.1007/978-94-007-6046-2_129

Kraft, Martin, & Rickert, Markus. (2017). How to Teach Your Robot in 5 Minutes: applying UX Paradigms to Human-Robot-Interaction. Human-Robot Collaboration and Human Assistance for an Improved Quality of Life: IEEE RO-MAN 2017: 26th IEEE International Symposium on Robot and Human Ineractive Communication: August 28-September 1, 2017, Lisbon, Portugal, 942–949. https://doi.org/10.1109/ROMAN.2017.8172416

Profanter, Stefan, Perzylo, Alexander, Somani, Nikhil, Rickert, Markus, & Knoll, Alois. (2015). Analysis and Semantic Modeling of Modality Preferences in Industrial Human-Robot Interaction. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2015): Hamburg, Germany, 28 September - 2 October 2015, 1812–1818. https://doi.org/10.1109/IROS.2015.7353613

Roitberg, Alina, Somani, Nikhil, Perzylo, Alexander, Rickert, Markus, & Knoll, Alois. (2015). Multimodal Human Activity Recognition for Industrial Manufacturing Processes in Robotic Workcells. ICMI ’15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, 259–266. https://doi.org/10.1145/2818346.2820738

Gaschler, Andre, Springer, Maximilian, Rickert, Markus, & Knoll, Alois. (2014). Intuitive Robot Tasks with Augmented Reality and Virtual Obstacles. 2014 IEEE International Conference on Robotics and Automation (ICRA), 6026–6031. https://doi.org/10.1109/ICRA.2014.6907747

Roitberg, Alina, Perzylo, Alexander, Somani, Nikhil, Giuliani, Manuel, Rickert, Markus, & Knoll, Alois. (2014). Human Activity Recognition in the Context of Industrial Human-Robot Interaction. Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific, 1–10. https://doi.org/10.1109/APSIPA.2014.7041588

Kupferberg, Aleksandra, Glasauer, Stefan, Huber, Markus, Rickert, Markus, Knoll, Alois, & Brandt, Thomas. (2011). Biological Movement Velocity Profile Increases Acceptance of Humanoid Robots as Human Partners in Motor Interaction. AI & Society, 26(4), 339–345. https://doi.org/10.1007/s00146-010-0314-2

Giuliani, Manuel, Müller, Thomas, Rickert, Markus, Knoll, Alois, & Lenz, Claus. (2010). Design Principles for Safety in Human-Robot Interaction. International Journal of Social Robotics, 2(3), 253–274. https://doi.org/10.1007/s12369-010-0052-0

Huber, Markus, Radrich, Helmuth, Wendt, Cornelia, Rickert, Markus, Knoll, Alois, Brandt, Thomas, & Glasauer, Stefan. (2009). Evaluation of a Novel Biologically Inspired Trajectory Generator in Human-Robot Interaction. RO-MAN 2009: The 18th IEEE International Symposium on Robot and Human Interactive Communication, 639–644. https://doi.org/10.1109/roman.2009.5326233

Huber, Markus, Rickert, Markus, Knoll, Alois, Brandt, Thomas, Glasauer, Stefan, & Kupferberg, Aleksandra. (2009). Video Observation of Humanoid Robot Movements Elicits Motor Interference. Adaptive and Emergent Behaviour and Complex Systems: Proceedings of the 23rd Convention of the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB), 81–85.

Foster, Mary Ellen, Giuliani, Manuel, Müller, Thomas, Rickert, Markus, Knoll, Alois, Erlhagen, Wolfram, Bicho, Estela, Hipólito, Nzoji, & Louro, Luís. (2008). Combining Goal Inference and Natural-Language Dialogue for Human-Robot Joint Action. Proceedings of the 1st International Workshop on Combinations of Intelligent Methods and Applications (CIMA), 25–30.

Huber, Markus, Rickert, Markus, Knoll, Alois, Brandt, Thomas, & Glasauer, Stefan. (2008). Human-Robot Interaction in Handing-Over Tasks. RO-MAN 2008: The 17th IEEE International Symposium on Robot and Human Interactive Communication, 107–112. https://doi.org/10.1109/ROMAN.2008.4600651

Lenz, Claus, Nair, Suraj, Rickert, Markus, Knoll, Alois, Rösel, Wolfgang, Gast, Jürgen, Bannat, Alexander, & Wallhoff, Frank. (2008). Joint-Action for Humans and Industrial Robots for Assembly Tasks. RO-MAN 2008: The 17th IEEE International Symposium on Robot and Human Interactive Communication, 130–135. https://doi.org/10.1109/ROMAN.2008.4600655

Rickert, Markus, Foster, Mary Ellen, Giuliani, Manuel, By, Tomas, Panin, Giorgio, & Knoll, Alois. (2007). Integrating Language: Vision and Action for Human Robot Dialog Systems. Universal Access in Human Computer Interaction: 4th International Conference in Universal Access Im Human-Computer Interaction, UAHCI 2007, Held as Part of HCI International 2007, Beijing, China, July 22-27, 2007; Proceedings, Part II, 987–995. https://doi.org/10.1007/978-3-540-73281-5_108

Foster, Mary Ellen, By, Tomas, Rickert, Markus, & Knoll, Alois. (2006). Human-Robot Dialogue for Joint Construction Tasks. Proceedings of the 8th International Conference on Multimodal Interfaces, 68–71. https://doi.org/10.1145/1180995.1181009