Natural Spoken Human-Machine-Interaction

The Natural Language Generation and Dialogue Systems group conducts research in the area of Conversational AI on the naturalness of speech-based human-machine interaction. The focus is on technology research and the question of what behaviour is perceived as natural by humans.

For natural spoken interaction, a system must have the following three properties:

• It must be able to understand and utter arbitrarily complex dialogue structures.
• It requires a highly dynamic knowledge base and world model and must be able to converse and learn new concepts interactively, regardless of the topic. Similarly, it must be able to adapt to and talk about a continuously changing environment and situation.
• It must be able to show natural behaviour and behave adequately (politely, kindly, humorously, etc.) and appropriately to the context.

Such a dialogue system must therefore be able to understand and express complex dialogue structures about any subject matter, so that the way the system reacts is perceived by humans as natural for an artificial system.

The implementation of the research will be achieved through

• a combination of Large Language Models with classical dialogue management technology, allowing fine and subtle nuances of speech to be represented while using structural information and procedures for dialogue control,
• Explainable Reinforcement Learning to explain the learned behaviour and to answer what exactly natural behaviour means in human-machine language interaction in order to draw conclusions about naturalness in human communication in general, and
• Active and Weakly Supervised Learning to minimally invasively determine the target variables needed to learn natural behaviour so that the interaction itself and its naturalness is not affected.