Generic filters
Exact matches only
Filter by content type

Interaction- and Communication-based Systems

Prof. Dr. Simon Mayer

In our research group, we explore interactions among devices and people in ubiquitous computing environments. Our focus is on the integration of physical things into the Web, on increasing the autonomy of Web-enabled devices, and on making interactions of connected devices intelligible for people.


Introducing Refashion

The Refashion circular fashion system — a project by SOLVE Studio that is supported by our research group — was featured last week on the European Commission’s website as an inspiring story of change contributing to the realization of the EU Textile Strategy. Refashion is a novel fashion design strategy that uses pre-designed multifunctional fabric blocks to create garments in a wide range of styles. This fashion design strategy aims to be zero-waste and sustainable. We are looking forward to exploring the potential of industrializing this circular design strategy together with SOLVE Studio!

Author: Andrei Ciortea

Date: 17. March 2023

Dagstuhl Seminar on Agents on the Web

This week, several members of our group are present at the Dagstuhl Seminar on the topic of Agents on the Web, which was proposed by Prof. Dr. Andrei Ciortea together with a team of international researchers. This Dagstuhl Seminar aims to consolidate and further investigate the research opportunities identified in the Dagstuhl Seminar 21072 (Autonomous Agents on the Web) , and to continue the transfer of knowledge and results across the involved research communities. We believe this seminar can break new ground in all these areas of research – and can help pave the way for a new generation of Web-based autonomous systems composed of people and intelligent agents interacting and collaborating through the Web.

Author: Simon Mayer

Date: 20. February 2023

Signifiers as a First-class Abstraction in Hypermedia Multi-Agent Systems

A new paper from our group has been published at the 22nd International Conference on Autonomous Agents and Multiagent Systems: Signifiers as a First-class Abstraction in Hypermedia Multi-Agent Systems

Abstract: Hypermedia APIs enable the design of reusable hypermedia clients that discover and exploit affordances on the Web. However, the reusability of such clients remains limited since they cannot plan and reason about interaction. This paper provides a conceptual bridge between hypermedia-driven affordance exploitation on the Web and methods for representing and reasoning about actions that have been extensively explored for Multi-Agent Systems (MAS) and, more broadly, Artificial Intelligence. We build on concepts and methods from Affordance Theory and Human-Computer Interaction that support interaction efficiency in open and evolvable environments to introduce signifiers as a first-class abstraction in Web-based MAS: Signifiers are designed with respect to the agent-environment context of their usage and enable agents with heterogeneous abilities to act and to reason about action. We define a formal model for the contextual exposure of signifiers in hypermedia environments that aims to drive affordance exploitation. We demonstrate our approach with a prototypical Web-based MAS where two agents with different reasoning abilities proactively discover how to interact with their environment by perceiving only the signifiers that fit their abilities. We show that signifier exposure can be inherently managed based on the dynamic agent-environment context towards facilitating effective and efficient interactions on the Web.

Author: Simon Mayer

Date: 19. February 2023

QRUco: Interactive QR Codes Through Thermoresponsive Embeddings

A new paper from our group has been published at the 2023 ACM CHI Conference on Human Factors in Computing Systems, Interactivity Track: QRUco: Interactive QR Codes Through Thermoresponsive Embeddings

Abstract: Due to their low cost and ease of deployment, fiducial markers – primarily Quick Response (QR) codes – gained widespread popularity over the past decade. Given their original use cases in logistics, these markers were created with the goal of transmitting a single static payload. We introduce QRUco as an approach to create cheap yet interactive fiducial markers. QRUco uses thermochromic paint to embed three secondary markers into QR code finder patterns. Users may interact with these markers through rubbing or pressing/touching, thereby changing the appearance of the marker while leaving the primary QR code intact. In this paper, we present the QRUco concept and demonstrate that our proposed approach is effective. We emphasize that QRUco markers can be created cheaply and that they do not require any specialized scanning equipment. We furthermore discuss limitations of the proposed approach and propose application domains that would benefit from QRUco.

Author: Simon Mayer

Date: 18. February 2023

Automatic Classification of High vs. Low Individual Nutrition Literacy Levels from Loyalty Card Data in Switzerland

A new paper from our group has been published at the International Workshop on Multimedia Assisted Dietary Management (MADiMa 2022): Automatic Classification of High vs. Low Individual Nutrition Literacy Levels from Loyalty Card Data in Switzerland

Abstract: The increasingly prevalent diet-related non-communicable diseases (NCDs) constitute a modern health pandemic. Higher nutrition literacy (NL) correlates with healthier diets, which in turn has favorable effects on NCDs. Assessing and classifying people’s NL is helpful in tailoring the level of education required for disease self-management/empowerment and adequate treatment strategy selection. With recently introduced regulation in the European Union and beyond, it has become easier to leverage loyalty card data and enrich it with nutrition information about bought products. We present a novel system that utilizes such data to classify individuals into high- and low- NL classes, using well-known machine learning (ML) models, thereby permitting for instance better targeting of educational measures to support the population-level management of NCDs. An online survey (n = 779) was conducted to assess individual NL levels and divide participants into high- and low- NL groups. Our results show that there are significant differences in NL between male and female, as well as between overweight and non-overweight individuals. No significant differences were found for other demographic parameters that were investigated. Next, the loyalty card data of participants (n = 11) was collected from two leading Swiss retailers with the consent of participants and a ML system was trained to predict high or low NL for these individuals. Our best ML model, which utilizes the XGBoost algorithm and monthly aggregated baskets, achieved a Macro-F1-score of .89 at classifying NL. We hence show the feasibility of identifying individual NL levels based on household loyalty card data leveraging ML models, however due to the small sample size, the results need to be further verified with a larger sample size.

Author: Simon Mayer

Date: 21. January 2023

Human-Like Movements of Industrial Robots Positively Impact Observer Perception

A new paper from our group and in collaboration with the Institute of Behavioral Science and Technology has been published in the International Journal of Social Robotics: Human-Like Movements of Industrial Robots Positively Impact Observer Perception

Abstract: The number of industrial robots and collaborative robots on manufacturing shopfloors has been rapidly increasing over the past decades. However, research on industrial robot perception and attributions toward them is scarce as related work has predominantly explored the effect of robot appearance, movement patterns, or human-likeness of humanoid robots. The current research specifically examines attributions and perceptions of industrial robots—specifically, articulated collaborative robots—and how the type of movements of such robots impact human perception and preference. We developed and empirically tested a novel model of robot movement behavior and demonstrate how altering the movement behavior of a robotic arm leads to differing attributions of the robot’s human-likeness. These findings have important implications for emerging research on the impact of robot movement on worker perception, preferences, and behavior in industrial settings.

Author: Simon Mayer

Date: 20. December 2022

BetterPlanet: Sustainability Feedback from Digital Receipts

A new paper from our group has been published at the International Conference on Advances in Mobile Computing and Multimedia Intelligence (MoMM 2022): BetterPlanet: Sustainability Feedback from Digital Receipts

Abstract: The global food system accounts for 25–30% of anthropogenic greenhouse gas emissions. A large share of these emissions is due to individual food shopping patterns. Despite the rising concern about the environment, many individuals fail to act upon it and change their food consumption. In this study, we attempt to motivate individuals to reduce their food-shopping-induced environmental footprint. To narrow the intention-behavior gap, we propose a novel technical system that gives automated near-term sustainability feedback on individuals’ food shopping recorded on digital receipts and communicates this feedback through the mobile application BetterPlanet. Based on a small sample (n = 8), we find a directional decrease in the overall CO2-Scores. Therefore, our study demonstrates the technical feasibility of automated sustainability feedback from digital receipts. The proposed energy-weighted CO2-Scoring Model contributes to the growing knowledge body of sustainability assessment.

Author: Simon Mayer

Date: 27. November 2022

Increasing the Intelligence of Low-Power Sensors with Autonomous Agents

A new paper from our group has been published and won the Best Paper Award at the 2022 Workshop on Challenges in Artificial Intelligence and Machine Learning for Internet of Things (in conjunction with ACM SenSys 2022): Increasing the Intelligence of Low-Power Sensors with Autonomous Agents

Abstract: Low-power sensors are becoming ever more powerful, increasing both their energy efficiency as well as their processing capabilities. Much work in recent years has focused on optimizing machine learning models to low-power systems, typically to locally process sensor data. Significantly less attention has been paid to other artificial intelligence fields such as knowledge representation and automated reasoning, which may contribute to building autonomous devices. In this work, we present a low-power sensor node with an autonomous belief-desire-intention agent. This kind of agent simplifies the implementation of both proactive and reactive behaviors, promoting autonomy in our target applications. It does so by locally perceiving and reasoning, and then wirelessly broadcasting an intention, which can be forwarded to an actuator. The capabilities of the autonomous agent are demonstrated with a light-control application. Experiments demonstrate the feasibility of running intelligent agents in low-power platforms with little overhead.

Author: Simon Mayer

Date: 26. November 2022



Full Professor, Interaction and Communication based Systems


Research Assistant, Hypermedia Multi-agent Systems