We investigate technology-mediated interactions in complex socio-technical systems. Within this broad field, we are interested in a great many aspects of the interaction of (computer) systems with other systems as well as with people, and specifically in pervasive computing environments.
We invite you to browse our research, teaching, and members on this page! Also, have a look here for open topics for your Bachelor or Master thesis. You can find quick links to our publications at the very bottom. Some of our publications are available as (mostly AI-generated) songs as well.
Ongoing PhD Theses
In our group, the PhD work of Danai (Google Scholar) is on the bridging of affordance-driven and hypermedia-driven interaction in the design of adaptive interfaces for human and artificial agents in Web-based Multi-Agent Systems while Jérémy (Google Scholar) studies how heterogeneous agents can combine their own specialized knowledge and cognitive abilities in order to improve collaboration efficiency. Based on a fascination for bio-inspired technology in general and its application to software specifically, Alessandro is working on socio-technical systems as well, and currently applies this to the design and creation of information management systems for circular economy and sustainability. And Ganesh (Google Scholar) is striving to achieve autonomy in industrial control systems by making system and domain knowledge accessible to the automation agents, with a specific focus on autonomous buildings.
More autonomous systems might behave in a way that is not so intelligible for their human users, and hard to interpret for other machines - hence, Sanjiv (Google Scholar) works towards explaining contextual influences on the behavior of cyber-physical systems using multimodal scene understanding. Turning more towards people, Damian (Google Scholar) investigates how the behavior of an industrial robot can be adapted to the implicit and explicit responses of humans to promote satisfactory human-robot interactions. Such personalization also features heavily in Jannis' (Google Scholar; Personal Website) research - Jannis studies how ubiquitous personalization systems can make our interactions with our environment more efficient, safer and more inclusive, and how they can be built in a responsible and societally beneficial way. Luka (Google Scholar) focuses on understanding and addressing the systemic risks posed by Very Large Online Platforms, combining methods from tech law, computer science, and social sciences to study how platforms personalized recommender systems influence us, while developing tools to enhance transparency and accountability in their operation. Benefitting individuals and society is also core to Jing (Google Scholar) - her research work is about analytics-based behavior interventions to promote human health, specifically in the fields of nutrition and ambient lighting.
Postdoctoral Researchers
Kim (Google Scholar) designs and studies systems that we refer to as Digital Companions; these are created to assist and protect their users while being aware of their privacy, and Kim uses a combination of machine-learning systems and symbolic approaches such as Knowledge Graphs in neuro-symbolic ensembles to achieve this. And Kenan (Google Scholar) brings in the human eye - quite literally, as since his PhD and during his postdoctoral studies he has been using eye tracking as an interaction mechanism and to study the workload of operators in human-robot interactions. All our work interacts with society, and societal processes, and hence regulation is a cross-cutting aspect - Clement (Google Scholar) brings in this dimension: He investigates the extent to which regulations (standards, laws, etc.) can and should be turned into an automatically processable format, for instance to automate compliance or facilitate their comprehension via querying possibilities.
Finally, our group is comprised of Andrei and Simon. Andrei (Google Scholar) is Assistant Professor for Web-based Systems and his research focuses on designing a new class of Web-based multi-agent systems (MAS) that inherit the architectural properties of the Web, preserve the properties of MAS, and are human-centric. And Simon (Google Scholar) is fascinated by the integration of concepts and approaches from across the fields of pervasive computing, hypermedia, human-computer interaction, and embedded systems to realize ideal interfaces between machines and animals.
To support bringing our research to practice, Lukas is our team's Research Entrepreneur - he is interested in exploring the commercial potential of newly emerging technologies and is particularly engaged in the fields of robotics (see Understanding Robots) and applications in the field of health care. And Jan is a software developer who maintains our lab's infrastructure and supports other team members in their research work wherever possible.
Come Visit Us!
We regularly welcome students from other international universities and other groups at the University of St.Gallen in our group. If you would like to work with us - for a Bachelor or Master thesis, a summer internship, or an idea that you'd like to realize together - get in touch with Simon!
And who has been a member in the past?
Since the creation of our group in the year 2018, we were privileged to have already had many short-term visitors. And there are also several colleagues who have left us again to take up their next (academic) challenge. These include Andres Gómez (Google Scholar, Personal Website) who is today an Associate Professor of Reliable Software and Hardware Systems at the Technical University of Braunschweig's Institute of Computer and Network Engineering.
The Chair of Interaction- and Communication-based Systems offers a series of challenging but rewarding courses on the topics of Ubiquitous Computing and Web-based Autonomous Systems as well as introductory courses to Computer Systems, Computer Networks, Distributed Systems, and to Computer Science more broadly.
In Autumn Semesters, we offer these courses:
In Spring Semesters, we offer these courses:
We are always looking for motivated students who would like to write their thesis at our group!
See the topics below for currently available thesis topics.
If you have a topic in mind that is not listed here, feel free to contact Simon (simon.mayer@unisg.ch)!
Diseases like Type-1 diabetes (T1D) and spinal muscular atrophy (SMA) affect peripheral nerve function and structure. While high-resolution ultrasound can assess nerve morphology, the current annotation process is time-intensive, requires specialized expertise, limits clinical evaluation to just 1-3 points, and misses variations along the nerve trajectory. Our project develops a methodology to comprehensively study the median nerve, which controls motor and sensory functions in the upper limb. This approach aims to enhance understanding of pediatric nervous system development and neuronal degeneration mechanisms in T1D and SMA patients by:
This project focuses on steps 2-3 of our methodology: systematically evaluating video object segmentation pipelines for pediatric median nerve ultrasound, and developing a robust pipeline for annotation, segmentation, and 3D reconstruction of ultrasound scans. In this interdisciplinary project, you will work with researchers at the University of St. Gallen's Interactions Research Group and clinicians at the Children's Hospital of Eastern Switzerland.
This project is great for you, if you are
If you want to further familiarize yourself with the topics, you can find more details in
Reach out if you are interested!
Interactions Research Group @ HSG
Dr. Kenan Bektas: kenan.bektas@unisg.ch
Prof. Simon Mayer: simon.mayer@unisg.ch
Children’s Hospital of Eastern Switzerland St. Gallen:
PD Dr. Philip Julian Broser: http://www.drbroser.org | PhilipJulian.Broser@kispisg.ch
Marc-Robin Gruener: marc-robin.gruener@unisg.ch
Social media platforms have become the new public squares, wielding massive influence over how billions of people consume information, form opinions, and take decisions. Yet, beneath the surface of these seemingly personalized experiences lies a profound challenge: the algorithms powering these platforms don’t just recommend content—they shape worldviews, amplify divisions, and steer societal discourse.
This thesis examines the systemic risks associated with personalized recommender systems on Very Large Online Platforms (VLOPs). While these systems are designed to enhance user engagement through content personalization, they often lead to unintended consequences, such as the formation of filter bubbles, distortion of public opinion, and increased societal polarization. Despite new regulations in the EU, such as the Digital Services Act (DSA), which promote data transparency and accountability from VLOPs, researchers continue to face significant barriers in accessing platform data from platforms like X (formerly Twitter), Instagram, TikTok, and others.
By employing black-box testing as a methodological framework, this project focuses on identifying, analyzing, assessing, and mitigating systemic risks. Additionally, it evaluates the effectiveness of the DSA in enabling meaningful research on these issues. Given the pivotal role of social media in shaping public discourse—especially during the major global elections of 2024 and the enforcement of the DSA—this research is both timely and critical.
In this interdisciplinary project, you will work at the crossroads of Computer Science, Legal/Platform Regulations, and Social Sciences. While the main focus lies in Computer Science, the project integrates insights from platform governance and societal impact to build a system/framework for evaluating systemic risks in Personalized Recommender Systems.
This framework will address key challenges in assessing the societal, ethical, and technical implications of recommender systems, contributing to state-of-the-art methods for evaluating their impact. You will have the opportunity to leverage your technical skills while exploring the broader regulatory and social dimensions of this critical field.
This project is great for you, if you are:
Unsure about the topic? See here for additional readings:
Reach out if you're interested!
Luka Bekavac: lukajurelars.bekavac@unisg.ch
Prof. Simon Mayer: simon.mayer@unisg.ch
Personalized nutrition typically requires users to continuously log what they eat, creating substantial user burden. Loyalty card data from retailers such as Migros and Coop could provide a strong starting point by capturing a significant share of household food purchases. However, household purchases do not equal individual consumption. This thesis explores how loyalty card data and multimodal user input can be combined using AI agents to estimate individual dietary behavior, balancing self-reporting effort, accuracy, and long-term user engagement. Instead of logging from scratch, users validate and enrich an automatically generated dietary profile through lightweight, adaptive interactions.
Keywords: Personalized nutrition, loyalty card data, dietary assessment, food logging, AI agents, large language models, multimodal input, human-in-the-loop annotation, user engagement, retrieval-augmented generation (RAG)
This project investigates a new approach to food logging, starting with what the system already knows and ask only what is missing.
You will build upon:
You will explore:
Increasingly, we are surrounded by smart devices that process (personal) data about us. We see a growing trend of such smart environments not only in industrial settings but also in educational spaces and at home. In our private spaces, we typically consent to the data processing of such devices when we set them up (e.g., a smart speaker, or a smart TV). However, other devices may process our personal data without explicit consent (e.g., in public spaces, rental homes, or friends’ houses). While these data can be used in favor of the user, for example to provide personalized experiences (e.g., making a space comfortable after a long day of work), these data can also be used for ad targeting, or it can be sold to third parties. To make users aware of potential privacy risks, researchers have proposed different types of privacy labels (e.g., printed on the box of smart devices). Although these privacy labels are useful for creating awareness, they do not help users to take action for protecting their privacy. Based on early efforts to create Mixed Reality applications to display such privacy labels in the context of use, this project aims at creating automatic means for detecting privacy risks for different type of users of smart environments (e.g., based on privacy personas). Once these risks are identified, possible options that can help people reduce those risks should be offered (e.g., the option to disable a microphone, blur a camera stream, or enact the user's legal right to be forgotten).
Context
In a previous project, we explored how different users might prefer different Mixed Reality user interfaces. Hence, a system exists that you will build upon, and a lot of literature has already been explored. For this project, we propose that you make use of visual object detection to find and categorize IoT devices in the environment, analyze their potentially privacy-threatening functionality, and provide users with options to act upon their rights (e.g., data deletion or transfer requests) through a Mixed Reality interface (by enriching the existing one).
Project topics
Learning Objectives
Literature list
[1] Garcia, Kimberly, Zaira Zihlmann, Simon Mayer, Aurelia Tamo-Larrieux, and Johannes Hooss. "Towards privacy-friendly smart products." In 2021 18th International Conference on Privacy, Security and Trust (PST), pp. 1-7. IEEE, 2021
[2] Emami-Naeini, Pardis, Janarth Dheenadhayalan, Yuvraj Agarwal, and Lorrie Faith Cranor. "An informative security and privacy “nutrition” label for internet of things devices." IEEE Security & Privacy 20, no. 2 (2021): 31-39.
Reach out if you are interested!
Jannis Strecker-Bischoff: jannis.strecker-bischoff@unisg.ch
Kimberly Garcia: kimberly.garcia@unisg.ch
Smart glasses (e.g., Ray-Ban Meta or Google's Android XR glasses) are equipped with cameras, microphones, and sometimes eye trackers that sense the wearer and their surroundings throughout the day [2, 4]. Yet, consent to such data collection is typically given once, when the device or an app is set up, and remains valid until the wearer actively revokes it. Some platforms already offer coarse conditions. For instance, Android XR lets people grant location access "only while using the app" instead of "all the time". However, such conditions are few, predefined by the platform, and tied to the state of the app rather than to the situation of the wearer. For smart glasses, this situation matters. People want to restrict sensors in specific contexts (e.g., at home, around confidential information, or in outdoor public locations) [3], and their attitudes toward data collection by Augmented Reality (AR) glasses vary with the context [4]. In this project, we explore the idea of *conditional consent*, i.e., consent interfaces in which wearers also specify when their consent is valid. A condition can define a start only ("valid as soon as …"), a stop only ("valid until …"), or both ("valid while …"). Conditions may refer to the wearer themselves (e.g., their location, the time of day, or who else is in the room). They may also refer to other people (e.g., what share of one's friends also use the app) or to the company behind the glasses (e.g., its headquarters location, governance structure, or revenue). Beyond that, conditions may concern third parties (e.g., data may be used only as long as it is not shared beyond research) or even politics (e.g., the current government, new legislation, or world events). Finally, some conditions are external, i.e., set by others for the wearer (e.g., a ban on smart glasses on public transport, as Hamburg is moving to introduce [9]).
Context
The affirmative consent framework [1] states that consent should be voluntary, informed, revertible, specific, and unburdensome. Conditional consent touches on several of these properties at once. Conditions may make consent more specific and revert it automatically when circumstances change. Yet, setting, understanding, and monitoring conditions may also make consent more burdensome. Recent AR research has moved beyond one-off, binary consent, for instance with fine-grained permissions [2], context categories in which people want to restrict sensors [3], and a framework of consent mechanisms for spontaneous AR interactions [5]. The theory of contextual integrity holds that whether a flow of information is appropriate depends on the norms of its context. For camera glasses, these norms vary with the setting and with the relationship between wearer and bystander [6]. Legally, the General Data Protection Regulation (GDPR), which also covers data collected by Mixed Reality devices [8], defines consent in terms similar to those of the affirmative consent framework. Consent must be freely given, specific, informed, and unambiguous (Art. 7 and Recital 32), and as easy to withdraw as to give [7]. Conditional consent may help meet these requirements, but it also raises the question of whether consent that becomes valid automatically still counts as a clear affirmative action. In previous work, we developed a web prototype that lets people set up an app on a pair of smart glasses through the smartphone the glasses are paired with. It shows Android XR-style permission requests and explains what the app can infer from each sensor, both about the wearer and about others. For this project, we propose that you map out the space of possible consent conditions and extend our prototype so that people can set such conditions. With the extended prototype, you will then study which conditions people want and whether they can express them with reasonable effort.
Project topics
- Usable Privacy and Consent
- Smart Glasses and Mixed Reality
- Human-Computer Interaction
- Privacy and Legal Context
Learning Objectives
- Review how far existing consent interfaces and permission systems (e.g., Android, iOS, or the companion apps of smart glasses) already support conditions
- Analyze how conditional consent can be designed to meet the consent requirements of regulation such as the GDPR (e.g., specificity, easy withdrawal, and a clear affirmative action)
- Develop a taxonomy of consent conditions, ranging from the individual to world politics
- Extend our smart glasses consent prototype so that people can specify start, stop, and "while" conditions
- Investigate how the glasses can detect when a condition is met, and how wearers are informed when their consent becomes valid or expires
- Explore conditions that involve bystanders (e.g., consent that holds only while no one else is in the room)
- Evaluate the prototype in a user study, using the properties of affirmative consent as criteria
Literature list
[1] Im, Jane, Jill Dimond, Melody Berton, Una Lee, Katherine Mustelier, Mark S. Ackerman, and Eric Gilbert. "Yes: Affirmative Consent as a Theoretical Framework for Understanding and Imagining Social Platforms." In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, 2021.
[2] Abraham, Melvin, Mark McGill, and Mohamed Khamis. "What You Experience is What We Collect: User Experience Based Fine-Grained Permissions for Everyday Augmented Reality." In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. ACM, 2024.
[3] Abraham, Melvin, Mohamed Khamis, and Mark McGill. "Don't Record My Private pARts: Understanding the Role of Sensitive Contexts and Privacy Perceptions in Influencing Attitudes Towards Everyday Augmented Reality Sensor Usage." In 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 2024.
[4] Gallardo, Andrea, Chris Choy, Jaideep Juneja, Efe Bozkir, Camille Cobb, Lujo Bauer, and Lorrie Cranor. "Speculative Privacy Concerns About AR Glasses Data Collection." Proceedings on Privacy Enhancing Technologies 2023, no. 4 (2023): 416-435.
[5] Windl, Maximiliane, et al. "Designing Effective Consent Mechanisms for Spontaneous Interactions in Augmented Reality." In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. ACM, 2025.
[6] Wang, Xueyang, Kewen Peng, Xin Yi, and Hewu Li. "Mind the Gap: Mapping Wearer-Bystander Privacy Tensions and Context-Adaptive Pathways for Camera Glasses." 2026.
[7] Kurteva, Anelia, Tek Raj Chhetri, Harshvardhan J. Pandit, and Anna Fensel. "Consent through the lens of semantics: State of the art survey and best practices." 2024.
[8] Mendelsohn, Juliane, Stephan Werner, Philipp Richter, Thomas Köllmer, Tobias Schwandt, and Wolfgang Broll. "Towards Privacy-Preserving Mixed Reality: Legal and Technical Implications." 2024.
[9] Rapporteur. "Hamburg Moves to Ban Smart Glasses on Public Transport." https://www.rapporteur.com/news/hamburg-moves-to-ban-smart-glasses-on-public-transport/
Reach out if you are interested!
Jannis Strecker-Bischoff: jannis.strecker-bischoff@unisg.ch
Alice Palmieri alice.palmieri@student.unisg.ch
Full Professor, Interaction and Communication based Systems
Assistant Professor
Research Assistant
Senior Researcher
Research Associate