In our research group, we explore interactions among devices and people in ubiquitous computing environments. Our focus is on the integration of physical things into the Web, on increasing the autonomy of Web-enabled devices, and on making interactions of connected devices intelligible for people.
The Refashion circular fashion system — a project by SOLVE Studio that is supported by our research group — was featured last week on the European Commission’s website as an inspiring story of change contributing to the realization of the EU Textile Strategy. Refashion is a novel fashion design strategy that uses pre-designed multifunctional fabric blocks to create garments in a wide range of styles. This fashion design strategy aims to be zero-waste and sustainable. We are looking forward to exploring the potential of industrializing this circular design strategy together with SOLVE Studio!
This week, several members of our group are present at the Dagstuhl Seminar on the topic of Agents on the Web, which was proposed by Prof. Dr. Andrei Ciortea together with a team of international researchers. This Dagstuhl Seminar aims to consolidate and further investigate the research opportunities identified in the Dagstuhl Seminar 21072 (Autonomous Agents on the Web) , and to continue the transfer of knowledge and results across the involved research communities. We believe this seminar can break new ground in all these areas of research – and can help pave the way for a new generation of Web-based autonomous systems composed of people and intelligent agents interacting and collaborating through the Web.
Abstract: Hypermedia APIs enable the design of reusable hypermedia clients that discover and exploit affordances on the Web. However, the reusability of such clients remains limited since they cannot plan and reason about interaction. This paper provides a conceptual bridge between hypermedia-driven affordance exploitation on the Web and methods for representing and reasoning about actions that have been extensively explored for Multi-Agent Systems (MAS) and, more broadly, Artificial Intelligence. We build on concepts and methods from Affordance Theory and Human-Computer Interaction that support interaction efficiency in open and evolvable environments to introduce signifiers as a first-class abstraction in Web-based MAS: Signifiers are designed with respect to the agent-environment context of their usage and enable agents with heterogeneous abilities to act and to reason about action. We define a formal model for the contextual exposure of signifiers in hypermedia environments that aims to drive affordance exploitation. We demonstrate our approach with a prototypical Web-based MAS where two agents with different reasoning abilities proactively discover how to interact with their environment by perceiving only the signifiers that fit their abilities. We show that signifier exposure can be inherently managed based on the dynamic agent-environment context towards facilitating effective and efficient interactions on the Web.
Abstract: Due to their low cost and ease of deployment, fiducial markers – primarily Quick Response (QR) codes – gained widespread popularity over the past decade. Given their original use cases in logistics, these markers were created with the goal of transmitting a single static payload. We introduce QRUco as an approach to create cheap yet interactive fiducial markers. QRUco uses thermochromic paint to embed three secondary markers into QR code finder patterns. Users may interact with these markers through rubbing or pressing/touching, thereby changing the appearance of the marker while leaving the primary QR code intact. In this paper, we present the QRUco concept and demonstrate that our proposed approach is effective. We emphasize that QRUco markers can be created cheaply and that they do not require any specialized scanning equipment. We furthermore discuss limitations of the proposed approach and propose application domains that would benefit from QRUco.
Abstract: The increasingly prevalent diet-related non-communicable diseases (NCDs) constitute a modern health pandemic. Higher nutrition literacy (NL) correlates with healthier diets, which in turn has favorable effects on NCDs. Assessing and classifying people’s NL is helpful in tailoring the level of education required for disease self-management/empowerment and adequate treatment strategy selection. With recently introduced regulation in the European Union and beyond, it has become easier to leverage loyalty card data and enrich it with nutrition information about bought products. We present a novel system that utilizes such data to classify individuals into high- and low- NL classes, using well-known machine learning (ML) models, thereby permitting for instance better targeting of educational measures to support the population-level management of NCDs. An online survey (n = 779) was conducted to assess individual NL levels and divide participants into high- and low- NL groups. Our results show that there are significant differences in NL between male and female, as well as between overweight and non-overweight individuals. No significant differences were found for other demographic parameters that were investigated. Next, the loyalty card data of participants (n = 11) was collected from two leading Swiss retailers with the consent of participants and a ML system was trained to predict high or low NL for these individuals. Our best ML model, which utilizes the XGBoost algorithm and monthly aggregated baskets, achieved a Macro-F1-score of .89 at classifying NL. We hence show the feasibility of identifying individual NL levels based on household loyalty card data leveraging ML models, however due to the small sample size, the results need to be further verified with a larger sample size.
Abstract: The number of industrial robots and collaborative robots on manufacturing shopfloors has been rapidly increasing over the past decades. However, research on industrial robot perception and attributions toward them is scarce as related work has predominantly explored the effect of robot appearance, movement patterns, or human-likeness of humanoid robots. The current research specifically examines attributions and perceptions of industrial robots—specifically, articulated collaborative robots—and how the type of movements of such robots impact human perception and preference. We developed and empirically tested a novel model of robot movement behavior and demonstrate how altering the movement behavior of a robotic arm leads to differing attributions of the robot’s human-likeness. These findings have important implications for emerging research on the impact of robot movement on worker perception, preferences, and behavior in industrial settings.
Abstract: The global food system accounts for 25–30% of anthropogenic greenhouse gas emissions. A large share of these emissions is due to individual food shopping patterns. Despite the rising concern about the environment, many individuals fail to act upon it and change their food consumption. In this study, we attempt to motivate individuals to reduce their food-shopping-induced environmental footprint. To narrow the intention-behavior gap, we propose a novel technical system that gives automated near-term sustainability feedback on individuals’ food shopping recorded on digital receipts and communicates this feedback through the mobile application BetterPlanet. Based on a small sample (n = 8), we find a directional decrease in the overall CO2-Scores. Therefore, our study demonstrates the technical feasibility of automated sustainability feedback from digital receipts. The proposed energy-weighted CO2-Scoring Model contributes to the growing knowledge body of sustainability assessment.
Abstract: Low-power sensors are becoming ever more powerful, increasing both their energy efficiency as well as their processing capabilities. Much work in recent years has focused on optimizing machine learning models to low-power systems, typically to locally process sensor data. Significantly less attention has been paid to other artificial intelligence fields such as knowledge representation and automated reasoning, which may contribute to building autonomous devices. In this work, we present a low-power sensor node with an autonomous belief-desire-intention agent. This kind of agent simplifies the implementation of both proactive and reactive behaviors, promoting autonomy in our target applications. It does so by locally perceiving and reasoning, and then wirelessly broadcasting an intention, which can be forwarded to an actuator. The capabilities of the autonomous agent are demonstrated with a light-control application. Experiments demonstrate the feasibility of running intelligent agents in low-power platforms with little overhead.
We research systems of autonomous agents for the Web: software agents able to meet their design objectives through flexible autonomous use of Web resources, and to support transparent and trustworthy interactions with people. We explore application areas ranging from industrial manufacturing to counteracting online disinformation.
Deep Insights into Real-World Objects and Environments
We explore how real-world objects might provide insights into their state, behavior, and provenance for users and how these insights might be used by users to derive smarter choices when interacting with such objects.
Enhanced and Intuitive User-Object Interactions
The geographic boundaries and economic limitations can be breached in virtual spaces.
Given ever smarter environments, we are more and more exposed to devices that process (personal) data about ourselves. Smart devices are no longer exclusive to high-end buildings and industrial settings, they are becoming ubiquitous in our everyday environments, such as in educational and private spaces (e.g., households). While, for some of these devices, specifically the ones we own and have installed ourselves, we might have consented to the processing of our data, we do not have that choice for devices installed in public spaces. Moreover, previous research has shown that users tend to accept privacy policies, even if they do not completely understand them, given that the perceived value of the services they want to use outweighs their data processing concerns (if any). In this project, you will be creating a Digital Privacy Companion that looks after its users, by making them aware of the data processing activities of smart devices and by acting on their behalf to ensure that their privacy concerns are considered in the environment they are located in. To this end, the companion will be capable of:
Recognising, through computer vision, smart devices that process (personal) user data;
Semantically understanding the privacy policies of such observed devices;
Communicating in an understandable way (through Mixed Reality) the privacy processing activities of diverse smart devices;
Identifying “legal rights” that can be exercised by the users while in an environment enabled with devices that process users (personal) data;
Utilising uniform interfaces to act upon the smart devices that process users (personal) data (i.e., actionable privacy)
You are:
A computer science or electrical engineering student interested in privacy
Excited to work in a technology project that could have a positive societal impact
Familiar with machine learning / Interested in computer vision
Interested or familiar with Mixed Reality
Interested in participating in state of the art research
Interested in publishing your research work in academic venues
The increasing number of connected (IoT-) devices in everyday environments calls for methods that enables users to intuitively and homogeneously interact with them. Mixed Reality head-mounted displays, such as the Microsoft HoloLens 2, are a suitable mean, since they allow users to perform hands-free interactions and they can augment the physical space of a user. To provide a homogeneous way to interact with a plethora of devices that have been made by different manufactures, we propose the usage of the Web of Things Thing Description (TD), a standardized way of describing the programming interface of a device (Thing).
In this project, you will create a Unity application for the HoloLens 2, capable of:
accessing the TD of a Thing,
parsing the TD from JSON to C#,
semantically understanding the TD's content, and
providing means (e.g., buttons, levers, text fields) to interact with a Thing through Mixed Reality in an intuitive way.
If you are a Master's student, you will enhance this application by making it additionally capable of:
remotely accessing a Thing's properties,
remotely interacting with a Thing,
live-streaming a video of the Thing to a remote user in which the interaction with the Thing is embedded,
granting access to different parts of the Thing's interaction possibilities based on a user's permission.
This project is great for you, if you are:
Strongly interested or familiar in programming Mixed/Augmented Reality applications
Interested in Semantic Technologies
Interested in participating in state of the art research
Interested in publishing your research work in academic venues
In our research group, we explore interactions among devices and people in ubiquitous computing environments. We are offering Master's and Bachelor's Thesis topics across a wide range of fields. Typically, a thesis at our group will include conceptual as well as implementation work, and we strive for achieving integration of our students with the rest of our research team.
If you would like to work with us, get in touch with Simon or another researcher in our team.
Due to their low cost and ease of deployment, fiducial markers – primarily Quick Response (QR) codes – gained widespread popularity over the past decade. Given their original use cases in logistics, these markers were created with the goal of transmitting a single static payload. We introduce QRUco as an approach to create cheap yet interactive fiducial markers. QRUco uses thermochromic paint to embed three secondary markers into QR code finder patterns. Users may interact with these markers through rubbing or pressing/touching, thereby changing the appearance of the marker while leaving the primary QR code intact. In this paper, we present the QRUco concept and demonstrate that our proposed approach is effective. We emphasize that QRUco markers can be created cheaply and that they do not require any specialized scanning equipment. We furthermore discuss limitations of the proposed approach and propose application domains that would benefit from QRUco.
get_appDanai Vachtsevanou, Andrei Ciortea, Simon Mayer, Jérémy Lemee
Forthcoming
Hypermedia APIs enable the design of reusable hypermedia clients that discover and exploit affordances on the Web. However, the reusability of such clients remains limited since they cannot plan and reason about interaction. This paper provides a conceptual bridge between hypermedia-driven affordance exploitation on the Web and methods for representing and reasoning about actions that have been extensively explored for Multi-Agent Systems (MAS) and, more broadly, Artificial Intelligence. We build on concepts and methods from Affordance Theory and Human-Computer Interaction that support interaction efficiency in open and evolvable environments to introduce signifiers as a first-class abstraction in Web-based MAS: Signifiers are designed with respect to the agent-environment context of their usage and enable agents with heterogeneous abilities to act and to reason about action. We define a formal model for the contextual exposure of signifiers in hypermedia environments that aims to drive affordance exploitation. We demonstrate our approach with a prototypical Web-based MAS where two agents with different reasoning abilities proactively discover how to interact with their environment by perceiving only the signifiers that fit their abilities. We show that signifier exposure can be inherently managed based on the dynamic agent-environment context towards facilitating effective and efficient interactions on the Web.
To enable people to interact more efficiently with virtual and physical services in their surroundings, it would be beneficial if information could more fluently be passed across digital and non-digital spaces. To this end, we propose to combine semantic technologies with Optical Character Recognition on an Augmented Reality (AR) interface to enable the semantic integration of (written) information located in our everyday environments with Internet of Things devices. We hence present SOCRAR, a system that is able to detect written information from a user’s physical environment while contextualizing this data through a semantic backend. The SOCRAR system enables in-band semantic translation on an AR interface, permits semantic filtering and selection of appropriate device interfaces, and provides cognitive offloading by enabling users to store information for later use. We demonstrate the feasibility of SOCRAR through the implementation of three concrete scenarios.
get_appAurelia Tamo-Larrieux, Andrei Ciortea, Simon Mayer
Journal paper
Intelligent machines surprise us with unexpected behaviors, giving rise to the question of whether such machines exhibit autonomous judgment. With judgment comes (the allocation of) responsibility. While it can be dangerous or misplaced to shift responsibility from humans to intelligent machines, current frameworks to think about responsible and transparent distribution of responsibility between all involved stakeholders are lacking. A more granular understanding of the autonomy exhibited by intelligent machines is needed to promote a more nuanced public discussion and allow laypersons as well as legal experts to think about, categorize, and differentiate among the capacities of artificial agents when distributing responsibility. To tackle this issue, we propose criteria that would support people in assessing the Machine Capacity of Judgment (MCOJ) of artificial agents. We conceive MCOJ drawing from the use of Human Capacity of Judgment (HCOJ) in the legal discourse, where HCOJ criteria are legal abstractions to assess when decision-making and judgment by humans must lead to legally binding actions or inactions under the law. In this article, we show in what way these criteria can be transferred to machines.
get_appMarc Blöchlinger, Jing Wu, Simon Mayer, Klaus Fuchs, Melanie Stoll, Lia Bally
Journal paper
The increasingly prevalent diet-related non-communicable diseases (NCDs) constitute a modern health pandemic. Higher nutrition literacy (NL) correlates with healthier diets, which in turn has favorable effects on NCDs. Assessing and classifying people's NL is helpful in tailoring the level of education required for disease self-management/empowerment and adequate treatment strategy selection. With recently introduced regulation in the European Union and beyond, it has become easier to leverage loyalty card data and enrich it with nutrition information about bought products. We present a novel system that utilizes such data to classify individuals into high- and low- NL classes, using well-known machine learning (ML) models, thereby permitting for instance better targeting of educational measures to support the population-level management of NCDs. An online survey (n = 779) was conducted to assess individual NL levels and divide participants into high- and low- NL groups. Our results show that there are significant differences in NL between male and female, as well as between overweight and non-overweight individuals. No significant differences were found for other demographic parameters that were investigated. Next, the loyalty card data of participants (n = 11) was collected from two leading Swiss retailers with the consent of participants and a ML system was trained to predict high or low NL for these individuals. Our best ML model, which utilizes the XGBoost algorithm and monthly aggregated baskets, achieved a Macro-F1-score of .89 at classifying NL. We hence show the feasibility of identifying individual NL levels based on household loyalty card data leveraging ML models, however due to the small sample size, the results need to be further verified with a larger sample size.
get_appYousouf Taghzouti, Danai Vachtsevanou, Simon Mayer, Andrei Ciortea
Journal paper
Content negotiation aims at enabling a server to provide a client with a representation of a resource that meets its needs. However, client and server might desire to negotiate constraints that go beyond the media type or language of the alternative representation. This is especially true in the Semantic Web, as a resource can be described with a single media type, but with different vocabularies (FOAF, schema.org, etc.), and may match specific patterns. In this paper, we propose an approach to increase the flexibility when negotiating a representation between client and server. Our approach follows the goals of the World Wide Web and uses a set of existing technologies: SHACL and profile-based negotiation. We define the mechanism (in terms of protocol and algorithm) for clients to announce their expectations and for servers to react and respond to them. We then explain, through a use case, how the same approach could be used in Web-based Multi-Agent Systems to help autonomous agents achieve their goals on the Web.
A higher degree of automation - and autonomization - of agricultural processes is expected to lead to productivity gains, especially in light of more environmentally-friendly farming practices, while improving the safety of agricultural processes. To exploit the potential of this development, it should be possible to flexibly integrate devices and services within service mashups, and thereby enable them to provide higher-value services together, However, current farm automation tools instead tend to reinforce vertical functional silos and tight coupling within often proprietary systems that manage the farm environment information. We propose to describe capabilities of individual devices and services and interlink them across components and with the description of the farm environment. We posit that this will better enable autonomous agents - software agents as well as humans - to perform complex agricultural tasks while integrating heterogeneous devices and services across multiple vendors. Concretely, we describe - and demonstrate in a laboratory setting - the usage of a Knowledge Graph to describe the environment and equipment used to perform farming tasks. We show how a multi-agent-based automation system for smart farming uses this graph to reason about the state of the environment and the agents to plan the achievement of user-specified goals. Furthermore, we show how such knowledge-driven autonomous systems may include human agents alongside artificial agents as first-class citizens, towards realizing "Social Machines" in the agriculture domain.
get_appClement Guitton, Aurelia Tamo-Larrieux, Simon Mayer
Journal paper
The field of computational law has increasingly moved into the focus of the scientific community, with recent research analysing its issues and risks. In this article, we seek to draw a structured and comprehensive list of societal issues that the deployment of automatically processable regulation could entail. We do this by systematically exploring attributes of the law that are being challenged through its encoding and by taking stock of what issues current projects in this field raise. This article adds to the current literature not only by providing a needed framework to structure arising issues of computational law but also by bridging the gap between theoretical literature and practical implementation. Key findings of this article are: (1) The primary benefit (efficiency vs. accessibility) sought after when encoding law matters with respect to the issues such an endeavor triggers; (2) Specific characteristics of a project—project type, degree of mediation by computers, and potential for divergence of interests—each impact the overall number of societal issues arising from the implementation of automatically processable regulation.
get_appNaomi Stricker, Jan Albert Liam, Md Masoon Rabbani, Simon Mayer, Andres Gomez
Book Section
Batteryless sensors have recently been proposed as an energy-efficient and cost-effective alternative to battery-powered sensors. By harvesting and immediately consuming ambient energy, it becomes unnecessary to design systems with large energy storages, which significantly increases their form factor, cost, and environmental impact. Many works have focused on designing batteryless systems that sense data, process it, and wirelessly broadcast processed data. Yet wireless security aspects of batteryless applications are only now receiving attention. In this work, we propose a secure communication system based on symmetric key encryption, that enables batteryless sensors to securely broadcast data. Furthermore, we demonstrate this system on a batteryless smartcard in two application scenarios: static sensor deployment for secure data gathering, and mobile device for identification purposes. Our experimental results demonstrate not only the feasibility of secure communication with batteryless devices but also the small overheads of introducing security in wireless beaconing applications.
get_appJérémy Lemee, Danai Vachtsevanou, Simon Mayer, Andrei Ciortea
Conference or Workshop Item
The ecological psychologist James J. Gibson defined the notion of affordances to refer to what action possibilities environments offer
to animals. In this paper, we show how (artificial) agents can discover and use affordances in a Multi-Agent System (MAS) environment to achieve their goals. To indicate to agents what affordances are present in their environment and whether it is likely that these may help the agents to achieve their objectives, the environment may expose signifiers while taking into account the current situation of the environment and of the agent. On this basis, we define a Signifier Exposure Mechanism that is used by the environment to compute which signifiers should be exposed to agents in order to permit agents to only perceive signifiers that are likely to be relevant to them, and thereby increase their efficiency. If this
is successful, agents can interact with partially observable environments more efficiently because the signifiers indicate the affordances they can use towards which purposes. Signifiers thereby facilitate the exploration and the exploitation of MAS environments. An implementation of signifiers and of a Signifier Exposure Mechanism is presented within the context of a Hypermedia Multi-Agent System and the utility and efficiency of this model is presented through the development of a scenario.
The Chair of Interaction- and Communication-based Systems offers a series of challenging but rewarding courses on the topics of Ubiquitous Computing and Web-based Autonomous Systems as well as introductory courses to Computer Systems and to Computer Science.
Computersysteme
Dieser Kurs fokussiert auf die Hard‑ und Softwarekomponenten sowie die Übersetzungsprozesse, welche zusammen die Grundlage moderner Computersysteme bilden. Wir erarbeiten dadurch das notwendige Basiswissen und ‑verständnis bezüglich der Darstellung und Verarbeitung von Informationen in modernen Computersystemen.
Introduction to Computer Systems and Networks
The goal of this hands-on course is to equip participants with the fundamental knowledge and tools required to design, implement, analyze, and take decisions on distributed computer systems, in particular in the context of the Internet of Things.
Ubiquitous Computing
This course covers fundamental concepts, technologies, drivers, trends, and implications of Ubiquitous Computing. The course gives both, an overview of UbiComp methods and deeper dives into foundational technologies of UbiComp that are integrated with practical exercises, a seminar, and a course project. In addition, we discuss implications of the proliferation of UbiComp on businesses and on society as a whole.
Collaborations and Funding
This website uses cookies. If you continue to use this website, we will assume your consent. OKPrivacy Policy
Privacy Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.