A new paper from our group and with colleagues from OST – Eastern Switzerland University of Applied Sciences was published at the Workshop AutomationXP22: Engaging with Automation which took place during the ACM CHI Conference on Human Factors in Computing Systems (CHI 2022).

Abstract: Mixed Reality (MR) is becoming an integral part of many context-aware industrial applications. In maintenance and remote support operations, the individual steps of computer-supported (cooperative) work can be defined and presented to human operators through MR headsets. Tracking of eye movements can provide valuable insights into a user’s decision-making and interaction processes. Thus, our overarching goal is to better understand the visual inspection behavior of machine operators on shopfloors and to find ways to provide them with attention-aware and context-aware assistance through MR headsets that increasingly come with eye tracking (ET) as a default feature. Toward this goal, in two industrial scenarios, we used two mobile eye tracking devices and systematically compared the visual inspection behavior of novice and expert operators. In this paper we present our preliminary findings and lessons learned.

Link to the full paper

An impression from the workshop:

The Refashion Collection, a project by Solve Studio that was supported by members from our group, has been unveiled today. We are looking forward to exploring the potential of industrializing this circular design strategy together with Solve!

A new paper with involvement of a member of our group was awarded Best Demo at the International Conference on Information Processing in Sensor Networks (IPSN).

The work, titled “DPP3e: A Harvesting-based Dual Processor Platform for Advanced Indoor Environmental Sensing“, has been a collaboration with researchers from ETH Zurich. The demonstrator showcases the DPP3e dual-processor indoor energy-harvesting platform. The DPP3e (and its advanced sensors for indoor environment sensing) can be integrated via its BLE interface (with the ability to send BLE packets every 5 seconds while consuming only 37 𝜇W) and also features a sub-GHz LoRa transceiver to facilitate data collection over long distances.

Link to the full paper.

A new paper from our group and with colleagues from the University of Bologna was published at the 10th International Workshop on Engineering Multi-Agent Systems.

Abstract: In this paper we introduce and discuss an approach for multi-agent-oriented visual programming. This aims at enabling individuals without programming experience but with knowledge in specific target domains to design and (re)configure autonomous software. We argue that, compared to procedural programming, it should be simpler for users to create programs when agent abstractions are employed. The underlying rationale is that these abstractions, and specifically the belief-desire-intention architecture that is aligned with human practical reasoning, match more closely with people’s everyday experience in interacting with other agents and artifacts in the real world. On top of this, we designed and implemented a visual programming system for agents that hides the technicalities of agent-oriented programming using a blocks-based visual development environment that is built on the JaCaMo platform. To further validate the proposed solution, we integrate the Web of Things (WoT) to let users create autonomous behaviour on top of physical mashups of devices, following the trends in industrial end-user programming. Finally, we report on a pilot user study where we verified that novice users are indeed able to make use of this development environment to create multi-agent systems to solve simple automation tasks.

Link to the full paper

Link to the presentation at EMAS

A new paper from our group was published at the 10th International Workshop on Engineering Multi-Agent Systems.

Abstract: The ecological psychologist James J. Gibson defined the notion of affordances to refer to what action possibilities environments offer to animals. In this paper, we show how (artificial) agents can discover and use affordances in a Multi-Agent System (MAS) environment to achieve their goals. To indicate to agents what affordances are present in their environment and whether it is likely that these may help the agents to achieve their objectives, the environment may expose signifiers while taking into account the current situation of the environment and of the agent. On this basis, we define a Signifier Exposure Mechanism that is used by the environment to compute which signifiers should be exposed to agents in order to permit agents to only perceive signifiers that are likely to be relevant to them, and thereby increase their efficiency. If this is successful, agents can interact with partially observable environments more efficiently because the signifiers indicate the affordances they can use towards which purposes. Signifiers thereby facilitate the exploration and the exploitation of MAS environments. An implementation of signifiers and of a Signifier Exposure Mechanism is presented within the context of a Hypermedia Multi-Agent System and the utility and efficiency of this model is presented through the development of a scenario.

Link to the full paper

Link to the presentation at EMAS

A new paper with involvement from a member of our group was published at the ESWC 2022 Industry Track.

Abstract: Building automation (BA) systems orchestrate and monitor the functioning of a wide variety of utilities in a building so that living spaces are kept comfortable, safe, and secure. The complexity of such a system which involves multiple disciplines (heating, air-conditioning, lighting, fire safety, security etc.), coming from multiple vendors, is compounded by the fact that each building differs in the way the equipment operate and coordinate. So far, efforts involving semantic modeling of BA systems, like Haystack, IFC, or Brick, have been focusing on the description of the building topology, installed equipment, and to a lesser extent, the control strategy, the modeling of the physical process and the role of the control program. In addition, the semantics of interaction with the devices used in BA, which is essential to establish technical interoperability, has so far not been coupled to the BA semantic models. As a result, planners, project engineers, technical operators, and service technicians have to design and understand the working of the system by piecing together information from different sources. From our experience at the Smart Infrastructure division of Siemens AG, we describe briefly in the following sections some key use-cases, the challenges faced by us while applying semantic data in BA, and finally describe our approach and its evaluation in real-life buildings.

Link to the full paper

A new paper from our group was published at the Florida Artificial Intelligence Research Society: An Overview on the Explainability of Cyber-Physical Systems.

The paper argues that today’s Explainable Artificial Intelligence approaches, when applied to Cyber-Physical Systems, overlook the impact of the CPS’ physical and virtual context when explaining the outputs of decision-making software models, which are essential factors in explaining CPS’ behavior to stakeholders. It then proposes to enrich explanations of CPS behavior with contextual information using semantic technologies, user feedback, and enhanced explanation visualization techniques to improve their understandability. To that end, context-aware explanation and better explanation presentation based on knowledge graphs might be a promising research direction for explainable CPS.

Link to the full paper

The first workshop on Consumers of Distributed Knowledge Graphs in Digital, Industry and Space was organised as part of the activities of the Distributed Knowledge Graphs COST action (DKG).

On February 16th, over 35 researchers from across Europe gathered online to present their work on Distributed Knowledge Graphs in Digital, Industry and Space. Seven contributions were presented in two plenary sessions: Knowledge Graphs and Logic and Knowledge Graphs and Agents.  After the great presentations, which you can watch here, researchers took the opportunity to engage in lively and enriching discussions.

This workshop was organised by Dr. Kimberly Garcia (University of St. Gallen), Prof. Dr. Andreas Hart (FAU Erlangen-Nuremberg and Fraunhofer IIS, Germany) and Dr. Andras Micsik (SZTAKI, Hungary).