Improving the trustworthiness of critical systems with AI: CAI and IDP are part of the EU HORIZON project AI4REALNET
In the project AI4REALNET, ZHAW researchers focus on the interaction of humans and AI-based solutions for critical systems like electricity, railway, and air traffic control. A key question that will be addressed is: What technological and ethical challenges arise from this human-AI cooperation?
The AI4REALNET project is a collaboration between the Centre for Artificial Intelligence (CAI), the Institute of Data Analysis and Process Design (IDP), several international universities and industry partners. It brings together a plethora of expertise from eight countries. Along with three other projects, it was selected from 114 submissions in a highly competitive European call.
Human-centered and robust development at the ZHAW
Critical infrastructure networks for mobility or electricity are usually operated by humans, yet increasingly the human expertise is augmented by control and supervision software and different levels of automation. “Because we are dealing with sensitive infrastructure, the stakes are very high. The AI systems have to be reliable, so that the critical applications are not endangered”, says Ricardo Chavarriaga from the CAI. This is why one team from the ZHAW that includes Prof. Dr. Thilo Stadelmann, Dr. Manuel Renold and Julia Usher will implement a powerful method, called reinforcement learning, that adapts to challenges. At the same time, a team consisting of Prof. Dr. Christoph Heitz, Dr. Ricardo Chavarriaga, and a PhD student, supervised by Prof. Teresa Scantamburlo from the University of Venice, focus on how to formalize and address the ethical aspect of the interplay between humans and AI in the context of critical infrastructure. The challenges of human-AI cooperation arise, for instance, when dealing with increasing uncertainty due to weather, asset age or demand. Also, the increasing automation needs to be overseen by humans who can intervene, when necessary. The project addresses these issues by developing trustworthy systems.
Robust operation for rail, air traffic and energy
The main objective of AI4REALNET is to successfully create an overarching multidisciplinary approach and to test and benchmark AI in industry-relevant use cases. The project team will combine emerging AI algorithms, existing open-source AI-friendly digital environments, socio-technical design of AI-based decision systems and human-machine interaction (HMI) to improve the operation of network infrastructures in real-time and predictive mode. The research aspects will be developed along three critical infrastructures, whose virtual and physical assets, systems and networks are considered vital in Europe and whose disruption would have a debilitating effect on society. These infrastructures are from the energy (electricity grid) and mobility (rail and air traffic management) sectors, two of the five priority sectors identified in the European national AI strategies. The project partners therefore include railway companies such as SBB and Deutsche Bahn, as well as air traffic services in various countries. In the use cases, the project team focuses not only on the critical challenges and tasks of network operators, but also considers strategic long-term goals such as decarbonisation, digitalisation and resilience.
Better decision-making through human-AI collaborations
Ricardo Chavarriaga explains the project’s vision: "Through AI4REALNET we want to explore the coexistence of human control and AI-based automation at different levels - from full human control with AI support to co-learning and trustworthy AI-based control. Traditionally, in safety-critical applications, the AI systems have been trained, tested and then frozen, unable to change. We want to develop systems that can adapt and improve over time.” The project therefore aims to empower humans, improve human performance, and achieve higher levels of critical infrastructure reliability and security. How will this be achieved? The project team will build on, train and test existing state-of-the-art algorithms to develop novel AI algorithms powered by reinforcement learning (RL) and supervised learning (SL). This approach combines the benefits of existing heuristics, physical modelling of these complex systems and learning methods, and a set of complementary techniques to improve transparency, safety, explainability and human acceptance. The team also includes human-in-the-loop decision making, which promotes co-learning between AI and humans, and autonomous AI systems that rely on human supervision. The CAI and IDP will address both the technical and ethical aspects of human-AI interaction in the context of critical infrastructures and develop the necessary algorithms and methods to enable and promote trust in these systems.