Monitoring diverse sensor signals of patients in intensive care can be key to detect potentially fatal emergencies. But in order to perform the monitoring automatically, the monitoring system has to know what is currently happening to the patient: if the patient is for example currently being moved by medical staff, this would explain a sudden peak in the heart rate and would thus not be a sign of an emergency.

To create such annotations to the data automatically, the CVPC-Team has teamed up with University Hospital Zurich’s Intensive Care Unit (ICU) under the lead of Prof. Emanuela Keller to equip the ICU-Cockpit-Software with video analysis capabilities: based on cameras in the patient room that deliver a constant, privacy-reserving video stream from the patient’s bed (i.e., no person can be identified based on the video resolution), the location of patient and medical staff shall be automatically detected and tracked to extract simple movement patterns. Based on these patterns, it shall be classified if and what medical intervention is currently performed on the patient. The research challenge in this project is to realize such a system without access to many labels, i.e., to learn the detection, tracking and classification in mainly un- and self-supervised ways.

Project AUTODIDACT is supposed to start in spring 2022 and deliver results until the year’s end. The project is funded by the Digitalization Initiative of the Zurich Universities (DIZH) under the Rapid Action Call in its Innovation Programme. Project lead is USZ’s Dr. Gagan Narula.