We are currently creating an AI algorithm using modern computer vision methods to assess movement quality in people after stroke. Our project, as many others, depends on therapist’s manual movement quality rating to create the ground truth. While therapists are trained to reliably assess movement quality in person (3-dimensional, 3D), it is unclear if they can do so when rating is based on videos (2-dimensional, 2D). The aim of this nested DFF project is to assess the reliability of video-based observations when evaluating compensatory movements in the upper extremities and trunk during a drinking task performed by post-stroke clients. Therefore, we recruit 25 therapists to assess 7 anonymized video recordings of patients after stroke and let them rate compensatory movements on a scale. We will analyze intra-rater and inter-rater reliability to contribute to reliable ground truth in AI applications for clinical decision-making processes.