Validating (Pearson) PTE Core speaking criteria and rating against non‑teacher perceptions of general English proficiency for workplace and everyday communication

This validation study examines whether PTE Core speaking assessment reflects how non‑experts judge English used in everyday and workplace communication. Using verbal protocol analysis, it explores which aspects of speech shape layperson judgments and how these relate to automated scoring.

Key data

Description

Background

The Pearson PTE Core test is used to assess general English proficiency for workplace and everyday communication and is aligned with the Canadian Language Benchmarks (CLB). Its speaking scores are generated through automated scoring models trained on expert ratings. However, for a test designed for general communication, it is essential that the underlying criteria reflect how non‑experts—ordinary listeners—perceive spoken English. This project addresses the gap between expert‑driven criteria and real‑world listener perceptions.

Objectives

The project aims to determine whether PTE Core speaking criteria and automated scores align with the judgments of non‑teachers. It investigates:

how linguistic laypersons evaluate PTE Core speaking performances
which aspects of speech influence their judgments, and
how their evaluations compare to official automated scores.

The goal is to provide evidence for the validity of PTE Core speaking assessment.

Approach

The study uses Verbal Protocol Analysis (VPA) to capture participants’ thought processes while evaluating authentic PTE Core speaking samples from CLB levels 3 to 10. Participants rate each performance and verbalise the reasons behind their decisions. These verbal reports are transcribed, coded, and analysed using code‑frequency and correspondence analysis. Layperson ratings are compared with official PTE Core scores to identify alignment or divergence. The findings will contribute to improving the representativeness and fairness of automated speaking assessment.