Assessing products and processes
Massey, Gary; Jud, Peter; Ehrensberger-Dow, Maureen (2015). Assessing products and processes: Developing process-oriented criteria to measure translation performance. In: 5th IATIS Conference: Innovative Paths in Translation and Intercultural Studies. (7-10 July). Belo Horizonte (Brazil): Universidade Federal de Minas Gerais.
Since Krings’ (1986) groundbreaking exploration of translators’ cognitive processes, translation researchers have been developing tools and techniques to investigate the processes behind translation products, and the effects of those processes on target-text quality. Process research methods have also found their way into translator education, serving to complement traditional product-oriented teaching by encouraging metacognition and self-regulation. Alongside more established techniques to access and evaluate translation processes, such as written commentaries and dialogue protocols, those currently proposed and successfully deployed in recent didactic and diagnostic experiments include screen recording combined with various forms of retrospection, self-evaluation, peer evaluation and trainer-student dialogue (e.g. Angelone 2013a, 2013b; Enríquez Raído 2013; Massey & Ehrensberger-Dow 2013). Over the past few years, all the compulsory entrance tests for our institute’s MA in Professional Translation have been recorded on-screen, introducing a process-oriented component to the diagnostic assessment of, and the formative feedback on, the performance and potential shown by candidates. Building on studies of process-oriented teaching and testing methods already implemented at our institute (cf. Massey & Ehrensberger-Dow 2013), as well as on work indicating how certain process measures may correlate with translation quality and even predict subsequent performance (e.g. Massey & Ehrensberger-Dow 2014), we have been attempting to identify indicators and predictors of performance in the processes of candidates taking and retaking our MA entrance tests. After reporting on the design and results of these exploratory studies, this paper discusses the possible applications and implications of our findings in the diagnostic, formative and summative assessment of translation competence. The ultimate objective of our research is to extend and refine traditional product-oriented measures by generating readily applicable criteria with which to evaluate observable screen-recorded actions and behaviour. It is hoped that these will offer hard-pressed staff and institutions an efficient, feasible means of assessing translation performance based not only on target-text products, but also on the processes that went into their making.
Angelone, E. (2013a). The impact of process protocol self-analysis on errors in the translation product. Translating and Interpreting Studies 8 (2), 253–271.
Angelone, E. (2013b). Watching and learning from 'virtual professionals': utilizing screen recording in process-oriented translator training. In: Kiraly, D., Hansen-Schirra, S. & Maksymski, K. (eds.), New Prospects and Perspectives for Educating Language Mediators. Tübingen: Narr, 139-155.
Enríquez Raído, V. (2013). Screen recording as a diagnostic tool in early process-oriented translator training. IN: Kiraly, D., Hansen-Schirra, S. & Maksymski, K. (eds.), New Prospects and Perspectives for Educating Language Mediators. Tübingen: Narr, 121-138.
Krings, H. P. (1986). Was in den Köpfen von Übersetzern vorgeht. Eine empirische Untersuchung zur Struktur des Übersetzungsprozesses an fortgeschrittenen Französischlernern. Tübingen: Narr.
Massey, G. & Ehrensberger-Dow, M. (2013). Evaluating translation processes: opportunities and challenges. Kiraly, D., Hansen-Schirra, S. & Maksymski, K. (eds.), New Prospects and Perspectives for Educating Language Mediators. Tübingen: Narr, 157-177.
Massey, G. & Ehrensberger-Dow, M. (2014). Looking beyond text: the usefulness of translation process data. In: Heine, C., Knorr, D. & Engberg, J. (eds.). Methods in Writing Process Research. Frankfurt am Main: Peter Lang, 157-177.