Modeling rater cognition in translation assessment

Abstract In this exploratory study, we investigated rater cognition in English–Chinese translation assessment, drawing on think-aloud, eye-tracking, and interview data. We designed a 3 × 2 × 2 experiment in which experienced raters assessed eighteen renditions of three levels of quality for each translation direction, using a Likert-type scale or analytic rubric scale. We found that: (a) the raters heeded meaning transfer more frequently than other contents; (b) they utilized a variety of processing actions, but a core subset involving eight actions constituted the mainstay; (c) to make a scoring decision, the raters mainly consulted the source text, the target texts, and the rating scale, but also displayed other patterns of interaction (e.g., relying on target texts only); (d) they fixated more frequently per time unit and proportionally longer on the target texts; and (e) translation direction and scoring method seemed to have modulated rater cognition. The implications of these findings for translation assessment are discussed.