Umeå University's logo

When the results of international comparative studies such as PISA or TIMSS get published, they are discussed broadly in media and are used to influence politics and public opinion. To solve mathematics PISA tasks, students have to read and understand the task text. Still, since the mathematics tasks are primarily supposed to measure mathematical ability and not reading ability, it is important to avoid unnecessary demands of reading ability in the tasks. In addition, the different language versions of a task used in PISA might vary in reading difficulty. Such differences can result in differential item functioning (DIF), that is, that students with the same mathematical ability but from different countries have a different probability of answering the item correctly. One reason for DIF between language versions is that linguistic features can differ between language versions. In this study we focus on four different linguistic features that in earlier studies have shown connections to the difficulty of solving mathematics tasks (e.g., Abedi, Lord, & Plummer, 1997).

• Grammatical person, that is, if the text is written in first, second, or third person.
• Voice, that is, if active or passive voice is used in the text.
• Sentence structure, that is, how the sentences are built of main and subordinate clauses.
• Word order, that is, the order of subject, finite verb, and object in the sentence.

This study is part of a larger project examining the relation between the language used in mathematics tasks and both the tasks’ difficulty and demand of reading ability. The research questions in this study are: Which differences in the four linguistic features investigated occur between PISA tasks in English, German, and Swedish? Which of these differences are related to DIF between the task versions? The English (USA), German, and Swedish language versions of 83 mathematics tasks of the PISA 2012 assessment are analyzed. The first step of the analysis was to search for differences in the four linguistic features between the different language versions of the tasks. The next steps will be quantitative analyses of the differences, a statistical analysis to detect DIF between the versions, and then statistical analyses to investigate possible relations between the differences and DIF. The first step showed that some differences occur sporadically, for example, the use of third person (he/she/it) in one language version and second person (you) in another language version. Other differences occur much more frequently. For example, differences in word order are quite common, in particular since the finite verb always is at the last position in subordinate clauses in German but not in English and Swedish. The next steps of the analysis are at present (January 2017) ongoing.