Umeå University's logo

This study uses results from TIMSS 1995 to examine the attitudes of Swedish girls and boys to physics and mathematics more closely. To be more precise, the attitude to mathematics and physics among girls and boys at different achievement levels are compared. At the time of testing the students were in the 7th grade. In TIMSS 1995 each student completed an achievement test and a questionnaire. In this study the results from the achievement test are used to divide the students into achievement levels – high and low. A number of statements about mathematics and physics were selected from the questionnaire and used to examine the differences in attitudes.

It turns out that the high-achieving boys and girls have a similar attitude when statements concerning mathematics are compared but not the statements concerning physics. The high-achieving girls’ attitude to physics is more similar to that of low-achieving boys. The statements that most clearly reveal this pattern can be divided into two categories: statements concerning the student’s attitude to physics and statements about the importance of physics to the student’s future studies and work.

In October 2009 a requirement to obtain a driver’s certificate was introduced for riding a moped class II. In order to obtain a driver’s certificate test-takers are required to complete a compulsory course and pass a test. Those who applied for permission to arrange tests for moped class II had to submit an example of a test to the Swedish Transport Agency with their applications. We have studied a sample of those tests. The content of the tests were analysed in order to determine to what extent the stipulated content of the regulations was covered. Compliance with general guidelines for item construction and whether the regulations concerning number of questions were met was also examined.

Among the 34 selected examiners 11 had developed their own test. The remaining examiners stated that they used the test developed by the Swedish National Association of Driving Schools (STR). Regulations for the test (TSFS 2010:80) stipulate that it should consist of 25 to 35 items. All tests in the study contained at least 25 items. With a couple of exceptions there are items from all four content areas outlined in the regulations. The main emphasis in the tests is on Traffic rules. The goals stipulated for the training and the goals stipulated for the test do not correspond entirely which probably is the reason why many items contain content not defined in the requirements for the test. However, most of such items can be linked to goals for the training.

As to shortcomings in the test some items could for example have been phrased more clearly and contained more information. Some response options deviate from the others or are just too unlikely which unfortunately makes the number of selectable options smaller.

Both content and design of the tests vary considerably why it would be best to let the Swedish Transport Agency develop the test for the conditional driving licence for mopeds class II. The test would be more uniform and equivalent, which would mean a fairer testing. It would be possible to strive for parallel tests. It would also be easier to supervise the results and the quality of the tests than is possible with the current system of supervision.

As the consistency of assessment of the practical driving test has been called into question by the Swedish National Audit Office the purpose of this study was to examine the consistency of assessment of test-takers’ performance on the driving test. In order to examine inter-rater reliability the study was designed so that two raters assessed the same test-taker. The assessment was done on a two-step rating scale (pass/fail). Since the result can be affected by factors associated with the test-taker and the two raters, questionnaires were developed and these were filled in by the test-takers and the raters. Information about the administration of the test was collected via a specially designed form filled in by the second rater. The ordinary examiners’ performance was also rated by the supervising raters on a number of aspects. The result from the study indicated that the agreement between the assessments was very good. For 93 per cent of the tests the two raters chose the same mark on the two-grade scale. In the cases where ratings differed the analyses indicated only a few systematic differences among variables designed to provide possible explanations for differences in opinion. However none of these was problematic with respect to consistency of assessment. Results indicated that most tests were carried out in a satisfactory manner. All in all, the results indicate high consistency of assessment, both from a national and international perspective.

A number of studies have been conducted in Sweden on the issue of postgraduate studies. However, the point of departure has mainly been the view of the postgraduate students. Few investigations have been conducted from the perspective of the supervisors. The main purpose of this study was to present a model for development of a program for assessment of postgraduate studies at Umeå University. The model suggested a questionnaire should be developed and distributed to postgraduate students and supervisors in all the faculties at the university. The questionnaires should focus on selected aspects considered to be essential, in order to reach a deeper understanding of interacting and under laying mechanisms as obstacles for success in postgraduate studies. The selected aspects are: The relation between the postgraduate student and the supervisor, the surrounding working climate, and the information and introduction given to the postgraduate students. Nevertheless, a number of theoretical and practical issues are to be considered before a program for assessment of postgraduate studies is to be implemented. A recommendation is made for how to design such a program.

Utifrån en rad perspektiv är det angeläget att antagningssystemet och de instrument som används för urval till högre utbildning har legitimitet hos systemets avnämare. Den undersökning som redovisas har genomförts inom ramen för VALUTAprojektet (Validering av den högre utbildningens antagningssystem). Syftet är att belysa antagningssystemets konsekvenser ur ett avnämarperspektiv genom att undersöka hur olika grupper av högskolestudenter ser på högskoleprovets relevans som urvalsprov samt om de studerande använt sig av strategier, i form av upprepat provdeltagande och komplettering av gymnasiebetyg. Huvudresultaten visar, oberoende av utbildning, kön och social bakgrund, att det delprov som anses vara det mest relevanta är LÄS medan det minst relevanta delprovet är ORD. Vidare att de studerande genomfört och upprepat högskoleprovet samt kompletterat sina gymnasiebetyg, men att dessa strategier använts på olika sätt av olika studerandegrupper vilket är relaterat till utbildning, kön och social bakgrund.

Våren 2004 infördes på försök två olika versioner av de nationella kursproven i matematik på kurserna B, C och D. På varje kurs skilde sig de båda versionerna åt i några av de ingående uppgifterna, medan övriga uppgifter var identiska. Syftet med denna studie är att undersöka om, hur och varför dessa förändringar i matematikuppgifterna påverkar uppgifternas svårighetsgrad.

Resultaten visar att en förändring av de i uppgifterna ingående talen endast i ett fåtal fall påverkat uppgifternas svårighetsgrad i någon större utsträckning. Dessa fåtal fall studeras vart och ett för sig. När två uppgifters sammanhang och formulering skiljer sig åt, även om det matematiska innehållet är i stort sett identiskt, visar ett exempel på att skillnaden i svårighetsgrad kan vara mycket stor.

A fairly large study of attitudes towards mathematics among Swedish students at secondary level was conducted during 2001–2004. A newly developed instrument was used that was designed to capture gender stereotyped attitudes among students related to various aspects of mathematics in education and future life. The new scale and its development are described with reference to the original Australian studies. The scale builds on the Fennema–Sherman attitude scale "Mathematics as a male domain" but allows mathematics to be viewed as female, male or gender neutral, reflecting a different societal and educational situation than in the seventies when attitudes towards mathematics as a male domain were first investigated. The Swedish study shows that mathematics is perceived as gendered, mostly as a male domain, by large minorities of students at secondary level. However, the results are complex, with clear differences in responses from female and male students. Furthermore, mathematics is also viewed as female in some aspects. A comparison with Australian data shows that Swedish students are less inclined to view mathematics as a female domain than Australian students of the same age. The relevance of the study is related to the lack of equity in mathematics in education and as a professional field in the Swedish society, documented by earlier research.

I denna tredje delrapport inom projektet Kön och matematikutbildning (GeMa) redovisas resultatet av en enkätundersökning bland elever i gymnasiets år två. Den aktuella studien har som mål att kartlägga vilka föreställningar det finns om matematiken som könsneutralt alternativt manligt eller kvinnligt område bland gymnasister. Tidigare har vi inom projektet gjort motsvarande studie bland elever i grundskolans år nio. På det sättet hoppas vi få en bild av utvecklingen i attityderna under de år som eleverna gör sina viktigaste utbildningsval.

Utgångspunkten är den vid det här laget klassiska Fennema-Sherman-skalan, en attitydskala från 70-talet med en delskala som kallas "Matematik som en manlig domän". Skalan har reviderats mot bakgrund av de stora förändringar som skett i skolan och samhället sedan 70-talet och som motiverar en könsneutral utformning av enkäten. Alternativet "Matematik som kvinnlig domän" kan vara aktuellt idag, då flickor i många länder presterar bättre resultat i matematik än pojkarna. Den nya skalan har utvecklats av forskare i Australien och den svenska studien görs i samarbete med dessa. I rapporten ges en bakgrund till hur könsskillnader kan beskrivas inom olika teoretiska ramar och hur begreppet "manligt område" kan förstås.

Undersökningen har omfattat omkring 550 elever i år två i gymnasieskolan på de direkt studieförberedande programmen i tre olika regioner. Enkätundersökningen har kompletterats med intervjuer, som kommer att redovisas i senare rapporter. Enkätsvaren visar sammanfattningsvis att majoriteten av eleverna uppfattar matematiken som könsneutral. Men det finns betydande minoriteter som uppfattar matematiken som i olika avseenden "manlig".

Exempel på resultat är att många, både kvinnor och män, tror att det är vanligare att män tycker matematik är lätt och intressant än kvinnor och att män behöver matematik för att få bra jobb när de blir vuxna. Resultaten ligger i linje med de attityder som grundskoleeleverna gav uttryck för, men föreställningen om matematik som manlig är mer uttalad hos gymnasieeleverna.

En sammanfattande slutsats är att en stor del av eleverna har anammat ett jämställdhetsperspektiv samtidigt som många av eleverna i sina svar ger uttryck för en föreställning om existerande könsskillnader.

The primary aim of this randomized controlled clinical trial was to compare the outcome from two types of short-term psychodynamic psychotherapy. The participants were thirty-nine women with depression. Half of the participants (n = 18) received art psychotherapy and the other half received verbal psychotherapy (n = 21). Data was collected before and after psychotherapy, and at a 3-month follow-up using self-rating scales and interviewer-based ratings. Results showed that art and verbal psychotherapies were comparable, and at follow-up, the average participant in both groups had few depressive symptoms and stress-related symptoms. The conclusion was that short-term psychodynamic art therapy could be a valuable treatment for depressed women.

Using the expectancy-value model of achievement motivation as a basis, this study’s purpose is to develop, apply, and validate scores from a self-report instrument measuring student test-taking motivation. Sampled evidence of construct validity for the present sample indicates that a number of the items in the instrument could be used as an indicator of student test-taking motivation. Exploratory factor analyses suggests that the test-taking motivation construct is distinct from general attitudes toward a subject and that task value perceptions are distinct from task performance expectancies. The instrument needs further development to consolidate its psychometric properties and to laborate on the testtaking motivation construct in relation to the expectancy-value theory of achievement motivation.

The main objective of this thesis was to explore issues related to student achievement motivation in the Swedish TIMSS 2003 (Trend in International Mathematics and Science Study) context. The thesis comprises of five empirical papers and a summary. The expectancy-value theory of achievement motivation was used as the general theoretical framework in all empirical papers, and all papers are concerned with construct validation in one form or another. Aspects of student achievement motivation were measured on a task-specific level (motivation to do well on the TIMSS test) and on a domain-specific level (self-concept in and valuing of mathematics and science) and regressed on test performance.

The first paper reports the development and validation of scores from an instrument measuring aspects related to student test-taking motivation. It was shown that a number of items in the instrument could be interpreted as a measure of test-taking motivation, and that the test-taking motivation construct was distinct from other related constructs. The second paper related the Swedish students’ ratings of mathematics test-taking motivation to mathematics performance in TIMSS 2003. The students in the sample on average reported that they were well motivated to do their best on the TIMSS mathematics test and their ratings of test-taking motivation were positively but rather weakly related to achievement. In the third and the fourth papers, the internal structure and relation to performance of the mathematics and science self-concept and task value scales used in TIMSS internationally was investigated for the Swedish TIMSS 2003 sample. For mathematics, it was shown that the internationally derived scales were suitable also for the Swedish sample. It was further shown that ratings of self-concept were rather strongly related to mathematics achievement while ratings of mathematics value were basically unrelated to mathematics achievement. For the science subjects, the internal structure of the scales was less simple, and ratings of self-concept and valuing of science were not very strongly related to science achievement. The study presented in the fifth paper used interviews and an open-ended questionnaire item to further investigate student test-taking motivation and perceptions of the TIMSS test. The results mainly corroborated the results from study II.

In the introductory part of the thesis, the empirical studies are summarized, contextualized, and discussed. The discussion relates obtained results to theoretical assumptions, applied implications, and to issues of validity in the TIMSS context.

The present study investigated whether two summated rating scales used in the Trends in International Mathematics and Science Study (TIMSS 2003), Mathematics Self-Concept (MSC), and Students' Valuing of Mathematics (VoM) seemed appropriate, meaningful, and useful in a Swedish context. The 4,256 Swedish eighth-grade students involved in TIMSS 2003 participated in the study. Items in the TIMSS Student Background Questionnaire asking for motivational beliefs were subjected to item analyses, principal components analyses, and confirmatory factor analyses. Factor scores were saved and regressed on mathematics score. The results indicated that the MSC scale was internally consistent, showed a simple factor structure that fitted the data rather well, and was positively related to mathematics achievement. The VoM scale was unrelated to mathematics score, showed a poor model-data fit, and the items in the scale were rather heterogeneous. The findings are discussed from an expectancy-value perspective and suggestions for researchers performing secondary analyses on TIMSS data are presented.

This study explores the reported level of test-taking motivation and the relation between test-taking motivation and mathematics achievement in a sample of Swedish eighth-grade students (n = 343) participating in Trends in International Mathematics and Science Study (TIMSS) 2003. A majority of students reported that they were motivated to do their best in TIMSS. Test-taking motivation was positively related to mathematics achievement, but the effect was small and not statistically significant when other relevant variables were held constant. Further, gender comparisons showed that test-taking motivation was positively, but not significantly related to achievement in boys, and was unrelated to achievement in girls. This result was probably due to the larger variability in boys' ratings. It was concluded that the Swedish TIMSS mathematics result is unlikely to be affected by a lack of student motivation, but that further research on the relation between test-taking motivation and test achievement is warranted.

The study’s objective was to investigate student test-taking motivation in the Swedish TIMSS 2003 context. Swedish eighth-grade students formed the study sample and the focus was on mathematics. Test-taking motivation was measured using questionnaire items and interviews, and reported level of test-taking motivation was regressed on test score. The questionnaire study showed that the Swedish students in general reported that they were well motivated to do their best in TIMSS. According to regression analysis, test-taking motivation was positively and significantly though rather weakly related to mathematics achievement. The interview study mainly corroborated the main results from the questionnaire study but added some complexity to the results. Although most interviewed students reported that they were well motivated to do their best in TIMSS and that they valued a good performance, they nevertheless seemed aware of the fact that the test was low-stakes for them personally. Many students further reported competitive, comparative, or social responsibility reasons as motivating, while other students seemed more intrinsically motivated to do their best. Findings from quantitative as well as qualitative analyses suggest that the Swedish TIMSS result is unlikely to be negatively affected by a lack of student motivation. However, nothing is known about student test-taking motivation in other countries participating in TIMSS, and further research exploring this issue in an international context is warranted.

The study’s objective was to investigate the test-taking motivation of students in the Swedish TIMSS 2003 context. Swedish Grade 8 students formed the study sample, and the focus was on mathematics. Test-taking motivation was measured using self-report measures (Likert-scale and open-ended questionnaire items), and reported level of test-taking motivation was regressed on test score. The quantitative part of the questionnaire study showed that the Swedish students in general reported that they were well motivated to do their best in TIMSS. According to regression analysis, test-taking motivation was positively and significantly, although rather weakly, related to mathematics achievement. Qualitative analysis of an open-ended questionnaire item mainly corroborated obtained findings but added some complexity to the results. In their answers to the open-ended item, most students reported they were well motivated to do their best in TIMSS and that they valued a good performance. Many students reported competitive, comparative, or social-responsibility reasons as motivating, while other students seemed more intrinsically motivated to do their best. Findings from quantitative as well as qualitative analyses suggest that the Swedish mathematics result in TIMSS 2003 is unlikely to be negatively affected by a lack of student motivation. However, nothing is known about student test-taking motivation in other countries participating in TIMSS, and further research exploring this issue in an international context is warranted.

Det är 63 lärare som besvarat enkäten och huvuddelen av dem, 66 %, är lärare vars elever tillhör komvux. Den näst största gruppen, 16 %, är lärare vars elever tillhör samhällsprogrammet. Om man ser till hela läsår så är det normalt sett en mindre andel av eleverna som gör höstterminsprovet än som gör vårterminsprovet. På vårterminsprovet brukar den andel av eleverna som tillhör samhällsprogrammet vara störst. Det är 67 % av lärarna som tycker att poänggränsen för betyget Godkänd är lämplig och de övriga, 33 %, tycker att den är för låg. När det gäller totalpoänggränsen samt minsta antal vg-poäng för provbetyget Väl godkänd tycker 81 % respektive 78 % att gränserna är lämpliga. 91 % tycker att det är bra eller mycket bra att bedömningen sker med både g- och vg-poäng. Det är 76 % som anser att provet till ganska eller mycket stor del kan användas som ett viktigt underlag för att sätta kursbetyget Mycket väl godkänd. 81 % tycker till ganska eller mycket stor del att informationen och anvisningarna om hur kursprovet skall användas som underlag för bedömning med betyget Mycket väl godkänd var tillräcklig. De flesta, 89 %, anser att det är bra eller mycket bra med en miniräknarfri del och 89 % tycker att en ganska eller mycket stor andel av uppgifterna på denna del var av en typ som borde ingå där. En stor andel av lärarna, 92 %, anser att kursprovet i ganska eller mycket stor utsträckning speglar ämnessynen i kursplanen och 73 % anser att kursprovet påverkar deras tolkning av kursplanens mål och kriterier i ganska eller mycket stor utsträckning. När det gäller i vilken mån som kursprovet påverkar det innehåll, i betydelsen delmål och delfärdigheter, som tas upp i undervisningen anser 64 % av lärarna att det sker i ganska eller mycket stor utsträckning och 73 % av lärarna anser att kursprovet i ganska eller mycket stor utsträckning påverkar det slutbetyg som de ger enskilda elever. På samma sätt som för provet som gavs våren 2003 är det för höstens prov en ganska stor andel lärare som tror att mängden text i provet påverkade deras elevers resultat negativt. 46 % av lärarna anser att det påverkade eleverna negativt i ganska eller mycket stor utsträckning. Av enkäten framgår att många elever, 76 %, i ganska liten eller mycket liten utsträckning fått ta del av innehållet i den generella bedömningsmatris som ligger bakom aspektbedömningen av provets sista uppgift. Det har varit ungefär på samma vis de senaste åren. Bland de positiva allmänna kommentarerna kan nämnas att flera lärare tycker att det är ett bra prov och att provet ger ett bra underlag för att se var G/VG- och VG/MVG-gränserna ligger. Det finns också några negativa omdömen som att det är för mycket text i uppgifterna och att språket borde vara något enklare och då särskilt med tanke på elever som inte går på teoretiska program. Några kommentarer antyder att det inte finns ett tillräckligt fylligt underlag för betygssteget IG/G, vilket skulle kunna tolkas som att de finns lite för få uppgifter för de elever som ligger nära gränsen för betyget Godkänd.

Av de 329 lärare som besvarat enkäten för Matematik B har 35 % angett att deras elever tillhör samhällsprogrammet och 11 % att deras elever tillhör estetiska programmet. För dessa program är kursprovet i Matematik B obligatoriskt. Av de svarande lärarna tycker 72 % att poänggränsen för betyget Godkänd är lämplig och bland de övriga är det något fler, 18 %, som tycker att den är för låg. När det gäller totalpoänggränsen samt minsta antal vg-poäng för provbetyget Väl godkänd tycker 77 % respektive 82 % att gränserna är lämpliga. 90 % av lärarna tycker att det är bra eller mycket bra att bedömningen sker med både g- och vg-poäng och 75 % anser att provet till ganska eller mycket stor del kan användas som ett viktigt underlag för att sätta kursbetyget Mycket väl godkänd. 80 % tycker till ganska eller mycket stor del att informationen och anvisningarna om hur kursprovet skall användas som underlag för bedömning med betyget Mycket väl godkänd var tillräcklig. Några enkätfrågor behandlar uppgifternas språkliga utformning till exempelvad gäller ordval, meningsbyggnad, textmängd och layout. Av lärarsvaren att döma är ordval och meningsbyggnad mer problematiskt för eleven än textmängd, och layouten upplevs inte som något problem utom i något enstaka fall. Av enkäten framgår att många elever, 59 %, i ganska liten eller mycket liten utsträckning fått ta del av innebörden i de tre aspekter som ligger bakom bedömningen av provets sista uppgift. Det är en minskning jämfört med tidigare år. Bland de positiva allmänna kommentarerna kan nämnas att flera lärare tycker att det är ett mycket bra prov och att provet ger ett bra underlag för att se var G/VG- och VG/MVG-gränserna ligger. Det finns också några negativa omdömen som att provet är för teoretiskt och då särskilt med tanke på elever som går på samhälls- och estetprogrammet. Några kommentarer antyder att det inte finns ett tillräckligt fylligt underlag för betygssteget IG/G, vilket skulle kunna tolkas som att de finns lite för få uppgifter för de elever som ligger nära gränsen för betyget Godkänd.

Over the last few years there have been discussions regarding whether the Swedish system for selection to higher education is suitable for selecting the most appropriate students. The reasons for these discussions are problems with recruitment, student completion and selection. In the light of these problems and within the limits of the alternative selection, the Swedish National Agency for Higher Education started a project in 2003, with the ambition of developing an entrance test for the technology section in higher education. The prototype consisted of three separate tests: a verbal test, a problem-solving test and a knowledge test.

The knowledge test, which was developed at the Department of Educational Measurement at Umeå University, was intended to measure general knowledge in technology. The assumption was that there is a relationship between knowledge and interest, and one related question was whether the result from this test also could be an indirect measure of interest. If there is a positive relationship between interest, motivation and study success, the idea of using such a test for recruitment and selection is supported.

The first purpose of this study was to examine if there is any evidence for a positive relationship between the score on a general knowledge test in technology and study success in education in technology. The second purpose was to find relevant psychological literature on interest and motivation which could be used in the verification of the hypotheses. For this purpose, a literature search in several databases was conducted. The third purpose was, given that there was a positive relationship between knowledge and study success, to describe the possible uses of the test in the Swedish selection system.

The result, concerning the first purpose, indicated evidence for the hypotheses and therefore the conclusion was that there is also a positive relationship between knowledge and study success. Regarding the third purpose, the conclusion was that the knowledge test could work as an instrument for recruitment, selection and prediction, with a few exceptions.

Finally, some limitations of the study were discussed and some suggestions for further research were given.

Resultaten från Trends in International Mathematics and Science Study (TIMSS) 2003 visar på en markant nedgång i resultaten från 1995 för svenska elever i årskurs 8 i både matematik och naturorienterade ämnen (NO). Inom NO har pojkar bättre resultat än flickor medan det i matematik inte finns någon könsskillnad i resultaten. Utifrån detta undersöktes i denna studie uppgiftsformatets och ämnesinnehållets betydelse för prestationsskillnader mellan flickor och pojkar inom NO, i Sverige, och i jämförelse med andra länders resultat. Lösningsproportioner och en IRT-baserad DIF-metod användes för att studera detta. Resultaten är entydiga, flickor presterar relativt sett bättre på uppgifter där de ska skriva ner ett svar och pojkar presterar bättre på uppgifter där det finns färdiga svarsalternativ. Sverige är tillsammans med Moldavien och Bahrain de länder där uppgiftsformatet har störst betydelse för prestationsskillnader

mellan flickor och pojkar. I biologi, som setts som ett ämne till flickornas fördel, är prestationsskillnaden mellan uppgiftsformaten så stor att pojkar presterar lika bra som flickor om endast flervalsuppgifter beaktas. I fysik, som setts som ett ämne till pojkars fördel, presterar flickor lika bra om endast uppgifter där de

ska skriva ner ett svar tas hänsyn till.

Since 1990, the Department of Educational Measurement at Umeå University has been commissioned to study the Swedish drivinglicense test by the Swedish National Road Administration, SNRA. Over the past few years several studies have been conducted in order to develop and improve the Swedish driving-license test. The focus of the majority of the studies has been the theory test.

The aims of this paper were threefold: firstly to describe the development of the driver education and the driving-license test in Sweden during the past century; secondly, to summarize the findings of our research, which is related to important issues in test development; and finally, to make some suggestions for further research.

The students who graduated in the spring of 1997, after three years’ of upper secondary school, was the first cohort who graduated with the new, criterionreferenced, grades. The new grading system is characterised by having defined goals for the grade levels. The purpose of this study is to investigate the relation between the new upper secondary school grades in English, Mathematics and Swedish, and the SweSAT subtests, administrated in the autumn of 1996 and spring of 1997, taken by the students who graduated from a Swedish upper secondary school in 1997. The result shows that there is a positive correlation between grades and SweSAT scores. The subjects English and Mathematics have a stronger correlation with the SweSAT, compared with Swedish. The English B-course have the strongest correlation with the subtests WORD, READ and ERC, and with the overall SweSAT score, while the Mathematics A-course have the strongest relation to the subtests DS and DTM. The same results show for both the SweSAT administered in the autumn of 1996 and the test administered in the spring of 1997. The students are ranked similarly by the SweSAT test scores as by their grades from the courses English A and B, Mathematics A, B, C, D, E and Swedish A and B. In two of the courses included in the study, English C and Swedish C, students with the grade level IG (fail) are ranked higher by their SweSAT scores than students with the grade level G (pass). The students who scored the highest on the SweSAT have also the highest grade level (pass with special distinction) in the courses English B and/or Mathematics E. Students from the Natural Science oriented programme are in majority among these students. The students who score the lowest on the SweSAT have the grade IG in English A and/or Mathematics A. When performing a factor analysis, three factors were identified. One factor was found in the

subtests WORD, READ, ERC and English grades. Another factor was found in the subtests DS, DTM and the Mathematics grades. The third factor was found only in the grades from the courses Swedish A, Swedish language B, and Swedish literature B. The aim would be to modify the SweSAT by introducing an additional dimension, a suggestion is to focus on the factor that is defined by the Swedish courses. However, making such a modification is a considerable step to take, and it is only by research that new and necessary knowledge can be obtained, that will make it possible to change the SweSAT in a longer perspective.

Since 1977 the SweSAT has been used as an instrument for selection in admissions to higher education. Parallel to the development of new versions of tests, extensive research and development work is undertaken to assess and assure the quality of the SweSAT. One aspect of the work with quality assurance is to examine to what extent there are sequential effects in the subtest Data Sufficiency DS.

All newly constructed items are tried out in connection with the regular tests. The day of the test consists of five test periods of 50 minutes each, one of which is the try out items. This means that all test-takers are doing two versions of one of the tests.

Consequently some test-takers will do two subtests of the same kind one after the other – where one is an ordinary test and one is a try-out test. The test-takers do not know which of these will contribute to their official test score and which one consists of new items that are being assessed and that will be used in future SweSATs. The try-out tests are randomly distributed over the 24 universities and colleges.

The aim of this study has been to examine to what extent there are sequential effects in the subtest Data Sufficiency, i.e. if the first DS subtest affects achievement on the second DS subtest when the two tests are distributed with only a short break in between.

The sequential effect has been analysed with the use of a structural equation model and an index that relates the results for different groups of test-takers to the result of all test-takers.

The results show a clear tendency – if two DS subtests are given in succession, with a short break of around 30-minutes in between tests, the average result of the second test will be higher than when the tests are not administered in succession. On average there is a difference of approximately one point out of a total of twenty-two. The index method shows that the test-takers at all the ten examined universities and colleges performed better on the second DS-subtest when the tests were administered in succession.

In this study the equal ability distribution assumption associated with therandomly-equivalent groups equating design was investigated in the contextof a selection test for admission to higher education. Test-takers’ scores onanchor items from two subtests were estimated using information abouttest-taker performance on the regular subtests. The results showed that theestimated anchor test scores varied sufficiently so that the equal ability distributionassumption could be questioned. Consequently, we call for morecaution when applying the randomly-equivalent groups design in the equatingof tests. Equal ability groups is a convenient assumption to make but itcan lead to systematic bias in the equating of test scores and this study providesa demonstration of that point.

College admission tests are of great importance for admissions systems in general and for candidates in particular. The SweSAT (Högskoleprovet in Swedish) has been used for college admission in Sweden for more than 30 years, and today it is alongside with the upper-secondary school GPA the most widely used instrument for selection of college applicants. Because of the importance that is placed on the SweSAT, it is essential that the scores are reliable and that the interpretations and uses of the scores are valid. The main purpose of this thesis was therefore to examine some assumptions that are of importance for the validity of the interpretation and use of SweSAT scores. The argument-based approach to validation was used as the framework for the evaluation of these assumptions.The thesis consists of four papers and an extensive introduction with summaries of the papers. The first three papers examine assumptions that are relevant for the use of SweSAT scores for admission decisions, while the fourth paper examines an assumption that is relevant for the use of SweSAT scores for providing diagnostic information. The first paper is a review of predictive validity studies that have been performed on the SweSAT. The general conclusion from the review is that the predictive validity of SweSAT scores varies greatly among study programs, and that there are many problematic issues related to the methodology of the predictive validity studies. The second paper focuses on an assumption underlying the current SweSAT equating design, namely that the groups taking different forms of the test have equal abilities. The results show that this assumption is highly problematic, and consequently a more appropriate equating design should be applied when equating SweSAT scores. The third paper examines the effect of textual item revisions on item statistics and preequating outcomes, using data from the SweSAT data sufficiency subtest. The results show that most kinds of revisions have a significant effect on both p-values and point-biserial correlations, and as a consequence the preequating outcomes are affected negatively. The fourth paper examines whether there is added value in reporting subtest scores rather than just the total score to the test-takers. Using a method derived from classical test theory, the results show that all observed subscores are better predictors of the true subscores than is the observed total score, with the exception of the Swedish reading comprehension subtest. That is, the subscores contain information that the test-takers can use for remedial studies and hence there is added value in reporting the subscores. The general conclusion from the thesis as a whole is that the interpretations and use of SweSAT scores are based on several questionable assumptions, but also that the interpretations and uses are supported by a great deal of validity evidence.

This article reviews ten predictive validity studies of the Swedish Scholastic Assessment Test (SweSAT). A primary result is that the predictive validity of the SweSAT seems to be highly dependent upon the study programme being examined; that is, the predictive validity is better at some programmes than others. When compared with the upper-secondary school grade point average, the predictive validity of the SweSAT seems to be fairly good, but there are major differences between study programmes in this case as well. However, it is suggested that the validity of the results is to some extent threatened by methodological issues. A general conclusion is, therefore, that there is room for improving the test itself, as well as the way that predictive validity studies are carried out.

The added value of reporting subscores on a college admission test (SweSAT) was examined in this study. Using a CTT-derived objective method for determining the value of reporting subscores, it was concluded that there is added value in reporting section scores (Verbal/Quantitative) as well as subtestscores. These results differ from a study of the SAT I and a study of a basic skills test and thus highlight the need for practitioners and researchers to gather empirical evidence to support the reporting ofsubscores. The cause of the disparate results seems to be related to differences in the composition ofthe tests rather than differences in the composition of the examinee groups.

The purpose of this study was to examine the effect of textual item revisions on classical item statistics and the adequacy of preequating outcomes. Three forms of the SweSAT data sufficiency subtest, comprising a total of 66 items, were examined. The items were categorized after type of revision, and the differences in p-values and point-biserial correlations between regular test and pretest were averaged in each category. These averages were subjected to a t-test, and it was found that revisions have a significant effect on both difficulty and discrimination indices. Also, while the preequating method used in this study produced adequate results, the revisions seem to increase the amount of error in preequating outcomes.

The equal ability distribution assumption associated with the randomly-equivalent groups equating design was investigated in the context of a selection test for admission to higher education. Test-takers’ scores on anchor items from two subtests were estimated using information about test-taker performance on the regular subtests. The results indicated that the anchor test item performance varied sufficiently so that the equal ability distribution assumption could be questioned. Consequently, our conclusion is that more caution when applying the randomly-equivalent groups design in the equating of tests is needed. Assuming equal ability groups is a convenient assumption to make but it also can lead to systematic bias in the equating of test scores and this study provides a demonstration of that point.

This study is about the introduction of an international grading scheme from the perspective of the educational providers, with the aim of creating a theory that accounts for their problems and actions. Interview data was collected from two institutions in a country participating in the Bologna Process. The results show that the educational providers are faced with a number of concerns and that they address these concerns by adaptation and adjustment. These two core processes are complemented by five additional processes: information seeking, trusting, safeguarding, problem-solving, and evaluation.

This document is the conference proceedings from the multidisciplinary research conference ECTS and Assessment in Higher Education, Umeå University, Sweden, June 7-9, 2006. ECTS is an abbreviation for the European Credit Transfer System. The conference had 130 participants, from 28 countries, and 16 poster- and paper-presenters. The invited keynote speakers were: Marianne Hildebrand, Terence Karran, Margaret Price and Ulrich Teichler. The first part contains the acknowledgements, the programme and the keynote speakers’ biographies together with a description of the context for the conference and the key concepts that were discussed during the conference. The second part contains a selection of the posters and papers that were presented during the conference. The conference was organised by the Department of Educational Measurement, Umeå University, Sweden.

The two laws Equal Treatment of Students at Universities Act (Lagen likabehandling av studenter i högskolan, SFS 2001:1286) and Prohibition of Discrimination Act (Lagen om förbud mot diskriminering, SFS, 2003:307) stipulate that university departments must address equality and gender issues when planning, executing and evaluating their courses. This report discusses these laws, the European Union’s (EU) Seventh Framework Programme (European Commission, 2006), the EU’s roadmap for equality (European Commission, 2006) and two recent Governmental Bills (Prop. 2004/05:80 & Prop. 2005/06:155). Results from previous empirical studies are also reported. Finally, the report presents a short and easily applied model for departments to use when they begin to analyse their own courses from an equal opportunities perspective.

The results show that the educational providers have six major concerns. These concerns are about Compliance, Integration, Optimization, Acceptance, Learning and Fairness. The educational providers address these concerns by two core processes which I call Adaptation and Adjustment. The results also show that the educational providers use five additional processes to address these concerns. These additional processes are Information seeking, Trusting, Safeguarding, Problem solving and Evaluation.

This study is about the registration of study achievement (courses, credit points, degrees etc). It analyses the computer based student documentation systems for Swedish colleges and the statutory instruments (Swedish laws and ordinances) that governs these documentations. The National Agency for Higher Education is the central agency responsible for the official statistics about students’ achievements (www.hsv.se). The practical responsibility for coordinating and collecting the data rests with the Statistics Sweden (www.scb.se), a government authority for official statistics. The study shows that the colleges (i.e. at the local level) record study grades (e.g. Fail, Pass or Pass with distinction) but this information is not collected by the Statistics Sweden (on the national level). Attention is also given to the use different types of measurements of study achievement in Swedish research. The study ends with a discussion about future research. One future task is to investigate new ways to use available statistics students’ achievements. A second task is to develop and use new types of measurements of study achievement. Finally, a third task is to increase the accessibility of the official statistics on students’ achievements.

This study was about students and their study achievements. Data from available records for students admitted to the study programme of business administration was used. The students (N19393) were born in the period 1972–1984, living in Sweden and registered on the study programme sometime between 1993 and 2000. The students were divided into three groups on the basis of the grounds on which the students had been admitted. The first group had been admitted on the basis of their credits from uppersecondary school, the second on the basis of their SweSAT-scores and the third group on the basis of their SweSAT-scores with additional credit for work experience. Study achievement was defined as the number of credits the students managed to attain each semester. The purpose of the study was to see if there were any differences in academic achievement between the three groups. Attention was also given to social background and sex. The results showed that approximately 50% of the students were women. The proportion of students from Social Group 1 was highest among those who had been admitted on the basis of their SweSAT-scores. The most common upper-secondary school programme, for the students who had completed upper-secondary school 1997 or later, was the Programme for Social Science (Samhällsvetenskaplig utbildning). About one fifth of the students who had completed upper secondary school 1997-1999 had taken supplementary secondary-education-level courses after they had left upper-secondary school. The study showed that the female students attained more course points per semester than the male students. There were also differences, in terms of the number of credits attained per term, between the students from the three groups and between students with different social backgrounds. However, these differences were not as extensive as the differences between male and female study achievements.