Umeå University's logo

Since the national science tests for grade 9 (age 15-16) in Swedish Compulsory schools were obligated 2010, there has been validity and reliability discussions concerning the dilemma of having essay items in the tests. In the test reports from 2013 onwards, teachers express concerns to assess the essay items, arguing that students only show that they are “good coders” and general arguers instead of showing scientific knowledge and skills. The goals of the science curriculum do however state that science knowledge should include the ability to participate in discussions in ways where science, technology and society interrelate, e.g. decision-making in socioscientific issues (SSI). Therefore, these skills are also important to assess. With this study, we aim to increase the knowledge about what perspectives students use in their supporting reasoning in Swedish national science tests. This was done using the SEE-SEP model covering three aspects (knowledge, value, and personal experiences) and six subject areas (sociology/culture, economy, environment/ecology, science, ethics/morality, and policy) (Chang Rundgren, & Rundgren, 2010), and a test item where students choose and argue for their choice of material for a chocolate box. The choices are plastic, paper or aluminium. Results show that students can use multiple perspectives of scientific knowledge when they make decisions and SSI argumentation, and this can be tracked in their written answers. The core of students reasoning was based on knowledge and values in the subject areas sociology/culture, environment and science. We discuss possibilities to, instead of avoiding these essay items, learn more about how to assess them.

Education plays an important role in preparing students with the knowledge and skills that will help them deal with science-related issues. Both national and international studies have analysed the purposes of science education, showing that there are various messages in terms of visions and curriculum emphases.This study aims to elucidate the guidance the Swedish science syllabuses and the year 2017 national science tests in lower secondary school give teachers to choose focus of content, context and teaching methods. By analysing sentences in the syllabuses and items and assessment guidelines in the tests through the lens of Biesta’s functions of education and Roberts curriculum emphases, a detailed picture of what guidance the syllabuses and the tests give is presented. The study shows that, according to the syllabuses and the tests, the overall function of science education is to provide students with knowledge and skills, while sentences emphasising certain norms or independent critical thinking are uncommon. Further, differences were found in the extent to which the different specific aims of science education (operationalised as curriculum emphases) are stressed. The study discusses similarities and differences in syllabuses, between subjects and in relation to the items and the assessment guidelines in the tests. 

Education has an important role in preparing students personally, socially and intellectually, for future challenges as citizens. But what is the role of science education in preparing students for the future? During the last decades, conflicting ideas about the balance between developing science knowledge needed as citizens and knowledge for future studies in science and technology (Fensham 1988; Roberts, 2007), have led to several revisions of the science curriculum in many countries. The aim of this exploratory study is to illustrate and problematize how the functions and purposes of education are expressed in the curricula and in the different science syllabuses in compulsory and upper secondary school. First, a content analysis of the syllabuses was done with respect to functions of education, i.e., qualification, socialization and subjectification (Biesta, 2009), and with respect to curriculum emphases targeting different aims of science education (Roberts, 2007; Roberts & Östman, 1998; Östman, 1995). Each sentence in the documents were assigned categories in respective framework. This enabled a quantitative comparison of functions and purposes expressed in different documents and different parts of the documents (e.g. aims, core content, and knowledge requirements). Second, a qualitative analysis of each sentence and clusters of sentences in respective category was done. Preliminary results show that different science subjects, at the different school levels, have different accentuations of functions and purposes. Furthermore, the sentences regarding knowledge requirements for school levels 7-9 are more difficult to categorize than other parts in the syllabus. Most text concerning socialization and subjectification, i.e., how to develop as an independent citizen able to make decisions is presented in the curriculum and is scarce/rare in the syllabus. We are looking forward discussing the different aims of science education from the results of this study.

The aim of this presentation is to give anoverview of the Swedish examination system how the national science tests havedeveloped over time. The aim is also to introduce the model of thetransformation process from paper and pencil approach to a digital approach.The presenters are looking forward discussing how to build and ensure thequality of different items e.g. animations, film clips andsimulations. The presentation will all consider the alignment of the new testitems with the curriculum aims.

In Sweden the national tests are designed as a paper-based assessment, but some test parts give students the opportunity to present their proficiency orally and practically. In Sweden different universities are responsible for developing the national tests, commissioned by the National Agency of Education. In 2016 some universities were assigned to start the development of item examples for digital national tests. The idea is to gradually introduce digital national tests between 2018 and 2022. The tests are intended to measure student proficiency in relation to the Swedish curriculum and work as a support for consistent national assessment and grading.

A project group at Umeå University, which is responsible for the national science tests, has started work on a test model for a digital national test in science. The test model is supposed to be aligned with the curriculum and fulfil its national aim. The project group also wishes to fully challenge item types in the digital sphere, where e.g. animations, film clips such as courses of events and simulations are available.

The project group has in their process been inspired by the released digital item examples from PISA (http://www.oecd.org/pisa/test/other-languages/) and SimScientists (http://simscientists.org/home/index.php). Inspiration also comes from ATC21S (Griffin, Patrick, McGaw, Barry, Care, Esther (Eds.) 2012 and (http://www.atc21s.org/). The project group now wishes to be part of a larger network for further development work.

Questions like what do different countries’ test systems look like, how do they build and ensure the quality of different item examples in a digital sphere, what do innovative digital item examples look like and how are they developed, need to be answered.

In the first session of the workshop the presenters will give an overview of the Swedish test system and will present what some concrete examples of innovative paper-based items in science look like in the tests.

In the second session the participants will have an opportunity to present some concrete examples of innovative items and share their knowledge, experiences and issues concerning the development and design of different types of innovative items in a paper-based and/or a computer-based test system. In this session the participants also will get the opportunity to answer the items. This opportunity will bring space for wider reflections around the developing potentiality.

The third session will focus on collaboration and development of ideas. During this session the participants will work in small groups with practical tasks. The purpose is to develop concrete suggestions of ideas for some selected items, and see how the digital format can be used to given the purpose of the items e.g. start formulating items based on ideas presented in earlier sessions and develop items from new ideas. 

In the fourth session each group will will be given an opportunity to provide feedback concerning the items discussed in session 3 in terms of opportunities, constraints, challenges and threats.

Finally we will summarize the sessions by discussing what to bear in mind when developing different types of innovative computer-based items, how to move forward and how to create a future international network.

In order to produce sound and valid tests with respect to the Standards (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014) there are a lot of issues to consider and take into account. Especially when developing tests on a national level. In addition, there are often several stakeholders having an ambition to influence the tests in different directions. Sometimes these stakeholders agree, but often their requests are diametrically opposed and it is not unusual that the requests are not in line with a good measurement practice. In the midst are the test developing organisations, commissioned to develop products that are valid in relation to the aim/aims for the test and therefore being developed within a sound measurement practice but also accepted by all users.

As described in the theme for the conference this external influence from stakeholders on the tests is immense and sometimes described as only negative. Often there are politicians who uses educational assessments, like national tests or exams, to control the school system on the one hand but on the other hand using the tests to implement changes. At the same time the politicians are sensitive to reactions from the teachers, parents and other stakeholders since they are important groups of voters. In Sweden the debate is at the moment focused on the large number of national tests and the workload they entails to teachers but also students. In a recently published government-appointed inquiry (SOU 2016:25, 2016) it is suggested that the number of national tests should be reduced, that the remaining tests should be less extensive and that the tests should be easier to administer and mark, which probably will affect the validity of the tests.

These external influences could, from a test developing perspective, be seen as problematic since it often introduces (rapid) changes of the tests. On the other hand, one could argue that these external influences are necessary prerequisites to have an ongoing process in order to develop the tests so that they become even more cost effective, valid and seen as valuable for the users.

We think it would be interesting to discuss this complex system of, on the one hand, social and political agents trying to influence and change the national assessment systems and, on the other hand, the test developing organisations aiming to develop assessments that are valid. But at the same time these organisations are dependent of getting resources from the agents to fulfil the commission, which might affect which changes that are implemented and not.

This is a proposal for a discussion group based on the broad question posed as title. Below we have specified some themes that would be interesting to discuss getting perspectives from different countries and testing systems.

• How are the products, i.e. the tests, and the processes developing the tests affected by the influences from different social and political agents?
• Are there stakeholders having greater impact, and if there are, is it a necessity or a risk? Why How?
• Finally, is it maybe necessary to have this continuous external validation of the tests in order to develop, strengthen and legitimise them or does it “ruin the work”?

Sweden has since 2009 had national tests for school year nine in the science (i.e. biology, physics and chemistry). These tests aim to assess the three competencies described in the curricula namely, review information, communicate and take a stand in questions related to the subject, conduct systematic investigations and use concepts theories and models to explain connections. Beginning from the end, the third competence is assessed trough a constructed response test, a test that can be seen as rather traditional science test. The second competence is assessed through a practical investigation that the students plan, conduct and evaluate. The first competence is assessed by a more complex item where the students are served some information which they are supposed to review and use in their argumentation when they take a stand. These tests include a lot of the competences aimed at when discussing assessment for the 21st century world; creativity, critical thinking, problem solving, communication, and decision-making.

The assessment of laboratory work has been an obligatory part of the national tests from the beginning in 2009. Despite all problems with the practical handling in the school there has been an important part of the aim that the national tests should be exemplary. If the tests do not include a practical test why should the schools work with laboratory work in class? There was also statistics indicating that many of the schools did not work practical at all.

When the new curriculum was introduced 2011 the competence to communicate was highlighted. Beside the more general definitions of communication several of the syllabi included a subject specific communication component. In the first version of the tests developed in relation to the new curriculum there has been a part assessing the students’ ability to communicate. In these items the students are supposed to scrutinize and analyse given information, communicate and make a decision. The items are connected to questions concerning energy, environment, resource use and health. One of the challenges is to really assess science communication and not communication in general.

In this presentation we will discuss the rationales for this test model and show how the assessment looks like.