The aim of this presentation is to give anoverview of the Swedish examination system how the national science tests havedeveloped over time. The aim is also to introduce the model of thetransformation process from paper and pencil approach to a digital approach.The presenters are looking forward discussing how to build and ensure thequality of different items e.g. animations, film clips andsimulations. The presentation will all consider the alignment of the new testitems with the curriculum aims.

In Sweden the national tests are designed as a paper-based assessment, but some test parts give students the opportunity to present their proficiency orally and practically. In Sweden different universities are responsible for developing the national tests, commissioned by the National Agency of Education. In 2016 some universities were assigned to start the development of item examples for digital national tests. The idea is to gradually introduce digital national tests between 2018 and 2022. The tests are intended to measure student proficiency in relation to the Swedish curriculum and work as a support for consistent national assessment and grading.

A project group at Umeå University, which is responsible for the national science tests, has started work on a test model for a digital national test in science. The test model is supposed to be aligned with the curriculum and fulfil its national aim. The project group also wishes to fully challenge item types in the digital sphere, where e.g. animations, film clips such as courses of events and simulations are available.

The project group has in their process been inspired by the released digital item examples from PISA (http://www.oecd.org/pisa/test/other-languages/) and SimScientists (http://simscientists.org/home/index.php). Inspiration also comes from ATC21S (Griffin, Patrick, McGaw, Barry, Care, Esther (Eds.) 2012 and (http://www.atc21s.org/). The project group now wishes to be part of a larger network for further development work.

Questions like what do different countries’ test systems look like, how do they build and ensure the quality of different item examples in a digital sphere, what do innovative digital item examples look like and how are they developed, need to be answered.

In the first session of the workshop the presenters will give an overview of the Swedish test system and will present what some concrete examples of innovative paper-based items in science look like in the tests.

In the second session the participants will have an opportunity to present some concrete examples of innovative items and share their knowledge, experiences and issues concerning the development and design of different types of innovative items in a paper-based and/or a computer-based test system. In this session the participants also will get the opportunity to answer the items. This opportunity will bring space for wider reflections around the developing potentiality.

The third session will focus on collaboration and development of ideas. During this session the participants will work in small groups with practical tasks. The purpose is to develop concrete suggestions of ideas for some selected items, and see how the digital format can be used to given the purpose of the items e.g. start formulating items based on ideas presented in earlier sessions and develop items from new ideas. 

In the fourth session each group will will be given an opportunity to provide feedback concerning the items discussed in session 3 in terms of opportunities, constraints, challenges and threats.

Finally we will summarize the sessions by discussing what to bear in mind when developing different types of innovative computer-based items, how to move forward and how to create a future international network.

In order to produce sound and valid tests with respect to the Standards (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014) there are a lot of issues to consider and take into account. Especially when developing tests on a national level. In addition, there are often several stakeholders having an ambition to influence the tests in different directions. Sometimes these stakeholders agree, but often their requests are diametrically opposed and it is not unusual that the requests are not in line with a good measurement practice. In the midst are the test developing organisations, commissioned to develop products that are valid in relation to the aim/aims for the test and therefore being developed within a sound measurement practice but also accepted by all users.

As described in the theme for the conference this external influence from stakeholders on the tests is immense and sometimes described as only negative. Often there are politicians who uses educational assessments, like national tests or exams, to control the school system on the one hand but on the other hand using the tests to implement changes. At the same time the politicians are sensitive to reactions from the teachers, parents and other stakeholders since they are important groups of voters. In Sweden the debate is at the moment focused on the large number of national tests and the workload they entails to teachers but also students. In a recently published government-appointed inquiry (SOU 2016:25, 2016) it is suggested that the number of national tests should be reduced, that the remaining tests should be less extensive and that the tests should be easier to administer and mark, which probably will affect the validity of the tests.

These external influences could, from a test developing perspective, be seen as problematic since it often introduces (rapid) changes of the tests. On the other hand, one could argue that these external influences are necessary prerequisites to have an ongoing process in order to develop the tests so that they become even more cost effective, valid and seen as valuable for the users.

We think it would be interesting to discuss this complex system of, on the one hand, social and political agents trying to influence and change the national assessment systems and, on the other hand, the test developing organisations aiming to develop assessments that are valid. But at the same time these organisations are dependent of getting resources from the agents to fulfil the commission, which might affect which changes that are implemented and not.

This is a proposal for a discussion group based on the broad question posed as title. Below we have specified some themes that would be interesting to discuss getting perspectives from different countries and testing systems.

• How are the products, i.e. the tests, and the processes developing the tests affected by the influences from different social and political agents?
• Are there stakeholders having greater impact, and if there are, is it a necessity or a risk? Why How?
• Finally, is it maybe necessary to have this continuous external validation of the tests in order to develop, strengthen and legitimise them or does it “ruin the work”?

Sweden has since 2009 had national tests for school year nine in the science (i.e. biology, physics and chemistry). These tests aim to assess the three competencies described in the curricula namely, review information, communicate and take a stand in questions related to the subject, conduct systematic investigations and use concepts theories and models to explain connections. Beginning from the end, the third competence is assessed trough a constructed response test, a test that can be seen as rather traditional science test. The second competence is assessed through a practical investigation that the students plan, conduct and evaluate. The first competence is assessed by a more complex item where the students are served some information which they are supposed to review and use in their argumentation when they take a stand. These tests include a lot of the competences aimed at when discussing assessment for the 21st century world; creativity, critical thinking, problem solving, communication, and decision-making.

The assessment of laboratory work has been an obligatory part of the national tests from the beginning in 2009. Despite all problems with the practical handling in the school there has been an important part of the aim that the national tests should be exemplary. If the tests do not include a practical test why should the schools work with laboratory work in class? There was also statistics indicating that many of the schools did not work practical at all.

When the new curriculum was introduced 2011 the competence to communicate was highlighted. Beside the more general definitions of communication several of the syllabi included a subject specific communication component. In the first version of the tests developed in relation to the new curriculum there has been a part assessing the students’ ability to communicate. In these items the students are supposed to scrutinize and analyse given information, communicate and make a decision. The items are connected to questions concerning energy, environment, resource use and health. One of the challenges is to really assess science communication and not communication in general.

In this presentation we will discuss the rationales for this test model and show how the assessment looks like.

Föreläsningen presenterar resultat, analyser och slutsatser från de fyra genomförda ämnesproven, 2009-2012, enligt Lpo94. Utifrån erfarenheter från ämnesproven 2009-2012 presenteras därefter en inblick i de nya ämnesproven enligt Lgr11, så som provens utformning, följt av en diskussion om de utmaningar, möjligheter och problem som proven medför.