Change search
ReferencesLink to record
Permanent link

Direct link
Agreement in assessment of prior learning related to higher education: an examination of interrater and intrarater reliability
Umeå University, Faculty of Social Sciences, Department of Psychology.
2013 (English)In: International Journal of Lifelong Education, ISSN 0260-1370, E-ISSN 1464-519X, Vol. 32, no 4, 535-547 p.Article in journal (Refereed) Published
Abstract [en]

This paper focuses on reliability in assessment of prior learning (APL) related to higher education. APL refers to a process where adults’ prior learning, formal as well as informal, is assessed and acknowledged. In higher education APL is used for the purpose of gaining admission, advanced standing or credits. In this paper studies investigating reliability in APL are presented. The studies provide data of inter- and intrarater reliability among assessors in a specific APL scheme used in higher education. Percentage of agreement and Cohen’s kappa were used to estimate reliability, and the results show that the agreement between assessors, i.e. interrater reliability, is poorer compared to the agreement on repeated assessment by the same assessor, i.e. intrarater reliability. Further, the results reveal that there are differences in agreement between different aspects of prior learning. A conclusion is that reliability in this type of assessment ought to be further investigated.

Place, publisher, year, edition, pages
2013. Vol. 32, no 4, 535-547 p.
Keyword [en]
Assessment, Access to HE, Workplace learning, Validity, Reliability
National Category
Social Sciences
Research subject
didactics of educational measurement
URN: urn:nbn:se:umu:diva-43458DOI: 10.1080/02601370.2013.778077OAI: diva2:414024
Available from: 2011-05-02 Created: 2011-05-02 Last updated: 2016-09-26Bibliographically approved
In thesis
1. As valid as it can be?: The assessment of prior learning in higher education
Open this publication in new window or tab >>As valid as it can be?: The assessment of prior learning in higher education
2011 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Assessment of prior learning (APL) represents the task to identify and acknowledge an individual’s knowledge and skills regardless of how it has been obtained. In higher education this type of assessment is primarily used for the purpose of awarding access, credits or advanced standing. Because of the impact the results from APL have on the future working career for individuals claiming APL, it is of great importance that these result of APL is valid. The question of interest in this thesis is to what extent APL in higher education is a valid assessment.

The thesis is written in the field of educational measurement and comprises four papers and an extensive introduction with summaries of the papers. The most recent views of validity theory were used as the general theoretical framework in all papers, and all papers are concerned with APL in higher education. Study I reviews the research area of APL in higher education from a validity perspective. The general conclusion from the review is that the majority of the studies conducted in this area primarily provide theoretical rationales and theories for a variety of APL practices, and that there is a need for empirically based studies examining and evaluating validity of APL. Studies II, III and IV are empirical studies based on, and exemplified with, an APL scheme related to higher education in Sweden. Study II examines validity issues identified from claimants (individuals or students claiming APL) view of APL. The claimants’ experiences from the specific APL scheme were examined using a questionnaire developed for that purpose. Conclusions drawn from the results are that possible threats to validity may exist in the administration of APL procedures, as well as in consequences of APL. Study III focuses on validity of admission decisions based on APL. The study examines decisions made by different higher education institutions for approximately 600 individuals applying for higher education based on their prior learning. The results show that the existing practice of APL needs improvements in order to obtain validity and trustworthiness in the decisions made in relation to APL. Finally, Study IV focuses on reliability in APL related to higher education. The study provides data of inter- and intra-rater reliability among judges in the specific APL scheme. The results show a lack of especially inter-rater reliability, and a conclusion is that reliability in this type of assessment should be further investigated.

The general conclusion from this thesis is that there is a need to take validity issues in APL seriously, and that APL in higher education may not be as valid as it could be.

Place, publisher, year, edition, pages
Umeå: Institutionen för tillämpad utbildningsvetenskap, Umeå Universitet, 2011. 50 p.
Academic dissertations at the department of Educational Measurement, ISSN 1652-9650 ; 6
Access to Higher Education, Recognition, Accreditation, Informal learning, Validity, Reliability, Validation
Research subject
didactics of educational measurement
urn:nbn:se:umu:diva-43461 (URN)978-91-7459-209-2 (ISBN)
Public defence
2011-06-01, Norra Beteendevetarhuset, Hörsal Hs1031, Umeå Universitet, Umeå, 10:00 (English)
Available from: 2011-05-11 Created: 2011-05-02 Last updated: 2011-05-11Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full text

Search in DiVA

By author/editor
Stenlund, Tova
By organisation
Department of Psychology
In the same journal
International Journal of Lifelong Education
Social Sciences

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 114 hits
ReferencesLink to record
Permanent link

Direct link