Replication Data for PlosOne 2020 manuscript Tempelaar ea

DOI

This archive contains data files and computer code belonging to: Subjective data, objective data and the role of bias in predictive modelling: lessons from a dispositional learning analytics application. Abstract: For decades, self-report measures based on questionnaires have been widely used in educational research to study implicit and complex constructs such as motivation, emotion, cognitive and metacognitive learning strategies. However, the existence of potential biases in such self-report instruments might cast doubts on the validity of the measured constructs. The emergence of trace data from digital learning environments has sparked a controversial debate on how we measure learning. On the one hand, trace data might be perceived as “objective” measures that are independent of any biases. On the other hand, there is mixed evidence of how trace data are compatible with existing learning constructs, which have traditionally been measured with self-reports. This study investigates the strengths and weaknesses of different types of data when designing predictive models of academic performance based on computer-generated trace data and survey data. We investigate two types of bias in self-report surveys: response styles (i.e., a tendency to use the rating scale in a certain systematic way that is unrelated to the content of the items) and overconfidence (i.e., the differences in predicted performance based on surveys’ responses and a prior knowledge test).We found that the response style bias accounts for a modest to a substantial amount of variation in the outcomes of the several self-report instruments, as well as in the course performance data. It is only the trace data, notably that of process type, that stand out in being independent of these response style patterns. The effect of overconfidence bias is limited. Given that empirical models in education typically aim to explain the outcomes of learning processes or the relationships between antecedents of these learning outcomes, our analyses suggest that the bias present in surveys adds predictive power in the explanation of performance data and other questionnaire data.

Identifier
DOI https://doi.org/10.34894/L1UMWV
Related Identifier https://doi.org/10.1371/journal.pone.0233977
Metadata Access https://dataverse.nl/oai?verb=GetRecord&metadataPrefix=oai_datacite&identifier=doi:10.34894/L1UMWV
Provenance
Creator Tempelaar, D ORCID logo
Publisher DataverseNL
Contributor Tempelaar, D; Stapleton, G; Dirk Tempelaar
Publication Year 2020
Rights CC0 Waiver; info:eu-repo/semantics/openAccess; https://creativecommons.org/publicdomain/zero/1.0/
OpenAccess true
Contact Tempelaar, D (Maastricht University); Stapleton, G (Maastricht University)
Representation
Resource Type Dataset
Format text/plain; text/csv
Size 761; 10999456; 1077126; 25184
Version 1.0
Discipline Agriculture, Forestry, Horticulture, Aquaculture; Agriculture, Forestry, Horticulture, Aquaculture and Veterinary Medicine; Life Sciences; Social Sciences; Social and Behavioural Sciences; Soil Sciences
Spatial Coverage Maastricht