From Reporting Guideline to Reporting Quality Instrument: Developing RAISE-QI for Artificial Intelligence Research in Education

Allison, Jordan ORCID: https://orcid.org/0000-0001-8513-4646 (2026) From Reporting Guideline to Reporting Quality Instrument: Developing RAISE-QI for Artificial Intelligence Research in Education. In: 6th International Conference on Educational Technology and Online Learning (ICETOL) Proceedings. ICETOL. (In Press)

Text 16631 Allison, J (2026) From Reporting Guideline to Reporting Quality Instrument Developing RAISE-QI for Artificial Intelligence Research in Education.pdf - Accepted Version Restricted to Repository staff only Available under License Creative Commons Attribution 4.0. Download (252kB)

Abstract

The rapid growth of artificial intelligence (AI) in education has intensified the need for transparent and comprehensive reporting of empirical research. Inconsistent reporting of AI systems, pedagogical design, human involvement, ethical considerations, and evaluation methods limits reproducibility, hinders evidence synthesis, and reduces confidence in research findings. The Reporting AI Studies in Education (RAISE) framework was developed to improve reporting quality through 30 reporting recommendations spanning ten domains. Although RAISE provides structured guidance for reporting AI in education research, no corresponding instrument currently exists to evaluate adherence to these recommendations. This paper presents the development of RAISE-QI (Reporting AI Studies in Education Quality Instrument), a methodological extension of the original RAISE framework designed to facilitate systematic assessment of reporting quality in AI in education research. The objective is to transform RAISE from a reporting guideline into a practical appraisal instrument that can support researchers, peer reviewers, journal editors, and evidence synthesis teams in evaluating the completeness and transparency of published studies. The instrument was developed by operationalising each of the 30 RAISE reporting recommendations into an observable assessment criterion, accompanied by standardised assessment questions and a proposed ordinal scoring framework to promote consistent application across reviewers. The operationalisation was guided by established principles underpinning reporting guidelines and quality appraisal instruments while maintaining fidelity to the conceptual intent of the original RAISE framework. The resulting instrument enables each reporting recommendation to be evaluated systematically rather than interpreted solely as narrative guidance. The principal outcome of this work is RAISE-QI, a reporting quality instrument specifically designed for AI in education research. By converting reporting recommendations into measurable appraisal criteria, the instrument provides a structured foundation for assessing reporting completeness, identifying common reporting deficiencies, and supporting future investigations of reporting quality, inter-rater reliability, and construct validity. It is concluded that reporting guidelines can serve a dual purpose as both author guidance and quality assessment instruments. Future research should evaluate the psychometric properties of RAISE-QI and examine its application across diverse AI in education studies. The development of RAISE-QI represents an important methodological contribution towards improving research transparency, reproducibility, and evidence quality within the rapidly evolving field of AI in education.

Item Type:	Book Section
Uncontrolled Keywords:	Artificial intelligence in education,; Reporting guidelines; Research quality; RAISE; methodological assessment
Related URLs:	Organisation
Subjects:	L Education > LC Special aspects of education Q Science > Q Science (General) > Q336 Artificial intelligence Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:	Schools and Research Institutes > School of Business, Computing and Social Sciences
Depositing User:	Kamila Niekoraniec
Date Deposited:	05 Nov 2026 11:43
Last Modified:	08 Nov 2026 09:30
URI:	https://eprints.glos.ac.uk/id/eprint/16631

University Staff: Request a correction | Repository Editors: Update this record