Machine Learning Driven Developments in Behavioral Annotation: A Recent Historical Review

Watson, Eleanor, Viana, Thiago ORCID: 0000-0001-9380-4611 and Zhang, Shujun ORCID: 0000-0001-5699-2676 (2024) Machine Learning Driven Developments in Behavioral Annotation: A Recent Historical Review. International Journal of Social Robotics, 16. pp. 1605-1618. doi:10.1007/s12369-024-01117-1

Text 13773 Watson et al (2024) Machine Learning Driven Developments - AAM.pdf - Accepted Version Restricted to Repository staff only until 7 April 2025. (Publisher Embargo). Available under License Publisher's Licence. Download (883kB)

Abstract

Annotation tools serve a critical role in the generation of datasets that fuel machine learning applications. With the advent of Foundation Models, particularly those based on Transformer architectures and expansive language models, the capacity for training on comprehensive, multimodal datasets has been substantially enhanced. This not only facilitates robust generalization across diverse data categories and knowledge domains but also necessitates a novel form of annotation—prompt engineering—for qualitative model finetuning. This advancement creates new avenues for machine intelligence to more precisely identify, forecast, and replicate human behavior, addressing historical limitations that contribute to algorithmic inequities. Nevertheless, the voluminous and intricate nature of the data essential for training multimodal models poses significant engineering challenges, particularly with regard to bias. No consensus has yet emerged on optimal procedures for conducting this annotation work in a manner that is ethically responsible, secure, and efficient. This historical literature review traces advancements in these technologies from 2018 onward, underscores significant contributions, and identifies existing knowledge gaps and avenues for future research pertinent to the development of Transformer-based multimodal Foundation Models. An initial survey of over 724 articles yielded 156 studies that met the criteria for historical analysis; these were further narrowed down to 46 key papers spanning the years 2018-2022. The review offers valuable perspectives on the evolution of best practices, pinpoints current knowledge deficiencies, and suggests potential directions for future research. The paper includes six figures and delves into the transformation of research landscapes in the realm of machine-assisted behavioral annotation, focusing on critical issues such as bias.

Item Type:	Article
Article Type:	Article
Uncontrolled Keywords:	Annotation; Behavior; Foundation models; LLMs; Machine learning; Robotics; Social
Subjects:	Q Science > QA Mathematics > QA76 Computer software
Divisions:	Schools and Research Institutes > School of Business, Computing and Social Sciences
Research Priority Areas:	Applied Business & Technology
Depositing User:	Susan Turner
Date Deposited:	21 Mar 2024 12:33
Last Modified:	14 Aug 2024 13:15
URI:	https://eprints.glos.ac.uk/id/eprint/13773

University Staff: Request a correction | Repository Editors: Update this record