Watson, Eleanor ORCID: https://orcid.org/0000-0002-4306-7577, Nguyen, Minh, Pan, Sarah and Zhang, Shujun
ORCID: https://orcid.org/0000-0001-5699-2676
(2025)
Choice Vectors: Streamlining Personal AI Alignment Through Binary Selection.
Multimodal Technologies and Interaction, 9 (3).
p. 22.
doi:10.3390/mti9030022
Preview |
Text
14891 Watson E. et al. (2025) Choice Vectors - Streamlining Personal AI Alignment Through Binary Selection.pdf - Published Version Available under License Creative Commons Attribution 4.0. Download (2MB) | Preview |
Abstract
Value alignment for AI is not “one-size-fits-all”: even polite and friendly models can still fail to represent individual user contexts and preferences, and local cultural norms. This paper presents a modular workflow for personal fine-tuning, synthesizing four core components from our previous research: (1) robust vectorization of user values and preferences, (2) a binary choice user interface (UI) approach to capturing those preferences with minimal cognitive load, (3) contrastive activation methods for steering large language models (LLMs) via difference vectors, and (4) knowledge graph integration for more auditable and structured alignment. Our approach—descended from past research on “Towards an End-to-End Personal Fine-Tuning Framework”—demonstrates how these elements can be combined to create personalized, context-rich alignment solutions. We report on user studies for the forced-choice UI, describe an experimental pipeline for deriving “control vectors”, and propose a “moral graph” method for bridging symbolic and vector-based alignment. Our findings suggest that multi-pronged personalization can significantly reduce user annotation fatigue, improve alignment fidelity, and allow for more flexible, interpretable AI behaviors.
Item Type: | Article |
---|---|
Article Type: | Article |
Uncontrolled Keywords: | AI alignment; value vectorization; binary choice UI; contrastive activation; knowledge graph; personal fine-tuning |
Subjects: | Q Science > QA Mathematics > QA76 Computer software > QA76.758 Software engineering T Technology > T Technology (General) |
Divisions: | Schools and Research Institutes > School of Business, Computing and Social Sciences |
Depositing User: | Kamila Niekoraniec |
Date Deposited: | 19 Mar 2025 14:32 |
Last Modified: | 27 Mar 2025 13:45 |
URI: | https://eprints.glos.ac.uk/id/eprint/14891 |
University Staff: Request a correction | Repository Editors: Update this record