What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment

Vallego, Jorge, Tiernan, Eleanor, Rukh, Mah ORCID: https://orcid.org/0000-0001-7660-1150, Roccia, Mariana ORCID: https://orcid.org/0000-0001-9631-9910 and Fiebig-Lord, Sabina (2026) What Stories Do Language Models Tell About Nature? A Multi Layer Evaluation Framework for Ecological Alignment. In: NLP4Ecology @ LREC 2026 Workshop Proceedings. NLP4Ecology 2026 The 2nd Workshop on Ecology, Environment, and Natural Language Processing (2). ELRA Language Resources Association (ELRA, pp. 60-68. ISBN 9782493814531

Text 16278 Vallego, J et al (2026) What stories do language moidels tell about nature.pdf - Accepted Version Restricted to Repository staff only Available under License Creative Commons Attribution Non-commercial 4.0. Download (382kB)

Abstract

Large language models increasingly generate environmental discourse, yet there is no standardised framework for evaluating the ecological narratives they produce. We introduce a structured prompt corpus and a reproducible multi layer evaluation framework grounded in ecolinguistic theory, operationalising five dimensions of ecological alignment: anthropocentrism, agency attribution, erasure of non human impacts, evaluation of growth, and responsibility framing. The framework integrates human judgement, an ecosophy aligned model judge, and automated semantic metrics, and is applied to outputs from ChatGPT, DeepSeek, and Ecophora, our ecosophy guided model. Ecophora achieves the highest alignment across all layers, with near ceiling judge scores of 159/160 and 142/160, together with the strongest automated composite performance. Divergences between automated metrics and holistic judgement indicate that ecological vocabulary alone does not guarantee ecological reasoning. The proposed framework provides a scalable methodology for benchmarking ecological alignment and assessing narrative shifts in language models.

Item Type:	Book Section
Uncontrolled Keywords:	Ecological Alignment; Ecolinguistics; Large Language Models
Subjects:	H Social Sciences > H Social Sciences (General) Q Science > Q Science (General) > Q336 Artificial intelligence T Technology > T Technology (General)
Divisions:	Schools and Research Institutes > School of Business, Computing and Social Sciences
Depositing User:	Mah Rukh
Date Deposited:	20 May 2026 12:15
Last Modified:	30 May 2026 10:00
URI:	https://eprints.glos.ac.uk/id/eprint/16278

University Staff: Request a correction | Repository Editors: Update this record