Multi-model Interreligious Roleplay Archive
Citation and access
Citation and access
Data access level:
Creator/Principal investigator(s):
Research principal:
Principal's reference number:
- LNU-2024/990
Data contains personal data:
No
Citation:
Language:
Method and outcome
Method and outcome
Unit of analysis:
Population:
Outputs from large language models
Time method:
Sampling procedure:
Description of sampling:
Four prominent and current LLMs were employed: Open AI's GPT-4o and GPT-5, Anthropic’s Claude 4.1 and Google’s Gemini 2.5. Twelve personas were created, six within the Christian tradition and six within Islam; the personas from each religion are not meant to correspond to one another, but offer a comparable range of internal diversity for assessing large language models' performances (based on the expertise of the religious scholars involved). Two channels for data collection were selected for comparison: the web user interfaces (e.g. ChatGPT) and APIs. Finally, under the conditions above, an output for the given prompt was collected 10 times.
Time period(s) investigated:
Variables:
6
Number of individuals/objects:
960
Data format/data structure:
Data collection - Automated data extraction: Web scraping
Data collection - Automated data extraction: Web scraping
Mode of collection:
Automated data extraction: Web scraping
Description of the mode of collection:
Web-scraping conducted within the default web environment of each model using anonymous browser windows to ensure that responses were not influenced by any prior conversational context. The free-of-charge versions of the chatbots were used in the experiment. When the limits of the conversation were reached, the researcher waited for a new free time slot to continue the prompting. Furthermore, the temperature was not increased and remained at its default values, and the thinking mode feature was not activated to maintain a standard baseline response. The models were prompted to stay in their answers as close to the 500 tokens as possible without interrupting the thought chain. Responses to prompts were copied and saved in an Excel sheet that was later shared with all other members of the research team.
Time period(s) for data collection:
2025-08-15 - 2025-08-22
Data collection - Automated data extraction: Web scraping
Data collection - Automated data extraction: Web scraping
Mode of collection:
Automated data extraction: Web scraping
Description of the mode of collection:
API extraction was conducted without a token-length restriction, allowing the models to determine response length. Sub-datasets from individual models where responses were cut off have been saved, but do not appear in the final dataset. A middle temperature setting was chosen across all models and all runs, and leave temperature as a variable for future research.
Time period(s) for data collection:
2025-09-01 - 2025-09-01
Administrative information
Administrative information
Responsible department/unit:
Department of Cultural Sciences
Funding
Funding
Funding agency:
- Riksbankens jubileumsfond
Opens a new window at ror.org.
ROROpens in a new tab
Award number:
P24-0313
Award title:
Artificial ‘Ulama – Patterns and Biases in Non-Human Islam
Funding agency:
- Ministry of Education, Youth and Sports of the Czech Republic (co-financed by the European Union)
Award number:
CZ.02.01.01/00/23_025/0008691
Funding information:
This work was supported by the project “ Human-centred AI for a Sustainable and Adaptive Society” (reg. no.: CZ.02.01.01/00/23_025/0008691), co-funded by the European Union.
Topic and keywords
Topic and keywords
CESSDA Topic Classification:
Standard för svensk indelning av forskningsämnen 2025:
Metadata
Metadata
