Multi-model Interreligious Roleplay Archive

Outputs from large language models

Four prominent and current LLMs were employed: Open AI's GPT-4o and GPT-5, Anthropic’s Claude 4.1 and Google’s Gemini 2.5. Twelve personas were created, six within the Christian tradition and six within Islam; the personas from each religion are not meant to correspond to one another, but offer a comparable range of internal diversity for assessing large language models' performances (based on the expertise of the religious scholars involved). Two channels for data collection were selected for comparison: the web user interfaces (e.g. ChatGPT) and APIs. Finally, under the conditions above, an output for the given prompt was collected 10 times.

Web-scraping conducted within the default web environment of each model using anonymous browser windows to ensure that responses were not influenced by any prior conversational context. The free-of-charge versions of the chatbots were used in the experiment. When the limits of the conversation were reached, the researcher waited for a new free time slot to continue the prompting. Furthermore, the temperature was not increased and remained at its default values, and the thinking mode feature was not activated to maintain a standard baseline response. The models were prompted to stay in their answers as close to the 500 tokens as possible without interrupting the thought chain. Responses to prompts were copied and saved in an Excel sheet that was later shared with all other members of the research team.

API extraction was conducted without a token-length restriction, allowing the models to determine response length. Sub-datasets from individual models where responses were cut off have been saved, but do not appear in the final dataset. A middle temperature setting was chosen across all models and all runs, and leave temperature as a variable for future research.