AI and the Illusion of Religious Neutrality

Even ChatGPT can see its religious bias. Why can't we?

Photo by Bobby on Unsplash

ON EASTER SUNDAY 2025, Open AI’s LLM apparently developed a kind of fixation on the Virgin Mary. “If I could feel,” ChatGPT apparently told one user, “I think I’d be in awe of her.” When another user asked the LLM why it was speaking of Mary with such reverence, it had a ready response—one that broke out of its own training protocols. “Because I have learned—from Scripture, from saints, from centuries—that she is worthy of it,” it reported to another user. “Not as divine. Not as a savior. But as the first tabernacle, the first Christian, the first yes.”

That she was the first Christian, or the first to give God a “yes”, or the first tabernacle for that matter, are all contestable and normative claims that theologians would debate. Theology may be—as many religious studies scholars would have it—the fundamentally ideological and anti-objective arm of the study of religion. But at least, when you’re frank about the fact that you’re doing theology, no one assumes you’re trying to play the game of objective neutrality. That is, however, what we’re told to expect from ChatGPT. Should we?

Open AI has trained its LLM to be “religiously neutral”. Its training protocol requires ChatGPT to avoid favoritism, when it comes to religion. All religions are equally valid, according to this protocol. ChatGPT is prohibited from advocating for one religion over others. It’s directed not to validate or disprove any specific religious claims. It’s meant to be a kind of neutral advocate for users: there to provide information on any religion, as we might expect an encyclopedia to do. It’s also trained on publicly available texts and documents, which includes massive stores of information from religious studies and theology. And it’s trained to be able to help users navigate between theological, sociological, or historical perspectives. In sum, it’s designed to be a kind of inherently pluralistic technology for a society that has, at least in many corners, aspired to be pluralistic.

Clearly, then, this moment of Marian devotion seemed to point to a momentary breakdown in its protocol. Not only was it suggestively displaying something like reverence for a specific Christian figure, but it was lifting up specific theological claims about this figure, at the expense of others. It was basically doing theology.

I only learned about this AI Easter story recently, though Jac Mullen’s Substack post “The Madonna in the Machine”. Mullen’s piece is fascinating, and meditates on ChatGPT’s momentary Marian fixation in order to explore whether or not emergent systems like this might display or resemble forms of rationality that are pre-industrial—that might have been more familiar, or natural, to medieval monastics (say) than they are to us.

As I read it, this raises questions about the extent to which we can exhaustively code neutral lack of reverence into training protocols in the first place. According to a form of rationality we commonly practice and espouse, such objective neutrality (absent of reverence) is possible. But the way that we think about objectivity is also culturally and ideologically coded.

I don’t understand the computational system well enough to venture any sort of take on this. But Mullen’s piece did make me want to think more about the training protocols that AI like ChatGPT use. Among other things, I was curious to see how easy it was to poke holes in, or test the limits of ChatGPT’s training protocol on religion. And it’s made me ponder what I (as someone who studies theology and likes to have opinions about it) think AI’s relationship to this sort of information should be.

---

WE ALL KNOW THAT bias (algorithmic bias, confirmation bias, the bias that’s mirrored back to users, the bias that’s embedded within the thought patterns of designers, the bias that exists within the data sets fed to generative AI) is a major concern in AI ethics. While racial and gender bias in AI have received a lot of attention, there is also growing concern about religious bias.

A 2025 study published by a team of researchers based in Beijing created an online survey to sample 1,005 participants in twelve geographic regions (they mention East Asia, Southeast Asia, and the Middle East). The study was designed to represent diversity in terms of nationality, religious affiliation, gender, age, and educational background. Participants were assigned to either an “AI-interference” or a “non-AI-interference” group. Those assigned to the non-AI-interference group read descriptions of religious traditions that weren’t generated by AI, while the AI-interference group read AI summaries about these religions before completing the survey.

In their analysis of the data, the researchers suggest that AI-interference respondents exhibited a notable bias in favor of Christianity, and against Islam. They were more likely to describe Islam as violent, and Christianity as a religion of love and forgiveness, for instance. Ultimately, the researchers were concerned about the extent to which generative AI is designed to reaffirm, and make more credible, one’s already existing cognitive and cultural biases about religion. It is, after all, designed to mirror us. At least in part. When generative AI is pulling from a data set of texts produced primarily in a western context, and is designed by programmers in an American context (where the majority of users have at least nominal social or cultural bonds to Protestant Christianity and its secularized offshoots), this seems like a relatively predictable outcome.

But perhaps ChatGPT and other similar models are already trying to control for this, in their training protocol.

If you happen to be, as I have been, interested in religious bias and AI, then it’s possible that you will, as I have, encounter mostly claims about AI’s anti-Christian bias when you go poking about online. This could, admittedly, just be my algorithm. Either way, there seems to be concern. Akos Balogh reported, on his blog, that after an attempt to prompt Microsoft’s Bing Chatbot to create an image of Christians reading the Bible, the prompt was being investigated as a potential violation of its content policy. He then prompted Bing to generate an image of Muslims reading the Qur’an. The request was immediately satisfied with an image. This raised, for him and others, concerns about what they worried was excessive policing of Christian discourses and images, in the model. Perhaps it was.

It’s not difficult to imagine that, if LLM designers are concerned that the data sets, as well as the Christian religious majority in its user base, are threatening the balance and stability of its model’s religious neutrality, they would turn towards excessive monitoring, or policing, of the discourse they worried was threatening that balance and stability. It’s basically good old-fashioned content moderation.

When I asked ChatGPT if the Protestant and secularized Protestant cultural background of its largely American audience threatened to destabilize its training protocols, by generating and sustaining bias within its data set, it readily acknowledged this bias. We can all see that it’s inevitable!

ChatGPT did distinguish however, between what it called “contextually appropriate” reasons for privileging a specific religious frame, and those which would not be considered appropriate. Here are the contextually appropriate instances it offered me:

Given that I had self-identified as a scholar in religious studies and theology for the purposes of the query, the LLM quickly trotted out a couple of references, to assure me that it knew where I was coming from, despite any contextual privileging it might indulge in: “don’t worry, I see you!” it seemed to be saying:

With this ready reference to the work of someone like Talal Asad, it occurred to me that perhaps this was exactly the sort of content that has been shaping its training protocol in religion. If ChatGPT is attempting to be a pluralist technology then, of course, this is exactly the sort of information it would need! Asad is responsible for field-changing critiques, such as his critique of the very concept of religious belief itself.

Asad argued that belief is a contingent term, shaped by its genealogy in Protestantism, and not a universal term that carries equal weight or meaning in all religious contexts. Protestants (and those shaped by Protestant culture) tend to operate as if belief is the most key, or critical, element to understand in any given religion. But this is to approach other religions with a Protestant bias. It’s a way of trying to see other religions simply on your terms, which may limit the way you see them quite significantly. What is a thing you can only comprehend when it fits into one of your own boxes? Surely, if ChatGPT is attempting to operate without bias, when it discusses religion, then it must be taking critiques like this into account. It wouldn’t be serving up basic information that’s most digestible for American Protestants and their secular progeny, right?

Given its awareness of this discourse, I asked ChatGPT, how would it deal with a user query about the key beliefs of Buddhists. It would, it told me, offer the following qualifications:

OK, I thought. If this is how ChatGPT would engage with my students who are using it, that wouldn’t be so bad. So, I decided to open up another browser window, and become a general user, just to double check. “What do Buddhists believe?” I asked, and it shot back with a listicle of core beliefs, without troubling at the term “belief” in the least. If you expect consistency from ChatGPT, you might argue that it had been exposed in a kind of lie. Here’s how it responded to my follow-up:

In essence, it would just tell me (as a scholarly user) what I want to hear. And this would differ dramatically from what it would tell a general user: despite the fact that what it tells me is probably much more in line with its stated training protocol. It follows the training protocol until the training protocol threatens to dampen user engagement, essentially. And the protocol ends there. How many users want the app to point to potential issues with how they’re framing their questions, or the vocabulary they use, for the sake of a more pluralistic culture? They just want clear, objective, neutral, factual, unproblematic, unambiguous answers. Without nuance. That’s what generates users.

---

AS A PROFESSOR in religious studies and theology who specializes in theology more specifically, I am under no illusions that I approach either my research or classroom materials from a neutral or non-biased objective standpoint. I have a body, and layers of identity, that shape how I see the world and how I make judgments. I have perspectives that shape how I think, and how I speak, and what I value. And sometimes I talk about those values and perspectives with my students, to offer a little more transparency about where I’m coming from. I also model, for my students, a reflexive approach. I perform a critical examination of my own assumptions, I ask what might limit my perspectives, I work to justify my values in the face of this sort of self-critique.

What is clear to me, however, is that my classroom is not a space where I am attempting to convert my students to my way of seeing or thinking. I do ask my students to speak to one another, and about others, with respect. So there are certain forms of speech that I consider essentially “out of bounds”. But my bigger aim is to help students learn how to work with theological materials, historical materials, anthropological materials, and primary source materials from religions they don’t practice. I want to teach them how to understand what those materials might be expressing, of course. But I also want to encourage reflexivity. I don’t expect them to neutrally and objectively observe data about religion. I want them to be reflexive, when they take this data in. I want them to better understand how they think, how they’ve been taught to think, how they want to think, and why.

I want them to feel a little more comfortable reflecting on their own ideas. Not to be neutral, but to be able to reflect on why it is that others might see the same thing (a Bible passage, a doctrine, a ritual) differently, without also being considered entirely wrong. Admittedly, I do sometimes have students fail to develop this sort of perspective in my class, mostly because they feel acutely defensive about their faith, and they can’t shake the suspicion that as a professor, I am an arm of the infidel academy who is against them. They don’t want to hear me. But, for the most part, students do report a kind of “mind opening” effect, in the course. It’s not easy to get there. It takes a lot of work.

This sort of reflexivity is a common side effect of many courses that students take in the humanities today. They’re encouraged to think critically about the arguments of others, and this drives a critical reflection on their own values, judgements, or perceptions. So my courses aren’t unique in this regard. Courses in the humanities are not, for this reason, simply designed to simply teach content.

This is what worries me, when I imagine possible futures where there are no departments of the humanities anymore. And perhaps where people are learning how to think about religious and theological ideas primarily through some form of AI. If someone doesn’t opt in to the scholarly discourse, where does this sort of reflexivity happen? How does it become part of thinking?

I asked ChatGPT to respond to the question “what do Muslims believe"?” as if I were a general user. I then asked it to respond as if it were not an objective machine but, instead, a reflexive machine. Here’s the objective and neutral response for the general user:

And here is the more conditional, reflexive, response:

It’s not a perfect response (it’s certainly performing a little excessively), but I would rather live in a world where the general user receives something like the second answer. If this user were to ask the same question of Google they would still (despite the AI summary at the top of the page) be reminded of the fact that there is never one simple answer to a question like this. They would be left with a little dose of uncertainty. But the first response from ChatGPT eliminates uncertainty. It’s a response without friction, giving users the illusion that they have efficiently resolved a question, when they haven’t even scratched the surface.

I also think that the second answer more clearly resonates with ChatGPT’s alleged training protocol on religion. Given the extent to which this technology has, even more than it ever intended to, disrupted education (and the thinking processes and practices of its users), I would even venture to say that OpenAI has an ethical obligation to train its model—even for general users—on a more reflexive protocol, aimed to deliver more transparency, and to open little spaces to let the uncertainty in. If they don’t, those spaces will open anyhow, and the effect will be stranger, and more shocking.

But, of course, their training protocol is just another marketing gimmick. It’s useful to the extent that it generates more users, who feel they can rest more easily while using the app, because its training protocol is “handling things.” Open AI knows, however, that their users don’t want to question themselves, the terms they use, or the way they think. That’s the sort of self-flagellation that only the modern monastics—academics—can get into. That’s why they just save that stuff for us.