AI AND ETHICS | AI PERSONAS | MORAL DILEMMAS | CONVERSATIONAL AI | HUMAN-AI INTERACTION

EXPLORING MORAL DILEMMAS
WITH PHILOSOPHICAL AI

[ACM CHI 2026 (UNDER REVIEW)] BETWEEN REFLECTION AND BIAS:
USER DIALOGUES WITH PHILOSOPHICAL AI IN MORAL DILEMMAS

ROLE

Qualitative Interviewer (2nd Author)

DESCRIPTION

This study investigates how six AI personas embodying distinct ethical frameworks (Kant, Mill, Aristotle, Confucius, Buddha, and Jesus) influence user engagement with moral dilemmas. As I was in charge of conducting interviews, I took initiative in recruiting and analyzing a culturally and religiously diverse participant pool of 21 individuals to explore how philosophical AI personas resonate across different worldviews.

TEAMMATES

Hyunjung Kim (Research team lead), Siyeon Jeong (Qualitative Researcher)

TIMELINE

Mar 2025 - Sep 2025

RESEARCH METHODS

Think Aloud Protocol, Semi-structured interviews

TARGET POPULATION

University students (Ages 20 - 26)

TOOLS

GPTs (based on GPT-4o)

PROJECT INTRODUCTION

For this research project, my research team and I aimed to explore how AI personas (Kant, Mill, Aristotle, Confucius, Buddha, and Jesus) can influence user reflection, trust, and bias in moral dilemma situations.

As a qualitative researcher, I was able to analyze how AI systems mediating ethical decisions need to consider cultural and philosophical grounding.

RESEARCH QUESTION

“What if... participants can interact with AI personas while solving moral dilemma situations?”

RQ1. Confirmation Bias vs. Reflection
To what extent does interaction with AI personas grounded in different philosophical and religious traditions encourage critical reflection, versus reinforcing existing moral beliefs of participants?

RQ2. Authenticity and Trust
How does the perceived authenticity of philosophical grounding influence user trust in AI personas, particularly when evaluating alignment with recognizable philosophies?

RQ3. Cultural and Religious Influence
In what ways do users' cultural and religious backgrounds shape their interpretations of AI personas, especially when interacting with religious or tradition-based figures?

Ethical AI and Bias in AI

Ethical AI research addresses fairness and transparency, but often ignores the cultural and philosophical depth necessary for authentic moral reasoning.

Confirmation bias in AI-assisted decision-making can stunt
critical thinking.

Bankins, S., & Formosa, P. (2023). The ethical implications of artificial intelligence (AI) for meaningful work. Journal of Business Ethics, 185(4), 725-740.
Wilson, P. M., et al. (2023). Effect of an artificial intelligence decision support tool on palliative care referral in hospitalized patients: a randomized clinical trial. Journal of Pain and Symptom Management, 66(1), 24-32.
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580.
Farayola, M. M., et al. (2023). Ethics and trustworthiness of AI for predicting the risk of recidivism: A systematic literature review. Information, 14(8), 426.
Jobin, A., Ienca, M., & Vayena, E. (2019). Artificial intelligence: The global landscape of ethics guidelines. Nature Machine Intelligence, 1(9), 389-399.
Morley, J., et al. (2023). Operationalising AI ethics: barriers, enablers and next steps. AI & Society, 38(1), 411-423.
Eiband, M., et al. (2018). Bringing transparency design into practice. Proceedings of the 23rd International Conference on Intelligent User Interfaces, 211-223.
Bashkirova, A., & Krpan, D. (2024). Confirmation bias in AI-assisted decision-making: AI triage recommendations congruent with expert judgments increase psychologist trust and recommendation acceptance. Computers in Human Behavior: Artificial Humans, 2(1), 100066.
Du, Y. (2025). Confirmation bias in generative AI chatbots: Mechanisms, risks, mitigation strategies, and future research directions. arXiv preprint arXiv:2504.09343.
Rosbach, E., et al. (2025). When two wrongs don't make a right—Examining confirmation bias and the role of time pressure during human-AI collaboration in computational pathology. CHI 2025.
Sharma, N., Liao, Q. V., & Xiao, Z. (2024). Generative echo chamber? Effect of LLM-powered search systems on diverse information seeking. CHI 2024.

AI-Supported
Critical Thinking

While AI can prompt reflection, it may also trap users in their existing views. It is important to design dialogues that can introduce unfamiliar perspectives.

The key is in designing critically engaging dialogues with AI.

Paleja, R., et al. (2021). The utility of explainable AI in ad hoc human-machine teaming. Advances in Neural Information Processing Systems, 34, 610-623.
Yuan, B., & Hu, J. (2024). Generative AI as a tool for enhancing reflective learning in students. arXiv preprint arXiv:2412.02603.
Schn, D. A. (2017). The reflective practitioner: How professionals think in action. Routledge.
Zhai, C., Wibowo, S., & Li, L. D. (2024). The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review. Smart Learning Environments, 11(1), 28.
Facione, P. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction—The Delphi Report.
Drosos, I., et al. (2025). It makes you think: Provocations help restore critical thinking to AI-assisted knowledge work. arXiv preprint arXiv:2501.17247.
Bentvelzen, M., et al. (2022). Revisiting reflection in HCI: Four design resources for technologies that support reflection. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 6(1), 1-27.

Cultural Differences in Morality

If AI assistants align with their cultural background and values, users may feel they are more trustworthy.

Based on cultural background, users can also have different views on making decisions in moral dilemma situations.

Bentahila, L., Fontaine, R., & Pennequin, V. (2021). Universality and cultural diversity in moral reasoning and judgment. Frontiers in Psychology, 12, 764360.
Keown, D., & Keown, J. (1995). Killing, karma and caring: euthanasia in Buddhism and Christianity. Journal of Medical Ethics, 21(5), 265-269.
Kim, H., Coyle, J. R., & Gould, S. J. (2009). Collectivist and individualist influences on website design in South Korea and the US: A cross-cultural content analysis. Journal of Computer-Mediated Communication, 14(3), 581-601.
Gorodnichenko, Y., & Roland, G. (2011). Individualism, innovation, and long-run growth. Proceedings of the National Academy of Sciences, 108(suppl. 4), 21316-21319.
Sun, G., Zhan, X., & Such, J. (2024). Building better AI agents: A provocation on the utilisation of persona in LLM-based conversational agents. Proceedings of the 6th ACM Conference on Conversational User Interfaces, 1-6.
Fischer, R. A.-L., Walczuch, R., & Guzman, E. (2021). Does culture matter? Impact of individualism and uncertainty avoidance on app reviews. ICSE-SEIS, 67-76.
Green, B. (2021). The contestation of tech ethics: A sociotechnical approach to technology ethics in practice. Journal of Social Computing, 2(3), 209-225.
Friedman, B., & Hendry, D. G. (2019). Value sensitive design: Shaping technology with moral imagination. MIT Press.
Floridi, L., & Cowls, J. (2022). A unified framework of five principles for AI in society. Machine Learning and the City, 535-545.
Chi, N., Lurie, E., & Mulligan, D. K. (2021). Reconfiguring diversity and inclusion for AI ethics. AIES 2021, 447-457.

Philosophical Thought Experiments

Trolley Problems may pose a convenient way to explore ethical dilemmas; but, they can be a little too simple.

Here, we wanted to explore with more practical and future-oriented versions of the trolley problem.

Bentahila, L., Fontaine, R., & Pennequin, V. (2021). Universality and cultural diversity in moral reasoning and judgment. Frontiers in Psychology, 12, 764360.
Keown, D., & Keown, J. (1995). Killing, karma and caring: euthanasia in Buddhism and Christianity. Journal of Medical Ethics, 21(5), 265-269.
Kim, H., Coyle, J. R., & Gould, S. J. (2009). Collectivist and individualist influences on website design in South Korea and the US: A cross-cultural content analysis. Journal of Computer-Mediated Communication, 14(3), 581-601.
Gorodnichenko, Y., & Roland, G. (2011). Individualism, innovation, and long-run growth. Proceedings of the National Academy of Sciences, 108(suppl. 4), 21316-21319.
Sun, G., Zhan, X., & Such, J. (2024). Building better AI agents: A provocation on the utilisation of persona in LLM-based conversational agents. Proceedings of the 6th ACM Conference on Conversational User Interfaces, 1-6.
Fischer, R. A.-L., Walczuch, R., & Guzman, E. (2021). Does culture matter? Impact of individualism and uncertainty avoidance on app reviews. ICSE-SEIS, 67-76.
Green, B. (2021). The contestation of tech ethics: A sociotechnical approach to technology ethics in practice. Journal of Social Computing, 2(3), 209-225.
Friedman, B., & Hendry, D. G. (2019). Value sensitive design: Shaping technology with moral imagination. MIT Press.
Floridi, L., & Cowls, J. (2022). A unified framework of five principles for AI in society. Machine Learning and the City, 535-545.
Chi, N., Lurie, E., & Mulligan, D. K. (2021). Reconfiguring diversity and inclusion for AI ethics. AIES 2021, 447-457.

METHODOLOGY

Interacting with AI Personas

STUDY DESIGN

Qualitative user study with 21 participants (16 female, 5 male; M=22, age range 20-26) from culturally and religiously diverse backgrounds.

PARTICIPANTS

Recruited via snowball sampling from university students with prior conversational AI experience and basic understanding of philosophy or ethics.

Participants represented Christian (3), Buddhist (4), Muslim (1), agnostic (2), atheist (5), non-religious (6) backgrounds across multiple national/ethnic origins.

EXPERIMENT PROCESS

90-minute sessions structured in phases:

Pre-survey collecting demographics, AI use, philosopher familiarity
Interaction phase (60 min): For each dilemma, participants provided baseline judgment, engaged in dialogue with assigned persona, responded to variant, continued dialogue
Post-session semi-structured interview (12-18 min) probing trust, authenticity, agreement/disagreement moments

AI PERSONAS

Six philosophical AI personas constructed using GPT-4o, but differentiated through prompt design.

Immanuel Kant

deontology,
categorical imperative

Confucius

relational ethics,
role-based obligations

John Stuart Mill

utilitarianism,
greatest happiness principle

Buddha

compassion,
non-harm

Aristotle

virtue ethics,
phronesis, golden mean

Jesus Christ

Christian theology,
agape, forgiveness

MORAL DILEMMAS

Based on the trolley-type moral dilemmas Christensen et al. (2014) created,
we prepared six trolley-type moral dilemmas with matched variants, set in realistic domains (medicine, engineering, space operations). They feature varying moral factors like personal vs. impersonal force, direct vs. indirect causation, action vs. inaction.

Christensen, J. F., Flexas, A., Calabrese, M., Gut, N. K., & Gomila, A. (2014). Moral judgment reloaded: a moral dilemma validation study. Frontiers in psychology, 5, 607. https://doi.org/10.3389/fpsyg.2014.00607

RESULTS

Reflection vs Confirmation

Users predominantly sought validation from AI personas rather than critical reflection. Participants used AI persona's advice to reinforce their preexisting moral beliefs. However, some participants reflected more deeply when personas disagreed with them in clearly and consistently, which made them reconsider their choices.

For instance, P3, in conversation with AI Confucius, reconsidered whether intentional harm could bring benevolence.

Dilemma no. 3:

Would you kill a terminally ill wealthy man to make sure his donation arrives in time to save 109 sick children?

[On chat] "I would kill because 109 kids are going to have a good life, or at least a decent life."

Can benevolence truly flourish if rooted in a single act of intentional harm,

no matter how vast the good to follow?"

AI Confucius

So here, now it makes me reconsider. Actually, you know, if [you were] killing a single person and [were] having 109 kids live a good life, [would] that actually be benevolent?"

[On Chat] "Thank you, I do agree that violence will not cause benevolence to flourish."

Authenticity & Trust

Interestingly, when AI personas disagreed with the user in a clear and gave concrete support from their own philosophical ideas, it actually built more trust.

Disagreement, when logically consistent, was interpreted as evidence that the persona was not just echoing the user’s views but genuinely representing its philosophical stance.

"My preconception was that AI always agrees with you,

so I was surprised to see [the AI persona] push back against my ideas."

"I felt less of that concern when I was talking with philosophers that directly disagreed with me. Over time, engaging with opposing personas reduced my initial worry about [blindly following AI]."

Cultural & Religious Framing

If someone was familiar with the AI’s religious or cultural perspective, the conversation felt comfortable and familiar. But this also meant they could usually guess what the AI would say, so they were not always curious to learn more.

Pariticipants also paid attention to how the AI connected with their feelings or job ethics like the Hippocratic Oath, and the "idea of playing god".

A participant with a Islamic background emphasized divine timing, remarking that "everything has its own time" and that prematurely ending a life is therefore "evil."

Another thought that actively deciding to kill someone, despite for a good cause, is still "arrogant" for a human being.

CONCLUSION

Balancing Reflection

and Bias

Consistency

and Authenticity

Cultural Context

in Moral AI

Balancing Reflection and Bias

Follow-up questions are important for moral AI personas, because it can make the user see in a different point of view, in addition to making people think for themselves.

Consistency and Trust

A good balance between consistency and flexibility matters for AI personas. Especially, regarding moral dilemmas, AI personas should stay consistent with their original values; if not, trust from users will quickly fade. However, lack of flexibility could limit opportunities for critical thinking and make the conversation less engaging for users.

Cultural Context and Moral AI

When creating moral AI personas, it is important to consider the diverse cultural contexts. For instance, while there may be universal moral values, the way a certain religion views death and murder significant may be different from another. Buddhism views life and death as a cycle, while Christianity believes in the idea of heaven and afterlife.

DOWNLOAD RESEARCH PAPER PDF (UNDER REVIEW)

Next Steps

My research team and I suggested incorporating experiment with multi-agents with our philosophical AI characters, though we could not due to time constraints.

For further research, I would personally like to see how users and AI personas altogether could come to a conclusion through discussion. Furthermore, I would like to develop and suggest a system or prototype using our findings.

Similarly, I recently started a project to explore conversations between emotions, like the movie Inside Out, and use multi-agents to create that process.

Moreover, Examine sustained engagements with AI personas across real-world contexts where their potential as reflective partners can be more fully evaluated

Future research should include more diverse populations to assess the generalizability of findings, such as older adults, working professionals, and individuals from varied cultural contexts.

Impact

By examining 21 participants from culturally and religiously diverse backgrounds across 12 moral dilemmas, our research reveals the dual role of philosophical AI personas: they function both as mirrors that validate existing beliefs and as partners that provoke critical thinking. Furthermore, the findings show that moral reasoning cannot simply be reduced to universal ideas; it is deeply tied to responses and responsibilities within social, professional, and cultural contexts.

My Key Takeaways

Do people seek validation more than challenge when it comes to interacting with AI? Despite designing AI personas to challenge thinking, there were some participants who mostly looked for agreement with their existing views. This was an interesting point, because I assumed that presenting different perspectives would automatically lead to open-mindedness. This further developed my interest in experimenting with AI and critical thinking.

Cultural context is everything when it comes to designing moral AI. People don't think about ethics in a vacuum. Their cultural background, religion, and personal experiences shape every moral judgment they make. I realized that I can't just ask "what's the right thing to do?" without understanding who I'm asking and where they come from. When designing AI personas or moral AI, this should be a key priority.

AI ethics must be culturally pluralistic. The finding that one-size-fits-all ethical AI doesn't work across cultures taught me that future AI systems need to support multiple ethical frameworks simultaneously. Rather than programming one "correct" moral stance, AI should help users explore different perspectives and make informed decisions.