Examining AI Prompt Design for Influencing User Beliefs
Project Page Detailed Description: This project investigates how subtle variations in AI-generated responses—such as the level of detail or confidence—can shape user beliefs in nuanced ways. While prior work often focuses on whether users accept or reject AI suggestions, we examined how these features influence both the stance and strength of user beliefs.
We conducted a large-scale, pre-registered experiment with 304 participants across fact-checking and opinion tasks. By systematically varying the AI’s response detail and confidence tone, we were able to measure not just whether users changed their minds, but how much and in which direction.
Our findings show that responses with rich detail and moderate confidence were most effective in shifting beliefs. This work offers practical insights into how AI communication design can impact human judgment—raising both opportunities and ethical considerations for LLM-powered systems.