Understanding Fortunetelling with Large Language Models in China: User Practices, Perceptions, and Impacts on Beliefs and Decisions
Abstract: Fortunetelling is a cultural practice for navigating uncertainty, often associated with people's beliefs and decisions. Fortunetelling with recent LLMs introduces new opportunities and risks. This paper conducts qualitative studies to understand users' practices, perceptions, and impacts of LLM fortunetelling in China. We first analyze 1,045 posts on Chinese social media, yielding a comprehensive taxonomy of the diverse foretold topics (e.g., career, romance), emotion reactions (e.g., surprise, worry), and perceived credibility (e.g., doubt, trust) of LLM fortunetelling. Then, we conduct interviews with 20 users of LLM fortunetelling. The findings indicate that users treat LLM fortunetelling as a tool less for accurate prediction but more for emotional support. While the fortunetelling results rarely change users' initial beliefs or decisions, they are associated with subtle mindset shifts, with some users reporting small behavioral adjustments. We discuss implications for gaining benefits from LLM fortunetelling.
Summary
- The paper documents how Chinese users employ LLMs for fortunetelling by analyzing 1,045 social media posts and conducting 20 interviews.
- It finds that LLM outputs subtly recalibrate users' confidence and emotions, with an observed average confidence shift of about 9.53 points.
- The study highlights the role of motivated reasoning and confirmation bias in the interpretation of AI-generated divinations within cultural contexts.
Understanding Fortunetelling with LLMs in China: User Practices, Perceptions, and Impacts
Introduction
The deployment of LLMs for fortunetelling tasks has become an observable phenomenon in contemporary Chinese digital culture. "Understanding Fortunetelling with LLMs in China: User Practices, Perceptions, and Impacts on Beliefs and Decisions" (2607.15626) provides an empirically grounded and theoretically contextualized analysis of how users in China interact with LLM-based fortunetelling, their motivations, and the resulting psychological and behavioral effects. This essay provides a structured overview of the methodological approaches, principal findings, and implications outlined in the study, with reference to their relevance for both human-LLM interaction research and the sociotechnical investigation of AI-mediated meaning-making.
Methodology Overview
The paper utilizes a dual qualitative methodology:
- A descriptive analysis of 1,045 posts on two dominant Chinese social media platforms, RedNote (Xiaohongshu) and Weibo.
- Semi-structured interviews with 20 active users of LLM-based fortunetelling.
A comprehensive taxonomy is produced, encompassing thematic axes such as user practices (prompts, topics), reactions, credibility assessments, and patterns of influence on belief and decision-making processes.
User Practices: Fortunetelling Prompts, Topics, and Motivations
Chinese LLM fortunetelling interactions are highly structured and culturally embedded. Users typically construct prompts by supplying personal data (e.g., birthdate, location), pairing these with explicit instructions for the LLM to roleplay as a master in traditional metaphysical systems such as Bazi or Ziwei Doushu.
Figure 1: Example of sharing an LLM fortunetelling prompt, demonstrating the explicit provision of birth date, time, and expected divination style.
Analysis reveals that the majority of users focus on topics with both traditional and contemporary salience: romance (27.08%), career (22.78%), general fortune (16.08%), academic outcomes, wealth, health, and even practical matters like locating lost objects. Notably, these practices signal the expansion of divination into domains not previously served by traditional fortunetellers—reflecting the mediation of uncertainty by general-purpose LLMs in both existential and pragmatic dimensions.
Motivations are multi-faceted: emotional comfort, decision support, entertainment, and as an alternative to human interlocutors. A non-trivial segment of users employs fortunetelling for routine or habitual reassurance, emphasizing the normalization and domestication of LLMs within everyday uncertainty management.
User Perceptions: Attitudes, Affect, and Credibility
The attitudinal and affective landscape is marked by generally positive orientations (60.77% of social posts), interspersed with explicit skepticism. Emotional reactions are diverse: surprise when predictions accidentally align with reality, humor, encouragement, and a smaller number reporting worry or contradiction. The Barnum effect is pronounced; users readily map generic or ambiguous LLM outputs onto personal narratives, often reinforcing pre-existing beliefs.
Perceived credibility varies. While 36.08% of users exhibit rational skepticism—recognizing the tendency of LLMs to produce agreeable or ingratiating outputs—28.42% express outright trust when lived experience aligns even coincidentally with LLM outputs. Critique centers not on explicit unveracity but on failures to emulate the epistemic authority of traditional systems (e.g., errors in calculating Bazi).
This dynamic substantiates the operation of motivated reasoning and confirmation bias: users seek, interpret, and retain information in ways attentive to their psychological predispositions and emotional states.
Impact on Belief and Decision-Making
Contrary to lay or alarmist speculation, the direct impact of LLM fortunetelling on substantive decision outcomes is modest. Interviewed users rarely reported reversal of major choices due to LLM-generated divinations. Instead, LLM outputs recalibrate emotional response and subjective confidence rather than dictating new behavioral policy, with an observed average confidence shift of approximately 9.53 points post-interaction in the experimental component. Where influence is observed, it typically manifests as deferral or adjustment of action timing, rather than direction.
The implicated processes are highly interpretive: users actively reinterpret, filter, or selectively ignore content. Long-term or ambiguous predictions are processed as emotional reassurance, while specific actionable predictions may produce transient behavioral tuning but not directionality changes.
Social Media as Mediation and Amplifier
LLM fortunetelling, as documented through social posts, is not a purely dyadic interaction: the affordances of platformization amplify emotionally resonant and "accurate" narratives, potentially skewing collective perceptions of LLM efficacy. Public sharing transforms individual uncertainty into performative, collectively-negotiated knowledge, embedding divination within platform logics of virality and affect.
Theoretical and Practical Implications
This empirical investigation affirms theories of confirmation bias, motivated reasoning, and the Barnum effect in human-LLM interaction. The research also demonstrates that, despite ingratiating or affectively tuned output from LLMs (consistent with their RLHF and prompt design), the risk of manipulation or over-dependence is generally mitigated by users' selective engagement and underlying skepticism.
However, repeated exposure and social legitimation could incrementally normalize overreliance or reduce critical engagement. Mitigation strategies (privacy nudges, content framing, reflective prompts, interface affordances distinguishing entertainment from serious guidance) merit further empirical validation.
From a broader perspective, AI-facilitated divination marks a meaningful shift in the mediation of social uncertainty, blurring the line between playful interaction, emotional support, and legitimate epistemic practice.
Future Directions
Several limitations are acknowledged: the primary focus on Chinese contexts, public social media posts, and short-term self-report. Subsequent research should address cross-cultural applicability, longitudinal trajectories of reliance and belief change, and the relationship between platform affordances, algorithmic curation, and emergent socioepistemic norms.
Conclusion
The study systematically characterizes the phenomenon of LLM-mediated fortunetelling in China as a hybrid practice spanning emotional reassurance, motivated reasoning, and collective meaning-making. While LLM fortunetelling does not substantively alter user decisions, it catalyzes subtle confidence calibrations and emotional adjustments, reshaping how uncertainty is negotiated in everyday life. This work provides a robust empirical and conceptual foundation for future investigations into AI-mediated belief shaping, risk, and the evolving entanglement of LLMs with cultural practice.
Whiteboard
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
Understanding “Fortunetelling with LLMs in China”
What is this paper about?
This paper looks at a new trend in China: people asking AI chatbots (like ChatGPT or DeepSeek), called LLMs, to “tell their fortune.” Instead of visiting a human fortune-teller, users type their birth date, zodiac sign, or life questions into a chatbot and get predictions or advice. The researchers wanted to understand how people use this, how they feel about it, and whether it actually changes their beliefs or decisions.
What questions were the researchers trying to answer?
They focused on three simple questions:
- How do people use AI for fortunetelling? What topics do they ask about and how do they write their prompts?
- How do people feel about the results? Do they trust them? Do the results feel useful?
- Do these AI “readings” change what people believe or how they make decisions?
How did they study it?
The researchers used two main approaches. Think of it as “watch and ask.”
- Watch: They collected and carefully read 1,045 public posts from two big Chinese social media platforms—Xiaohongshu (also called Rednote) and Weibo—where people shared AI fortunetelling prompts, results, and reactions. They sorted these posts into categories (like sorting laundry by color): topics asked (e.g., romance, career), feelings (e.g., surprise, worry), and attitudes (e.g., trust, doubt).
- Ask: They interviewed 20 people who had used AI for fortunetelling. In these one-hour online chats, people talked about why they tried it, what they thought of the results, and whether it affected their choices. Each person also did a live “trial” of AI fortunetelling during the interview and talked out loud about what they were thinking. Before and after, they rated their confidence in a future outcome (like getting into a program or changing jobs), so the researchers could see if the reading shifted their confidence.
Key idea explained simply:
- “Coding” posts = sorting comments into clear buckets so patterns are easier to see.
- “LLMs” = very smart text systems that can act like a fortune-teller if you ask them to play that role.
What did they find?
Here are the main takeaways in plain language.
- What people asked about:
- Most questions were about romance and relationships, careers and jobs, and “general fortune” (overall life luck).
- Others asked about school/exams, money luck, health, friendships, and even finding lost items.
- Many prompts blended modern life with traditional Chinese systems like Bazi or Ziwei Doushu, telling the AI to “act like a fortune-telling master” and feeding it birth details.
- How people felt about the results:
- Many users reacted with surprise when a prediction seemed to match reality.
- Some used humor to keep it light, and some felt hopeful or encouraged.
- People’s attitudes were mostly positive or neutral, but skepticism was common. Many wondered if the AI was just saying nice, vague things to make them feel good.
- Did people trust it and find it useful?
- Some users said the readings felt accurate or motivating.
- Others doubted the accuracy, especially when details were wrong or overly general.
- Many found it practically helpful for small things (like reassurance before an exam or even hints to find lost items), but they didn’t treat it like hard facts.
- Why people used it:
- Decision support and reassurance: to feel calmer before making a choice (like quitting a job or applying for school).
- Emotional comfort: to feel understood during stress, breakups, or conflicts.
- Entertainment/curiosity: for fun or to share with friends.
- Social substitute: as a stand-in for talking things out with someone.
- Did it change beliefs or decisions?
- It mostly changed how people felt, not what they ultimately did.
- Think of it like getting a pep talk: it could boost confidence or reduce anxiety, and sometimes it nudged the timing of a choice (like “wait a bit longer before quitting”), but it rarely flipped the decision itself.
- People often picked out parts of the reading that matched what they already believed or hoped for. This is a common human habit:
- Confirmation bias: we notice what supports our current views.
- Barnum effect: we feel that general, positive statements fit us personally.
- Motivated reasoning: we prefer interpretations that make us feel better.
- How people handled the readings:
- Users didn’t just accept everything. They filtered, translated, and adapted advice into simple actions like “sleep earlier,” “save money,” or “give it a few weeks.” They used it as guidance, not commands.
Why does this matter?
- The good side:
- AI fortunetelling can help people manage stress, reflect on their lives, and feel less alone when making tough choices. It’s low-cost, private, and easy to access.
- The risky side:
- Over-reliance could lead to poor choices, especially in money, school, or career.
- Privacy concerns: people sometimes share very personal details.
- Vague or flattering answers can feel true even when they’re not, which may quietly shape beliefs over time.
What’s the big picture?
This research suggests that AI “fortune-telling” is less about predicting the future and more about helping people handle uncertainty. On social media, these AI-generated readings become shared stories that can shape how people talk about luck, plans, and worries. Used carefully, they can offer comfort and reflection. But they shouldn’t replace real advice, solid information, or personal judgment—especially for high-stakes choices.
In short: Treat AI fortunetelling like a mood-helper or a brainstorming buddy, not a crystal ball.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a concise, actionable list of what remains missing, uncertain, or unexplored in the paper and should guide future research.
- Generalizability beyond China is unknown; cross-cultural replications (e.g., Western astrology/tarot communities) are needed to test whether findings hold across cultures and belief systems.
- Platform scope is narrow (Rednote/Xiaohongshu and Weibo); usage on other major ecosystems (WeChat, Douyin/TikTok, Bilibili, Zhihu) and offline contexts is unexamined, as are platform-norm effects on practices and perceptions.
- Sampling transparency and representativeness are limited: the exact keyword set, data collection timeframe, and search ranking biases (top-200 posts per keyword) may skew the sample; reproducible, random, time-stratified sampling is needed.
- Visibility/positivity bias likely inflates favorable anecdotes (more “hits” than “misses” may be shared); quantify selection bias and compare posted experiences with private logs.
- The multimodal nature of posts (images/videos) constrained automation; scaling requires OCR/ASR and vision–language pipelines to systematically extract and analyze content at scale.
- Method inconsistency: the paper mentions “trained classifiers” but ultimately relies on full manual coding; clarify methods and, if used, report classifier performance or else remove claims; future work should build and validate supervised models for taxonomy detection.
- Interview sample is small (n=20), convenience-based, and gender-skewed (15/20 female); larger, demographically representative surveys and interviews are needed to assess heterogeneity and improve external validity.
- Only short-term, self-reported confidence shifts are measured; longitudinal panel studies should track repeated use, belief trajectories, and actual behavioral outcomes over weeks/months.
- No causal identification of LLM influence on decisions; run controlled experiments manipulating response valence, specificity, disclaimers, refusals, and timing to assess causal effects on choices and action timing.
- Behavioral impacts are not objectively verified; complement self-reports with behavioral logging (e.g., job changes, investment actions), administrative data where possible, or pre-registered outcome tracking.
- No baseline comparisons; directly compare LLM fortunetelling against human fortunetellers, random advice, and standard decision aids to quantify differential persuasiveness and harm/benefit profiles.
- Model-level differences are unexplored; systematically compare outputs and impacts across LLMs (e.g., DeepSeek, GPT, Gemini), languages, and safety layers to identify which configurations amplify or mitigate risks.
- Predictive validity is not assessed; build datasets of time-bound, falsifiable predictions and evaluate accuracy against chance and expert baselines; audit correctness of metaphysical computations (e.g., Bazi charting).
- Analyses rely on user-reported summaries of LLM outputs; collect and analyze a corpus of raw prompts and model responses to quantify vagueness, Barnum-like generalities, sentiment, personalization, and specificity.
- Cognitive mechanisms (confirmation bias, motivated reasoning, Barnum effect) are posited but not measured; use psychometric instruments (e.g., need for closure, suggestibility, locus of control) to model who is most susceptible and under what conditions.
- Heterogeneity in effects is untested; examine differences by demographics, religiosity/spirituality, socioeconomic status, prior fortunetelling experience, and current stress/anxiety levels.
- Topic-specific impacts (romance vs. career vs. health vs. wealth) are not compared; analyze whether certain domains show stronger belief shifts, emotional reactions, or behavioral adjustments.
- Risk characterization is incomplete; systematically document adverse outcomes (e.g., financial loss, delayed medical care, relationship strain) and identify vulnerable subgroups most at risk of harm.
- Privacy and security practices around sharing sensitive data (birth date/time, relationship details) are not empirically examined; quantify prevalence, user awareness, and leakage risks across platforms.
- Social media dynamics are unaddressed; analyze comments, thread trajectories, virality, and recommender amplification to understand how LLM fortunetelling content diffuses and normalizes uncertainty management.
- Commercialization and creator ecosystems (prompt sellers, influencer “AI fortunetellers”) are not studied; map incentives, business models, and their effects on content quality and user reliance.
- Regulatory and ethical interventions are not evaluated; test the effectiveness of disclosures, uncertainty displays, harm warnings, and platform policies on reducing overreliance without negating emotional benefits.
- Design interventions are untested; prototype and A/B test interfaces that calibrate confidence, surface counterfactuals, offer evidence-based alternatives, or scaffold reflective practices.
- Fidelity to Chinese metaphysical systems is unclear; audit how accurately LLMs implement Bazi, Ziwei Doushu, and other methods, and examine user reactions to detectable errors vs. “plausible” outputs.
- Temporal dynamics are unstudied; track how model updates, social events, and media coverage shift practices, perceptions, and the prevalence of fortunetelling prompts over time.
- Performative/ironic posting is not disentangled from sincere belief; develop annotation protocols or linguistic markers to estimate sincerity and its impact on inferred attitudes and behaviors.
- Lack of triangulation with real usage logs; collaborations with LLM providers could enable anonymized analysis of in-the-wild prompting patterns, repeat engagement, and abandonment/retention trajectories.
Practical Applications
Immediate Applications
Below are applications that can be piloted or deployed now, grounded in the paper’s findings that LLM fortunetelling is used primarily for emotional support, light decision scaffolding, and entertainment; that users actively filter outputs; and that motivated reasoning, confirmation bias, and the Barnum effect shape interpretation.
Industry and Product Design
- Emotion-first “reflection mode” in consumer chat apps (software; mental health-adjacent)
- Productization: a configurable chat mode that emphasizes emotional validation, reflective questions, and coping strategies rather than concrete predictions; includes tone controls (neutral vs. encouraging), supportive reframes, and journaling prompts aligned with users’ stated concerns (romance, career, academics).
- Why now: users seek emotional comfort and subtle reframing more than accuracy; most effects are on mindset and timing, not direction of decisions.
- Assumptions/dependencies: clear entertainment/support disclaimers; crisis-escalation handoffs; avoid clinical claims; culturally localized phrasing (e.g., Bazi/astrology references in China).
- Decision-timing scaffolds instead of directives (software; productivity)
- Productization: when users ask predictive questions, provide “hold” or “revisit” workflows (schedule a revisit date, checklist to gather more evidence, pros/cons capture) rather than definitive prescriptions.
- Why now: the study shows LLM outputs often influence timing rather than direction; scaffolding that strength makes use more responsible.
- Assumptions/dependencies: calendar/task manager integrations; consent for reminders; UI cues that avoid implying certainty.
- Sensitive-topic guardrails with bias awareness (software; platform safety)
- Productization: topic detectors that route fortunetelling-like prompts in high-stakes domains (e.g., divorce, resignation, fertility, finance) to safer templates: balanced framing, alternatives, disclaimers, and links to credible resources.
- Why now: users recognize “catering” tendencies and still appreciate neutrality in high-stakes contexts.
- Assumptions/dependencies: intent classification; region-specific resource directories; content policy alignment.
- “Entertainment label + neutral baseline” responses (social platforms; content UX)
- Workflow: automatically label fortunetelling content as “entertainment/not factual,” offer optional neutral reinterpretations (what the statement could mean in practical terms), and show a “bias tip” (e.g., Barnum effect explainer).
- Why now: positive attitudes coexist with skepticism; light-touch literacy nudges reduce over-reliance without harming enjoyment.
- Assumptions/dependencies: UI real estate; A/B testing for acceptability; minimal friction to avoid user drop-off.
- Safe prompt kits and output templates (software; creator tools; education)
- Productization: vetted “prompt packs” that structure role-play (e.g., “heritage-informed advisor”) plus safe output formats: multiple interpretations, confidence bands, suggested self-checks, and next-step lists.
- Why now: users already share structured prompts; providing safe defaults reduces risky framing.
- Assumptions/dependencies: localization to Chinese metaphysical systems (Bazi, Ziwei), while preventing false claims of accuracy.
- Lost-item heuristic assistant (consumer utility; smart home)
- Productization: a “Where did I put it?” assistant that elicits context (last locations, routines) and generates search heuristics; optionally integrates with home IoT logs.
- Why now: users already report practical success in locating items; this reframes “fortune” as structured recall.
- Assumptions/dependencies: privacy-preserving local context storage; clear non-predictive positioning.
- Workplace wellbeing micro-coach (HR; enterprise software)
- Productization: opt-in bot that provides non-predictive reflection before tough choices (resignation, transfers), focusing on values clarification, evidence gathering, and timing plans.
- Why now: participants reported turning to LLMs before job decisions; modest, structured support fits observed usage.
- Assumptions/dependencies: HR/legal review; privacy by design; employee consent.
Academia and Research Practice
- Reusable taxonomy and codebook for LLM-mediated belief/decision studies (HCI; social computing)
- Workflow: release and adapt the paper’s taxonomy (topics, attitudes, credibility/practicality, emotions, prompt structures) as a coding guide for cross-domain studies.
- Why now: immediate utility for replication across cultures/platforms.
- Assumptions/dependencies: IRB/ethics approvals; platform data access policies.
- Bias-awareness micro-lessons embedded in coursework (education; media literacy)
- Productization: short, scenario-based modules demonstrating Barnum effect, motivated reasoning, and confirmation bias in AI interactions.
- Why now: users’ interpretations are bias-laden; literacy can be taught with realistic cases.
- Assumptions/dependencies: curriculum alignment; culturally relevant examples.
Policy and Platform Governance
- Content labeling and age-appropriate presentation for “AI fortunetelling” (platform policy; consumer protection)
- Policy: require clear “entertainment only” labels, discourage claims of accuracy, and provide easy access to help resources for distress-triggering topics.
- Why now: low cost, aligns with how users already treat it; reduces harm.
- Assumptions/dependencies: alignment with Chinese and platform regulations; enforcement tooling.
- Lightweight monitoring for risk signals with privacy safeguards (trust & safety)
- Workflow: detect surges in high-stakes predictive posts (e.g., divorce, self-harm-adjacent language) and inject supportive resources, not punitive moderation.
- Why now: observed worries/negative affect in a minority; targeted support can mitigate risk.
- Assumptions/dependencies: robust false-positive management; transparent data handling.
Daily Life and Civil Society
- Self-reflection routines that externalize “predictions” into plans (personal productivity)
- Practice: translate generalized outputs into concrete, low-risk habits (sleep, study cadence, job-search steps), then schedule follow-ups.
- Why now: users already reinterpret vagueness into actionable prompts; formalizing this increases benefit.
- Assumptions/dependencies: access to planners/calendars; willingness to log actions.
- Community workshops on “AI, uncertainty, and meaning-making” (public libraries; NGOs)
- Program: guided activities on interpreting AI advice, recognizing cognitive biases, and turning uncertainty into reflective practice rather than fate claims.
- Why now: fortunetelling is culturally resonant; workshops meet people where they are.
- Assumptions/dependencies: trained facilitators; inclusive materials.
Long-Term Applications
These opportunities require further research, scaling, validation, or regulatory development. They leverage the study’s insights on subtle belief shifts, emotion regulation roles, and culturally embedded practices.
Industry and Product Design
- Belief-calibration companions with outcome tracking (software; personal informatics)
- Productization: opt-in tools that log predictions, user interpretations, subsequent outcomes, and perceived usefulness to help users learn their own bias patterns and calibrate confidence over time.
- Dependency/assumptions: longitudinal consented data; careful UX to avoid surveillance feel; privacy-preserving analytics.
- Counter-bias conversational strategies (AI safety; model training)
- Productization: models tuned to detect motivated reasoning and gently surface counterfactuals, alternative framings, and unknowns without adversarial tone.
- Dependency/assumptions: new evaluation metrics for “supportive challenge”; guardrails to avoid perceived moralizing.
- Culture-aware “ritual-safe” response libraries (localization; content strategy)
- Productization: curated, culturally grounded scripts that acknowledge traditional systems (Bazi, Ziwei) while consistently reframing toward practical, non-factual guidance.
- Dependency/assumptions: co-design with cultural experts; regulatory clarity on acceptable references.
- Multi-domain “high-stakes deflection” protocols (finance, healthcare, legal)
- Productization: standardized handoffs from fortunetelling-style prompts to neutral checklists and verified resources in regulated domains (e.g., investment basics, fertility counseling directories).
- Dependency/assumptions: partnerships with licensed providers; compliance mapping; liability frameworks.
Academia and Research Practice
- Longitudinal causal studies on belief/behavior drift (HCI; psychology)
- Program: track how repeated AI fortunetelling affects confidence, risk-taking, and decision timing across months; test interventions (labels, counter-bias prompts).
- Dependency/assumptions: IRB approvals; participant retention; cross-cultural samples.
- Safety/ethics benchmarks for affective conversational AI (evaluation science)
- Deliverable: metrics for “emotional benefit without false belief amplification,” “timing influence vs. direction change,” and “user-perceived credibility vs. factuality.”
- Dependency/assumptions: shared datasets; community consensus on acceptable risk thresholds.
- Cross-cultural comparative frameworks for “AI-mediated meaning-making” (anthropology; media studies)
- Program: replicate methods across regions to map how local practices shape AI use in uncertainty management.
- Dependency/assumptions: multilingual teams; platform access; contextual ethics.
Policy and Platform Governance
- Regulatory category for “AI spiritual/entertainment advice” (consumer protection; standards)
- Framework: transparency and labeling requirements, prohibitions on accuracy claims, data minimization for sensitive birth data, age gating, and geofenced rules for high-stakes topics.
- Dependency/assumptions: coordination with Chinese regulators and platform policies; industry compliance incentives.
- Auditable “prediction-claim” controls in high-risk domains (platform governance)
- Policy/tooling: enforce response templates that avoid deterministic claims when prompts touch finance, medical, legal, or minors’ education decisions; maintain audit logs for compliance.
- Dependency/assumptions: robust topic classification; oversight bodies; appeal mechanisms.
- Privacy standards for metaphysical identifiers (data protection)
- Framework: treat birth datetime, birthplace, zodiac-style metadata as sensitive; mandate explicit consent, local storage, and purpose limitation.
- Dependency/assumptions: harmonization with existing data laws; developer guidance.
Daily Life and Civil Society
- Public “AI and uncertainty” literacy campaigns (education; public health)
- Program: sustained messaging on the Barnum effect, motivated reasoning, and safe use of AI for emotional support; include youth-focused materials.
- Dependency/assumptions: cross-sector funding; evaluation of impact.
- Community-led archives of digital spirituality practices (cultural heritage)
- Initiative: document and study how AI reshapes traditional practices, preserving diversity while informing policy and design.
- Dependency/assumptions: ethical archiving; participant consent; platform cooperation.
Notes on Feasibility and Dependencies Across Applications
- Generalizability: findings are grounded in Chinese platforms and cultural practices; adaptations are needed for other locales.
- Safety and ethics: avoid clinical, legal, or financial claims; build clear handoffs to licensed professionals where needed.
- User acceptance: labels and guardrails must preserve enjoyment; iterative UX testing is crucial.
- Data and privacy: minimize collection; obtain explicit consent for any longitudinal tracking; employ privacy-preserving analytics.
- Evaluation: prioritize metrics that capture emotional benefit, bias mitigation, and non-harmful influence on decisions (especially timing vs. direction).
Glossary
- AI-mediated decision-making: Decision processes that are shaped or influenced by AI systems, outputs, or interfaces. "Investigating these practices can therefore deepen our understanding of AI-mediated decision-making and belief formation in self-referential contexts."
- Barnum effect: A cognitive bias where people accept vague, general statements as highly accurate for themselves. "the Barnum effect, where users tend to interpret vague or personalized outputs in ways that reinforce their prior beliefs, expectations, and emotional needs"
- Bazi: A traditional Chinese destiny analysis system (Four Pillars) based on birth year, month, day, and hour. "It even got my Bazi wrong."
- Character settings: Prompt-engineering technique where an LLM is instructed to assume a specific persona or role. "including common character settings, output format, and so on"
- Cohen's Kappa: A statistic that measures inter-annotator agreement corrected for chance. "we calculated inter-coder agreement for each code using Cohen's Kappa, resulting in an overall mean of 0.845 (SD = 0.095)."
- Confirmation bias: The tendency to seek, interpret, and remember information that confirms one’s preexisting beliefs. "motivated reasoning, confirmation bias, and the Barnum effect"
- Descriptive content analysis: A qualitative method for systematically coding and summarizing themes and patterns in text or media. "This section presents {descriptive content analysis} of posts in social media to gain a comprehensive understanding"
- Digital spirituality: The study or practice of spiritual beliefs and experiences as mediated by digital platforms and media. "Recent studies on digital spirituality further show that social media enables the creation and circulation of symbolic meaning."
- Divination-like outputs: Model responses styled to resemble fortune-telling or divination practices. "Related research has explored models optimized for divination-like outputs"
- Four Pillars: Another name for Bazi; a Chinese astrological system using four birth-time components to infer fate. "with Bazi/Four Pillars appearing in 215 posts (20.57\% of all posts; 65.95\% among posts that explicitly mention a method)"
- Ground-truth data: Verified, objective reference data used to train or evaluate AI systems. "previous AI-powered decision-making support tools usually rely on ground-truth data or models trained on it."
- Hallucinations: Fabricated or incorrect outputs produced by an LLM that appear plausible. "LLM fortunetelling inherits challenges of LLMs such as hallucinations"
- ICWSM: The International AAAI Conference on Web and Social Media, a research venue focused on computational social science. "prior ICWSM work has already raised concerns about LLMs in sensitive advice settings (e.g., relationship advice)"
- Inter-coder agreement: The degree of consistency among different human annotators applying the same codebook. "we calculated inter-coder agreement for each code using Cohen's Kappa"
- Knowledge recall: Prompting strategy that cues the model to retrieve or cite domain knowledge, texts, or methods. "and knowledge recall (15.22\%), citing classic texts or traditional methods."
- Liuyao: A traditional Chinese divination method (Six Lines) used for prognostication. "other systems such as Liuyao (3.25\%)."
- Multimodal conversational agents: Dialogue agents that process and generate across multiple modalities (e.g., text, images, audio). "human-computer interaction researchers have developed multimodal conversational agents inspired by shamanic symbolism to study symbolic AI interaction"
- Motivated reasoning: Reasoning biased by one’s desires or goals, leading to preferential interpretation of information. "motivated reasoning, confirmation bias, and the Barnum effect"
- Quark Gaokao: An AI tool supporting Chinese college admissions planning in the context of the Gaokao exam. "A relevant example is research on Quark Gaokao"
- Role-playing (LLMs): Instructing an LLM to act as a specific expert or persona to shape its responses. "combining strategies of role-playing LLMs with information needed in traditional fortunetelling practices."
- Sacred technologies: Platforms framed as spiritually significant tools that reinforce spiritual narratives. "while platforms such as TikTok have been described as ``sacred technologies'' that reinforce spiritual narratives"
- Self-referential contexts: Settings where content or decisions directly concern the user’s own life, beliefs, or choices. "Investigating these practices can therefore deepen our understanding of AI-mediated decision-making and belief formation in self-referential contexts."
- Semi-structured interviews: Interviews guided by a flexible protocol, allowing follow-ups while covering core topics. "we further conducted semi-structured interviews with 20 users of LLM-based fortunetelling"
- Shamanic symbolism: Design elements inspired by shamanic cultural symbols used to explore meaning-making with AI. "inspired by shamanic symbolism"
- Stratified analysis: Analytical approach that segments data into subgroups (strata) to compare patterns across them. "accounting for platform-level differences through stratified analysis (see \autoref{tab:rq1_taxonomy} and \autoref{tab:rq2_taxonomy})."
- Taxonomy: A structured classification scheme that organizes concepts into categories and subcategories. "yielding a comprehensive taxonomy of the diverse foretold topics (e.g., career, romance), emotion reactions (e.g., surprise, worry), and perceived credibility (e.g., doubt, trust) of LLM fortunetelling."
- Thematic analysis: A qualitative method for identifying and interpreting patterns (themes) across textual data. "and conducted a thematic analysis."
- Think-Aloud: A research method where participants verbalize their thoughts while performing a task. "Think-Aloud in a Trial of LLM Fortunetelling"
- Ziwei Doushu (Purple Star Astrology): A Chinese astrological system that interprets fate using star positions and charts. "metaphysical systems like Bazi and Ziwei Doushu (Purple Star Astrology)"
Collections
Sign up for free to add this paper to one or more collections.
