Consciousness, AI, and the Limits of Scientific Explanation
Abstract: Science is constitutively third-personal: its findings are in principle reproducible by any observer, independent of perspective, and answerable to measurement. This is the source of its power and also its limit when it comes to phenomena that are first-personal. While it is obvious that a science of the Meaning of Life is unattainable, researchers have not drawn the same conclusion for consciousness -- in its phenomenal dimension, the qualia of seeing red, of feeling pain, of being anything at all. I argue they should. The hard problem of consciousness is not a scientific problem awaiting better tools or a more ambitious theory, but a category error. The same structural problem applies to machine consciousness: neither attribution nor denial is scientifically adjudicable. I situate science within a broader ecology of understanding and argue that a unified framework that addresses both the objective and the subjective may be unattainable.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Explain it Like I'm 14
A Simple Explanation of “Consciousness, AI, and the Limits of Scientific Explanation”
What the paper is about
This paper asks a big question: Can science fully explain what it feels like to be conscious? The author’s main point is no. Science is great at studying things we can measure from the outside, but the “feel” of an experience from the inside (like the redness of red or the hurt of pain) is a different kind of thing. The paper argues that mixing these two kinds of questions is like using the wrong tool for the job.
The key questions, in everyday terms
- What can science, which studies the world from an outside view, say about conscious experience, which is felt from the inside?
- Why is the “hard problem of consciousness” (explaining why brain activity comes with an inner feeling at all) so stubborn?
- Can we decide, using scientific tests, whether a machine or a brain-in-a-dish is conscious?
- Are there better ways to think about these questions without asking science to do what it can’t?
How the author approaches the problem
The paper is a careful argument, not a lab experiment. It uses clear definitions, examples, and thought experiments.
- First-person vs. third-person: Third-person means an outside, measurable view (like a camera recording a soccer game). First-person means the inside view (what it feels like to be the player in the game). Science is built for third-person views.
- The “Meaning of Life test”: If a claim about “consciousness” still makes sense when you swap in “Meaning of Life,” then you might be asking science to answer a question it can’t. For example, “We can measure the Meaning of Life with a number” sounds silly—and so, the author says, does “We can measure consciousness with a number.”
- The paper examines popular strategies:
- Deflationary view (Dennett): says our idea of raw inner feelings (“qualia”) is confused. The author replies that this denies what seems most obvious to us—our own experience.
- Measuring consciousness (IIT and the number Φ): says consciousness is “integrated information.” The author argues that turning a first-person feeling into a third-person number can’t be checked against the thing that matters—the inner feel itself.
- Panpsychism (Chalmers): says experience is a basic part of reality. The author asks how we could ever test such laws or combine tiny experiences into a person’s rich experience, from a scientific standpoint.
- Classic thought experiments:
- “What is it like to be a bat?” We can learn everything about a bat’s brain and sonar, but we still won’t know what it feels like to be a bat from the inside.
- The “Chinese Room”: A system can shuffle symbols and produce the right answers without understanding. This shows that correct behavior (third-person) doesn’t guarantee inner understanding (first-person).
What the paper finds and why it matters
The paper makes four main claims:
- Science is an “outside” tool. It’s designed to give results anyone could check, no matter who they are. That’s its power.
- Conscious experience is an “inside” fact. It’s about what it feels like to be you.
- There’s a structural gap. No amount of outside facts automatically gives you the inside feel. That’s why the “hard problem” isn’t just hard—it’s the wrong kind of question for science to solve.
- The same goes for machine consciousness. No brain scan, behavior test, or math formula can settle whether an AI or a neuron dish “feels like something from the inside.” Both saying “yes” and saying “no” go beyond what science can prove.
Why this matters:
- It warns against “scientism,” the idea that science can answer every kind of question. Respecting science’s limits protects its strengths.
- It helps focus research on what science can do very well: finding neural correlates of consciousness, studying anesthesia, modeling attention and memory, and building useful AI—without promising answers science can’t deliver.
- It clarifies public debates about AI sentience so we don’t mistake convincing behavior for proof of inner experience.
What this means for the future
The author suggests we should see science as one way of understanding the world, alongside others like art, ethics, religion, and careful first-person reflection. We might never get one “master method” that covers both outer facts and inner feelings. Just as some math truths can’t be proved inside a single system, understanding may also have built-in limits.
In practice:
- Neuroscience and AI research can keep making real progress on the “easy problems” (how attention works, how information is integrated, when consciousness fades under anesthesia).
- We should not expect a scientific formula to tell us “what it is like” from the inside or to declare definitively that a machine is (or isn’t) conscious.
- Conversations about meaning, value, and experience remain important, but they belong to different kinds of inquiry than measurement-based science.
In short: Science is a powerful camera pointed at the world. It can show us an incredible amount. But it can’t step into the photo and become the person having the experience. Recognizing that boundary helps us do better science—and have more honest, thoughtful discussions about minds, both human and machine.
Knowledge Gaps
Knowledge gaps, limitations, and open questions
Below is a consolidated list of concrete gaps and unresolved questions that the paper leaves open for future researchers to address.
- Demarcation criterion: Provide a precise and operational criterion for when a question about mind is “constitutively first-personal” versus legitimately third-personal, beyond illustrative examples and intuitions.
- Formal no-go result: Develop a rigorous impossibility theorem (or identify its limits) showing that third-person observational data cannot, even in principle, determine first-person facts about phenomenal consciousness.
- Scope conditions: Specify whether the “in-principle unavailability” claim applies to all aspects of phenomenality or only to some (e.g., does it block only qualia identity, or also any coarse-grained mapping between neural states and phenomenal categories?).
- Ontological vs explanatory gap: Clarify whether the argument targets an explanatory gap (epistemic) or an ontological gap (metaphysical), and formalize the modal assumptions required for either claim.
- Dependence on a particular view of science: Test the argument’s robustness under alternative conceptions of science (e.g., pragmatism, constructive empiricism, perspectival realism, process metaphysics). Would the conclusion still hold if realism or strict physicalism is relaxed?
- Intersubjective first-person methods: Engage systematically with neurophenomenology, trained introspection, second-person methods, and heterophenomenology to assess whether any intersubjective protocols could narrow the gap without collapsing into “creeping scientism.”
- Standards of rigor outside science: Articulate explicit epistemic standards (replicability, calibration, training, error models) for non-scientific or extra-scientific inquiry into consciousness so that cumulative progress is possible.
- Translation constraints: Propose and evaluate formal translation rules or invariants (if any) that could relate subjective predicates to objective descriptions, and specify why such rules must fail or how far they can succeed.
- Partial-bridge hypotheses: Investigate whether dual-aspect monism, neutral monism, structural realism, or indexical/centered-worlds frameworks can yield limited, testable bridge principles without claiming full reduction.
- Relational/perspectival physics: Provide a deeper analysis of whether perspectival or agent-centered formalisms in physics (e.g., QBism, relational QM) might supply meta-level resources for integrating first- and third-person claims, beyond the brief dismissal offered.
- Illusionism and representationalism: Address, in detail, whether robust illusionist or representationalist accounts dissolve the hard problem without denying experience, and what decisive considerations would arbitrate among these positions.
- IIT critique generality: Move beyond the IIT case and argue (or prove) more generally why any third-person scalar or functional invariant must fail as a measure of consciousness, specifying minimal assumptions and edge cases.
- Combination and decomposition problems: For panpsychism and related views, identify concrete empirical signatures or decision procedures (even indirect) that could differentially support or undermine candidate psychophysical laws.
- Machine consciousness governance: Provide actionable policy frameworks (decision-theoretic, precautionary, evidential) for allocating moral status and risk mitigation under acknowledged epistemic undecidability about AI consciousness.
- Clinical decision-making under uncertainty: Specify protocols for disorders of consciousness, anesthesia awareness, and neonatal or non-verbal patients when first-person facts are (allegedly) in-principle inaccessible, including error costs and default rules.
- Criteria for moral patienthood: Offer non-consciousness-centric criteria (e.g., welfare proxies, capacity for interests, vulnerability) or a hierarchy of proxies to guide ethical treatment when consciousness cannot be adjudicated scientifically.
- Abductive constraints: Analyze whether and how inference to the best explanation could legitimately support consciousness attributions (human, animal, machine) under strict third-person evidence constraints, or why such abduction must fail.
- Self-report status: Clarify the normative role of self-reports (human and machine) under the thesis that first-person evidence is incommunicable; determine what, if anything, reports license in scientific or ethical contexts.
- Meaning-of-Life test validation: Empirically and analytically evaluate the “Meaning of Life test” as a diagnostic of scientism; specify false positive/negative rates and boundary conditions for its appropriate use.
- Ecology of understanding: Provide a worked taxonomy of “systems of understanding” (science, art, religion, mathematics, phenomenology), with interfaces, conflict-resolution rules, and case studies showing how they can coordinate without a master framework.
- Success criteria for neuroscience: State positive research goals, success metrics, and claim boundaries for neuroscience of consciousness that respect the paper’s limits while still enabling cumulative, action-guiding progress.
- Chinese Room conflation risks: Disentangle understanding/semantics from phenomenality in the Chinese Room discussion; identify which parts of the argument depend on which distinction and how conclusions change under different separations.
- Indexicality and self-location: Explore whether formal treatments of indexical information and self-locating uncertainty (centered worlds, anthropic reasoning) can model anything essential about first-person facts without overstepping scientific constraints.
- Meta-epistemic governance: Propose institutional and peer-review guidelines that prevent overclaiming about consciousness while encouraging research on correlates, functions, and clinical tools.
- Boundary-testing case studies: Identify concrete empirical domains (e.g., dreams, psychedelics, locked-in syndrome, split-brain) to stress-test the paper’s claims and to delineate which questions remain scientifically tractable and which do not.
Practical Applications
Overview
The paper argues that phenomenal consciousness (what it is like to have experiences) is constitutively first-personal and therefore not scientifically adjudicable from a third-person perspective. It recommends focusing scientific work on tractable “easy problems” (mechanisms of attention, reportability, neural correlates, anesthesia), cautions against measuring or legislating “consciousness,” and proposes a pluralistic “ecology of understanding.” Below are practical applications that follow from these claims.
Immediate Applications
- Research prioritization and funding criteria — Sectors: academia, neuroscience, AI
- What: Reframe calls, proposals, and peer review to target mechanistic, third-person questions (attention, reportability, neural correlates, anesthesia) rather than asserting solutions to phenomenal consciousness.
- Tools/workflows: Grant and journal checklists that flag claims failing the “Meaning of Life test”; reviewer rubrics that distinguish “capacity explanations” from “qualia claims.”
- Assumptions/dependencies: Funders and journals accept the first-/third-person distinction as policy; community norms shift away from overclaiming.
- Measurement and benchmarking standards — Sectors: AI/ML, software, robotics
- What: Replace “sentience/consciousness tests” with capability- and impact-based evaluation (e.g., honesty, autonomy, situational awareness, robustness, deception).
- Tools/workflows: Model/Robot cards include an explicit “Consciousness: non-adjudicable by scientific evidence” section; capability scorecards tied to safety and governance.
- Assumptions/dependencies: Standards bodies (e.g., NIST, ISO) and labs align incentives to prioritize measurable functional risks over metaphysical claims.
- Product UX to calibrate anthropomorphism — Sectors: software, robotics, education, healthcare, finance
- What: Reduce unwarranted attributions of sentience in chatbots and social robots where harmful; provide transparent persona settings and in-UI reminders that coherence is not evidence of consciousness.
- Tools/workflows: “Anthropomorphism risk score” integrated into design reviews and A/B tests; style guides limiting first-person declarations in sensitive domains (clinical, financial).
- Assumptions/dependencies: Acceptable trade-offs between user engagement and accurate mental models; product teams adopt calibration over anthropomorphic framing.
- Marketing and disclosure policies — Sectors: technology, consumer protection, media
- What: Prohibit or heavily qualify “conscious”/“sentient” claims in ads, press releases, and documentation; require explicit disclaimers when language might mislead.
- Tools/workflows: Legal/compliance “no-sentience claims” policy templates; newsroom style guides avoiding headlines that infer consciousness from behavior or Φ-like metrics.
- Assumptions/dependencies: Regulators and platforms enforce truth-in-advertising standards; companies accept reputational risk of overclaiming.
- Clinical practice for disorders of consciousness (DoC) — Sectors: healthcare
- What: Base diagnosis, prognosis, and decisions on observable behavioral and physiological markers (e.g., EEG/fMRI paradigms), not assertions about phenomenal experience.
- Tools/workflows: Protocols and consent documents that communicate evidential limits; decision-support checklists emphasizing uncertainty and surrogate values rather than “is the patient conscious?”
- Assumptions/dependencies: Professional societies update guidelines; clinicians trained to communicate first-/third-person limits clearly to families.
- Organoid and animal research oversight — Sectors: biotech, ethics review
- What: Govern research using observable welfare proxies (nociceptive circuitry, stress markers, complex learning) rather than declaring “sentience.”
- Tools/workflows: IRB/IACUC addenda with functional-threshold triggers for enhanced monitoring; precautionary principle framed around measurable signals.
- Assumptions/dependencies: Agreement on proxy thresholds; evolving science of nociception and integrative function.
- Policy and law decoupled from consciousness attributions — Sectors: public policy, AI governance, law
- What: Draft laws and standards that regulate AI by behavior, capability, and impact (safety, accountability, rights-of-users) without relying on consciousness determinations.
- Tools/workflows: Template statutory language (“No regulatory obligations shall hinge on adjudications of machine consciousness.”); impact-based risk tiers.
- Assumptions/dependencies: Legislative appetite for pragmatic, observable criteria; courts accept non-metaphysical bases for rights/duties.
- Education and reviewer training on “scientism risk” — Sectors: academia, science communication
- What: Incorporate the “Meaning of Life test” as a heuristic to spot category errors in proposals, papers, and public talks.
- Tools/workflows: Short pedagogical modules in methods courses; reviewer checklists for conferences/journals.
- Assumptions/dependencies: Departments prioritize philosophy-of-science literacy; minimal added burden for reviewers.
- AI safety assessments focused on externalities — Sectors: AI safety, enterprise IT
- What: Evaluate systems for misuse, autonomy, escalation, and deception rather than “consciousness risk.”
- Tools/workflows: Red-team playbooks; impact matrices keyed to measurable behaviors; release gates tied to capability thresholds.
- Assumptions/dependencies: Org leadership aligns incentives; external auditors recognize impact-centric frameworks.
- Daily-life guidance for users — Sectors: public education, mental health
- What: Encourage users to treat AI systems as tools; provide resources to avoid unhealthy attachments based on perceived sentience.
- Tools/workflows: In-product tips; consumer-facing AI literacy materials; clinician guidance for addressing AI parasociality.
- Assumptions/dependencies: Platforms cooperate; mental health providers incorporate new media literacy.
- Communications for complex networks (e.g., power grids) — Sectors: energy, research communication
- What: Avoid implying consciousness in large interconnected systems based on integrated-information-like metrics.
- Tools/workflows: Research abstracts and public summaries with explicit caveats about interpretability limits.
- Assumptions/dependencies: Research institutions adopt communications standards; media adheres to guide rails.
Long-Term Applications
- International standards on “consciousness claims” — Sectors: standards bodies, policy, industry consortia
- What: ISO/NIST-like guidance specifying that product safety, liability, and rights should not hinge on consciousness determinations; certification for compliant disclosures.
- Tools/workflows: Consensus-driven standard; conformance audits.
- Assumptions/dependencies: Broad, cross-sector buy-in; alignment with consumer protection laws.
- Legal doctrines for AI and synthetic agents grounded in harm and interests, not consciousness — Sectors: law, policy
- What: Develop rights/duties frameworks tied to demonstrable capacities and stakeholder harms (e.g., fiduciary duties for AI advisors) rather than metaphysical status.
- Tools/workflows: Model statutes; judicial reference guides.
- Assumptions/dependencies: Jurisprudential shift; empirical research on harm and remedy effectiveness.
- Neurophenomenology 2.0 (pluralistic research programs) — Sectors: academia (cognitive science, philosophy, neuroscience)
- What: Build structured mixed-methods programs that pair first-person reports (as data about reports) with third-person measures, while explicitly refraining from claiming to solve qualia.
- Tools/workflows: Protocols that map subjective reports to task performance and neural signatures; public datasets with interpretability caveats.
- Assumptions/dependencies: Interdisciplinary funding; careful framing to avoid category errors.
- Organoid governance with dynamic functional triggers — Sectors: biotech, ethics, policy
- What: Create evolving oversight regimes where specific, validated functional markers automatically escalate review, monitoring, or limits.
- Tools/workflows: Registries; longitudinal monitoring platforms; standardized assays.
- Assumptions/dependencies: Scientific consensus on markers; regulatory capacity for adaptive governance.
- Anthropomorphism risk modeling at scale — Sectors: software, robotics, human–computer interaction
- What: Develop ML tools to detect and quantify anthropomorphic cues in interfaces and media; optimize for calibrated user beliefs.
- Tools/workflows: SDKs for cue detection; design linting in CI pipelines.
- Assumptions/dependencies: Reliable labeling of cues; industry adoption in design workflows.
- Public evaluation ecosystems for AI capabilities (not consciousness) — Sectors: AI/ML, benchmarking organizations
- What: Establish open, regularly updated capability benchmarks covering reportability, attention-like control, tool use, and self-monitoring, with standardized uncertainty reporting.
- Tools/workflows: Shared leaderboards; “capability cards” with caveated interpretability statements.
- Assumptions/dependencies: Community coordination; funding for maintenance and governance.
- Treaty-level AI governance centered on outcomes — Sectors: international policy
- What: Embed the paper’s perspective into multilateral agreements that regulate AI by safety, security, and socioeconomic impact without invoking consciousness.
- Tools/workflows: Model treaty text; compliance mechanisms tethered to auditable behaviors.
- Assumptions/dependencies: Geopolitical alignment; verification infrastructure.
- Consumer protection for affective and companion AI — Sectors: consumer electronics, mental health, child safety
- What: Regulate emotional-design features that manipulate users via perceived sentience; age-gating and clear labeling for companion bots.
- Tools/workflows: Labeling standards (“simulated persona” disclosures); third-party audits of manipulative patterns.
- Assumptions/dependencies: Evidence of harm; legislative appetite for design regulation.
- Advanced clinical decision support for brain injury — Sectors: healthcare, health IT
- What: Tools that synthesize behavioral and physiological proxies with calibrated uncertainty to support family decisions, explicitly avoiding metaphysical interpretations.
- Tools/workflows: EHR-integrated decision aids; uncertainty visualizations.
- Assumptions/dependencies: Validation datasets; clinician training and liability coverage.
- Academic norms and curricula — Sectors: education
- What: Widespread inclusion of the first-/third-person distinction and “Meaning of Life test” across STEM curricula to reduce scientism and improve public communication.
- Tools/workflows: Core modules; cross-listed courses co-taught by philosophers and scientists.
- Assumptions/dependencies: Curriculum reform cycles; faculty incentives.
Notes on feasibility:
- Many immediate applications are policy or process changes requiring institutional will, not new technology.
- Long-term items depend on consensus, empirical maturation of proxies, and international coordination.
- All applications presume acceptance of the paper’s central thesis: phenomenal consciousness is not scientifically adjudicable; practical governance should therefore focus on observable functions, impacts, and harms.
Glossary
- Anomalous monism: A view (due to Davidson) that mental events are physical but there are no strict laws reducing mental descriptions to physical ones. "Davidson's anomalous monism"
- Anosognosia: A neurological condition in which a person is unaware of their own deficits or impairments. "Neuroscientists have long questioned the unity of consciousness through the study of split-brain \citep{gazzaniga1967}, anosognosia \citep{vuilleumier2004}, and blindsight patients \citep{weiskrantz1986}."
- Blindsight: The ability to respond to visual stimuli without conscious visual experience, typically due to damage in visual cortex. "Neuroscientists have long questioned the unity of consciousness through the study of split-brain \citep{gazzaniga1967}, anosognosia \citep{vuilleumier2004}, and blindsight patients \citep{weiskrantz1986}."
- Cartesian Theater: Dennett’s metaphor for a central internal stage where experiences are observed by a homunculus; invoked to critique such models. "Inside us, there has never been a homunculus at home watching the Cartesian Theater \citep{dennett1991}."
- Category error: A mistake of applying a concept or explanatory framework to the wrong kind of thing. "The hard problem of consciousness is not a scientific problem awaiting better tools or a more ambitious theory, but a category error."
- Chinese Room: Searle’s thought experiment arguing that symbol manipulation (syntax) is not sufficient for understanding (semantics). "Consider Searle's Chinese Room thought experiment \citep{searle1980}."
- Combination problem: In panpsychism, the challenge of explaining how micro-level experiences combine into unified macroscopic consciousness. "The combination problem is at least as intractable as the original."
- Deflationary account: An approach that explains away or reduces the target phenomenon (here, qualitative experience) by denying its purported properties. "Deflationary Accounts"
- First-person perspective: The subjective, experiential point of view of a conscious subject. "the first-person perspective of the bat"
- Formal and material modes of speech: Carnap’s distinction between talking about linguistic expressions (formal) versus talking about the world (material). "Carnap's distinction between the formal and material modes of speech"
- Gödel's incompleteness theorems: Results showing that any sufficiently expressive formal system contains true statements unprovable within the system. "something akin to Gödel's incompleteness theorems applied to explanation itself"
- Homunculus: A hypothetical “little observer” inside the mind; used to criticize theories that implicitly posit an inner observer. "Inside us, there has never been a homunculus at home"
- Integrated Information Theory (IIT): A theory proposing that consciousness corresponds to integrated information measured by Φ. "Integrated Information Theory (IIT, \cite{tononi2004, tononi2016})"
- Multielectrode array: A device with many electrodes used to record from or stimulate multiple neurons simultaneously. "cultured 800,000 human cortical neurons on a multielectrode array"
- Neural correlates of consciousness: The measurable brain states reliably associated with conscious experience. "Behavioral measures, architectural analyses, information-theoretic quantities like , and the full battery of neural correlates of consciousness are all third-person instruments."
- Organoids: Lab-grown miniaturized and simplified versions of organs (e.g., brain organoids) derived from stem cells. "whether it involves patients with neurological injury, AI systems, or organoids."
- Panpsychism: The view that consciousness is a fundamental and ubiquitous feature of reality. "This leads naturally to panpsychism"
- Perspectival physics: The idea that some physical theories (e.g., relational quantum mechanics) represent states relative to other systems or perspectives. "Perspectival physics might seem to offer a way out."
- Phenomenal consciousness: The qualitative, subjective “what-it’s-like” aspect of experience. "I will argue that a science of phenomenal consciousness is wrong for exactly the same reason"
- Phenomenological inquiry: A systematic first-person method for describing and analyzing structures of experience. "including art, religion, mathematics, and phenomenological inquiry."
- Physicalist: Committed to the doctrine that everything is ultimately physical, allowing no non-natural causal entities. "It is physicalist: no ghosts, no magic, no causally efficacious entities outside the natural order."
- Psychophysical laws: Hypothetical laws that relate physical states to phenomenal (mental) states. "proposes psychophysical laws relating physical states to phenomenal states"
- Qualia: The qualitative aspects of experience (e.g., the redness of red, the painfulness of pain). "the qualia of seeing red, of feeling pain, of being anything at all."
- Relational quantum mechanics: An interpretation of quantum theory where states are defined relative to other systems. "However, the laws governing how perspectives relate in relational quantum mechanics are themselves stated from no particular perspective."
- Scientism: The claim that science can in principle explain all aspects of human experience. "Scientism --- the claim that science can in principle address all of human experience --- is not an extension of science."
- Sentience: The capacity for subjective experience or feeling. "In 2022, a Google engineer concluded that the LaMDA LLM was sentient"
- Split-brain: A condition (often after severing the corpus callosum) used to study dissociations in consciousness across brain hemispheres. "Neuroscientists have long questioned the unity of consciousness through the study of split-brain"
- Third-person perspective: The objective, observer-independent standpoint characteristic of scientific inquiry. "Science operates from a third-person perspective"
- Turing test: A behavioral test evaluating whether a machine’s responses are indistinguishable from a human’s. "more often than actual humans in a standard behavioral (i.e., Turing) test"
- View from nowhere: Nagel’s phrase for the fully objective perspective independent of any particular observer. "the view from nowhere"
- Φ (phi): IIT’s quantitative measure of integrated information intended to track the degree of consciousness. "quantified as "
Collections
Sign up for free to add this paper to one or more collections.