Self-Ask: Zero-Shot Maieutic Prompting

• Self-Ask is a prompting paradigm that employs recursive sub-question generation and abductive explanation trees to improve reasoning in large language models.
• The method structures the reasoning process into a tree with both supporting and counterfactual branches, later validated by converting nodes into a MAX-SAT problem for global consistency.
• Empirical results show up to 20-point accuracy gains over standard and chain-of-thought methods, underlining its potential in enhancing commonsense and arithmetic reasoning.

Self-Ask (Zero-Shot Maieutic Prompting) is a prompting paradigm for LLMs that structures multi-step reasoning tasks as recursive sub-question generation, answer extraction, and abductive explanation trees. The defining feature is its reliance on the model's ability to self-generate explanatory paths—often branching into both supporting and counterfactual explanations—and then select logical inferences by synthesizing these threads. Contemporary variants such as Maieutic Prompting and Hint of Thought (HoT) prompting formalize this process, often combining it with symbolic post-processing steps such as MAX-SAT to enforce global logical consistency. The approach addresses key limitations of standard and Chain-of-Thought (CoT) prompting, most notably in binary and multi-step commonsense, symbolic, or arithmetic reasoning tasks.

1. Conceptual Foundations and Context

Self-Ask and related maieutic prompting methods were introduced to mitigate the persistent issue that LLMs, when prompted in standard, zero-shot, or vanilla CoT formats, struggle with logical consistency and susceptibility to local, noisy errors in generative reasoning chains. While vanilla CoT prompting, e.g., appending “Let’s think step by step,” can elicit reasoning traces, it often produces undirected or semantically inconsistent thoughts without structure for logical verification. Program-of-Thought (PoT) prompting supplements arithmetic tasks by emitting Python code, achieving higher precision but lacking generalizability to non-quantitative domains (Lei et al., 2023).

Maieutic prompting, as in Self-Ask, frames inference as the generation of a tree of explanations: for each question or sub-statement, both a supporting (“True, because ...”) and an opposing (“False, because ...”) explanation are recursively generated, culminating in a global inference that is locally and globally consistent (Jung et al., 2022). This strategy enables the model both to thoroughly explore the hypothesis space and to filter out noise by enforcing logical self-consistency.

2. Formalism and Algorithmic Structure

Maieutic prompting renders the reasoning process as a formal explanation tree, recursively expanding nodes based on abductive prompts. The fundamental components are:

• Original Question $Q$ as root.
• Recursive child expansion: for a node $E$, generate children $E_T$ (“True, because …”) and $E_F$ (“False, because …”).
• Logical integrality test at each node: check model consistency between $E$, $\neg E$, and their recursively generated children; prune leaves not meeting integrity thresholds.
• Translation of all surviving nodes and their relations into Boolean clauses, constructing a (weighted) MAX-SAT instance:

$$\max_{\mathbf{x}: V \to \{0,1\}} \sum_{c \in \mathcal{C}} w_c \cdot \mathbf{1}\{c \text{ is true under } \mathbf{x}\}$$

where $\mathcal{C}$ is the union of unary (belief) and binary (consistency) constraints, and $w_c$ are model-derived weights.

In practical settings, the algorithm is as follows:

1. Initialize explanation tree with $Q$.
2. For each non-integral node and up to depth $E$0, expand children via abductive prompts.
3. At each leaf, test integrality (consistency of explanations).
4. Translate all nodes and parent-child relations into SAT clauses.
5. Solve MAX-SAT for a globally self-consistent assignment, which determines the final answer (Jung et al., 2022).

3. Prompt Templates, Sub-Question Engineering, and Example Interactions

Prompt templates for Self-Ask and related methods are rooted in systematic decomposition and abductive interrogation:

Abductive generation for node $E$1:

• “$E$2? True, because ...”
• “$E$3? False, because ...”

This yields two direct branches for each proposition, recursively expanded until integrality is established or the depth constraint is reached.

Concrete example (Com2Sense):

• Q: “Bananas are purple?”
  • “Bananas are purple? True, because” → “Some heirloom bananas have dark purple flesh.”
  • “Bananas are purple? False, because” → “The common banana peel is yellow when ripe.”

Each continuation is recursively further expanded, then subjected to the integrality test. Only integral branches remain in the explanation tree for the MAX-SAT phase.

4. Empirical Results and Benchmark Comparisons

Maieutic Prompting and its Self-Ask variant have been empirically validated on several binary fact/commonsense QA benchmarks, establishing clear gains over standard and Chain-of-Thought methods. Experimental findings include:

Dataset	Standard	Chain of Thought	Self-Consistency	GKP	Maieutic (Zero-Shot)
Com2Sense (dev/test)	58.1 /—	61.6 /—	61.4 /—	61.8 /—	72.5 / 75.0
CSQA 2.0 (dev/test)	54.1 /—	59.6 /—	60.8 /—	59.7 /—	69.5 / 68.3
CREAK (dev/test/contrast)	60.3 /— / 55.2	64.8 /— / 59.4	70.5 /— / 64.8	75.4 /— / 68.2	85.2 / 85.3 / 77.4

Key observations:

• Maieutic Prompting achieves up to 20 points higher accuracy than baselines.
• Competitive with parameter-heavy, supervised models, despite fully unsupervised methodology (Jung et al., 2022).

In tasks blending math and commonsense, Hint of Thought (HoT) prompting further demonstrates that enforcing fixed-length, human-readable sub-questions and coupling each with pseudo-code can yield substantial empirical improvements. For instance, on GSM8K, zero-shot CoT achieves 40.5% versus HoT’s 67.8%; on StrategyQA, zero-shot CoT yields 52.3% versus HoT’s 82.96% (Lei et al., 2023).

5. Analysis, Insights, and Relation to Alternative Paradigms

The core strengths of zero-shot maieutic prompting are attributable to structured hypothesis exploration and post hoc logical validation:

• Structured Decomposition: Breaking down the question into sub-questions or abductive support/attack threads focuses the model’s generative process and forces explicit isolation of intermediate assumptions.
• Noise Filtering and Robustness: The integrality check prunes semantically or logically unreliable branches, while MAX-SAT enforces mosaic global consistency.
• Interpretability: The surviving explanation tree provides a transparent, traceable rationale path, enhancing trust and error analysis.
• Empirical Robustness: Accuracy maintains stability under semantic perturbations and contrast sets, with reduced variance to prompt-order changes (Jung et al., 2022).

When compared to related approaches:

• Chain-of-Thought (CoT): Produces one linear chain, vulnerable to early errors; lacks validation or branching.
• Program-of-Thought (PoT): Imposes strict code execution but is heavily arithmetic-specific.
• Hint of Thought (HoT): Blends explicit sub-question decomposition with pseudo-code, eliminating external retrieval but imposing intra-model logic, extending past mere math into commonsense QA (Lei et al., 2023).

A plausible implication is that enhancements to Self-Ask might adopt fixed-length sub-question loops, explicit pseudo-code consolidation, or final answer-extraction prompts, as demonstrated effective in HoT (Lei et al., 2023).

6. Practical Implementation Guidelines and Limitations

Reproducibility and deployment of Self-Ask/maieutic prompting entail:

• LLM: e.g., GPT-3 (text-davinci-001 or derivatives), sometimes with in-context examples.
• Expansion control: max depth $E$4 to limit exponential growth; breadth at each depth controlled via sampling.
• Integrality: consistency prompts for both $E$5 and $E$6; only branches passing integrality propagate.
• SAT solving: use of an off-the-shelf weighted MAX-SAT solver (e.g., RC2); optional incorporation of NLI-based constraints for added logical links.
• Typical computational cost: $E$7 20 LM calls per question, followed by one SAT solve; amenable to batching (Jung et al., 2022).

Identified limitations are:

• Applicability is largely restricted to binary (True/False) QA due to tree complexity.
• Depth and breadth limitations restrict the granularity of hypothesis exploration.
• Cross-question generalization and scalability to open-form answers remain open challenges.

7. Impact, Interpretability, and Future Directions

Zero-shot Maieutic Prompting (Self-Ask) and structurally similar approaches like Hint of Thought constitute robust strategies for enhancing LLM reasoning on multi-step and commonsense inference tasks. Empirically, they achieve large gains in accuracy and stability while preserving or improving interpretability. Surviving rationales exhibit high grammatical and relevance quality, and even in error cases, a substantial proportion of explanations remain factually correct or helpful (Jung et al., 2022).

Potential future work includes generalization to broader answer formats, dynamic regulation of tree depth/breadth, and integration with external NLI verifiers. The consistent empirical and robustness advantages underscore the foundational role of explicit sub-question frameworks—human-readable, logic-enforced, and model-native—across the spectrum of LLM reasoning paradigms.