Answer Type Modification (ATM)

• Answer Type Modification (ATM) is a type system discipline that tracks changes in computation answer types during evaluation and control flow transitions.
• ATM plays a central role in typed delimited continuations and effect handlers, enabling shift/reset operators and CPS translations that transform answer contexts.
• ATM increases expressiveness by allowing distinct answer types in different clauses while supporting rigorous verification, soundness proofs, and termination guarantees.

Searching arXiv for recent and foundational papers on answer-type modification and related effect-handler work. I’ll look up relevant arXiv records to ground the article. Answer-type modification (ATM) is the ability of a type system to track that the answer type of a computation changes during evaluation. In typed delimited control, this is expressed by judgments of the form $\Gamma \vdash e : \tau \;;\; \alpha,\beta$, meaning that under an initial answer type $\alpha$, evaluating $e$ produces a value of type $\tau$ and modifies the answer type to $\beta$; in algebraic effects and handlers, the same idea is internalized as answer-type annotations on computation types and control effects (Kobori et al., 2016). ATM is central to the typing of delimited continuations, shift/reset-style operators, and effect handlers because captured continuations can suspend, resume, discard, duplicate, or recontextualize computations in ways that alter the type expected by the enclosing context. More recent work further refines ATM into answer-refinement modification (ARM), where only refinements change while the underlying base type remains fixed, and answer-effect modification (AEM), where temporal effects rather than ordinary result types are modified (Kawamata et al., 2023).

1. Core definition and formal role

In the surface calculus for typed delimited-control operators $\mathsf{shift}$ and $\mathsf{reset}$, ATM is introduced by replacing the ordinary judgment $\Gamma \vdash e : \tau$ with $\Gamma \vdash e : \tau \;;\; \alpha,\beta$. The intended meaning is explicit: evaluation starts in an ambient answer type $\alpha$ and ends in an answer type $\alpha$0. This is the sense in which ATM “modifies” answers rather than merely returning values. The same paper gives the effectful function type $\alpha$1, meaning a function that takes a $\alpha$2 and, under answer type $\alpha$3, returns a $\alpha$4 while changing the answer type to $\alpha$5 (Kobori et al., 2016).

In effect-handler calculi, ATM is encoded directly in computation types. One formulation uses $\alpha$6, where $\alpha$7 is a pure computation returning $\alpha$8, and $\alpha$9 is an effectful computation that changes its answer type from $e$0 to $e$1. Another formulation packages the same information as a control effect

$e$2

where $e$3 means “no operation,” and $e$4 means that when the computation is plugged into a context expecting answer type $e$5, with $e$6 bound to the computation’s result, the handler will see answer type $e$7 (Endo et al., 18 Aug 2025).

ATM is therefore not merely a property of operators such as $e$8 or of a specific handler clause. It is a typing discipline for the interaction between a computation and its surrounding context. This suggests that ATM is best understood as a structured account of context transformation: the program does not simply produce a value, but changes what the rest of the computation is allowed or required to be.

2. Typing judgments, subtyping, and handler interaction

A recurring feature of ATM systems is that computation types carry both a return component and an answer-type transformation. In the effect-handler setting, the judgment

$e$9

assigns to a computation a type $\tau$0 that may be pure or may carry an annotation $\tau$1. In the refinement-based system for OCaml-5-style algebraic effects, the corresponding judgment is

$\tau$2

where $\tau$3 is an operation signature, $\tau$4 is a return type, and $\tau$5 is a control effect. The operation rule has the form

$\tau$6

which makes ATM visible already at the point of invoking an effectful operation (Kawamata et al., 2023).

Subtyping is the technical device that allows ATM to compose. In the finitary PCF system, the key rules include:

• $\tau$7: $\tau$8.
• $\tau$9: $\beta$0, $\beta$1, and $\beta$2 imply $\beta$3.
• $\beta$4: $\beta$5 and $\beta$6 imply $\beta$7.

The last rule is especially important because it says that any pure computation may be viewed as an effectful one, provided subtyping holds on results and answer types. In the refinement formulation, the corresponding embedding rule is

$\beta$8

and the core ATM subtyping rule weakens assumptions contravariantly and guarantees covariantly on answer types (Endo et al., 18 Aug 2025).

Handler typing is where ATM departs most sharply from no-ATM systems. Under a standard no-ATM system, every clause of a handler—both the return clause and each op-clause—must produce the same answer type. ATM frees the typing discipline from that requirement. In the finitary PCF account, each operation $\beta$9 may have its own answer type $\mathsf{shift}$0, and the with-handle construct itself may modify the ambient answer type from $\mathsf{shift}$1 to $\mathsf{shift}$2. In the refinement system, a handler transforms a computation typed as $\mathsf{shift}$3 into $\mathsf{shift}$4 once the return clause is checked at $\mathsf{shift}$5 and each operation clause is checked against its own continuation type (Kawamata et al., 2023).

3. ATM in algebraic effects, handlers, and refinement typing

The refinement-type system for algebraic effects and handlers was proposed to address what the paper describes as the absence of a satisfactory refinement type system for algebraic effects and handlers. Its central claim is that the expressivity and usefulness of algebraic effects and handlers come from their ability to manipulate delimited continuations, but those same continuations complicate control flow and make verification harder. The proposed solution is ARM, a notion that allows the refinement type system to precisely track what effects occur and in what order when a program is executed, and to reflect such information as modifications to the refinements in the types of delimited continuations (Kawamata et al., 2023).

The distinction between ATM and ARM is explicit. Full ATM allows the entire answer type $\mathsf{shift}$6 of a computation to change. ARM is the restricted case where only the refinement part of a base-type answer changes and the underlying base type itself remains the same. The paper states that the system supports full ATM, but that in “ordinary” verification style one typically only needs ARM, namely the case where $\mathsf{shift}$7 and $\mathsf{shift}$8 have the same base type but different refinements.

A worked OCaml-5 example, “choose-max,” shows how ATM is used to verify a nondeterministic program:

$\alpha$16

The account states that every call perform decide() has a type of the form $\mathsf{shift}$9, where $\mathsf{reset}$0 and $\mathsf{reset}$1 are refinement types selected by the predicate argument $\mathsf{reset}$2. For the first call, the instantiation

$\mathsf{reset}$3

and

$\mathsf{reset}$4

permits ATM subtyping via $\mathsf{reset}$5 to match the handler’s demands, yielding the whole program type $\mathsf{reset}$6. The paper emphasizes that every movement of the answer type from $\mathsf{reset}$7 to $\mathsf{reset}$8 is recorded by ATM as a whole computation-type change, even though only the refinement changes.

This use of ATM supports an interpretation of handlers as refinement transformers rather than only control delimiters. That interpretation is not stated verbatim, but it is a plausible implication of the way answer-type changes are reflected into continuation types.

4. CPS and prompt-passing translations

A major line of work on ATM studies whether ATM can be eliminated by translation into calculi or host languages without native answer-type modification. One result is a typed prompt-passing-style translation from a calculus with $\mathsf{reset}$9 and $\Gamma \vdash e : \tau$0 with ATM into a target calculus with multi-prompt $\Gamma \vdash e : \tau$1 and $\Gamma \vdash e : \tau$2 but no ATM. The translation maps

$\Gamma \vdash e : \tau$3

and

$\Gamma \vdash e : \tau$4

It creates two fresh prompts at each source-level reset, with one prompt corresponding to the new answer type after the reset and the other to the old one. The paper proves that the translation preserves typing and gives an optimized translation that generates prompts only when needed (Kobori et al., 2016).

The same elimination idea reappears for algebraic effects and handlers in CPS form. In the ATM-typable finitary PCF setting, types are translated by

$\Gamma \vdash e : \tau$5

$\Gamma \vdash e : \tau$6

Here $\Gamma \vdash e : \tau$7 is a record type encoding the operation signature. Operation invocation becomes record lookup and direct call, and withhandle constructs are compiled by supplying a record of handler bodies together with the continuation for the previous answer type. The paper describes this CPS as derivation-directed: each subtyping or typing derivation is turned into a target term (Endo et al., 18 Aug 2025).

The refinement-type paper for algebraic effects and handlers also proves bidirectional refinement-type preservation for its CPS transformation. The statements are:

• Forward preservation: if $\Gamma \vdash e : \tau$8 then $\Gamma \vdash e : \tau$9.
• Backward preservation: if $\Gamma \vdash e : \tau \;;\; \alpha,\beta$0 in the handler-free target system, then there is some $\Gamma \vdash e : \tau \;;\; \alpha,\beta$1 in the source such that $\Gamma \vdash e : \tau \;;\; \alpha,\beta$2 and $\Gamma \vdash e : \tau \;;\; \alpha,\beta$3.
• Simulation: $\Gamma \vdash e : \tau \;;\; \alpha,\beta$4.

These results show that ATM is not only a source-language typing discipline; it can also serve as a guide for semantics-preserving translations to handler-free or no-ATM targets. The data additionally states that the 2016 prompt-passing work generalizes Kiselyov’s direct-style implementation of typed printf, which uses two prompts to emulate the modification of answer types.

5. Verification, soundness, decidability, and termination

ATM has been used as a verification mechanism, not only as a typing convenience. In the refinement system for algebraic effects and handlers, the main meta-theoretic result is type safety in the form of subject-reduction and progress: if $\Gamma \vdash e : \tau \;;\; \alpha,\beta$5 and $\Gamma \vdash e : \tau \;;\; \alpha,\beta$6, then either $\Gamma \vdash e : \tau \;;\; \alpha,\beta$7 with $\Gamma \vdash e : \tau \;;\; \alpha,\beta$8, or $\Gamma \vdash e : \tau \;;\; \alpha,\beta$9 for some $\alpha$0, or $\alpha$1 can step further. The proof sketch uses standard induction on evaluation together with $\alpha$2 to justify contravariant and covariant manipulations of answer types (Kawamata et al., 2023).

In temporal verification, the corresponding notion is answer-effect modification rather than answer-type modification. The system extends temporal effects to shift$\alpha$3/reset$\alpha$4 by giving computations a temporal effect $\alpha$5 together with a control effect $\alpha$6, written

$\alpha$7

The control effect is either $\alpha$8 or $\alpha$9. The key interpretation is that $\alpha$00 records what the captured continuation’s context must satisfy, while $\alpha$01 records what the enclosing reset$\alpha$02 expects. The paper proves soundness for finite event sequences via type safety and for infinite event sequences using a logical relation, and emphasizes that AEM enables reasoning about traces yielded by captured continuations (Sekiyama et al., 2022).

A different consequence of ATM appears in finitary PCF with effect handlers. The paper proves the theorem: if $\alpha$03 is a closed ATM-typable finitary PCF program with effect handlers, then deciding whether $\alpha$04 holds is decidable. The proof proceeds by CPS-transforming $\alpha$05 into a simply-typed, handler-free finitary PCF program $\alpha$06, using typability preservation and simulation, and then appealing to the known decidability of reachability in plain simply-typed, handler-free finitary PCF due to Ong ’06 and Kobayashi ’09 (Endo et al., 18 Aug 2025).

The same paper also proves that any ATM-typable nonrecursive finitary PCF program with effect handlers is strongly normalizing. The argument is that if the source has no recursion, then its CPS transform also has no recursion; the target is then an ordinary simply-typed $\alpha$07-calculus term with records and without recursion or effects, hence strongly normalizing; simulation transfers termination back to the source.

6. Expressiveness, restrictions, and related notions

ATM is sometimes presented as increasing expressiveness because it permits programs that standard no-ATM systems cannot type. The finitary PCF work confirms this intuition by exhibiting programs typable with ATM but not without it. One example is

$\alpha$08

The paper states that no-ATM simple typing fails because the return clause body x has type unit while the op-clause body (k x; true) has type bool, and the standard handler rule requires a single answer type for all clauses. ATM succeeds by assigning

$\alpha$09

This is a direct instance of ATM enabling distinct answer types in return and op clauses (Endo et al., 18 Aug 2025).

The same paper also shows the converse: there are programs typable without ATM that become untypable with ATM. Its example is

$\alpha$10

where

$\alpha$11

Under the simple system one can give $\alpha$12 and $\alpha$13, with both clauses producing unit. Under ATM, however, the subtyping constraints force the answer-type transformation $\alpha$14 to become pure, which conflicts with the effectful return clause invoking op (). The paper states that a contradiction follows, so $\alpha$15 is untypable in ATM.

This bidirectional separation matters for interpretation. ATM is not simply “more permissive” than traditional handler typing. The data explicitly states that there are programs typable with ATM but not without it, and also programs typable without ATM that become untypable with ATM. A plausible implication is that ATM imposes a finer structural discipline on continuations: it enlarges expressiveness along some dimensions while ruling out programs whose control flow cannot be assigned coherent answer-type transformations.

Related notions clarify this landscape. ARM is the case where only refinements change while the underlying base type remains the same (Kawamata et al., 2023). AEM is the temporal-effect analogue, where what changes is the answer effect of the continuation rather than an ordinary answer type (Sekiyama et al., 2022). Across these variants, the unifying pattern is a require–guarantee view of delimited control: a captured continuation assumes one answer specification before resumption and establishes another afterward.