MemGUI-3K: Mobile GUI Context Management Dataset

• MemGUI-3K is a supervised dataset of 2,956 mobile GUI agent trajectories annotated with the full ConAct protocol, enabling proactive context management.
• It integrates end-to-end supervision for UI actions, history folding, and memory operations, addressing challenges in long-horizon, cross-app tasks.
• The dataset underpins models like MemGUI-8B-SFT, yielding significant improvements in pass rates and context retention on memory-intensive mobile GUI benchmarks.

MemGUI-3K is a supervised dataset of mobile GUI agent trajectories built specifically for Context-as-Action (ConAct), the proactive context-management mechanism introduced in "MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management" (Liu et al., 18 Jun 2026). It consists of 2,956 trajectories with full ConAct annotations, and is designed to support supervised training and offline analysis of long-horizon mobile GUI interaction in which an agent must retain intermediate facts across many steps and app transitions. The dataset is collected in the MemGUI-Bench Android snapshot environment and is closely tied to the memory-centric evaluation setting introduced by "MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments" (Liu et al., 3 Feb 2026).

1. Definition and scope

MemGUI-3K is defined as “a 2,956-trajectory dataset with full ConAct annotations, supporting supervised training and offline analysis of proactive context management across model scales” (Liu et al., 18 Jun 2026). Its immediate purpose is to make ConAct trainable, especially at the 8B scale. The dataset is not merely a larger corpus of demonstrations: each trajectory contains explicit supervision for standard UI actions, context-management actions, self-describing step outputs, and the resulting structured state transitions of the context fields.

The dataset addresses a gap identified in prior mobile GUI work. Existing mobile GUI datasets and benchmarks primarily supervise UI actions and often rely on ReAct-style histories or unstructured multi-turn dialogues, but they do not provide explicit supervision for context-management decisions such as when to summarize history, what to persist in memory, or how to maintain a compact but sufficient context (Liu et al., 18 Jun 2026). In the broader benchmark context, MemGUI-Bench was introduced because existing mobile GUI agent benchmarks contain only 5.2–11.8% memory-related tasks, lack pass@k protocols for cross-session learning, and rely on evaluation methods that do not scale well to long, cross-app trajectories (Liu et al., 3 Feb 2026). MemGUI-3K is therefore best understood as the training substrate for the memory-centric regime that MemGUI-Bench exposed.

A common misconception is to treat MemGUI-3K as simply a larger benchmark. The primary paper instead presents it as a supervised dataset for ConAct. MemGUI-Bench remains the evaluation benchmark, whereas MemGUI-3K provides annotated trajectories from which a model can learn proactive context management (Liu et al., 18 Jun 2026).

2. ConAct formalism and annotation schema

MemGUI-3K is organized around the ConAct state and action formulation. The structured context state at step $t$ is

$\mathcal{S}_t = (G, H_t, M_t, L_t),$

and the policy output is

$$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$

Here, $G$ is the task goal, $H_t$ is folded action history, $M_t$ is folded UI state, $L_t$ is the recent step record, and $I_t$ is the screenshot (Liu et al., 18 Jun 2026).

For each step, MemGUI-3K includes the following inputs: the task goal $G$, folded action history $H_t$, folded UI state $\mathcal{S}_t = (G, H_t, M_t, L_t),$0, recent step record $\mathcal{S}_t = (G, H_t, M_t, L_t),$1, and current screenshot $\mathcal{S}_t = (G, H_t, M_t, L_t),$2. It also includes the gold assistant output in a five-part ConAct structure: <thinking> for $\mathcal{S}_t = (G, H_t, M_t, L_t),$3, <folding> for $\mathcal{S}_t = (G, H_t, M_t, L_t),$4, <tool_call> for $\mathcal{S}_t = (G, H_t, M_t, L_t),$5, <ui_observation> for $\mathcal{S}_t = (G, H_t, M_t, L_t),$6, and <action_intent> for $\mathcal{S}_t = (G, H_t, M_t, L_t),$7 (Liu et al., 18 Jun 2026).

Three context fields are central.

First, Folded Action History $\mathcal{S}_t = (G, H_t, M_t, L_t),$8 is a list of folded records generated by prior folding directives. A folding directive is written as

$\mathcal{S}_t = (G, H_t, M_t, L_t),$9

where $$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$0 specifies the interval to compress and $$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$1 is a natural-language summary. The update rule is

$$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$2

This supports both step-level distillation ($$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$3) and span-level abstraction ($$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$4) (Liu et al., 18 Jun 2026).

Second, Folded UI State $$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$5 stores persistent UI-derived facts. Each memory item is

$$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$6

with a unique identifier, a short description, and full content. Memory actions are defined by

$$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$7

The update rule is

$$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$8

MemGUI-3K explicitly logs which UI facts are preserved, updated, or deleted (Liu et al., 18 Jun 2026).

Third, Recent Step Record $$y_t = (\tau_t,\phi_t,a_t,o_t,\iota_t) \sim \pi_\theta(\cdot \mid I_t,\mathcal{S}_t).$$9 contains a grounded UI observation and an action intent. The observation includes “exact visible text, numbers, names, and task-relevant UI facts,” while the action intent summarizes what the step is meant to accomplish. Together with the executed action and tool result, the update is

$G$0

The complete next context state is then

$G$1

This annotation design makes context management part of the same policy that selects UI actions, rather than an external or post-hoc mechanism (Liu et al., 18 Jun 2026).

3. Construction pipeline and data curation

MemGUI-3K is collected in the snapshot-based Android environment of MemGUI-Bench, with 26 apps and the same device abstraction and standard touch actions used in that framework (Liu et al., 18 Jun 2026). The teacher model is Qwen3-VL-235B-Thinking running the full ConAct protocol. Each selected task is executed once, with a step budget of

$G$2

where $G$3 is the golden-step count from MemGUI-Bench (Liu et al., 18 Jun 2026).

Task generation begins from the 128 seed tasks of MemGUI-Bench. The authors apply three transformations. Entity substitution preserves task pipelines while replacing entities such as product names or search queries, producing 456 expanded tasks. Memory-operation augmentation explicitly constructs tasks that require memory_update and memory_delete in addition to memory_add, producing 289 tasks. Task simplification decomposes complex multi-app tasks into shorter, single-objective variants while preserving the need for context management, producing 6,558 simplified tasks (Liu et al., 18 Jun 2026). These processes yield 7,303 candidate tasks, of which 5,293 are selected for rollout.

The final dataset contains 2,566 simplified tasks, 234 entity substitution tasks, and 156 memory-operation tasks (Liu et al., 18 Jun 2026). Trajectory-level filtering then retains only rollouts judged correct by the MemGUI-Eval pipeline: 2,959 of 5,293 pass this stage. A subsequent sanity filter removes one abnormal 321-step outlier and two tasks with extremely low-frequency apps, leaving the final 2,956 trajectories (Liu et al., 18 Jun 2026).

The dataset also applies step-level filtering. Even successful trajectories may contain exploratory or counterproductive steps, so each step is labeled with reasonableness, impact, and an explanation. Only steps labeled reasonable are used as supervised fine-tuning samples. This produces 64,430 reasonable-step samples, or about 21.8 reasonable steps per trajectory, corresponding to a 75.7% ratio (Liu et al., 18 Jun 2026). This suggests that MemGUI-3K is curated not only for success at the trajectory level but also for local action quality and context-management quality.

4. Scale, composition, and long-horizon properties

MemGUI-3K contains 2,956 successful rollouts, split into 2,661 train trajectories and 295 test trajectories. At the step level it contains 64,430 reasonable-step samples, divided into 57,951 train and 6,479 test (Liu et al., 18 Jun 2026). Every trajectory is a full interaction from initial state to termination in the MemGUI-Bench Android snapshot environment.

The average trajectory length is 28.8 steps with median 25, which the paper reports as about 1.9× GUIOdyssey (15.3 steps) (Liu et al., 18 Jun 2026). Task difficulty inherits the MemGUI-Bench categorization: Easy (1–20 golden steps), Medium (21–40), Hard (41+). The source benchmark is predominantly memory-intensive: 89.8% of MemGUI-Bench tasks are explicitly memory-intensive and 78.1% require cross-application information transfer (Liu et al., 3 Feb 2026). MemGUI-3K is built from expansions of those tasks while remaining disjoint from the original 128 evaluation tasks (Liu et al., 18 Jun 2026).

The dataset spans 26 Android apps across 7 app categories and includes both single-app and cross-app workflows, with many transitions among applications such as Amazon, Joplin, Bing, Calculator, Calendar, and Messages (Liu et al., 18 Jun 2026). In the underlying benchmark, task structures include flows such as Amazon → Joplin → Calculator, Apartments.com → Bing → Citymapper → Joplin, and AP News → DeepL → Joplin, all designed to stress cross-temporal and cross-spatial retention (Liu et al., 3 Feb 2026).

The ConAct annotations reveal nontrivial context-management behavior. 23.8% of folds are span-level, with average span length 6.25 steps, median 4, and a maximum of 118; 88.7% of trajectories contain at least one span-level fold (Liu et al., 18 Jun 2026). Memory actions appear in 65.1% of trajectories, with averages of 1.17 memory_add, 0.03 memory_update, and 0.04 memory_delete per trajectory (Liu et al., 18 Jun 2026). These statistics indicate that MemGUI-3K is not limited to one-step memoization or trivial logging; it encodes teacher behavior for deciding when to abstract history and when to persist task-relevant UI facts.

5. Training role and empirical significance

MemGUI-3K is used to train MemGUI-8B-SFT, an 8B MemGUI-Agent obtained by supervised fine-tuning on the dataset’s reasonable steps (Liu et al., 18 Jun 2026). The base model is Qwen3-VL-8B-Instruct, trained with LoRA on 57,951 training steps for 1 epoch (Liu et al., 18 Jun 2026). The supervision target is the full assistant response, so the model is jointly trained to predict the next UI action, the folding directive, memory operations, UI observation, and action intent.

The paper emphasizes that proactive context management is not merely a prompt-format change. In zero-shot experiments, simply switching to ConAct helps only the strongest model, Qwen3-VL-235B-Thinking, and can hurt smaller backbones (Liu et al., 18 Jun 2026). MemGUI-3K is therefore presented as the mechanism that makes ConAct learnable below that scale.

On MemGUI-Bench, the improvement attributed to MemGUI-8B-SFT (ConAct + MemGUI-3K) over the Qwen3-VL-8B-Instruct ReAct-style baseline is substantial. Pass@1 rises from 9.4% to 23.4%, pass@3 from 20.3% to 35.9%, and IRR from 15.1% to 30.2% (Liu et al., 18 Jun 2026). On Hard tasks (41+ golden steps), pass@1 increases from 2.6% to 21.1%, pass@3 from 7.9% to 34.2%, and IRR from 9.5% to 32.6% (Liu et al., 18 Jun 2026). Since MemGUI-Bench was specifically designed to expose failures in memory-intensive, long-horizon, and cross-app tasks, these gains are directly relevant to the long-context problem formulation (Liu et al., 3 Feb 2026).

The same model also transfers to MobileWorld GUI-Only (117 tasks), where success rate rises from 9.4% for the baseline to 17.9% for MemGUI-8B-SFT, exceeding OpenMobile-8B (17.7%) and ClawGUI-2B (17.1%) in that reported comparison (Liu et al., 18 Jun 2026). This indicates that the learned behavior is not restricted to the MemGUI-Bench environment.

Offline analysis further shows that MemGUI-3K teaches context-control skills rather than only action syntax. With gold context, UI action match accuracy improves from 29.2% to 36.3%; memory timing trigger F1 improves from 19.9% to 48.0%; deep fold ratio rises from 8.8% to 26.1%; deep range accuracy improves from 45.2% to 58.9%; and full format compliance increases from 94.9% to 99.9% (Liu et al., 18 Jun 2026). The largest improvement is specifically described as occurring in context-control decisions, especially “promoting transient observations to durable memory.”

6. Relation to MemGUI-Bench, distinctiveness, and limitations

MemGUI-3K is structurally downstream of MemGUI-Bench. The benchmark paper defines memory for mobile GUI agents as the ability to retain, process, and utilize contextual information within tasks and experiential knowledge across tasks to enhance decision-making and task performance over time (Liu et al., 3 Feb 2026). It also introduces the core evaluation measures—SR, IRR, MTPR, pass@k SR, FRR, Average Step Ratio, Average Time per Step, and Average Cost per Step—and documents substantial memory deficits across evaluated agents (Liu et al., 3 Feb 2026). MemGUI-3K does not replace these benchmark functions; it complements them by supplying annotated trajectories that operationalize proactive context management.

Relative to prior datasets, MemGUI-3K differs in two main respects. First, it is reported as the longest open SFT dataset for mobile GUI agents currently reported, with average length 28.8 steps (Liu et al., 18 Jun 2026). Second, it provides end-to-end supervision for history folding ranges and summaries, structured UI memory actions inside the same policy, and explicit self-describing step outputs, which the paper states are absent from other datasets and agent frameworks such as ReAct-, Action-Thought-, rule-based-, or external-memory-agent designs (Liu et al., 18 Jun 2026).

Several limitations are explicit. MemGUI-3K is restricted to Android-style mobile GUI environments and 26 apps, with future work needed for iOS, desktop, and web interfaces (Liu et al., 18 Jun 2026). Although the dataset extends horizon length, the paper notes that harder regimes such as more app transitions or multi-session tasks are not yet captured. The annotations are generated by a 235B model + MemGUI-Eval, not by human experts, so residual suboptimal context actions or teacher bias may remain (Liu et al., 18 Jun 2026). Memory editing is also sparse: memory_update and memory_delete average only 0.03 and 0.04 per trajectory, which may limit generalization to settings requiring aggressive memory revision (Liu et al., 18 Jun 2026).

A further point of clarification concerns naming. MemGUI-Bench contains a section that extrapolates to a hypothetical “MemGUI-3K” as a scaled-up successor benchmark, but explicitly states that this is reasoned extrapolation and that the paper does not mention “MemGUI-3K” as an actual benchmark artifact (Liu et al., 3 Feb 2026). The concrete MemGUI-3K that exists in the literature is instead the ConAct-centric dataset introduced in the MemGUI-Agent work (Liu et al., 18 Jun 2026). This suggests that the term now denotes a training dataset rather than a benchmark, even though the benchmark paper had earlier used it hypothetically as a name for a future large-scale evaluation suite.