K‑Visitation Framework: Concepts & Applications

• K‑Visitation Framework is a family of formulations that formalizes behavior through visitation sets, frequencies, and measures across diverse domains.
• It employs greedy set-cover strategies to determine the minimal set of places needed for essential amenity exposure in urban mobility studies.
• Extensions in reinforcement learning, robotics, and stochastic processes demonstrate its versatility in optimizing, constraining, and balancing sequential visitations.

K‑Visitation Framework denotes a family of formulations in which behavior is represented through visitation sets, visitation frequencies, or visitation measures, and a cardinality, clustering, horizon, or counter variable governs how those visitations are summarized, constrained, or optimized. The explicit term is introduced in a behavioural urban-analytics setting as the minimal set of distinct visited places needed to cover essential amenity categories under either a frequency ordering or a distance-from-home ordering, yielding $K_{\text{freq}}$ and $K_{\text{dist}}$ (Zhang et al., 31 Aug 2025). Closely related constructions appear in reinforcement learning, stochastic processes, swarm robotics, sequential inference, and co‑visitation modeling, where the same underlying idea is to formalize trajectories through what is visited, how often it is visited, and how those visitations should be balanced, matched, diversified, or exactly counted (Nedergaard et al., 2022, Bolland et al., 2024, Patel et al., 11 Jun 2026).

1. Canonical definition in behavioural urban mobility

In the formulation introduced for proximity-centred urban analysis, the K‑Visitation framework asks, for each individual, what is the smallest set of distinct places that together provide exposure to at least one instance of every essential amenity category (Zhang et al., 31 Aug 2025). Places are operationalised as H3 level‑10 hexagonal cells, approximately $75$ m, and amenity exposure is defined through POIs within a $400$ m buffer. The place set excludes the home cell.

For a user with visited-place set $P$, amenity-category set $C$, and $\operatorname{Cat}(p)\subseteq C$ denoting categories around place $p$, the two central quantities are

$$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$

$$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$

where $K_{\text{dist}}$0 is $K_{\text{dist}}$1 sorted by descending visitation frequency $K_{\text{dist}}$2, and $K_{\text{dist}}$3 is $K_{\text{dist}}$4 sorted by ascending distance to home $K_{\text{dist}}$5 (Zhang et al., 31 Aug 2025). This is a greedy set-cover style construction: places are added until the union of their amenity categories covers the full required set.

The framework therefore separates two objects that standard accessibility analyses often conflate: a recurrent behavioural portfolio of places and a proximate opportunity portfolio. To measure the overlap between those portfolios, the paper defines the mobility-alignment coefficient

$K_{\text{dist}}$6

that is, a Jaccard similarity between the place sets selected by the two orderings (Zhang et al., 31 Aug 2025). A null model $K_{\text{dist}}$7 is also defined from a random ordering $K_{\text{dist}}$8 that preserves the global empirical distance–frequency pattern while removing individual-specific habit.

Within this urban interpretation, K is not a visit count threshold. It is the cardinality of a minimal functional portfolio of places. This definition is exact in the source formulation and is the clearest instance in which “K‑Visitation framework” is introduced as a named concept (Zhang et al., 31 Aug 2025).

2. Mathematical variants across disciplines

Taken together, the cited literature suggests an umbrella concept rather than a single canonical formalism. The object being counted or optimized differs by domain, even when the common language of “visitation” is retained.

Domain	Visitation object	Representative quantity
Urban mobility	Minimal set of distinct places	$K_{\text{dist}}$9
Continuous-state RL	Cluster or occupancy visitation	$75$0, $75$1, $75$2
Reward shaping in RL	Success/failure state-action visitations	$75$3
Exploration across episodes	Episodic state visitation discrepancy	$75$4
Active model estimation	State-action occupancy measure	$75$5
Constrained sequence inference	Exact visitation counts	$75$6

In continuous-state reinforcement learning, one variant is to discretize behavior by a learned or adaptive partition. “k‑Means Maximum Entropy Exploration” defines the discounted state visitation distribution

$75$7

and seeks to maximize its differential entropy $75$8 (Nedergaard et al., 2022). In the paper’s K‑Visitation view, the $75$9 clusters act as macro-states with visitation frequencies $400$0, and the balancing weights

$400$1

control empirical occupancy across those macro-states (Nedergaard et al., 2022).

A second family uses state-action occupancy measures directly. “Learning Process Rewards via Success Visitation Matching” defines successful and unsuccessful visitations $400$2 and $400$3, and uses the process reward

$400$4

which can be rewritten as an outcome reward plus a contrastive KL regularizer over occupancy measures (Tsao et al., 22 Jun 2026).

A third family uses conditional future visitation. “Off-Policy Maximum Entropy RL with Future State and Action Visitation Measures” defines

$400$5

and a feature-level future visitation distribution

$400$6

from which intrinsic rewards are defined via negative KL to a reference $400$7 (Bolland et al., 2024). “Maximum-Entropy Exploration with Future State-Action Visitation Measures” uses the same conditional-future idea and shows that the expected sum of these intrinsic rewards is a lower bound on the entropy of the discounted distribution of state-action features visited in trajectories starting from the initial states (Bolland et al., 19 Mar 2026).

In active model estimation, the focus shifts from entropy to optimal allocation of occupancy mass. “$400$8-Explorer” defines

$400$9

over state-action occupancy measures $P$0, with gradient

$P$1

so that underexplored and high-variance transitions are prioritized (Gu et al., 23 Feb 2026).

In constrained sequential inference, K becomes an exact visitation cardinality. “Controller-Augmented Hidden Markov Models” represents “visit at least / at most / exactly $P$2 times” via finite-state counters with controller space $P$3, update

$P$4

and appropriate gate $P$5 and terminal acceptance set $P$6 (Patel et al., 11 Jun 2026).

3. Reinforcement-learning realizations

The reinforcement-learning literature in the data block instantiates several technically distinct K‑Visitation mechanisms.

In k‑means maximum-entropy exploration, the continuous state space is partitioned by an additively weighted online k‑means, with assignment

$P$7

online center update

$P$8

and intrinsic reward equal to the increment in an entropy-surrogate objective

$P$9

where

$C$0

with $C$1 in practice (Nedergaard et al., 2022). The intended effect is to increase the entropy of the visitation distribution by encouraging well-separated centers and approximately balanced visitation frequencies across clusters.

Success visitation matching uses a discriminator trained between successful and unsuccessful state-action visitations at time step $C$2. The Bayes-optimal identity

$C$3

yields a density-ratio reward, and the RL objective becomes

$C$4

so the policy is simultaneously pulled toward successful occupancy and pushed away from unsuccessful occupancy (Tsao et al., 22 Jun 2026). The paper explicitly interprets this as a $C$5 instance of a more general visitation framework.

Rewarding Episodic Visitation Discrepancy instead compares entire episodes. It defines an episodic visitation discrepancy

$C$6

estimates it with k‑nearest neighbors in a fixed random embedding, and assigns intrinsic reward

$C$7

with

$C$8

to penalize low intra-episode diversity (Yuan et al., 2022).

Future-visitation maximum-entropy methods move the visitation object from past empirical counts to conditional discounted futures. Their central claim is that the conditional visitation distribution is the fixed point of a contraction operator, enabling off-policy estimation of $C$9 and intrinsic rewards of the form

$\operatorname{Cat}(p)\subseteq C$0

(Bolland et al., 2024), and that the expected sum of such intrinsic rewards is a lower bound on a trajectory-level entropy objective (Bolland et al., 19 Mar 2026).

A separate but related line uses visitation values rather than visitation entropy. “Long-Term Visitation Value for Deep Exploration in Sparse Reward Reinforcement Learning” defines

$\operatorname{Cat}(p)\subseteq C$1

so exploration is planned far into the future through discounted visitation-based rewards, while exploitation remains in a separate $\operatorname{Cat}(p)\subseteq C$2-function (Parisi et al., 2020). This decoupling is explicitly intended to avoid the non-stationary target induced by reward augmentation.

4. Extensions beyond reinforcement learning

Outside RL, the same logic appears in several structurally different settings.

In large-scale human mobility and retail analytics, the co‑visitation problem is explicitly pairwise. “NAICS-Aware Graph Neural Networks for Large-Scale POI Co-visitation Prediction” defines

$\operatorname{Cat}(p)\subseteq C$3

as the number of distinct devices that visit both brands $\operatorname{Cat}(p)\subseteq C$4 and $\operatorname{Cat}(p)\subseteq C$5 within a one-hour window during month $\operatorname{Cat}(p)\subseteq C$6, aggregated at the brand, state, and month level (Alrubyli et al., 25 Jul 2025). The paper frames this as a $\operatorname{Cat}(p)\subseteq C$7 special case of a broader K‑visitation view, with the learned edge score $\operatorname{Cat}(p)\subseteq C$8 interpretable as a pairwise affinity kernel for co‑visitation.

In robot swarms, K‑visitation becomes explicit coverage accounting. “A Visitation Grid for Complete Coverage Foraging in Robot Swarms” partitions the unknown search arena into an $\operatorname{Cat}(p)\subseteq C$9 grid, stores global visitation counts

$p$0

and selects for each uninformed trip the $p$1 block with minimum cumulative visitation

$p$2

(Gonzalez et al., 21 May 2026). The framework is a direct instance of visitation-driven control: robots are biased toward under-visited regions while remaining stochastic inside the selected block.

In stochastic-process theory, the relevant object is the number of distinct sites visited by time $p$3,

$p$4

“Complete Visitation Statistics of 1d Random Walks” derives the full multi-time distributions

$p$5

and the Laplace-space formula for the cumulative event

$p$6

thereby turning K‑th discovery events into explicit threshold events on $p$7 (Régnier et al., 2022). The K‑th discovery time $p$8 satisfies $p$9.

In constrained HMMs, K‑visitation is exact and symbolic rather than statistical. CHMMs compile “visit at least / at most / exactly $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$0 times” into finite-state controllers and then run standard forward–backward and Viterbi recursions on the augmented chain (Patel et al., 11 Jun 2026). This makes K‑visitation a first-class pathwise constraint in exact sequential inference.

5. Empirical findings and interpretive themes

The empirical record across the cited work is not uniform, but several patterns recur.

In the urban setting that explicitly introduces the term, $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$1 is strongly concentrated around $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$2–$$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$3 nationally, and shifts toward around $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$4 when work locations are removed (Zhang et al., 31 Aug 2025). Central districts in Helsinki, Turku, Tampere, and Oulu exhibit lower $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$5, while peripheral and suburban areas exhibit higher $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$6. In Helsinki, under $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$7, about $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$8 of residents could in principle access all daily amenities within $$K_{\text{freq}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{freq}} \right\},$$9 minutes by public transport, whereas under $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$0 only about $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$1 do so; in Espoo, the behaviourally grounded share is around $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$2 (Zhang et al., 31 Aug 2025). The explicit conclusion is that proximity is necessary but insufficient for achieving the proximity-living ideal.

In sparse-reward RL, cluster-, ratio-, and discrepancy-based visitation shaping often improves sample efficiency. KME substantially outperforms PPO, RND, and RE3 on Cheetah and Quadruped, is competitive on Walker, Cartpole, and Acrobot, and fails on Humanoid together with the other methods (Nedergaard et al., 2022). Success visitation matching yields significantly faster finetuning on simulated and real-world robotic manipulation tasks and about $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$3 fewer environment steps to reach high success when finetuning generalist VLAs (Tsao et al., 22 Jun 2026). REVD significantly improves the sample efficiency of reinforcement learning algorithms and outperforms the benchmarking methods on Atari and PyBullet (Yuan et al., 2022).

In swarm foraging, explicit visitation counting particularly improves the late stage of collection. Compared to CPFA, the visitation-grid strategy reduces total collection time by up to $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$4 and improves collection efficiency by more than $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$5 during the final stage of the mission (Gonzalez et al., 21 May 2026). In sequence inference, controller augmentation is uniquely able to recover globally feasible trajectories on cumulative-constraint regimes, while simpler decoders are matched in validity on locally-dominated regimes (Patel et al., 11 Jun 2026).

A plausible synthesis is that K‑Visitation constructions are most distinctive when the target property is cumulative rather than local: exact visit counts, long-horizon coverage, contrastive success occupancy, or multi-time discovery events. In such settings, stepwise or purely local heuristics are repeatedly described as insufficient or myopic (Parisi et al., 2020, Patel et al., 11 Jun 2026).

6. Limitations, ambiguities, and open directions

A recurrent source of ambiguity is that $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$6 does not denote the same thing in all formulations. In the Finnish mobility study, $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$7 is the cardinality of a minimal place portfolio (Zhang et al., 31 Aug 2025). In k‑means exploration, $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$8 is the number of clusters (Nedergaard et al., 2022). In CHMMs, $$K_{\text{dist}} = \min \left\{ k \,\middle|\, \bigcup_{i=1}^{k} \operatorname{Cat}(p_{(i)}) \supseteq C, \; p_{(i)} \in P_{\text{dist}} \right\},$$9 is a hard visitation-count threshold (Patel et al., 11 Jun 2026). In active model estimation, the analogous control parameter is the curvature $K_{\text{dist}}$00, not a count threshold (Gu et al., 23 Feb 2026). Taken together, the literature suggests a family resemblance organized around visitation representations, not a single universally standardized object.

The urban formulation carries several explicit caveats: mobility traces come from opt-in smartphone data; visited places are inferred from stays and $K_{\text{dist}}$01 m POI buffers rather than actual in-venue check-ins; the ten amenity categories are treated as equally essential; and overshoot may reflect quality, prices, brand loyalty, social ties, workplace locations, school catchments, or perceived safety rather than policy failure (Zhang et al., 31 Aug 2025).

The RL formulations also rely on substantial assumptions. KME’s theoretical arguments assume continuity of $K_{\text{dist}}$02, dense centers as $K_{\text{dist}}$03, and balanced Voronoi conditions that are described as idealized; the method also fails on very high-dimensional Humanoid and is sensitive to initialization and non-stationarity of $K_{\text{dist}}$04 (Nedergaard et al., 2022). Success visitation matching proves policy invariance only under deterministic transitions and countable state-action spaces, although the practical method uses function approximation in continuous robotic domains (Tsao et al., 22 Jun 2026). Conditional future-visitation maximum-entropy methods currently rely on biased off-policy approximations and, in the cited experiments, are demonstrated in MiniGrid rather than high-dimensional continuous control (Bolland et al., 2024, Bolland et al., 19 Mar 2026). $K_{\text{dist}}$05-Explorer is developed for tabular MDPs with perfect state observability and ergodicity assumptions, and its guarantees are occupancy-level rather than representation-learning guarantees (Gu et al., 23 Feb 2026). CHMMs remain exact only when the constraint can be compiled into a finite-state controller; combining many counters can increase controller size multiplicatively (Patel et al., 11 Jun 2026).

Across domains, the most stable interpretation is therefore methodological rather than terminological. K‑Visitation frameworks formalize sequential phenomena through what is visited, how often it is visited, and how those visitations are constrained or optimized. The precise mathematics then diverges according to whether the task is set cover over places, maximum-entropy exploration, success-conditioned occupancy matching, coverage control, exact pathwise counting, or multi-time discovery statistics.