GFlowNet-Based Personalized Feeds Algorithm

• The paper demonstrates a generative policy approach for personalized feeds by sampling actions proportional to reward, enabling diverse candidate generation in short-video systems.
• The methodology models feed generation as a sequential control process that integrates recommendation, prefetching, and delivery actions under network and user constraints.
• Empirical results reveal significant QoE improvements and reduced bandwidth usage, validating the effectiveness of multi-candidate generation over greedy selection.

to=arxiv_search.search wuregistration 彩神争霸网站json {"query":"all:(GFlowNets personalized feeds short video feeds Generative Flow Networks recommendation GFlowHF pGFlowMeta GFlowGR)", "max_results": 10, "sort_by": "submittedDate", "sort_order": "descending"} to=arxiv_search.search уйғурларғаjson {"query":"id:(Jin et al., 23 Aug 2025) OR id:(Li et al., 2023) OR id:(Ji et al., 2023) OR id:(Shen et al., 2023) OR id:(Maksimov et al., 4 Jun 2026) OR id:(Wang et al., 19 Jun 2025) OR id:(Niu et al., 1 Mar 2026)", "max_results": 10, "sort_by": "relevance", "sort_order": "descending"} A GFlowNet-Based Personalized Feeds Algorithm is a family of feed-generation and feed-control methods that uses Generative Flow Networks (GFlowNets) to sample actions, items, slates, or trajectories in proportion to a positive reward or utility signal, rather than greedily selecting only a single maximizer. In the literature, the most explicit direct instantiation appears in personalized short-video delivery, where a GFlowNet generates multiple candidate control actions and a selector executes the best one under a personalized quality-of-experience-minus-bandwidth objective (Jin et al., 23 Aug 2025). Closely related work treats human- or user-scored trajectories as terminal rewards, studies personalized meta-policies for heterogeneous tasks, and develops offline, policy-based, and partial-episode training schemes that make the same design pattern applicable to feed or slate generation more broadly (Li et al., 2023).

1. Scope and lineage

The term covers both a direct multimedia-systems algorithm and a broader recommendation-oriented interpretation of GFlowNet training. In the direct formulation, the feed problem is not conventional relevance ranking alone; it is a joint control problem over recommendation queue management, chunk prefetching, bitrate choice, pausing, and bandwidth use in short-video feeds (Jin et al., 23 Aug 2025). In the broader interpretation, a feed or slate is treated as a compositional object generated sequentially, so that a partially built personalized feed corresponds to a GFlowNet state and a completed feed corresponds to a terminal object with positive utility (Li et al., 2023).

Paper	Setting	Contribution to personalized feeds
"Generative Flow Networks for Personalized Multimedia Systems: A Case Study on Short Video Feeds" (Jin et al., 23 Aug 2025)	Short-video multimedia control	Direct GFlowNet-based personalized feeds algorithm
"GFlowNets with Human Feedback" (Li et al., 2023)	Human-scored trajectories	Proportional-to-score sampling rather than favorite-only optimization
"Meta Generative Flow Networks with Personalization for Task-Specific Adaptation" (Ji et al., 2023)	Heterogeneous tasks	Shared meta policy plus task-specific personalized policies
"GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks" (Wang et al., 19 Jun 2025)	Generative recommendation	GFlowNet fine-tuning for exposure-bias mitigation
"FedGrAINS: Personalized SubGraph Federated Learning with Adaptive Neighbor Sampling" (Ceyani et al., 22 Jan 2025)	Federated graph learning	GFlowNet-based personalized context selection

This literature establishes two technically distinct meanings of “personalized feeds.” One is personalized multimedia delivery, where the algorithm chooses feed-delivery actions under network and caching constraints. The other is personalized slate or item generation, where the algorithm samples complete recommendations or feed prefixes according to user-conditioned reward. The second meaning is often an adaptation rather than a directly evaluated feed benchmark; where that occurs, the literature explicitly presents it as transfer or extrapolation rather than a production-validated recommender recipe (Li et al., 2023).

2. Mathematical formulation

The common GFlowNet substrate is a directed acyclic graph or sequential construction process over states $\mathcal{S}$ and actions $\mathcal{A}$. A trajectory is written as

$\tau = (s_0,\dots,s_f),$

and the forward policy induced by a flow network is

$$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$

The target condition is that terminal objects are sampled proportionally to reward,

$\pi(x) \propto R(x),$

or, in the human-feedback setting,

$\pi(s_f) \propto \operatorname{score}(s_f).$

The detailed balance form used in the short-video formulation is

$$F(s) P_F(s' \mid s) = F(s') P_B(s \mid s'), \quad \forall (s \to s') \in \mathcal{A},$$

with terminal condition $F(x)=R(x)$ (Jin et al., 23 Aug 2025).

For personalized feeds, this admits a natural sequential interpretation. A state can denote a partially constructed feed, slate, or session; an action can denote selecting the next item, topic, creator, chunk, or module; and a terminal state can denote a completed feed page or recommendation session. In the GFlowHF formulation, this yields the feed-level analogue

$$\pi(\text{feed slate} \mid \text{user context}) \propto r_\phi(\text{slate}, \text{user context}),$$

where $r_\phi$ is a reward model learned from scores. The central contrast with RLHF is explicit: RLHF tends to maximize expected reward and can collapse onto one high-reward mode, whereas GFlowHF aims to learn the entire reward distribution and sample terminal states strictly proportionally to score (Li et al., 2023).

A second mathematically important line of work treats personalization as task heterogeneity. In pGFlowMeta, terminal-state sampling remains reward-proportional,

$\mathcal{A}$0

but each task $\mathcal{A}$1 receives its own personalized parameter $\mathcal{A}$2, regularized toward a shared meta parameter $\mathcal{A}$3 through

$\mathcal{A}$4

This supports the interpretation of users, cohorts, sessions, or contexts as distinct but related feed-generation tasks (Ji et al., 2023).

3. Direct instantiation: personalized short-video feed control

The clearest direct “GFlowNet-based personalized feeds algorithm” is the short-video system in which users may swipe at any time, making prefetch decisions uncertain and coupling recommendation-side and delivery-side control. At each decision step, the state includes video-related information such as chunk sizes, bitrate options, bitrate of cached segments, recommendation queue contents, and per-video buffer states; network-related information such as bandwidth or throughput traces, rebuffering time, download time, and current network condition category; and user-related information such as historical watch-time or retention behavior, swipe behavior, and personalized preference parameters (Jin et al., 23 Aug 2025).

The action space is not “choose the next content item” in the narrow ranking sense. Instead, each action determines whether to prefetch or pause, which video chunk to download next, and at what bitrate. The simulator also supports a pause or sleep action. Operationally, the GFlowNet generates a ranked list of candidate actions, for example top 10, and a selection module chooses the best one for execution. The implemented method is therefore a generative policy over feed-delivery actions, not a conventional full-feed ranker (Jin et al., 23 Aug 2025).

Its personalized objective is

$\mathcal{A}$5

with

$\mathcal{A}$6

and

$\mathcal{A}$7

Here $\mathcal{A}$8 personalize the quality, stall, and smoothness trade-offs, while $\mathcal{A}$9 controls the trade-off between QoE and bandwidth cost. The goals explicitly include maximizing perceived video quality, minimizing rebuffering, minimizing quality fluctuation, minimizing bandwidth usage, minimizing bandwidth wastage from unwatched prefetched data, adapting to user-specific preferences, and improving overall user satisfaction under network and caching constraints (Jin et al., 23 Aug 2025).

A distinctive implementation detail is the separation between candidate generation and candidate selection. The GFlowNet generates multiple plausible high-reward actions; the system computes the objective value for each candidate under the current personalized preference weights; it executes the single best candidate; and for training it uses the average objective value across all candidates as the reward signal. The paper presents this averaging step as the main adaptation for encouraging the candidate set as a whole not to collapse to a single action (Jin et al., 23 Aug 2025).

This direct short-video formulation is narrower than a full recommender-system slate generator, but it demonstrates an important principle: personalized feeds can be modeled as a sequential control process in which a GFlowNet proposes a distribution over good next actions, and downstream selection chooses one action while preserving broad candidate quality.

4. Learning from feedback, personalization, and offline data

One major route to a personalized feed algorithm is to learn the reward from feedback instead of assuming it is known. GFlowHF does exactly this: sample trajectories, collect human scores on either complete trajectories or final states, train a reward network $\tau = (s_0,\dots,s_f),$0 with $\tau = (s_0,\dots,s_f),$1 as input and $\tau = (s_0,\dots,s_f),$2 as output, and then train a GFlowNet so that terminal-state probability satisfies

$\tau = (s_0,\dots,s_f),$3

The discrete flow-matching loss given there is

$\tau = (s_0,\dots,s_f),$4

with $\tau = (s_0,\dots,s_f),$5 for non-terminal states. For feeds, the paper’s own transfer guidance maps human feedback to explicit or implicit user signals and maps the reward model to a personalized utility model over complete slates or sessions (Li et al., 2023).

Another route is explicit meta-personalization. pGFlowMeta uses a shared meta policy $\tau = (s_0,\dots,s_f),$6 and task-specific personalized policies $\tau = (s_0,\dots,s_f),$7, where each $\tau = (s_0,\dots,s_f),$8 minimizes a task-specific flow-matching loss plus a quadratic proximity term to $\tau = (s_0,\dots,s_f),$9. This is directly relevant when different users or contexts induce heterogeneous transitions, so a single global generator becomes a poor compromise. The paper’s practical lesson is that a strong personalized feed generator should not necessarily be a single global GFlowNet; it can instead be a meta-personalized GFlowNet with global transfer and localized adaptation (Ji et al., 2023).

A third route is offline training from logs without proxy reward queries. Proxy-Free GFlowNet introduces Trajectory-Distilled GFlowNet, which estimates edge-level rewards $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$0 from offline trajectories by inverse reinforcement learning, prunes the DAG using those edge scores, and then trains on backward-sampled trajectories using flow matching. This is especially relevant when unseen-feed reward queries are unsafe or expensive. The pruning rule is

$$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$1

and prioritized backward sampling uses

$$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$2

This suggests a feed analogue in which logged successful slates define high-value placement transitions and low-value transitions are filtered before GFlowNet training (Chen et al., 26 May 2025).

Policy-based training generalizes these ideas further. “GFlowNet Training by Policy Gradients” shows that trajectory-balance optimization can be reframed as policy-gradient reinforcement learning with policy-dependent rewards

$$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$3

and derives actor-critic and TRPO-style updates for the forward policy together with a coupled strategy for backward-policy design. “Evaluating GFlowNet from partial episodes for stable and flexible policy-based training” then introduces Subtrajectory Evaluation Balance (Sub-EB), which learns an evaluator $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$4 over partial episodes: $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$5 For feed generation, this is particularly significant because users often interact with prefixes rather than entire generated feeds, so partial-slate evaluation is structurally closer to observed data (Niu et al., 2024, Niu et al., 1 Mar 2026).

Additional adjacent work broadens the personalization toolkit. FedGrAINS uses a client-specific GFlowNet to adaptively sample informative graph neighbors during federated GNN training, effectively personalizing which contextual evidence influences feed-relevant predictions. GFlowGR treats generative recommendation as multi-step item-token generation and fine-tunes it with DB or TB objectives so that output probability becomes proportional to a reward composed of augmentation signal, collaborative score, and token similarity, explicitly targeting exposure-bias mitigation (Ceyani et al., 22 Jan 2025, Wang et al., 19 Jun 2025).

5. Empirical evidence

The most direct quantitative evidence comes from the short-video feed case study. On low, medium, and high bandwidth conditions, the reported normalized QoE values for the GFlowNet-based method are $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$6, compared with $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$7 for Incendio, $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$8 for DAM, and $$\pi(a_t \mid s_t) = \frac{F_{\theta}(s_t,a_t)}{F_{\theta}(s_t)}.$$9 for PDAS. Rebuffering for the GFlowNet-based method is $\pi(x) \propto R(x),$0 seconds; bandwidth usage is $\pi(x) \propto R(x),$1 MB; and bandwidth wastage is $\pi(x) \propto R(x),$2. The paper summarizes these results as roughly $\pi(x) \propto R(x),$3 QoE improvement in low bandwidth, $\pi(x) \propto R(x),$4 in medium bandwidth, $\pi(x) \propto R(x),$5 in high bandwidth, plus $\pi(x) \propto R(x),$6 lower bandwidth usage than the strongest baselines. Its multi-candidate ablation reports QoE $\pi(x) \propto R(x),$7 for single-candidate versus $\pi(x) \propto R(x),$8 for multi-candidate, rebuffering $\pi(x) \propto R(x),$9s versus $\pi(s_f) \propto \operatorname{score}(s_f).$0s, and bandwidth wastage $\pi(s_f) \propto \operatorname{score}(s_f).$1 versus $\pi(s_f) \propto \operatorname{score}(s_f).$2, identifying multi-candidate generation as the central mechanism (Jin et al., 23 Aug 2025).

The human-feedback formulation provides evidence for the same diversity-oriented principle in a different domain. In the Point-Robot task with two rewarding goals, GFlowHF explores both targets while RLHF-style baselines often explore only one, achieves more valid-distinctive answers, and maintains slightly higher average reward. Under noisy labels, a wrong high-score label injected at $\pi(s_f) \propto \operatorname{score}(s_f).$3 with score $\pi(s_f) \propto \operatorname{score}(s_f).$4 distracts RLHF baselines more than GFlowHF, which the paper interprets as evidence that learning the overall reward distribution is more robust than chasing isolated maxima (Li et al., 2023).

Meta-personalization experiments reinforce the importance of explicit task adaptation. On Grid World, Frozen Lake, and Cliff Walking, pGFlowMeta reports the lowest empirical $\pi(s_f) \propto \operatorname{score}(s_f).$5 error, finds modes faster, achieves the best average rewards, and shows that personalized policy performance is consistently better than meta-policy performance on task-specific evaluation. The paper’s strongest feed-relevant conclusion is that explicit personalized adaptation outperforms a single shared meta-policy when tasks have heterogeneous transitions (Ji et al., 2023).

Exposure-bias mitigation in generative recommendation shows a parallel pattern. GFlowGR-DB and especially GFlowGR-TB outperform SFT and several RLFT or preference-learning baselines on Amazon Beauty and Yelp. Representative $\pi(s_f) \propto \operatorname{score}(s_f).$6 values include $\pi(s_f) \propto \operatorname{score}(s_f).$7 for TIGER-SFT versus $\pi(s_f) \propto \operatorname{score}(s_f).$8 for GFlowGR-TB on Beauty, and $\pi(s_f) \propto \operatorname{score}(s_f).$9 versus $$F(s) P_F(s' \mid s) = F(s') P_B(s \mid s'), \quad \forall (s \to s') \in \mathcal{A},$$0 on Yelp. The paper also reports lower KL divergence to a normal item-distribution reference and a broader collaborative-score range, interpreting these as less concentrated and more diverse generation (Wang et al., 19 Jun 2025).

Work on training dynamics provides supporting evidence that better flow learning matters even when the end task is not a feed benchmark. “Towards Understanding and Improving GFlowNet Training” reports that prioritized replay training, relative edge flow policy parametrization, and guided trajectory balance can improve sample efficiency and mode discovery substantially, while Sub-EB-based policy evaluation improves stability on harder combinatorial tasks and allows parameterized backward policies and offline data-collection techniques to be integrated into policy-based training (Shen et al., 2023, Niu et al., 1 Mar 2026).

6. Interpretation, limitations, and common misconceptions

A frequent misconception is that a GFlowNet-based personalized feed is simply a ranking model with stochastic sampling. The direct short-video paper contradicts that simplification: its algorithm is a multimedia control system whose outputs are prefetch, pause, chunk, and bitrate actions, and it does not output a full ranked feed of content items in the conventional recommender sense (Jin et al., 23 Aug 2025). A second misconception is that GFlowNet personalization is equivalent to RLHF-style “optimize the favorite response.” The human-feedback formulation explicitly rejects that reading; its aim is to learn the entire reward distribution and satisfy $$F(s) P_F(s' \mid s) = F(s') P_B(s \mid s'), \quad \forall (s \to s') \in \mathcal{A},$$1, not merely maximize the single best mode (Li et al., 2023).

The literature also places clear boundaries on current evidence. Production feeds involve millions of candidate items, position bias, delayed feedback, non-stationary preferences, real-time latency limits, and substantial exposure confounding. The direct short-video study does not report confidence intervals or significance tests, training curves, sensitivity to number of candidates $$F(s) P_F(s' \mid s) = F(s') P_B(s \mid s'), \quad \forall (s \to s') \in \mathcal{A},$$2, latency or compute overhead, or online A/B evaluation; it also does not fully formalize how the average-candidate reward is inserted into FM or TB training for the deployed controller (Jin et al., 23 Aug 2025). Likewise, pGFlowMeta, GFlowHF, and several policy-based papers operate in small or stylized DAG environments, so their feed interpretations remain principled extrapolations rather than direct production validations (Ji et al., 2023, Niu et al., 1 Mar 2026).

A further limitation concerns reward specification. Many formulations assume a known positive terminal reward or a reward model that can be fit from explicit scores, while real feeds mostly observe noisy implicit behavior. Proxy-Free GFlowNet addresses this by learning edge-level supervision from offline trajectories rather than querying unseen rewards, but it still assumes deterministic DAG structure and does not solve recommender-specific confounding. FedGrAINS addresses privacy-sensitive graph heterogeneity by learning client-specific context samplers, yet its base task is node classification rather than ranking. These results suggest that a complete feed system would likely combine several ingredients: user-conditioned reward modeling, candidate retrieval, structured or graph-based context selection, flow-based or policy-based GFlowNet training, and a serving-time selector layered with safety and business constraints (Chen et al., 26 May 2025, Ceyani et al., 22 Jan 2025).

The most defensible synthesis is therefore narrow but substantial. A GFlowNet-Based Personalized Feeds Algorithm is best understood as a reward-proportional generative policy for personalized feed construction or feed-delivery control. Its defining property is the preservation of probability mass over multiple high-utility outcomes; its main technical realizations include flow matching, trajectory balance, policy-gradient and partial-episode evaluators, meta-personalized policies, and offline edge-reward training; and its strongest direct evidence currently comes from personalized short-video multimedia control, with broader slate-generation and recommendation formulations supported by adjacent GFlowNet research rather than by a single unified benchmark suite (Jin et al., 23 Aug 2025).