---
title: Personalized Content Transformation
url: https://www.emergentmind.com/topics/personalized-content-transformation
type: topic
---

# Personalized Content Transformation

Searching arXiv for recent and foundational papers on personalized content transformation and adjacent personalized generation systems.
Personalized content transformation denotes a class of systems that adapt, rewrite, stylize, rank, or synthesize content for a particular user, subject, or audience by combining user-specific context with task-specific instructions. In the unified perspective of personalized generation, the core ingredients are personalized contexts, users’ multimodal instructions, user modeling, and generative modeling, with the universal objectives of high quality, instruction alignment, and personalization [2503.02614]. Across recent work, the term covers subject-preserving style transfer in diffusion models, teacher-controlled educational rewriting, user-centric creative generation in advertising, preference-aligned multimodal synthesis, and recipient-side moderation that alters sensitive elements while preserving informational value [2403.19456] [2407.09484] [2508.18118] [2509.22861].

## 1. Conceptual scope and boundary conditions

A recurring distinction in the literature is between selecting existing content and transforming or generating new content. “Generate, Not Recommend” explicitly argues that recommendation is constrained to filtering existing items and cannot synthesize novel concepts tailored to a user’s latent tastes, whereas personalized multimodal generation aims to create a new item directly from multimodal interaction history [2506.01704]. “Personalized Image Generation with Large Multimodal Models” makes the same contrast for images, arguing that filtering systems only select among already-existing items, while personalized generation synthesizes a new image conditioned jointly on user history and a new multimodal instruction [2410.14170]. TailorMind sharpens the point by requiring generated content \(\mathcal{O}\) to satisfy \(\mathcal{O} \notin \mathcal{V}\), where \(\mathcal{V}\) is the existing item pool [2606.23643].

At the same time, many practical systems are better described as transformation pipelines than unconstrained generators. GPTutor starts from educator-supplied course material, then rewrites and extends it into a learner-specific curriculum, lesson content, analogies, and practice items [2407.09484]. “One-Topic-Doesn’t-Fit-All” starts from existing RACE-C reading-comprehension tests and transcreates them so that passages become semantically aligned with learner interests while remaining linguistically similar to the originals [2511.09135]. HLLM-Creator takes a source advertisement and generates a personalized ad title that preserves product facts but changes framing and emphasis to better match inferred user interests [2508.18118]. DIY-MOD operates even more explicitly at the transformation level: instead of suppressing content, it modifies sensitive elements in real time while preserving surrounding informational and social value [2509.22861].

This suggests a spectrum rather than a strict dichotomy. At one end are systems that generate new multimodal artifacts from user history; in the middle are systems that rewrite or restyle source content under user control; at the other end are systems that personalize presentation through ranking, filtering, or modulation of existing content. “Controlled Personalization in Legacy Media Online Services” is an example of the latter: personalization appears as a bounded ranking component inside an editorially governed presentation pipeline rather than as content rewriting [2510.09136]. “Real-Time Personalized Content Adaptation through Matrix Factorization and Context-Aware Federated Learning” goes further toward ranking and filtering, and the paper itself is best interpreted as a personalized content adaptation system rather than a full-fledged rewriting engine [2511.18489].

## 2. Personalization signals and user modeling

The survey formalizes personalized contexts as user profiles, user documents, user behaviors, personal face/body, and personalized subjects [2503.02614]. Recent systems instantiate these signals in markedly different ways. In education-oriented systems, the explicit user model is often lightweight. GPTutor collects interests and career goals from students and combines them with teacher-authored summaries, scope descriptions, learning goals, and example exercises [2407.09484]. PAGE expands the profile to academic major, year or grade level, general learning preferences, personal interests, institutional estimates of knowledge level, and self-reported information used to estimate motivation and engagement [2509.15068]. “One-Topic-Doesn’t-Fit-All” uses nine categories and 33 subcategories from the 2022 English curriculum in South Korea, with students rating their interests on a 7-point Likert scale and identifying top and least interesting topics [2511.09135].

Other systems infer user preferences from behavioral traces. HLLM-Creator represents a user by a historical click sequence \(H_u=\{I_1,I_2,\dots,I_n\}\), encodes each clicked ad with an Item LLM, aggregates them with a User LLM, and uses the resulting user embedding \(U\) to condition title generation [2508.18118]. Zero-Shot Reasoning for game personalization uses recent gameplay telemetry such as remaining moves, failed moves, clicks on the board, boosters used, and explicit 1–5 level ratings, then asks GPT-4 to infer a player type and output the next three levels as JSON-formatted parameters [2402.10133]. TailorMind begins from historical interactions \(\mathcal{D}_u\), enriches them with hypergraph collaborative filtering, textualizes multimodal traces, and optimizes a natural-language profile with ranking-error feedback and textual gradient descent [2606.23643]. TIMGen models the user as a temporally ordered multimodal interaction sequence \(H_u=\{x_1,x_2,\dots,x_T\}\), then learns dynamic interest states \(z_t=g(x_1,\dots,x_t)\) rather than a single static preference vector [2509.04330].

Psychographic and identity-centered profiling also appears. SoMin.ai predicts MBTI dichotomies—Introversion/Extroversion, Sensing/iNtuition, Thinking/Feeling, and Judging/Perceiving—from public Instagram and Twitter content using a “deep multi-view personality profiling framework” that fuses a BiGRU text branch with a fine-tuned VGG16 image branch [2011.14615]. GRAVITY constructs a structured profile \(P_u\) with demographics, product interests, intrinsic values and beliefs, and Big Five personality traits, grounding value inference in Hofstede’s cultural dimensions, Schwartz’s basic values, and the World Values Survey [2510.11952]. Pigeon and CRAFT-LoRA emphasize identity-bearing visual signals: Pigeon infers visual preference from a user’s interacted history images \(\mathcal{H}=\{\mathbf{x}_i\}_{i=1}^{N}\) plus a multimodal instruction \(\mathcal{R}=\{\mathbf{x}_0,\mathbf{txt}\}\), while CRAFT-LoRA separates personalized content concepts from style concepts so that identity or structure can be preserved under style transfer [2410.14170] [2602.18936].

A common pattern across these systems is the coexistence of explicit and implicit signals. Interests, major, career goals, or prompt markers such as `<c>` and `<s>` are explicit controls; clicks, reviews, search history, or image interaction sequences are implicit evidence. This suggests that personalized content transformation is increasingly built on hybrid user models rather than on a single source of preference evidence.

## 3. Architectures, optimization strategies, and transformation pipelines

A general workflow emerges from the survey: user modeling first, then generative modeling, with representation learning, prompt engineering, or retrieval-augmented generation on the user side, and foundation models plus instruction or structural guidance plus tuning-free, supervised, or preference-based optimization on the generation side [2503.02614]. Several recent systems make this workflow unusually explicit.

In image personalization, Break-for-Make rethinks the adapter parameter space itself. Instead of learning content and style in one shared LoRA-like adaptation space, it introduces partly learnable projection matrices, breaks adapters into “up projection” and “down projection,” trains content and style PLPs individually, preserves generalization through a multi-correspondence projection learning strategy, reconstructs the entity parameter space, and then fine-tunes the combined adapter [2403.19456]. In standard LoRA notation, the underlying low-rank adaptation is
\[
W' = W + \Delta W,\qquad \Delta W = BA,
\]
but Break-for-Make’s contribution is to divide the low-rank pathway into separate sub-parameter spaces for content and style before recombination [2403.19456]. CRAFT-LoRA advances the same problem by first producing a rank-constrained backbone \(W_{\text{init}}\), then training content and style LoRAs on disjoint U-Net layer subsets, and finally performing training-free selective adapter aggregation
\[
W^{\mathrm{agg}} = W_{\text{init}} + \sum_{i\in I_c}\mathsf{E}_i\!\left(\gamma_c\,\Delta W^{(c)}_i\right) + \sum_{i\in I_s}\mathsf{E}_i\!\left(\gamma_s\,\Delta W^{(s)}_i\right),
\]
with asymmetric classifier-free guidance during denoising [2602.18936].

In educational transformation, GPTutor uses a “curriculum-first, content-second” decomposition. Teachers upload curriculum content, summaries, scope descriptions, learning goals, and optional examples; students input interests or career goals; GPT-4 first generates a personalized curriculum and then generates personalized learning content and practice based on the student’s profile, the original course material, the tailored curriculum, and teacher examples [2407.09484]. PAGE implements a retrieval-augmented variant of this pattern: it creates a cognitive-affective student profile, generates 3–5 search keywords per content segment, retrieves and chunks external documents, embeds them into a personalized knowledge base \(K_{s_i,P}\), selects the top-\(k\) chunks with \(k=5\), and then adapts only eligible segments with a prompt explicitly constrained to make a maximum of 0–2 subtle adjustments while preserving core concepts and logical structure [2509.15068]. “One-Topic-Doesn’t-Fit-All” decomposes transformation into topic extraction, Bloom-level question analysis, linguistic feature extraction with 41 special tags, passage transcreation, question transcreation, and human validation of answerability [2511.09135].

Advertising and recommendation-adjacent systems show a different architecture. HLLM-Creator defines personalized title generation as \(p_{\Theta}(y\mid u,I)\), where the user is encoded hierarchically from historical clicked ads and the target ad is encoded by a Creative LLM; the final model is trained with a generative loss plus click-prediction, alignment, and reconstruction auxiliaries [2508.18118]. TailorMind factorizes the problem into a behavioral preference model
\[
f_R:(u,\mathcal{D}_u,\mathcal{V}) \rightarrow \mathbf{h}_u
\]
and a generation function
\[
f_G:(\mathbf{t}, \{\mathbf{m}_1,\mathbf{m}_2,\ldots\}) \rightarrow \mathcal{O},
\]
then adds retrieval-augmented style control and cross-modal cohesion reflection to stabilize the downstream multimodal generator [2606.23643]. TIMGen formalizes a temporal variant with a Transformer over interaction sequences, modality-specific encoders such as BERT, ViT, Video Transformer, and wav2vec 2.0, an attention-based multimodal fusion layer, and a VAE-based generator conditioned on a unified interest vector \(z_{\text{final}}\) and candidate item features \(x^*\) [2509.04330].

Finally, several systems show that “personalization” can be implemented through real-time serving logic rather than content rewriting. Controlled personalization in news introduces a personalized score \(s_4\) into a composite article score
\[
{\text{cs}_{i,u}} = \frac{\sum_{n=1}^{3} (w_n s_n) + w_4 s_4}{\sum_{n=1}^{3} (w_n) + w_4},
\]
with \(w_4=0.20\), thereby bounding personalization to at most 20% of the ranking score [2510.09136]. DIY-MOD uses a two-stage intervention pipeline with a predictive pruner and a scorer over transformation candidates such as blur, occlusion, inpainting, replacement, rewriting, and artistic stylization, selecting the final intervention by summed scores over transformation seamlessness, semantic fidelity, predicted emotional impact, and transformation appropriateness [2509.22861].

## 4. Modalities and application domains

The survey organizes the area by modality—text, image, video, 3D, audio, and cross-modal generation—and recent systems populate each branch with concrete transformation tasks [2503.02614]. Text-centric transformation includes personalized writing assistance, headline generation, dialogue, and educational rewriting. GPTutor personalizes explanations, examples, exercises, and feedback around learner interests such as *Jujutsu Kaisen* while preserving the original instructional target [2407.09484]. PAGE adapts lecture content to each student’s major and interests, but restricts personalization mostly to illustrative components like examples and analogies [2509.15068]. “One-Topic-Doesn’t-Fit-All” transcreates reading passages and multiple-choice questions so that the topic changes but the reading level, linguistic profile, and Bloom-level intent remain comparable [2511.09135]. GRAVITY rewrites generic book descriptions into profile-grounded personalized descriptions conditioned on demographics, interests, values, beliefs, and personality traits [2510.11952].

Image-centric transformation remains a major technical frontier. Break-for-Make and CRAFT-LoRA both address the composition of content and style under few-shot personalized generation [2403.19456] [2602.18936]. Pigeon frames the task as generating a personalized target image \(\mathbf{x}_{N+1}\) from user-interacted history images plus a multimodal instruction containing a reference image and text [2410.14170]. “Generate, Not Recommend” uses Janus-Pro-1B to generate personalized movie posters and micro-video covers directly from multimodal interaction histories rather than retrieving them from a catalog [2506.01704]. TailorMind likewise targets personalized multimodal content generation when relevant UGC is absent [2606.23643].

Video and audio transformation appear mainly in the survey’s taxonomy: subject-driven text-to-video, ID-preserving video generation, talking head generation, pose-guided video generation, video virtual try-on, music generation from user behavior, and face-to-speech generation are all treated as forms of personalized generation driven by personal face/body signals, subject exemplars, or behavioral preference traces [2503.02614]. A narrower but concrete multimodal transformation example appears in the federated social-media adaptation paper, where videos are transformed into frames, textual descriptions, and embeddings, then used in retrieval-augmented generation for “smart video querying” [2511.18489].

Application domains are correspondingly broad. Advertising and marketing include SoMin.ai’s personality-driven variant generation from branded assets, HLLM-Creator’s personalized search-ad titles, and TailorMind’s user-tailored multimodal content beyond existing UGC [2011.14615] [2508.18118] [2606.23643]. Education includes GPTutor, PAGE, and transcreated reading-comprehension tests [2407.09484] [2509.15068] [2511.09135]. Entertainment and games include Match-3 level generation from telemetry and personalized image or poster generation from historical taste [2402.10133] [2410.14170]. Moderation introduces a distinctive domain-specific meaning of transformation: DIY-MOD changes the presentation of harmful elements while preserving access to lawful content [2509.22861]. News personalization, by contrast, shows that legacy media may prefer ranking transformation over direct rewriting [2510.09136]. This variety suggests that personalized content transformation is less a single task than a systems pattern that recurs wherever user-specific relevance must be achieved without sacrificing core content constraints.

## 5. Evaluation practices and empirical findings

Evaluation is typically multi-dimensional. The survey highlights three universal objectives—high quality, instruction alignment, and personalization—and notes that only the target user can fully judge preference fit, which is why many systems mix automatic metrics with human evaluation or deployment metrics [2503.02614]. In practice, recent papers evaluate different slices of the problem.

In education, “One-Topic-Doesn’t-Fit-All” reports a controlled experiment with 20 Korean EFL learners. Group B, which received personalized transcreated readings, improved from \(73.33_{\pm 15.12}\) to \(86.67_{\pm 8.28}\), whereas Group A, with randomly assigned topics, went from \(76.50_{\pm 11.11}\) to \(83.00_{\pm 9.39}\); Group B also improved by \(+1.33_{\pm 0.71}\) on Analyze-level questions, reduced turnaround time from \(30.78_{\pm 7.60}\) to \(27.78_{\pm 5.30}\) minutes, and showed better motivation retention \((4.68_{\pm 0.66} \rightarrow 4.61_{\pm 1.04})\) than Group A \((4.82_{\pm 0.38} \rightarrow 4.44_{\pm 0.85})\) [2511.09135]. PAGE reports blind expert evaluation over 60 samples from five university-level courses, where PAGE scored 77.8 on Instructional Accuracy, 75.5 on Expressive Clarity, 76.2 on Logical Coherence, 87.0 on Student Engagement, 86.0 on Linguistic Naturalness, and 92.2 on Personalization Relevance, outperforming human-authored materials and multiple LLM baselines [2509.15068]. GPTutor, by contrast, is mainly a demonstration paper; its strongest empirical systems claim is stress testing up to 100 simultaneous calls without issues, while learning-effectiveness evidence is not yet reported [2407.09484].

Advertising and industrial systems are evaluated more directly on downstream behavior. HLLM-Creator reports an online A/B test on Douyin Search Ads with a 0.476% increase on Adss, a 0.297% increase on Advv, and a 0.510% increase on RankAdvv; for ads where personalized titles won the candidate-selection stage, CTR increased by 1.789% [2508.18118]. Zero-Shot Reasoning in a live Match-3 deployment reports overall completion rates of 55% for LLM-generated levels versus 35% for traditional PCG, and first-level completion of 34% versus 18%, with near-certain Bayesian support for the completion advantage [2402.10133]. Controlled personalization in legacy news shows a CTR increase from 0.524 to 0.601, a relative increase of +14.60%, while impressions per user decreased from 22.462 to 19.834 and daily click coverage increased from 0.902 to 0.938 [2510.09136].

In personalized image generation, Pigeon reports overall scores of 44.38 on sticker generation and 33.31 on movie-poster generation, outperforming strong baselines such as TI, and human evaluation found Pigeon strongly preferred on personalization while also beating TI and PMG on semantic alignment [2410.14170]. Break-for-Make states that it outperforms state-of-the-art single/multiple concept learning pipelines in terms of content-style-prompt alignment [2403.19456]. CRAFT-LoRA reports content similarity 0.79, style similarity 0.80, and GPT-4o combination score 0.83, compared with 0.70/0.69/0.73 for ZipLoRA and 0.74/0.70/0.77 for BLoRA; in user study, CRAFT-LoRA received 82.5, 85.0, and 87.5 on content fidelity, style fidelity, and coherence, respectively [2602.18936]. TailorMind reports competitive or stronger coherence, improved novelty and aesthetic quality over representative generation baselines and ground-truth UGC, and up to 29% Recall gains in reranking [2606.23643].

The following representative results illustrate how heterogeneous the evaluation landscape is.

| System | Domain | Representative finding |
|---|---|---|
| PAGE | Education | Personalization Relevance 92.2; Overall 82.4 [2509.15068] |
| One-Topic-Doesn’t-Fit-All | EFL assessment | \(73.33_{\pm 15.12} \rightarrow 86.67_{\pm 8.28}\) for personalized group [2511.09135] |
| HLLM-Creator | Search ads | Adss +0.476%; CTR +1.789% when personalized titles won [2508.18118] |
| Zero-Shot Reasoning | Games | 55% vs 35% overall completion; 34% vs 18% first-level completion [2402.10133] |
| Controlled Personalization | News | CTR 0.524 \(\rightarrow\) 0.601; daily click coverage 0.902 \(\rightarrow\) 0.938 [2510.09136] |
| CRAFT-LoRA | Personalized image generation | Content 0.79; Style 0.80; Combination 0.83 [2602.18936] |

A notable pattern is that evaluation often measures only part of the transformation problem. SoMin.ai reports Macro F1 for personality inference—e.g., BiGRU+VGG fused achieves EI 0.83, SN 0.57, TF 0.64, JP 0.60—but does not provide quantitative generation metrics [2011.14615]. Several generation papers likewise show strong perceptual or alignment results but limited evidence on long-term user satisfaction. This suggests that benchmarking still lags behind the breadth of the problem itself.

## 6. Limitations, controversies, and open directions

Several recurring limitations define the current state of the field. First is the tension between personalization quality and data availability. The survey identifies evolving user preference, user data management, benchmark fragmentation, and the lack of robust personalization metrics as core challenges [2503.02614]. TIMGen explicitly motivates dynamic interest modeling because static preference vectors cannot represent short-term drift or changing modality importance, yet the paper does not provide concrete experimental results validating the proposed architecture [2509.04330]. TailorMind addresses sparse histories with hypergraph collaborative filtering and profile optimization, which suggests that generation-ready preference inference remains a central bottleneck [2606.23643].

Second is the boundary between “generation” and “selection.” SoMin.ai describes personality-driven content generation, but its concrete pipeline is closer to personality-driven example selection plus style-based variant generation from existing branded assets rather than fully personality-conditioned GAN generation [2011.14615]. The federated social-media adaptation paper is primarily a ranking and filtering system with a multimodal video-understanding component, not a direct rewriting engine [2511.18489]. Controlled personalization in news is even more explicitly a presentation-layer intervention. These cases do not diminish the relevance of the work, but they complicate claims that all personalized content systems are generative in the same sense.

Third is safety, privacy, and fairness. SoMin.ai depends on MBTI, which the paper itself acknowledges is controversial and coarse for content personalization; it also infers personality from public social-media content, raising privacy and consent concerns [2011.14615]. Edge-assisted collaborative fine-tuning encodes prompts prior to transmission and assumes a non-colluding Training Edge Server and Inference Edge Server, but the paper is explicit that this is semantic obfuscation rather than a rigorous differential-privacy guarantee [2508.04745]. DIY-MOD foregrounds agency and recipient-side control, yet also raises questions about over-customization, cognitive avoidance, and transformation appropriateness in sensitive civic or traumatic contexts [2509.22861]. PAGE warns about over-personalization narrowing exposure to diverse perspectives [2509.15068]. Controlled personalization in legacy news is motivated precisely by the need to avoid loss of shared reality and filter-bubble effects [2510.09136].

Fourth is factuality and semantic stability. GPTutor relies on a curriculum-first generation stage partly to reduce hallucination [2407.09484]. “One-Topic-Doesn’t-Fit-All” found that 2.8% of transcreated questions were unanswerable and that all such failures came from a single question involving temporal relations between two events; fixing them required an expert to add on average 1.7 words per modified passage [2511.09135]. HLLM-Creator keeps hallucination control central, reporting hallucination pass rate as low as 9% when title input is removed and 75% for the full system [2508.18118]. In multimodal generation, “Generate, Not Recommend” reports reward hacking on MovieLens, where RL tended to generate popular posters such as *The Godfather*, and CRAFT-LoRA notes failure cases for extremely abstract styles or severe content-style mismatches [2506.01704] [2602.18936].

Open directions are therefore converging around several themes: scalable yet expressive user modeling, temporal preference adaptation, retrieval-generation synergy, stronger multimodal foundation models, evaluation that measures personalization rather than only generic quality, and governance mechanisms for privacy, fairness, and exposure diversity [2503.02614]. This suggests that the mature form of personalized content transformation will likely be neither pure recommendation nor pure unconstrained generation, but a layered system in which user modeling, retrieval, controllable generation, and post-generation validation operate together under explicit domain constraints.

Source: https://www.emergentmind.com/topics/personalized-content-transformation