TM-Rephrase: Data-Centric Text Rewriting

• The paper introduces TM-Rephrase as a data-centric pipeline that rephrases raw social media inputs to enhance model-agnostic topic modeling.
• It details two rephrasing strategies—general and colloquial-to-formal—that balance grammatical correction with preservation of domain-specific terms.
• Empirical results show significant improvements in topic coherence and diversity, achieving over 60% relative gain in LDA coherence using colloquial-to-formal rephrasing.

Searching arXiv for the specified papers and closely related uses of “TM-Rephrase” to ground the article in current literature. arXiv search query: id:(Xin et al., 21 Oct 2025) OR id:(Yang et al., 2024) OR id:(Mekala et al., 2023) OR id:(Shao et al., 2022) OR id:(Einolghozati et al., 2020) OR id:(Botha et al., 2018) OR id:(Al-Fedaghi, 2021) TM-Rephrase is a rephrasing-based intervention in which text is rewritten before a downstream analytic or predictive procedure. In the arXiv literature, the term is used explicitly for a model-agnostic, data-centric pipeline that standardizes short social-media posts prior to topic modeling, especially for noisy COVID-19 tweets, and separately for a zero-access protocol that estimates the uncertainty of closed-source LLMs through agreement across multiple rephrased queries (Xin et al., 21 Oct 2025, Yang et al., 2024). The common mechanism is external textual transformation rather than modification of the downstream model’s internal inference algorithm. In the topic-modeling setting, TM-Rephrase is designed to preserve original meaning and domain-specific terms while converting raw tweets into more formal, grammatically sound English, thereby improving topic coherence, topic uniqueness, topic diversity, and interpretability on short, informal texts (Xin et al., 21 Oct 2025).

1. Core definition and scope

In "Improving Topic Modeling of Social Media Short Texts with Rephrasing: A Case Study of COVID-19 Related Tweets" (Xin et al., 21 Oct 2025), TM-Rephrase is defined as a model-agnostic, data-centric pipeline positioned between raw tweet collection and downstream topic modeling. Its central idea is to leverage a LLM to rephrase each tweet into formal, grammatically sound English while preserving original meaning and domain-specific terms. The pipeline consists of six stages: data collection, optional preliminary cleaning, LLM-based rephrasing, standard NLP preprocessing, topic modeling, and quantitative plus qualitative evaluation.

This formulation is explicitly model-agnostic because it does not alter the internals of LDA, BERTopic, FASTopic, or TSCTM. Instead, it augments the raw textual inputs so that richer lexical and contextual signals are available to the topic model. A common misconception is that improvements in short-text topic modeling necessarily require redesigning the topic model itself. TM-Rephrase is presented as a counterexample: the intervention is upstream and data-centric rather than algorithm-centric (Xin et al., 21 Oct 2025).

A second use of the name appears in "Just rephrase it! Uncertainty estimation in closed-source LLMs via multiple rephrased queries" (Yang et al., 2024). There, TM-Rephrase denotes "Target-Model Rephrase for Uncertainty Estimation," a protocol that asks the same closed-source LLM several linguistically altered versions of a base question and interprets answer agreement as a confidence proxy. This suggests that, across distinct subfields, TM-Rephrase names a broader methodological pattern: rephrasing is used as an external control surface for systems that otherwise expose limited or noisy behavior.

2. Architecture for short-text topic modeling

For topic modeling, TM-Rephrase was evaluated on a dataset of 25,027 COVID-19-related Twitter replies collected from May 2020 to November 2022 as public replies to seven official CDC Twitter accounts (Xin et al., 21 Oct 2025). The initial corpus contained 180,090 interactions; after filtering out private mentions and deleted originals, the final dataset comprised 25,027 replies linked to 1,512 CDC posts. The replies had mean word count 27.33, standard deviation 14.23, minimum 1, and maximum 64.

Before rephrasing, the corpus underwent removal of URLs, emojis, and non-ASCII characters, followed by lowercasing, tokenization, stop-word removal, punctuation stripping, and lemmatization. Because the task is unsupervised, no train-test split was used; all 25K tweets were used to estimate topics and compute evaluation metrics (Xin et al., 21 Oct 2025).

Two prompt-driven rephrasing schemes were implemented with Google Gemini (gemini-2.5-flash):

Strategy	Transformation	Fidelity control
General rephrasing	Improves grammar, clarity, and tone while keeping meaning exactly the same	Output checked so named entities, hashtags, and @-mentions remain unchanged
Colloquial-to-formal rephrasing	Converts tweets into formal, professional, and standard English suitable for a public health report	Same entity check plus edit-distance thresholding on key terms

The general strategy performs light formalization: it corrects typos, fixes grammar, smooths informal phrasing, and preserves jargon and hashtags. The colloquial-to-formal strategy performs strong formalization: it replaces slang with formal equivalents, expands contractions, removes casual tone, and uses complete sentences and proper grammar. In both cases, outputs that alter or drop named entities, hashtags, usernames, or specific terms are discarded or re-submitted; the stronger strategy also uses a secondary edit-distance filter to reduce semantic drift (Xin et al., 21 Oct 2025).

3. Topic models and evaluation formalism

TM-Rephrase was tested with four representative topic modeling methods under uniform settings of $K = 8$ topics and $N = 15$ top words per topic (Xin et al., 21 Oct 2025). The three input conditions were original cleaned tweets, general-rephrased tweets, and colloquial-to-formal-rephrased tweets.

The four topic modeling back ends were:

• LDA: a generative model with topic-word distributions $\phi_k \sim \mathrm{Dirichlet}(\beta)$, document-topic proportions $\theta_d \sim \mathrm{Dirichlet}(\alpha)$, latent assignments $z_{d,n} \sim \mathrm{Multinomial}(\theta_d)$, and observed words $w_{d,n} \sim \mathrm{Multinomial}(\phi_{z_{d,n}})$.
• BERTopic: Sentence-BERT embeddings, UMAP dimensionality reduction, HDBSCAN clustering, and class-based TF-IDF keyword extraction.
• FASTopic: joint learning of topic and word embeddings with an optimal transport regularizer aligning semantic relations between document and topic embeddings.
• TSCTM: dense document representations and topic distributions coupled with a contrastive loss that encourages semantically similar documents in topic-distribution space to have closer embeddings.

Evaluation used four topic-quality metrics. Topic coherence $C_v$ is based on Normalized Pointwise Mutual Information over word co-occurrences in Wikipedia, with pairwise NPMI scores aggregated through NPMI-vector embeddings and average cosine similarity. Topic uniqueness is

$$TU = \frac{1}{K} \sum_{k=1}^{K} \left(\frac{1}{N} \sum_{x_i \in T_k} \frac{1}{\mathrm{freq}(x_i)} \right),$$

where $\mathrm{freq}(x_i)$ is the number of topics whose top-$N$ list contains $N = 15$0. Topic redundancy is

$N = 15$1

with lower values preferred. Topic diversity is

$N = 15$2

These metrics quantify complementary aspects of topic quality: semantic interpretability, distinctiveness of topic vocabularies, overlap across topics, and exclusivity of top words (Xin et al., 21 Oct 2025).

4. Empirical behavior and model-specific effects

The principal empirical finding is that TM-Rephrase improves topic coherence, topic uniqueness, and topic diversity while reducing topic redundancy for most topic modeling algorithms, with colloquial-to-formal rephrasing yielding the greatest performance gains and especially for LDA (Xin et al., 21 Oct 2025).

For LDA, the reported values were:

• without rephrasing: $N = 15$3, $N = 15$4, $N = 15$5, $N = 15$6
• with general rephrasing: $N = 15$7, $N = 15$8, $N = 15$9, $\phi_k \sim \mathrm{Dirichlet}(\beta)$0
• with colloquial-to-formal rephrasing: $\phi_k \sim \mathrm{Dirichlet}(\beta)$1, $\phi_k \sim \mathrm{Dirichlet}(\beta)$2, $\phi_k \sim \mathrm{Dirichlet}(\beta)$3, $\phi_k \sim \mathrm{Dirichlet}(\beta)$4

The paper characterizes this as LDA coherence increasing from 0.3094 to 0.4206 to 0.5004, a $\phi_k \sim \mathrm{Dirichlet}(\beta)$5 relative gain (Xin et al., 21 Oct 2025). At the same time, topic uniqueness and topic diversity remained similar for LDA, with a slight dip under rephrased inputs, while topic redundancy rose marginally. The stated interpretation is a small trade-off between distinctiveness and coherence gains, with stronger formality sometimes reusing high-frequency terms in multiple topics.

Across models, LDA exhibited the largest coherence improvement from rephrasing, and the paper attributes this to probabilistic models being most sensitive to richer lexical context. Colloquial-to-formal rephrasing was the single best scheme for LDA and also BERTopic because maximal standardization makes word co-occurrence statistics more reliable. By contrast, neural models such as FASTopic and TSCTM showed smaller coherence gains, but under colloquial-to-formal rephrasing they achieved perfect diversity ($\phi_k \sim \mathrm{Dirichlet}(\beta)$6) and zero redundancy ($\phi_k \sim \mathrm{Dirichlet}(\beta)$7), indicating fully disjoint topics, albeit with the possibility of some coherence drop for FASTopic (Xin et al., 21 Oct 2025).

5. Trade-offs, limitations, and analytic uses

TM-Rephrase is not presented as cost-free. Rephrasing 25K tweets through an LLM API adds latency and cost, which may be a barrier for very large datasets or low-budget projects (Xin et al., 21 Oct 2025). Semantic drift remains a risk, particularly for the colloquial-to-formal scheme, because it can omit or alter nuance, including emotive slang that may carry sentiment. Prompt gating and automatic checks help but do not eliminate this risk.

The dataset scope is also narrow: the study used only CDC-related COVID-19 replies, and the paper notes that results may vary across topics or languages. Likewise, only one LLM, Google Gemini, was evaluated; other systems such as GPT-4 or Claude may yield different rephrasing quality (Xin et al., 21 Oct 2025).

A central trade-off is interpretability versus authenticity. General rephrasing preserves user voice better and is therefore preferable when sentiment or stylistic variation is analytically important. Colloquial-to-formal rephrasing maximizes coherence and topic purity but at the cost of stylized nuance. In the public-health setting, the stated impact is that medically relevant keywords such as “adverse,” “risk,” and “efficacy” become easier to surface from informal chatter, enabling agencies to gauge public concerns and target communications more precisely (Xin et al., 21 Oct 2025).

The recommended practice is correspondingly pragmatic: choose the rephrasing scheme according to analytic goals, always include fidelity checks on named entities and domain terms, experiment with different LLMs and prompt variants, combine the method with qualitative expert review in high-stakes domains, and consider extending the same data-centric enhancement to other short noisy texts such as customer reviews, forum posts, and chat logs (Xin et al., 21 Oct 2025).

6. Related rephrasing paradigms in adjacent research

Although the topic-modeling framework is the most direct instantiation of TM-Rephrase in the present corpus, adjacent literature shows that rephrasing has been used for several technically distinct purposes.

For uncertainty estimation in closed-source LLMs, TM-Rephrase asks $\phi_k \sim \mathrm{Dirichlet}(\beta)$8 rephrased versions of a base query and defines empirical confidence $\phi_k \sim \mathrm{Dirichlet}(\beta)$9 as the fraction of rephrasings that produce the modal answer $\theta_d \sim \mathrm{Dirichlet}(\alpha)$0. On ARC-Challenge, ARC-Easy, and OpenBookQA, this method reduced Expected Calibration Error by 10–30%, reduced Brier score by 0.1–0.4, and improved AUROC by 10–40% relative to a naïve baseline; the best practical rephrasing rules were Reword and Expansion (Yang et al., 2024).

In in-context learning, "EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning" prompts a model to restate a query before answering it. The reported gains span numerical reasoning, reading comprehension, and logical reasoning, including average improvements of 5% in numerical tasks and 13% in reading comprehension tasks for zero-shot chain-of-thought prompting on code-davinci-002 (Mekala et al., 2023).

In non-autoregressive machine translation, "Rephrasing the Reference for Non-Autoregressive Machine Translation" introduces a rephraser that rewrites the training reference to better align with the NAT output while remaining close to the original reference, using reinforcement learning over fidelity and similarity rewards. The best reported variant reaches performance comparable to the autoregressive Transformer while remaining 14.7 times more efficient in inference (Shao et al., 2022).

Other related lines include sentence-level split-and-rephrase, where WikiSplit provides one million naturally occurring Wikipedia rewrites and yields a $\theta_d \sim \mathrm{Dirichlet}(\alpha)$1 BLEU improvement over the prior best on WebSplit (Botha et al., 2018), and dialog-system content rephrasing, where BART with copy mechanisms and distilled LSTM models were evaluated on a dataset of approximately 3,000 messaging examples to convert content into forms that “sound natural” to recipients (Einolghozati et al., 2020).

Taken together, these results suggest that rephrasing functions as a general-purpose interface layer: it can standardize noisy inputs before unsupervised modeling, expose latent uncertainty in black-box systems, improve prompting for inference-time reasoning, reshape training targets in sequence generation, or minimally transform user content for dialog execution. In the specific sense established by the COVID-19 tweet study, TM-Rephrase is a data-centric method for improving topic modeling without redesigning the topic model itself, and its strongest reported effect is the marked increase in LDA topic coherence under colloquial-to-formal rewriting (Xin et al., 21 Oct 2025).