TULIP Models: Diverse ML Applications

• TULIP models are a polysemous acronym representing distinct methods across vision-language modeling, uncertainty estimation, optimization, and domain-adapted language models.
• They include innovations such as token-length expansion via RoPE in CLIP-like systems, unified contrastive pretraining for fine-grained visual tasks, and novel OOD detection via perturbative uncertainty.
• Disambiguation of TULIP is crucial due to its varied usage in research, from advanced ML pipelines to statistical software and graph visualization tools.

Searching arXiv for the TULIP papers and adjacent uses of the acronym to ground the article in current literature. “TULIP models” does not denote a single research lineage. In contemporary arXiv usage, the name labels several technically unrelated systems across multimodal learning, uncertainty estimation, domain-adapted language modeling, optimization, statistical software, and programming-language semantics. The most prominent recent machine-learning uses include “TULIP: Token-length Upgraded CLIP” (Najdenkoska et al., 2024), “TULIP: Towards Unified Language-Image Pretraining” (Tang et al., 19 Mar 2025), “Transitional Uncertainty with Layered Intermediate Predictions” (Benkert et al., 2024), “TULiP: Test-time Uncertainty Estimation via Linearization and Weight Perturbation” (Zhang et al., 22 May 2025), and “TULIP: Adapting Open-Source LLMs for Underrepresented Languages and Specialized Financial Tasks” (Demirtaş et al., 22 Aug 2025).

1. Scope and nomenclature

The label is best treated as a polysemous acronym rather than a unified architecture family. In some papers it denotes a specific neural model, in others a training pipeline, a warm-start algorithm, a software toolbox, or a program logic.

TULIP designation	Domain	Core description
Token-length Upgraded CLIP (Najdenkoska et al., 2024)	Vision-language modeling	Extends CLIP-like text encoders beyond the 77-token cap
Towards Unified Language-Image Pretraining (Tang et al., 19 Mar 2025)	Multimodal pretraining	Unifies image-text, image-image, text-text contrastive learning and reconstruction
Transitional Uncertainty with Layered Intermediate Predictions (Benkert et al., 2024)	Uncertainty estimation	Uses intermediate-layer exits for single-pass uncertainty
Test-time Uncertainty via Linearization and Weight Perturbation (Zhang et al., 22 May 2025)	OOD detection	Post-hoc uncertainty from linearized training dynamics and weight perturbation
Turkish finance TULIP models (Demirtaş et al., 22 Aug 2025)	Domain-adapted LLMs	Continual pre-training and SFT for Turkish financial tasks
TULIP-algorithm (Markhorst et al., 2024)	Stochastic mixed-integer optimization	Two-step warm start via scenario reduction and cut reuse

This multiplicity matters because secondary literature can otherwise conflate incompatible objects: a CLIP upgrade, a SigLIP-like multimodal pretraining framework, and an OOD detector named TULiP are not variants of one another. A plausible implication is that “TULIP models” should be disambiguated by paper title or domain whenever the term appears in bibliographies or benchmark tables.

2. TULIP as Token-length Upgraded CLIP

In “TULIP: Token-length Upgraded CLIP”, TULIP is a method for converting a standard CLIP-like contrastive vision-LLM with a 77-token text limit into a model that can process arbitrarily long captions while preserving image-text alignment (Najdenkoska et al., 2024). The stated bottleneck is CLIP’s fixed, absolute positional embedding table, which truncates inputs beyond 77 positions and degrades performance on dense captions and long compositional descriptions.

The architectural intervention is the replacement of absolute positional encoding with Rotary Position Embedding (RoPE) inside every transformer block of the text encoder. Standard self-attention uses

$q_m = W_q x_m,\qquad k_n = W_k x_n,$

with attention weights derived from

$$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$

TULIP instead injects position through rotations,

$$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$

so that relative displacement is encoded through phase differences rather than a fixed-length embedding table. The paper’s interpretation is that this improves pairwise token modeling and makes the encoder, in principle, length-agnostic.

The training procedure is explicitly two-stage. The first stage, relative position distillation, keeps the original CLIP text encoder as teacher and trains the RoPE-based student on the first 77 tokens of captions using cosine similarity between teacher and student text embeddings. The second stage, relative position expansion, fine-tunes the distilled text encoder together with the image encoder on full long captions using a mixture of short- and long-caption contrastive losses. For extrapolation, the method uses NTK-aware scaled RoPE with original length $T_f=77$, new context length $T_g$, and hyperparameter $\alpha$. In the main experiments, the method is implemented on OpenAI CLIP / OpenCLIP ViT-B/16 and ViT-L/14, trained on ShareGPT4V, with 20 epochs of distillation and one epoch of expansion.

Evaluation spans short-caption retrieval, long-caption retrieval, and text-to-image generation. The long-caption benchmarks are ShareGPT4V test, Urban-1K, and Long-DCI, where Long-DCI is introduced as a 7,000-pair benchmark with captions averaging about 200 tokens. On long-caption retrieval, TULIP is reported to outperform CLIP and Long-CLIP. With ViT-L/14, it reaches 55.7 / 56.4 on Long-DCI, 99.0 / 99.0 on ShareGPT4V, and 90.1 / 91.1 on Urban-1K; with ViT-B/16, it reaches 50.2 / 50.6, 98.6 / 98.6, and 88.1 / 86.6, respectively. On ViT-L/14 Long-DCI, the reported baselines are 35.0 / 37.0 for CLIP, 51.6 / 50.7 for fine-tuned CLIP, 54.0 / 46.1 for Long-CLIP, and 55.7 / 56.4 for TULIP. Short-caption retrieval remains competitive; for example, on Flickr30k with ViT-L/14, TULIP achieves 56.7 / 79.5 for image-to-text $R@1/R@5$ and 41.6 / 64.3 for text-to-image $R@1/R@5$.

The paper also reports qualitative transfer to Stable Diffusion XL by replacing the standard CLIP text encoder without retraining the diffusion model. The cited examples show better preservation of details beyond token 77, including phrases such as “the red tulip inside a wooden box,” “an old man sits on a rock,” “playful grin,” and “inside a large glass jar.” Ablations support three design choices: RoPE outperforms CoPE, context expansion helps up to roughly 231 tokens before a mild plateau or decline at 308, and cosine loss outperforms L2 and MSE during distillation. The authors also report that useful image-relevant information is distributed throughout long captions, reinforcing the argument against truncation.

3. TULIP as unified language-image pretraining

In “TULIP: Towards Unified Language-Image Pretraining”, TULIP is an open-source, drop-in replacement for CLIP-like models intended to preserve global language-image alignment while substantially improving fine-grained visual understanding (Tang et al., 19 Mar 2025). The motivating claim is that CLIP/SigLIP-style contrastive training often privileges high-level semantics at the expense of counting, depth estimation, localization, fine-grained recognition, multi-view reasoning, and compositional reasoning.

The framework combines five elements: image-text contrastive learning, image-image contrastive learning, text-text contrastive learning, reconstruction regularization, and generative data augmentation. The contrastive core uses a SigLIP-style pairwise sigmoid objective. The total contrastive term is

$$L_\text{cont} = L_\text{I-T} + L_\text{I-I} + L_\text{T-T},$$

where $L_\text{I-I}$ operates on augmented image views and $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$0 on augmented text views. Reconstruction is added through an MAE-style decoder for images and a causal decoder based on T5 for text, giving

$$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$1

and the total objective

$$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$2

A central component is GeCo (GEnerative COntrastive view augmentation). For text, GeCo uses Llama-3.1-8B-Instruct to generate positive paraphrases and negative paraphrases with minimal semantic changes. For images, it fine-tunes an instruction-based image editing model with soft prompting, using sources such as WebVid-10M, MVImgNet, and InstructPix2Pix-style semantic edits to produce positive and negative views. This extends contrastive supervision beyond handcrafted augmentations like crop, flip, or color jitter.

The training corpus includes DataComp-1B, described as about 1.4 billion image-text pairs, together with Recap-DataComp-1B recaptioned data. The paper states that average caption length increases from about 10.22 words to 49.43 words, and that 20% of original captions are replaced with recaptioned data during augmentation, while 50% replacement is used for text reconstruction. Optimization uses Adam, learning rate $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$3, weight decay $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$4, gradient clipping norm 2, batch size 49,152, and training on up to 32 A100 GPUs. The framework scales to over 1B parameters.

The reported results are broad. In zero-shot classification, ViT-B/16 TULIP reaches 79.5 on ImageNet-1K val, 73.0 on ImageNet-v2, 86.2 on ImageNet-ReaL, 74.2 on ObjectNet, and 73.8 on ImageNet 10-shot. Larger variants report 85.0 and 85.3 on ImageNet val for So/14, 384 and g/16, 384, respectively. Retrieval also improves: for B/16, the paper reports 54.2 / 70.1 on COCO $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$5 and 81.8 / 93.9 on Flickr. The strongest vision-centric evidence appears in linear probing and multimodal evaluation. The paper emphasizes almost $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$6 improvement over SigLIP on RxRx1, and on MMVP reports a progression for SigLIP (ViT-So/14) from 5.9 at baseline to 17.4 with image-image/text-text contrastive learning, 18.2 with added reconstruction, and 20.3 with full GeCo. On Winoground, TULIP is reported as the only contrastive image-text model listed to exceed random chance on group score, reaching 17.75 for ViT-So/14, 384 and 18.50 for ViT-G/16, 384, compared with random chance at 16.67.

Conceptually, this TULIP is distinct from the long-caption CLIP upgrade. The former is a general pretraining framework for improving visual fidelity while preserving alignment; the latter is a retrofit procedure for extending CLIP-like text context length. Their overlap is limited to the broader CLIP-like multimodal setting.

4. TULIP and TULiP in uncertainty estimation

The acronym also appears in two separate uncertainty-estimation lines that share neither architecture nor derivation. “Transitional Uncertainty with Layered Intermediate Predictions” defines TULIP as a single-pass uncertainty estimation framework, whereas “Test-time Uncertainty Estimation via Linearization and Weight Perturbation” defines TULiP as a post-hoc OOD detector grounded in linearized training dynamics (Benkert et al., 2024, Zhang et al., 22 May 2025).

The single-pass TULIP paper argues that many deterministic uncertainty estimators, such as SNGP and DUQ, preserve feature distances at the final output in a way that can oppose the deep network’s compression objective. Its remedy is transitional feature preservation: rather than forcing the final representation to retain all uncertainty-relevant geometry, the method places internal classifiers (ICs) at intermediate layers and extracts uncertainty before subsequent layers collapse information. Training uses a weighted shallow-deep objective,

$$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$7

with equal weighting $$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$8 often reported to work well. Per-exit uncertainties are then aggregated as

$$\mathrm{softmax}\!\left(\frac{q_m^T k_n}{\sqrt{d}}\right).$$9

using a GP-style / Dempster-Shafer score

$$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$0

Combination weights are fit by logistic regression on proxy labels derived from prediction switches among internal exits, without requiring OOD validation data. The paper reports competitiveness or superiority over DNN, Energy-Based, DUN, Early Exit, SNGP, and DUQ across CIFAR10/CIFAR100, ImageNet, and MedMNIST CT settings, especially under class imbalance, deeper architectures, and modality shift.

By contrast, TULiP formulates OOD uncertainty as the sensitivity of a trained network’s final prediction to a hypothetical perturbation introduced before convergence. The theory is stated in an NTK / lazy-training regime. A linearized model

$$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$1

is perturbed, and the discrepancy between converged perturbed and unperturbed solutions is bounded by a gradient-space distance to the training set. The implementation then approximates this uncertainty post hoc by perturbing model parameters, generating surrogate predictions, and computing predictive entropy. The final score is

$$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$2

for the averaged predictive distribution over sampled perturbed outputs. The paper reports defaults $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$3 samples, $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$4, $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$5, $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$6, perturbation of all parameters including biases and BatchNorm $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$7, and evaluation on OpenOOD v1.5 using ResNet-18 and ResNet-50. On CIFAR-10, TULiP (ENT) is reported at FPR@95 40.77 / 28.40 and AUROC 89.36 / 92.25 for near/far OOD; on ImageNet-1K, it reports FPR@95 64.96 / 48.01 and AUROC 78.38 / 88.85. The paper’s stated emphasis is that gains are strongest on near-distribution OOD.

Taken together, these works show that the TULIP acronym spans both architectural single-pass uncertainty estimation and test-time perturbative uncertainty estimation. This suggests that citation by acronym alone is especially error-prone in safety- or robustness-oriented surveys.

5. TULIP as Turkish-finance-adapted LLMs

In “TULIP: Adapting Open-Source LLMs for Underrepresented Languages and Specialized Financial Tasks”, TULIP denotes a pair of Turkish-finance-adapted LLM families based on Llama 3.1 8B and Qwen 2.5 7B, together with their instruction-tuned variants (Demirtaş et al., 22 Aug 2025). The paper’s framing is application-driven: finance requires privacy-preserving, on-premise deployment, while Turkish is described as an underrepresented language for general-purpose open models.

The development pipeline has five stages: data collection, continual pre-training (CPT), benchmark design, synthetic data generation, and supervised fine-tuning (SFT). The CPT corpus totals about 2.19 billion tokens, with the following reported breakdown: Academic sources 1.1B, Financial institutions 150M, Textbooks and study materials 200M, Market and business data 350M, Legislation and regulations 50M, and Other reports and documents 340M. For SFT, the raw text sources are Academic, Central Bank, News, and Trade Registry Gazette.

CPT uses a QLoRA-style setup with learning rate $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$8, optimizer paged_adamw_8bit, bf16 precision, 4-bit quantization, LoRA alpha 128, and LoRA rank 64, with all linear layers, the head, and embedding layers optimized. SFT keeps the same general setup but reduces the learning rate to $$q_m = R_{\Theta,m} W_q x_m,\qquad k_n = R_{\Theta,n} W_k x_n,$$9. The resulting model names are TULIP-Llama-3.1, TULIP-Qwen2.5, TULIP-Llama-3.1-IT, and TULIP-Qwen2.5-IT.

Because no suitable evaluation suite existed, the paper introduces two benchmarks. FINTR-EXAMS is a 5-shot multiple-choice benchmark based on undergraduate finance courses across seven domains: Banking and Insurance, Foreign Trade and International Finance, Economics, Finance and Investment, Private Sector Finance, Public Finance and Tax, and Accounting and Financial Reporting. The paper states a human passing threshold of 60/100. The second benchmark is a Turkish Trade Registry Gazette QA dataset with 880 manually annotated question-answer pairs, 9 annotators, and Cohen’s kappa = 0.91. The event types are Change of Capital, Change of Management, Composition with Creditors, and Notice to Creditors.

Synthetic SFT data are generated rather than translated from English datasets. The paper states that translation quality was inadequate for specialized financial terminology and that human translation was too expensive. The synthetic procedure uses random source chunks, seed prompts, GPT-4o rephrasing, and filtering. The final SFT set contains about 23K instruction-answer pairs, with source distribution Academic 54%, Central Bank 5%, News 21%, Trade Registry Gazette 20%, and task distribution including fill-in-the-blank 19%, multiple-choice QA 16%, summarization 16%, true/false 14%, sentiment analysis 14%, and smaller shares for the remaining task types.

The results emphasize the value of CPT. On FINTR-EXAMS, TULIP-Llama3.1 improves mean accuracy from 0.544 to 0.583, and TULIP-Qwen2.5 from 0.600 to 0.668. The corresponding instruction-tuned variants reach 0.573 and 0.659. On the Trade Registry Gazette QA benchmark, Llama3.1-Instruct reports mean 0.866, TULIP-Llama3.1-IT 0.907, and TULIP-Llama3.1-Instruct 0.918; Qwen2.5-Instruct reports 0.775, TULIP-Qwen2.5-IT 0.858, and TULIP-Qwen2.5-Instruct 0.818. The paper also reports that untuned Qwen2.5 sometimes switches languages mid-answer or emits English/Chinese, a behavior said to disappear after adaptation. The authors note several limitations: SFT does not always improve over CPT alone, hard numeric reasoning remains difficult, and specialized terminology such as memzuç and temdit still challenges general models.

This TULIP family is therefore not a generic foundational model in the CLIP sense. It is a domain- and language-adaptation recipe for finance-focused, underrepresented-language deployment under privacy constraints.

6. Other TULIP usages and disambiguation outside model nomenclature

Several additional arXiv works use TULIP in ways that are not “models” in the narrow neural-network sense. In “A Two-Step Warm Start Method Used for Solving Large-Scale Stochastic Mixed-Integer Problems”, TULIP is a two-phase optimization framework: first reduce the scenario set and solve only the root node of the reduced problem to harvest cuts, then reuse those cuts to warm-start the full problem (Markhorst et al., 2024). The paper demonstrates this on the Stochastic Capacitated Vehicle Routing Problem and the Two-Stage Stochastic Steiner Forest Problem, positioning TULIP as a preprocessing mechanism for branch-and-cut rather than a predictive model.

In statistics, “TULIP: A Toolbox for Linear Discriminant Analysis with Penalties” presents TULIP as an R package integrating DSDA, ROAD, SOS, SeSDA, MSDA, and CATCH for high-dimensional, semiparametric, multiclass, tensor-valued, and covariate-adjusted discriminant analysis (Pan et al., 2019). Here the acronym denotes software for sparse discriminant estimation rather than a single classifier. In programming languages, “A denotationally-based program logic for higher-order store” uses TULIP, written as $T_f=77$0, for a higher-order separation logic over the typed equational theory of higher-order store (Aagaard et al., 2023). In graph rewriting, “Graph Creation, Visualisation and Transformation” describes GraphPaper as an editor built on the graph-visualization engine TULIP, which provides efficient graph storage, iteration, and dynamic visualisation (Fernández et al., 2010).

A related but secondary usage appears in LiDAR super-resolution. “Real-Time LiDAR Super-Resolution via Frequency-Aware Multi-Scale Fusion” treats TULIP as its strongest transformer-based baseline, describing it as a Swin Transformer–style U-Net on range images with row-based patching, non-square windows, and circular padding (Goo et al., 10 Nov 2025). In the reported comparison on KITTI, that baseline achieves MAE 0.4354, CD 0.1342, IoU 0.3819, and F1 0.5502, while FLASH improves these to 0.3899, 0.1161, 0.3972, and 0.5661. Since the baseline is described from the perspective of FLASH, this use is best understood as a benchmark reference rather than a fully reconstructed TULIP paper in its own right.

Finally, the word “tulip” also appears in combustion literature, but there it refers to tulip flames, a flame morphology rather than an acronymic model family. Recent papers analyze tulip flame formation through rarefaction-wave-driven hydrodynamics, ignition geometry, wall heat loss, or reduced chemical-diffusive modeling (Qian et al., 2 Feb 2025, Qian et al., 2023, Illacanchi et al., 2023, Yan et al., 9 Apr 2026, Xiao et al., 16 Apr 2026). That literature is terminologically adjacent but conceptually separate from TULIP acronyms in machine learning and computation.

Across these usages, the commonality is nominal rather than methodological. TULIP may designate a long-context CLIP retrofit, a unified multimodal pretraining framework, a single-pass uncertainty method, a perturbative OOD estimator, a Turkish-finance LLM adaptation pipeline, a stochastic-programming warm start, a statistical software toolbox, a program logic, or a graph-visualization backend. For technical reading, the paper title and domain are therefore indispensable disambiguators.