Can lightweight LLMs gain strong agentic capabilities via pre-training?

Determine whether lightweight large language models can acquire strong agentic capabilities solely through pre-training, without relying on post-augmentation approaches such as post-training procedures or external agentic frameworks, in order to establish the feasibility of native agentic competence in small-scale models.

Background

The paper motivates the development of lightweight LLMs that retain robust agentic behaviors while avoiding the cost of very large parameter counts. Prior work has largely focused on distillation or instruction tuning, which aligns outputs rather than cultivating intrinsic planning and reasoning competence.

Within this context, the authors highlight a central uncertainty in the field regarding whether strong agentic capabilities can be induced during pre-training alone, without resorting to post-training or externalized agent frameworks. This question frames the rationale for their proposed agentic pre-training strategy and the Youtu-LLM model.