Unknown resources for LLM inference hinder fair speed comparisons
Determine the computational resources required for inference by the large language models used in the FAMOSE experiments (AWS Bedrock Sonnet 3.5 V2 and Deepseek-R1) to enable fair and meaningful comparisons of algorithm inference times between LLM-based and classical feature engineering methods.
References
Because the resources needed to run LLM inference are unknown, and could affect LLM speed, we do not compare algorithm inference times, especially the speed of LLM methods against classical methods.
— FAMOSE: A ReAct Approach to Automated Feature Discovery
(2602.17641 - Burghardt et al., 19 Feb 2026) in Subsection LLMs and Resources
We did not measure runtime or resource use, so a direct efficiency comparison remains for future work.
— Inferring L1 Backgrounds from L2 English with LLM Token Probabilities
(2610.23309 - Uluslu et al., 16 Oct 2026) in Section 'Limitations', paragraph 'Computational cost'