Infusing Prompts with Syntax and Semantics

Abstract: Despite impressive success, LLMs often generate outputs with flawed linguistic structure. We analyze the effect of directly infusing various kinds of syntactic and semantic information into LLMs. To demonstrate the value of our proposals, we focus on the translation of natural language queries to SQL, in particular dealing with languages with less resources than English, to better investigate how much help we can get from low cost syntactic and semantic information. We show that linguistic analysis can significantly boost LLMs, to the point that we have surpassed previous best systems.