OmniFill: Domain-Agnostic Form Filling Suggestions Using Multi-Faceted Context (2310.17826v1)

Abstract: Predictive suggestion systems offer contextually-relevant text entry completions. Existing approaches, like autofill, often excel in narrowly-defined domains but fail to generalize to arbitrary workflows. We introduce a conceptual framework to analyze the compound demands of a particular suggestion context, yielding unique opportunities for LLMs to infer suggestions for a wide range of domain-agnostic form-filling tasks that were out of reach with prior approaches. We explore these opportunities in OmniFill, a prototype that collects multi-faceted context including browsing and text entry activity to construct an LLM prompt that offers suggestions in situ for arbitrary structured text entry interfaces. Through a user study with 18 participants, we found that OmniFill offered valuable suggestions and we identified four themes that characterize users' behavior and attitudes: an "opportunistic scrapbooking" approach; a trust placed in the system; value in partial success; and a need for visibility into prompt context.