Cross-Domain Generalization in Observation Perception

Establish methods for observation perception that enable LLM-based agents to generalize across fundamentally different domains, rather than only across similar environments such as different websites within the same domain.

Background

The survey distinguishes between observation perception (detecting/extracting meaningful environmental signals) and observation processing (structuring inputs for the backbone LLM). It notes that existing efforts primarily improve generalization within similar environments, such as across different websites in web-based tasks.

The authors emphasize that transferring observation perception across fundamentally different domains (e.g., from web tasks to household robotics) remains unresolved and is crucial for robust agent behavior, especially under dynamic or out-of-distribution conditions.