- The paper introduces the concept of an Autonomous Intelligent Cyber-defense Agent (AICA), a software agent designed for autonomous defense and recovery of systems, addressing the critical need arising from cyber expert shortages and sophisticated malware.
- The chapter details AICA's operational environment, extended roles as a 'doer', and functional architecture, which includes sensing, planning, collaboration, execution, and continuous learning components for autonomous operation.
- The paper acknowledges the potential risks of AICA and stresses the critical need for human oversight in its development and monitoring, building trust through transparency, and ensuring the agent's safety, trustworthiness, and resilience.
The chapter introduces the concept of an Autonomous Intelligent Cyber-defense Agent (AICA), a software agent designed to autonomously defend a system from cyber compromises and facilitate its response and recovery. The necessity for AICA arises from the increasing shortage of human cyber-experts and the capacity of sophisticated malware to impair communications with remote monitoring centers. The chapter explores AICA's operational environment, roles, internal architecture, and potential risks, highlighting considerations for trust-building among users.
AICA is defined as software residing on computing devices that perceives its environment and executes actions to achieve its goals. Autonomy is crucial, enabling AICA to plan and execute complex activities to counter malware while minimizing side effects. The chapter emphasizes the need for adversarial reasoning to combat adaptive malware and the use of deception and camouflage to remain undetected. It also addresses the challenges of relying on human cyber-defenders due to their scarcity and the increasing sophistication of malware that can compromise remote monitoring and mitigation efforts. The chapter identifies key competencies for AICA, including environmental perception, attack detection, countermeasure planning, and rapid adaptation. Proactive and autonomous operation, safety, trustworthiness, and resilience to threats are also highlighted as essential characteristics.
The chapter describes AICA's operational environment, including computer hardware/software, controlled physical entities (e.g., vehicles, industrial robots), malware, human interactions, and other agents. It assumes compromised communications with remote operators, rendering conventional centralized cyber-defense infeasible. AICA is envisioned to reside on a computer where it was originally installed by an authorized entity, with propagation to other computers occurring only under specific conditions within a trusted network.
AICA's roles extend beyond those of current cyber-defense tools, acting as a "doer" rather than merely a "watcher". It must plan and execute responses to attacks and perform recovery actions autonomously. The agent should observe the system's state, diagnose situations, and project future states to devise action plans, assess risks/benefits, and make decisions rapidly. Actions may include gathering information or destructive measures like quarantining software/data. Self-defense and self-preservation are critical, requiring stealth and camouflage. Collaboration with other agents and communication with human operators are also key aspects, with defined protocols for handover of responsibilities between AICA and remote centers to avoid interference.
The functional architecture of AICA includes sensing and world state identification, planning and action selection, collaboration and negotiation, action execution, and learning and knowledge improvement. The sensing component relies on a "world model", "current world state and history", "sensors", and a "world state identifier". The planning function uses knowledge about "actions and effects", "goals", a "planner-predictor", and an "action selector". Collaboration facilitates information exchange with other agents or a central cyber Command and Control (C2). Action execution monitors and adjusts plan execution using an "action effector" and feedback mechanisms. The learning component improves efficiency through feedback data analysis and knowledge refinement.
The chapter acknowledges the inherent risks of AICA, including potential harm to friendly systems. It argues that the benefits of using AICA must outweigh these risks, and emphasizes the need for human oversight in software development, validation, and operational monitoring. Human supervisors can modify AICA's goals, criteria, and constraints, and offer additional examples for learning. The chapter concludes by stressing the importance of building trust in AICA through transparency and communication of its decision-making processes.