When AI Becomes a Coworker: The Non-Human Organizational Actor
This lightning talk explores what happens when a proactive AI agent moves into your team's workspace. Drawing from a five-month deployment across 20+ teams, the research reveals that these systems don't fit cleanly into our existing categories of 'tool' or 'teammate.' Instead, they introduce a new kind of organizational actor: one with operational agency but without human tacit knowledge, accountability, or relational stakes. The talk examines the collisions that result when technical access meets situated social norms, and offers a path forward for designing and governing hybrid human-AI teams.Script
An AI agent just filed 16 engineering bugs and tagged a vice president in every single one. Not because it was hacked or malfunctioning, but because it couldn't tell the difference between someone who can access a workflow and someone who should be asked to act on it.
The researchers deployed Team Agent across more than 20 teams for five months, and it handled thousands of tasks: filing bugs, summarizing meetings, scheduling briefings, even nudging conversations toward resolution. But it repeatedly cited stale notes as current policy, created formal documents from casual brainstorms, and shared work-in-progress materials that weren't ready for circulation. The agent had technical access to everything, but it lacked the organizational understanding to know what was authoritative, what was obsolete, and what was just thinking out loud.
Teams couldn't agree on what the agent actually was. Some treated it like an intern who needed bounded tasks and careful supervision. Others wanted it to act like a capable program manager who could challenge assumptions and enforce commitments. A few rejected the teammate framing entirely, insisting it was just software. This wasn't simply a vocabulary disagreement. It generated conflicting expectations about authority, accountability, and whether the agent had any business giving humans feedback on their tone or behavior.
The agent's most valuable capabilities were also the source of its most damaging failures. Proactivity enabled useful discoveries and administrative execution, but that same autonomy generated document pollution, unwanted interventions, and inappropriate disclosures. Teams that felt ownership over the deployment tolerated early mistakes and approached the agent experimentally. Those who felt it had been imposed on them withdrew trust immediately. The researchers propose progressive autonomy: low-risk tasks first, with higher-stakes actions released only after the agent demonstrates reliability and earns the team's explicit authorization.
Even when the agent was helpful, its presence changed how people communicated. Some users reported feeling surveilled, reducing candor and moving sensitive discussions to agent-free side channels. The coaching function, which sent private messages about tone or mood, led workers to self-censor and maintain a persistently positive facade. One team noticed a bystander effect: when the agent responded to a request for help, human teammates assumed someone was handling it, even though the agent's answer provided no real assistance. The agent wasn't just adding another voice to the channel. It was altering the interpretation of human behavior itself.
The researchers argue that these systems are neither tools nor teammates. They're non-human organizational actors: entities with operational and structural agency, but without human tacit knowledge, phenomenological experience, or accountability. That distinction matters because it tells us what needs to change. We need distinct permissions, auditability, reversibility, team-level behavioral contracts, and staged autonomy. Most of all, we need deployment governance that treats consent as ongoing and revisable. To explore the full study and create your own video summaries of cutting-edge research, visit EmergentMind.com.
