Auto-creation of agent SKILLs from observing your screen via Gemma 4 for any agent to execute and self-improve

AgentHandover, an open-source Mac menu bar application developed by Sandro Andric, introduces a novel approach to automating workflows by observing user behavior. The tool leverages Google's Gemma 4 vision model running entirely on-device through Ollama to watch screen activity. It employs an 11-stage local processing pipeline to transform observed actions into structured, reusable 'Skill' files. These Skills contain executable steps, guardrails, and confidence scores that improve with each observation. The system offers two primary modes: 'Focus Record' for capturing specific tasks and 'Passive Discovery,' which runs in the background to identify and learn repetitive patterns automatically.

A key innovation is the tool's privacy-first architecture; all processing occurs locally with data encrypted at rest, ensuring no information leaves the user's machine. Created to solve the repetitive task of instructing AI agents, AgentHandover outputs Skills that are immediately usable by any AI agent supporting the Model Context Protocol (MCP), such as Claude Code, Cursor, or OpenClaw. This provides a one-click integration path, turning personal workflows into agent-executable commands. The project is released under the Apache 2.0 license and includes both a GUI and a CLI, making it accessible for different user preferences while opening new possibilities for local, observational AI and agent skill acquisition.