1:02:25
Matt Pocock’s Agentic Engineering Workflow (just copy him)
David Ondrej
Overview
This video explores the evolving landscape of software development in the age of AI, emphasizing a shift from focusing solely on AI models to optimizing the 'harness' – the prompts, skills, and environment AI operates within. It advocates for strategic programming over tactical, as AI excels at the latter. The discussion highlights how AI amplifies the capabilities of senior developers and introduces a 'teach skill' agent designed for personalized, on-demand learning. The conversation also delves into the practicalities of agentic workflows, including sandboxing, continuous integration, and the debate between human-in-the-loop and automated processes, ultimately stressing the enduring importance of fundamental software engineering principles.
How was this?
Save this permanently with flashcards, quizzes, and AI chat
Chapters
- Developers should focus more on the 'harness' (prompts, skills, environment) than the AI model itself.
- The harness offers more control and potential for optimization than the underlying AI model.
- Effective AI utilization involves providing the right prompts and skills, and improving the environment the model runs in.
- Optimizing for token spend is best achieved by having a more manageable and easier-to-change codebase.
Understanding where to direct your optimization efforts is crucial for maximizing AI's impact, as focusing on the harness yields greater control and efficiency than solely chasing the latest model.
The speaker uses 'Fable' as an example where the harness is as important as the model itself.
- Tactical programming involves day-to-day coding tasks like writing syntax, debugging, and creating commits.
- Strategic programming focuses on long-term goals, codebase architecture, and improving overall development velocity.
- AI has effectively 'eaten' tactical programming, performing these tasks more cheaply and efficiently than humans.
- To leverage AI, developers must excel at strategic programming, directing an 'infinite fleet' of AI tactical programmers.
By understanding the distinction between tactical and strategic programming, developers can reposition themselves to focus on higher-value, long-term thinking, leveraging AI to handle routine coding tasks.
AI is better at the 'on the ground day-to-day stuff' of coding, freeing up humans for 'winning the war, not the battle'.
- AI progress is rapid, necessitating continuous upskilling for developers.
- While tools and subscriptions are accessible to all, true advantage comes from enhanced human skills.
- Senior developers see a magnified benefit from AI, becoming 10x more productive, whereas junior developers gain a smaller boost.
- An individual's skills act as the ceiling for what AI can achieve for them; improving your domain expertise enhances AI's effectiveness.
Your personal skills and domain knowledge are the ultimate limiters of AI's potential benefit; investing in self-improvement is key to leveraging AI for significant growth.
A better teacher can use AI to teach people more effectively than a random person could, highlighting how existing skills amplify AI's utility.
- The 'teach skill' is an AI agent designed to create personalized learning courses on demand.
- It encodes teaching principles like the zone of proximal development and differentiates between knowledge, skills, and wisdom.
- The skill requires a 'mission' from the user to orient its learning path and tailor content.
- It generates structured learning materials, including cheat sheets and interactive lessons, stored locally for easy reference.
This tool democratizes personalized education, allowing anyone to learn complex topics efficiently by aligning AI instruction with their specific goals and learning style.
The speaker demonstrates the 'teach skill' by asking it to create a curriculum for a 'vibe coder' wanting to learn systems design, starting with Git.
- Agent skills can be categorized as 'procedures' (invoked by the user) or 'abilities' (invoked by the model).
- The speaker prefers procedures for greater user control, citing 'grill me' as an example.
- Skills are descriptions that can be leaked into the AI's context window, impacting performance.
- Good agent skills are effective, often concise, and tailored to specific use cases.
Distinguishing between skill types and understanding how they interact with AI models is crucial for designing effective and controllable agentic workflows.
The 'grill me' skill turns the model into an adversarial interviewer to help flush out ideas before implementation.
- Focusing on timeless software engineering fundamentals is more valuable than chasing the latest AI model.
- A well-designed, easily modifiable codebase allows for the use of simpler, cheaper AI models.
- Poor codebase architecture forces reliance on more sophisticated, expensive models.
- The 'harness' (codebase, prompts, skills) is as important as the AI model itself for effective AI utilization.
Adhering to established software design principles ensures long-term maintainability and cost-effectiveness, making AI tools more accessible and powerful regardless of model advancements.
Having a codebase that's easier to make changes in allows you to use a 'stupider model' and optimize for token spend.
- The speaker uses Claude Code with Opus 4.8 for planning and local implementation.
- A tool called 'Sandcastle' is used to run agents within sandboxes (like Docker) for safety and parallelization.
- Agents can be run locally or on remote sandboxes, with results committed back to a local workspace.
- GitHub Actions are integrated for automated agent reviews and other tasks, enabling parallel agent execution without local resource constraints.
Implementing robust agentic workflows with sandboxing and automation enhances safety, scalability, and efficiency, allowing for parallel processing and reduced risk.
Using Sandcastle with GitHub Actions to run an agent review on a pull request, which checks code quality and provides feedback.
- The concept of 'loops' in agentic workflows can be misleading; 'queues' better represent task management.
- AFK (Away From Keyboard) work, where agents perform tasks autonomously, is key to scaling output.
- A queue-based approach manages a backlog of tasks, allowing agents to pick them up and resolve them.
- Human-in-the-loop checkpoints are essential but should be strategically placed and pushed towards the final output stage.
Adopting a queue-based system for agent tasks, rather than relying on endless loops, provides a more realistic and manageable framework for development and automation.
Managing Sandcastle issues (bug reports, feature requests) as a queue, where agents explore, implement, and potentially auto-merge fixes, with human oversight at critical junctures.
- Building a successful business still relies on fundamental product design principles: talking to customers and solving real needs.
- AI can significantly accelerate implementation and delegation of procedures but doesn't replace the need for a core idea or vision.
- Focus on simplifying products and improving user experience, rather than adding excessive features.
- The key differentiator for success is having the right idea and building the right thing, informed by customer conversations.
While AI transforms implementation, the core principles of identifying market needs and building valuable products remain paramount for business success.
Instead of asking AI to build an app, ask it how to simplify the app or improve the user experience, focusing on core user needs.
Key takeaways
- The 'harness' (prompts, skills, environment) is as critical as the AI model itself for effective AI utilization.
- AI excels at tactical programming, shifting the developer's focus to strategic, long-term planning and architecture.
- Continuous self-upskilling, particularly in domain expertise, is essential to maximize the benefits AI offers.
- Personalized learning tools like the 'teach skill' can accelerate skill acquisition by tailoring education to individual needs and goals.
- Prioritizing timeless software engineering fundamentals over chasing the latest AI models leads to more robust and cost-effective solutions.
- Agentic workflows benefit from a queue-based task management system rather than endless loops, with strategic human oversight.
- Successful business building in the AI era still hinges on understanding customer needs and having a clear product vision, with AI as an implementation accelerator.
- The most significant differentiator for AI-powered success lies in the developer's strategic thinking, problem-solving skills, and ability to guide AI effectively.
Key terms
HarnessTactical ProgrammingStrategic ProgrammingAgentic EngineeringSkills (AI)Procedures (AI Skills)Abilities (AI Skills)SandcastleHuman-in-the-loopAFK (Away From Keyboard) WorkQueue-based Task ManagementZone of Proximal Development
Test your understanding
- How does the concept of the 'harness' differ from the AI model itself, and why is it important for developers to focus on it?
- Explain the distinction between tactical and strategic programming and how AI's capabilities influence this division.
- What is the 'teach skill,' and how does it leverage principles of education to facilitate personalized learning?
- Describe the difference between 'procedures' and 'abilities' in the context of AI skills and how this impacts agent control.
- Why is a codebase's design crucial for optimizing AI token spend and overall efficiency?
- Contrast the 'loop' and 'queue' approaches to managing agentic workflows. Which does the speaker advocate for and why?
- What fundamental principles of product development remain essential for building successful businesses in the age of AI?