Crash Course: Developing Software Factories
Mentor - Vyacheslav Koldovskyy, Founder of Dev AI Consulting
Building autonomous factories of AI agents has become the main trend of 2026. In this setup, agents work with almost no supervision: they carry a project from task definition through to the final release.
Mentor: Vyacheslav Koldovskyy, founder of Dev AI Consulting.
He has built several dozen factories for his clients in practice, on projects using a variety of technologies. Along the way, he has gained experience both from working independently and from working with development teams. The author prepared this course based on that experience.
Prerequisites
Daily work with a coding agent (CLI or IDE), Git and CI, and an understanding of at least one programming language. The mechanisms are not tied to a specific stack. Nice to have, but not required: completing one of the courses Agentic Engineering or Agentic Engineering for New Projects.
By the end of the course, you will:
- design your own factory and identify the components that matter for your tasks
- accept into work only what the factory is able to verify
- run quality gates, starting with the cheapest and fastest to check
- protect the oracle (the source of reference answers): tests and hidden holdouts, and know how they can leak
- set up review based on the maker ≠ checker principle
- limit the executor agent’s permissions: a policy hook, process isolation, and a plan for network egress
- measure the cost of one accepted change and outcome metrics rather than activity metrics
- build a dashboard for your agent that lets you visually monitor its work
- running an AI harness (Claude Code, Codex, etc.) autonomously without a reliable mechanism for verifying the result
- treating something as “done” just because the agent said “done”
- acceptance tests that the agent can read, modify, or see quoted in the retry prompt
- the change is verified by the same agent that wrote it
- lines in a prompt instead of mechanisms (“don’t edit the tests”)
- costs counted in tokens or merged PRs rather than in accepted changes
- granting autonomy to the whole project at once
- effective quality gates that agents cannot bypass
- tests that the executor agent cannot change, and hidden holdouts (tests the agent never sees)
- maker ≠ checker: a reviewer with a clean context who is tasked with refuting the change and must back up its conclusions with references
- a policy hook, process isolation, and egress rules (outbound network traffic) for everything the agent runs
- the cost of one accepted change — from the cost report
- autonomy granted separately for each type of change: it has to be earned with evidence
The course covers dozens of principles of the dark software factory (a factory where agents work without constant human supervision). They are grouped into separate areas and follow the order in which a real run goes through them: intent, context, control, verification, trust, transparency, autonomy, people, and evolution. You’ll understand why each principle is needed and how to build it and prove it works.
The author will share the code of his own factories, which you can use as a starting point to build yours. The demos are shown in Claude Code, but the principles carry over to any other harness (Codex, Cursor, GitHub Copilot, OpenCode...).
- Tech Leads, Solution Architects, and Staff Engineers who design how agents will work without supervision in their delivery
- Middle/Senior developers who already work actively with Claude Code, Codex, or Cursor and want agents to work more autonomously with less supervision
- QA engineers and test automation engineers: since a large part of the factory is focused on quality control (automated tests, holdouts, evals, acceptance criteria), this is where you can deepen your expertise
- DevOps and Platform Engineers who build sandboxes, CI gates, and ecosystems for running agents
- Engineering Managers who need to evaluate how effectively factories work in terms of cost, speed, and the quality of task implementation, and to find the balance between autonomy and engineer involvement
Two sessions: from the current state of the industry to your own factory
session-1 The current state of Software Factoriesabout 2.5–3 hours 1/2 ›
What is a software factory, and what kinds are there?
- – Five levels of autonomy; the same pipeline in three modes: lights-on, dim, dark
- – Who already runs factories (OpenAI, Anthropic, Stripe, Uber, and others), the types of factories, and their pros and cons
Anatomy of a factory
- – The factory zones (Control, Store, Target) and the path of a change from intent to release
- – The main rule: a principle works only when a mechanism enforces it, not an instruction in a prompt
What the industry’s experience teaches
- – Generating code is cheap; the bottleneck is now verification
- – Typical failures: “done” without evidence, tests fitted to the code, an agent checking its own work, sandbox escapes, invisible costs
Principles: what to build, how to verify, how much to trust
- – Intent, context, control: people decide what to build; the repository is the single source of truth; the factory sets the limits
- – Verification: evidence instead of claims; gates and tests the agent cannot bypass or change; hidden holdouts
- – Trust and transparency: minimal permissions, isolation, egress control; the cost of an accepted change instead of tokens
- – Autonomy, people, evolution: autonomy is earned per type of change; every mechanism must prove it is needed
Session summary
- – A checklist of principles and a self-assessment of your own projects
session-2 How a real factory is built: principles in actionabout 2.5–3 hours 2/2 ›
The principles in the order of a real run: the failure each one stops, the mechanism, and proof that it really says “no”. Slides, a live demo, and recordings of the author’s real factory runs.
Architecture and the run
- – A false “done” and the author’s three factories: from a trimmed-down “mega-factory” to one where every principle is backed by a mechanism
- – Three zones and a boundary around the executor; a person reviews the plan, not the diff
- – The model proposes, your code decides: cheapest checks first, run limits, a kill switch
Protect the oracle and earn every “passed”
- – No evidence — no “passed”; a check the agent can change is not a check
- – Hidden holdouts and where to store them; a reviewer with a clean context who must refute the change
- – Case study: the reviewer found holes in the author’s factory; discussion: holdout scenarios for your feature
Trust boundaries
- – No role may combine private data, untrusted content, and a channel to the outside (the lethal trifecta)
- – Isolation rings and allowlists treated as parsers; prevention is not detection
- – Each approval is tied to an artifact version and a named person; the author of a request never approves their own release
Demo: the factory at work
- – A live demo with scripted agents: an honest executor passes every gate, yet the release still waits for a person
- – A recorded run of the public factory project-factory-next, and a proof pack instead of a raw diff
Transparency, autonomy, and people
- – The cost of one accepted change from the factory report; a dashboard with outcome metrics first
- – Autonomy per type of change; discussion: your autonomy matrix
Evolution and summary
- – Automatic improvement of the factory and protection against its degradation
- – The whole factory on one diagram, a comparison of practices, where to start, and the final assignment explained
There is no homework after the first session. After the second, there is a final assignment: your own factory at work.
- Build a factory for your own repository: based on the code of the author’s factories or from scratch, on any harness
- Run it on a real project — new or existing
- Prove that the project was implemented successfully: before launch, fix the success criteria (acceptance cases), and afterward show that they have been met (a working result, a demo, or a recording)
Collect the evidence of the run:
- logs and traces of the runs
- a report on costs and on the cost of one accepted change
- check results: tests, hidden holdouts, review conclusions
- a proof pack for the key changes and a short description of what the factory stopped
- a dashboard of your factory (a screenshot or a link) that shows the progress of the runs
Materials:
- slides and video recordings of the sessions
- the code of the author’s factories as a starting point for yours
- a starter repository with templates, including a proof pack template for the final assignment
- a checklist of principles with first steps
A certificate is awarded upon completing the final assignment.
- A coding agent with a CLI and hooks. We recommend Claude Code (the author will use it for the course). Other harnesses (Codex, Cursor, GitHub Copilot, OpenCode) can also be used, but behavior may differ.
- Git and access to GitHub (GitHub or GitLab).
- A project for the final assignment: new or existing.
Vyacheslav Koldovskyy
- – 20+ years in IT
- – PhD / Candidate of Economic Sciences, associate professor, head of the Gen AI Center at IT STEP University
- – Founder of the Programming Mentor YouTube and Telegram channels; AI community leader on DOU; speaker at iForum and DOU Day
- – Builds software factories for Dev AI Consulting clients
- – Advises companies on moving their SDLC to AI; author of Fwdays Academy courses on Agentic Engineering
Event price
Access to 2 online broadcasts (~6 hours)
Presentations and materials
Access to the course recording on the learning platform
Access to the chat in the TG, where the speaker will answer questions
Participant certificate (subject to completing homework)
Free month in Fwdays Club
Course participants receive a 10% discount on participation in Fwdays conferences
Affordable payment in installments from Monobank and purchase in installments from PrivatbankIf after the first lesson you realize that your expectations do not match, we will refund the money in full.