How Software Engineering Teams Should Look in the AI Era

AI is changing more than how engineers write code. It is changing how software organizations should be structured.

For years, the default pattern has been simple: clone the same delivery team across every domain. Each silo gets its own front-end and back-end engineers, lead, QA, and product owner. That model works until it does not — duplicated tooling, siloed knowledge, inconsistent delivery, and limited reuse become the tax you pay on every new domain.

The AI era pushes a different question: not "how many full-stack teams can we afford?" but "how much leverage can each specialized squad create when shared AI infrastructure amplifies everyone?"

The before model: siloed domain teams

The traditional structure repeats the same roles in every domain team:

Domain Team A, Domain Team B, Domain Team C — same stack, same rituals, same tooling rebuilt from scratch.

That repetition creates predictable failure modes:

• Duplication of effort — every team builds its own scripts, dashboards, and AI experiments
• Siloed knowledge — architecture decisions stay local; patterns do not travel
• Inconsistent tooling — one team uses one agent framework; another uses something else entirely
• Limited reuse — MCP servers, RAG pipelines, and agent workflows live in individual repos instead of a platform

None of this is a people problem. It is an operating model problem.

The now model: connected operating squads

The connected model replaces repeated full stacks with specialized squads that share a central Efficiency Team building MCPs, RAG pipelines, agents, and agentic workflow utilities for the whole organization.

Think of it as hub-and-spoke platform leverage:

Specialized squads (architecture, execution, product, delivery, ops)
        ↕ shared AI infrastructure ↕
            Efficiency Team
        (MCPs · RAG · Agents · Workflows)

Efficiency Team — shared AI infrastructure

The Efficiency Team is not another feature team. It is the platform layer for AI-assisted engineering:

• MCPs — standardized tool surfaces so agents can reach approved systems safely
• RAG pipelines — retrieval over architecture docs, runbooks, and domain knowledge
• Agents — reusable assistants for code review, incident triage, test scaffolding
• Agentic workflows — multi-step pipelines with human approval gates where state changes

If you are building MCP servers in production, the same ideas apply on Kubernetes: host the server, scale it, and keep authorization in the application layer — see Running MCP Servers on Kubernetes.

The eight squads

Each squad owns a high-leverage slice of the delivery system. Every squad connects back to the Efficiency Team for shared AI capability — not duplicated local tooling.

This is not bureaucracy for its own sake. It is specialization with shared leverage — the same principle behind platform engineering, applied to the AI layer.

Partner functions

Engineering does not operate in isolation. Sales, Marketing, Account Management, and Finance remain partner functions — connected to outcomes, not buried inside every domain silo.

Finance in particular shapes how engineering leaders think about build-vs-buy, platform ROI, and cost of duplicated effort — themes I explore in What Finance Taught Me About Engineering Decisions.

What this is not

This model is not:

• A reason to add headcount without removing duplication
• An excuse to centralize every decision in one architecture committee
• A replacement for domain expertise — Solution Architect and Product Super squads exist precisely because domains still matter

The goal is to replace repeated effort with shared capabilities that improve the productivity of every squad.

AI will not eliminate engineering teams. It will force us to redesign how they work together — from AI model consumer habits to agentic workflows that retrieve, act, and pause for review.

A practical migration path

You do not flip org charts overnight. A realistic sequence:

• Name the duplication — inventory where each domain team rebuilt the same AI tooling, test harnesses, or release scripts
• Stand up a thin Efficiency Team — start with one MCP server, one RAG corpus, one approved agent workflow that two squads actually use
• Pull architecture out of silos — Platform and Solution Architect squads publish patterns; Execution ships against them
• Keep partner functions visible — Finance and account teams stay in the operating rhythm, especially when platform investments need ROI language
• Measure leverage, not activity — fewer rebuilt pipelines, faster time-to-production for new domains, higher reuse of MCP tools across teams

Start where you are

If your organization still looks like three identical domain teams, you are not behind — you are at the starting line most enterprises share.

The shift is conceptual first: treat MCPs, RAG, agents, and workflows as shared infrastructure, not side projects inside every team. Then let the org structure follow the leverage you want.

The diagram below summarizes the before-and-after operating model — siloed repetition on top, connected squads with shared AI infrastructure below.

---

Related reading

• From AI Model Consumer to AI Application Builder
• Model Context Protocol Tools, Resources, and Prompts Explained
• Running MCP Servers on Kubernetes
• Kubernetes Concepts Every Staff Engineer Should Understand