AI Systems Have Three Layers, Not Two

I started my career as a software engineer and spent many years building systems. Over the past several years, I have moved increasingly deeply into the modeling side of AI.

That combination has changed how I think about production AI.

We often talk as though an AI product has two layers: the model and the software system built around it.

I increasingly think there are three:

the model, the software system, and the model-system boundary between them.

And some of the hardest work is moving into that third layer.

The model and the system speak different languages

A software system wants symbols.

An order is ready or it is not. A customer is waiting or being served. An event happened at a particular time. An API returns a typed object. A state machine moves from one state to another.

Models do not naturally operate this way.

Their world is stochastic. They produce probabilities, embeddings, similarities, noisy predictions and uncertain associations. Their outputs can vary with context, distribution shift and sometimes seemingly insignificant changes in input.

Somewhere, those two worlds have to meet.

A sequence of uncertain predictions has to become an event. A confidence distribution has to become a decision. Noisy observations over time have to become a stable state that the rest of the software system can consume.

That translation is what I think of as the model-system boundary.

It is neither purely modeling nor conventional software engineering.

A lot of the solution is surprisingly unglamorous

One thing I have learned from building production AI systems is that the right answer at this boundary is often not another neural network.

It may be temporal smoothing. Hysteresis. Confidence gating. A state machine. Some geometry. A nearest-neighbor rule. A simple heuristic that incorporates knowledge about how the physical world behaves.

These mechanisms are often easy to implement.

Knowing which one to implement is not.

You need to understand why the model is producing the behavior you are seeing. Is the noise random? Is there distribution shift? Is the representation unstable? Is identity being lost? Is the threshold wrong? Is the model missing information entirely? Will smoothing remove noise or erase a real state transition?

This is where having worked on both sides has been particularly useful to me.

My software-engineering background makes me instinctively look for a clean abstraction that the system above can consume. My modeling experience makes me much more cautious about what assumptions can safely be made when turning stochastic predictions into that abstraction.

Increasingly, I believe this boundary needs to be owned by engineers with real ML depth.

A coding agent cannot make this decision for you

Coding agents make the implementation side dramatically easier.

Once I know what I want—a state machine, a filter, a particular matching algorithm or a new evaluation harness—a coding agent can save enormous amounts of time.

But that qualifier matters:

once I know what I want.

At the bleeding edge, the difficult question is often precisely what should be built.

I recently ran into this while working on tracking people in motion. An agent suggested a tracking approach commonly used for vehicles. The recommendation was reasonable based on existing practice, but vehicle motion is relatively constrained: trajectories tend to have stronger kinematic continuity.

Humans are different. We can abruptly turn, stop, reverse direction or change how we move without following the assumptions that make those methods work well for vehicles.

The suggested solution performed poorly.

The coding agent had done what it could reasonably do: retrieve and synthesize approaches that had worked elsewhere.

But the actual problem was new enough that the answer was not sitting in the training corpus waiting to be retrieved. I had to understand both the behavior of the model and the characteristics of the system and work out a different solution.

That experience reinforced something I have been thinking about for a while.

Using a coding agent does not make someone AI-native.

What does AI-native actually mean?

We increasingly use “AI-native” to describe organizations or engineers who make heavy use of AI tools.

I think that definition is too shallow.

Being good at prompting a coding agent is certainly useful. But the agent is largely amplifying an engineering process whose objective has already been specified.

To me, being AI-native means understanding what changes when a stochastic learned component becomes part of the software architecture.

What is the nature of the model’s output?

What can safely be treated as a fact?

What uncertainty needs to survive into the system?

When should multiple weak predictions become one symbolic event?

When should the system abstain?

Which failure should be fixed in data, which in the model, which at the model-system boundary and which in ordinary software?

These questions require understanding both worlds.

The goal is not merely to call a model from software. It is to translate stochastic intelligence into reliable symbolic machinery.

The low-hanging fruit is disappearing

The first phase of the current AI wave created enormous opportunity simply by wrapping increasingly capable models with software.

That work was important, and there is plenty of it left to do.

But as those patterns mature, the easy opportunities become easier for everyone. Agents, RAG, tool use, structured generation, evaluation harnesses and orchestration are increasingly well understood, and coding agents make implementing them cheaper still.

The difficult problems move toward the boundaries.

When a new model capability meets a system requirement nobody has encountered before, there may not be a Stack Overflow answer, an established design pattern or a GitHub repository to copy.

The engineer has to reason from first principles.

That requires enough understanding of the model to know what it can and cannot provide, and enough understanding of the system to know what the rest of the application actually needs.

This is why I think MLEs have an interesting opportunity—and a challenge.

The MLE needs to become a generalist

Earlier in my career at Google, one hiring principle I appreciated was an emphasis on generalist engineers.

You might have deep expertise somewhere, but you were expected to be able to cross boundaries, learn unfamiliar systems and solve the problem that actually existed rather than the one that happened to match your job title.

The ML world has evolved somewhat differently.

It is increasingly possible to spend an entire career inside one slice of ML. A recommendation-system engineer may rarely touch search. A search engineer may never work on vision. A training specialist may know relatively little about inference systems. An MLE working on models may rarely engage deeply with the software architecture consuming their outputs.

There are good reasons for specialization. Modern ML is enormously complex.

But many of the underlying ideas transfer surprisingly well: representations, distributions, ranking, calibration, retrieval, optimization, evaluation, uncertainty and generalization.

And the model-system boundary does not respect organizational silos.

A production failure might travel through data, representation learning, inference, geometry, temporal aggregation, an API contract and a downstream state machine before becoming visible to the user.

Someone needs to be able to follow it across those boundaries.

I increasingly think the most valuable MLEs will therefore resemble AI systems generalists: people with enough depth to understand what happens inside the model and enough breadth to understand the systems around it.

That does not mean every MLE needs to train foundation models, design distributed databases and build frontends.

It means resisting the temptation to define yourself too narrowly.

Learn neighboring modeling domains. Learn enough systems engineering to understand where your predictions go. Understand the business abstraction your model is supposed to support. Follow your output all the way until it becomes something the product actually uses.

The next frontier may be the boundary

We have spent the last several years getting extraordinarily good at models.

We have also built an increasingly sophisticated discipline around the systems surrounding them: agents, tools, retrieval, context engineering, evals, guardrails and orchestration.

The space between those two deserves equal attention.

The model produces stochastic representations of the world.

The software system wants stable symbols on which it can act.

Turning one into the other is not plumbing. It is a core part of AI engineering.

And as the easy applications are increasingly built, I suspect much of the difficult work ahead will belong to people who are comfortable living exactly there—at the boundary between models and systems.