API Gateways, Service Boundaries and the Diagrams That Explain Them

An API gateway is one of those pieces of architecture everyone agrees on in principle and nobody enforces in practice. Draw the diagram and it looks clean: clients talk to the gateway, the gateway talks to the services, nothing crosses that line. Six months later, half the services have a second, unofficial front door that nobody drew, because someone needed to ship something and the gateway was, that week, in the way.

This article is about that gap — between the gateway as an architectural intention and the gateway as an actual, verifiable fact about your system — and about what it takes to close it. Not with another diagram, and not with a policy document nobody rereads, but with a model of the dependency graph that is precise enough to check.

What an API gateway is actually for

Strip away the product marketing around any particular gateway technology and the architectural job is narrow and specific: an API gateway is the single point where external traffic is allowed to enter your service landscape. Everything that a client — a web app, a mobile app, a partner integration, another company's backend — needs from your services, it gets by going through that one point, never around it.

That single fact, single entry point, is what makes three other things possible:

• Centralized authentication and authorization. The gateway is where you decide, once, whether a caller is who they claim to be and what they are allowed to touch. Every service behind it can assume that decision has already been made, instead of each one implementing its own copy of the same auth logic — and inevitably implementing it slightly differently.
• Centralized rate limiting and traffic shaping. Throttling, quota enforcement, circuit breaking under load — all of it lives in one place that sees all the traffic, rather than being bolted onto each service individually, where it tends to be inconsistent or simply missing.
• Centralized routing, and therefore decoupling. The client knows one address. What actually serves a given request — which internal service, which version, which region — is the gateway's decision, and it can change that decision without the client ever finding out. Split a monolith into three services and the gateway absorbs the change; nothing external has to be told the internal topology moved.

Put together, those three things describe the real value of the pattern: it draws a hard line between "the shape of our public API" and "the shape of our internal service topology," and it lets those two shapes evolve independently. That decoupling is the entire point. A gateway that clients can route around isn't providing decoupling, centralized auth, or centralized rate limiting for the traffic that goes around it — it's providing those things only for the traffic that happens to still be going through the front door.

The anti-pattern: death by a thousand exceptions

Almost nobody sets out to bypass the gateway. It happens gradually, and every individual step looks reasonable at the time.

The pattern is familiar enough that most architects can narrate it from memory. A batch job needs to pull data from the orders service overnight, and someone decides the gateway's rate limits — sized for interactive traffic — are the wrong shape for a bulk export, so the job gets a direct connection instead, "just for now." A partner integration is under deadline pressure and the gateway team hasn't shipped the new route yet, so the partner is handed a direct URL and a promise to migrate later. An internal admin tool, technically a client of the inventory service, gets built by a team that didn't know the gateway existed, or knew and didn't think an internal tool counted. A reporting dashboard reads straight from a service's database because someone found that easier than calling an API at all.

None of these decisions were unreasonable in isolation. Each one was a small, local trade-off made under real constraints. What makes the pattern dangerous is that it's invisible in aggregate. Nobody ever decides "let's undermine the gateway" — the gateway gets undermined one exception at a time, each one forgotten roughly as soon as it ships, until the actual dependency graph looks nothing like the one on the architecture diagram.

The costs show up later, and they show up all at once, usually during an incident or an audit rather than during calm planning time:

• A security review asks "what can reach the payments database directly" and the honest answer requires someone to go find out, because it was never tracked anywhere that would tell you.
• The gateway team wants to roll out a new auth scheme and can't be sure which traffic would break, because they don't know who's actually behind the gateway versus who's routing around it.
• An incident on a backend service turns out to have three different callers with three different expectations about its contract, one of which nobody on the service team knew existed.
• A "simple" internal refactor of a service's schema breaks a reporting job that was reading the database table directly, and the first anyone hears of it is the report going blank.

The common thread is that the diagram said one thing and the system did another, and there was no mechanism that would have caught the divergence before it caused damage. That's not a tooling failure in the sense of "we used the wrong diagramming software." It's a structural failure: the architecture was documented as a picture, and pictures don't get checked against reality unless a person remembers to, and remembers correctly, and does it often enough to matter.

Modeling it so the bypass is visible, not assumed away

The fix isn't a stricter policy memo. It's making the actual traffic topology something you can look at and verify, rather than something you have to trust. That starts with treating "client," "gateway," and "service" as real elements in a model, with real relationships between them, rather than as boxes on a slide that happen to be arranged in the right order.

In Mooodels, that looks like a small set of elements and typed relationships: a Client (or Public Application) element, a Gateway element, one element per backend Service, and directed relationships between them that carry a type — "routes through," "calls directly," and so on — rather than being generic, undifferentiated lines. Drawn on the canvas, the intended topology is five elements and four connections: a Customer Portal tagged as publicly exposed, an API Gateway tagged as an approved entry point, and Auth, Orders and Inventory sitting behind it — with the portal connected to the gateway as "routes through," and the gateway connected to each of the three services as "routes to."

The tags are doing as much work there as the boxes. "Publicly exposed" and "approved entry point" are not decoration; they are the properties everything later in this article depends on, because they are what lets a check tell a client that must go through the gateway apart from an internal job that need not, and a sanctioned gateway apart from any other element that merely happens to sit in the middle of a path. Applying them at the moment the topology is first drawn costs nothing. Retrofitting them across a hundred services eighteen months later is a project, and it is a project that tends to get deferred exactly as long as it takes for the question to become urgent.

That's the intention, and on its own it's no more trustworthy than the diagram it replaces — it's still just a description of how things are supposed to work. What changes the picture is recording the exceptions the same way, as the same kind of typed relationship, instead of leaving them undocumented. The nightly reporting job that reads inventory data directly isn't a gap in the model; it's an element in the model, tagged as internal, with a connection to Inventory explicitly typed as a direct call rather than a routed one.

Once that edge exists in the model — instead of existing only as a connection string in a config file somewhere — it stops being invisible. It's a relationship the same graph queries, the same views, and the same rules see and can act on. The diagram below shows both halves of that graph at once: the intended path through the gateway, and the bypass sitting right next to it instead of hiding behind it.

The point of drawing it this way isn't aesthetic. A diagram that only ever shows the intended path is a diagram that agrees with you no matter what's actually happening. A diagram generated from a model that includes the bypass edge shows you the disagreement between intention and reality the moment it exists, because the bypass is a real relationship in the graph, not a fact someone would have to remember to draw in.

From "we think" to "we can check": the deterministic rule

Drawing the bypass so it's visible is a real improvement over hiding it, but it still depends on someone looking at the right view at the right time. The step that actually closes the gap is turning the intention — public applications use the approved gateway, full stop — into a rule the model itself can check, continuously, against every relationship that exists, not just the ones currently on screen.

That rule reads almost exactly like the sentence an architect would say out loud in a design review: public applications must use the approved API gateway. Expressed against the model, it's a query with three parts — find every element tagged as a public-facing application, look at what it connects to, and flag any connection to a backend service that doesn't route through an element tagged as an approved gateway.

Run against the model above, this rule doesn't flag the reporting job at all — it isn't tagged public, so it's out of scope by design, which matters: not every direct call is a violation, and a rule that flagged all of them would train people to ignore it. It would flag a public-facing client that somehow acquired a direct edge to a backend service, which is the actual failure mode worth catching. The value isn't that the rule is clever; it's that it's exhaustive. It doesn't check the three diagrams someone remembered to open. It checks every relationship touching every public application in the entire model, every time it runs, including the one added last week by someone who never saw the original design review.

That's the real difference between "we think everything goes through the gateway" and "we can verify that everything goes through the gateway." The first is a belief about a system, held by whoever last checked and possibly stale since. The second is the output of a query anyone can re-run, on demand, against the current state of the actual dependency graph — not against whatever a diagram claimed six months ago.

Where the AI assistant fits — and where it deliberately doesn't

The scenario this pattern comes up in constantly is exactly the one the gateway rule is meant to prevent: inserting a gateway in front of a service that a client currently calls directly. It's common enough, and mechanical enough once you see the shape of it, that it works as the canonical example of how AI-assisted modeling is supposed to behave in Mooodels — propose the change, never make it unilaterally.

Ask the assistant, in plain language, to "add an API gateway and move authentication behind it," and it doesn't touch the model. It produces a ModelPatch: a specific, reviewable, structured set of operations against the graph — add this element, add these relationships, remove that one — that a human looks at before anything is applied. For the portal-to-services topology used throughout this article, that patch looks like this:

{
  "patch": [
    { "op": "add-element", "type": "gateway", "id": "gw-1", "name": "API Gateway", "tags": { "approved-gateway": true } },
    { "op": "add-relationship", "from": "portal", "to": "gw-1", "type": "routes-through" },
    { "op": "add-relationship", "from": "gw-1", "to": "auth", "type": "routes-to" },
    { "op": "add-relationship", "from": "gw-1", "to": "orders", "type": "routes-to" },
    { "op": "remove-relationship", "from": "portal", "to": "orders", "type": "calls-directly" }
  ],
  "rationale": "Inserts an approved gateway between the public portal and the auth/orders services, and removes the portal's existing direct call to orders so traffic routes through the gateway instead."
}

Two things about that patch matter more than the fact that an AI wrote it. First, it's checkable before it's applied — run it against the gateway rule above in a dry run, and you can confirm the resulting model would actually pass, not just that it looks plausible. Second, it's exactly as auditable as a change a human drew by hand on the canvas, because it goes through the same patch mechanism either way. There's no separate, less-visible path where the AI edits the model directly — every proposed change, AI-authored or not, is a diff against the canonical model that a person approves, and Mooodels is provider-neutral and BYOK about which AI does the proposing, because the guarantee that matters is about the review step, not about which model generated the suggestion.

This is also where the earlier point about visibility and the rule-based check meet in practice. The assistant proposing the patch can see the same rule the architect can — it can check, before proposing, whether routing the portal through the new gateway would actually satisfy "public applications must use the approved gateway," rather than producing a plausible-looking diagram that happens to leave a relationship dangling. And after the patch is applied, the same rule keeps running against whatever gets added next, by anyone, whether or not they were in the room when the gateway was introduced.

A worked scenario, end to end

It's worth walking through a slightly larger version of this to see how the pieces compose, because the value of the approach is mostly visible at the point where a system has grown past what any one person can hold in their head.

Say a public-facing mobile app, an internal admin console, and a nightly ETL job all eventually need something from a service called Pricing. The mobile app was built correctly from day one and calls Pricing through the gateway. The admin console was built by a different team, six months later, under time pressure, and calls Pricing directly — reasoning, at the time, that "it's internal, the gateway is for public traffic." The ETL job was added later still, by someone who inherited the pipeline and had no context on the gateway policy at all, and also calls Pricing directly, because that's what the code it was copied from did.

Tagged and modeled honestly, this graph has three relationships into Pricing: one routed, two direct. Running the public-applications rule against it produces exactly one finding — the mobile app is public and it's compliant, so it passes silently; the admin console and the ETL job are both untagged as public, so by the letter of the rule as written, neither trips it. That's the moment a second, narrower rule usually gets added, because the real governance question wasn't only about public exposure — it was also "should Pricing be reachable at all except through the gateway," which is a stricter, service-scoped version of the same check. Its scope is every relationship that terminates at Pricing, whatever the source and whatever that source is tagged. Its condition is that the relationship either originates at the gateway itself or is a routed one. Its severity is a warning rather than an error, because an internal caller reaching Pricing directly is a conversation worth having rather than a boundary breach on the public edge. And its finding names the source element that reaches Pricing without going through the gateway, so the output is a list of callers to talk to rather than a count of problems.

Run that, and both the admin console and the ETL job light up — not because anyone remembered to go looking for them, but because the rule doesn't care whether anyone remembered. It's checking every relationship that terminates at Pricing, which is a small, exhaustive, mechanical thing to check once the relationships are real edges in a real graph, and an open-ended, unbounded thing to check by memory once a system has more than a handful of services and more than one team touching them.

What happens next is a normal engineering conversation — maybe the admin console gets migrated behind the gateway, maybe it gets an explicit, tagged exception because internal traffic really is out of scope for this particular rule, maybe the ETL job's direct read gets replaced with a batch-oriented endpoint the gateway actually supports well. The rule doesn't make that decision. What it does is guarantee the decision gets made deliberately, on a documented exception, rather than by omission — which is the entire difference between the failure mode this article opened with and a system where the gateway pattern is actually holding.

What this doesn't solve, and what it does

It's worth being precise about the boundaries here, because it's easy to oversell a modeling approach as a substitute for engineering discipline it can't actually replace.

A rule catching an unapproved direct call after the fact doesn't prevent the call from having been made in code and deployed to production — that's a runtime and network-policy problem, and it belongs to service meshes, network segmentation, and API management infrastructure, not to an architecture model. What the model gives you is the ability to know, reliably and continuously, that the call exists and that it violates policy, instead of finding out during an incident or a security audit. Closing the loop the rest of the way — failing a build, blocking a deploy, alerting a team — is a matter of wiring the rule's output into whatever pipeline already enforces other policies, which is straightforward once the check itself is deterministic and machine-readable rather than a line item in a design review someone has to remember to raise.

Nor does modeling the topology correctly guarantee anyone keeps modeling new relationships as they're added — a model that stops being updated degrades exactly like a diagram that stops being updated, just more slowly, because the rules keep working against whatever's actually in the model rather than silently going stale themselves. The realistic goal isn't a model that's magically self-maintaining; it's lowering the cost of keeping it current enough that doing so is the path of least resistance — importing from Archi or Sparx EA rather than starting from a blank canvas, letting an AI assistant propose the graph update alongside the code change instead of as a separate chore, and having every view resolve against the same underlying model, so there's only ever one place to update rather than a separate diagram per audience.

What the approach reliably does deliver is turning a question that used to require research — does everything really go through the gateway — into a question that requires a query. That's a meaningful shift in what "governance" means day to day. It stops being a periodic audit performed by someone with enough tribal knowledge to know where to look, and starts being something closer to a linter: fast, exhaustive, boring, and running against the actual state of the system rather than against whoever's memory of it is freshest.

The pattern generalizes

Everything here is specific to gateways because gateways are where the anti-pattern is most common and most costly, but the underlying shape — an architectural intention, a set of exceptions that accumulate invisibly, and a rule that makes the gap checkable — applies well beyond API traffic. "All production databases must be reachable only from their owning service," "all Tier-1 applications must have a designated owner," "nothing in the DMZ calls back into the internal network" are the same structure: a relationship-shaped rule, checked exhaustively against a graph that actually records the relationships, rather than trusted on the strength of a diagram that was accurate the day it was drawn.

The gateway is just the clearest version of the problem, because almost every architect has watched it happen at least once: a clean, sensible boundary, drawn with good intentions, worn down one reasonable exception at a time until the diagram and the system barely resemble each other. Modeling the boundary as something the system can check, instead of something a person has to remember, is what keeps that boundary real six months after the design review that established it.