The Post-Transformer Enterprise

Your Data. Your AI. Your Way. Page 1

Contents Executive Summary................................................................................................................................3 Part I — Foundations: How the Transformer Won, and What It Costs...........................................3 Why parallelism, not attention, was the breakthrough..................................................................................4 The efficiency lesson from the original benchmark....................................................................................... 4 Part II — The Serving Layer: Where Enterprise AI Economics Live..................................................6 PagedAttention: virtual memory for the KV cache.........................................................................................6 The number that reaches the CFO....................................................................................................................7 Part III — The Model Layer Is Fragmenting: Beyond the Transformer............................................9 Mixture-of-Experts, State Space, recurrent, and hybrid families..................................................................9 The two failure modes leadership must avoid............................................................................................... 11 Part IV — The Synthesis: The Sovereign, Self-Instrumenting Runtime........................................12 One substrate; self-instrumenting reliability; sovereignty by construction...............................................12 Part V — A Decision Framework for Technology Leadership.........................................................13 About Enclavia.ai & The Ask................................................................................................................14 Sourcing & methodology note......................................................................................................................... 15

Your Data. Your AI. Your Way. Page 2

Executive Summary

For a decade, the strategic question in enterprise AI has been which model to adopt. That question is now the wrong one. The model layer is fragmenting into a landscape of architectures, and the durable economics, reliability, and control of an AI system are decided one layer down — in the runtime that serves the model, holds its state, and watches it while it works. Leadership teams that keep optimizing at the model layer will keep paying the runtime's tax.

The thesis, in one line. Whoever owns an architecture-agnostic, selfinstrumenting runtime — running inside their own trust boundary — owns the cost curve, the reliability, and the sovereignty of enterprise AI. That runtime is what Enclavia is building.

This paper consolidates three bodies of technical evidence into a single argument for technology leadership. First, the foundations: the Transformer won not because attention is magical but because it removed recurrence and unlocked parallel hardware — and in doing so it made training compute-bound and inference memorybound. Second, the serving layer: because inference is memory-bound, the runtime that manages GPU memory determines unit cost, and a paged-memory runtime has been shown to cut serving cost per token by roughly two-thirds against a legacy stack.

Third, the diversification: the Transformer is no longer the only game — Mixture-of- Experts, State Space Models, recurrent, neuromorphic, Dragon Hatchling, and hybrid architectures each compute differently and each break the assumptions a Transformeronly runtime is built on.

Put together, these three lines converge on one conclusion. An enterprise AI platform should not be architected as an inference engine for one model family. It should be architected as a generalized computation substrate: one runtime, many architecture adapters, architecture-aware optimization, unified state management, and — critically for regulated and federal buyers — deployment inside the customer's own perimeter with continuous, self-generated authorization evidence. This is the difference between renting intelligence you cannot see into, and owning intelligence you can measure, correct, and govern.

The remainder of this paper develops the argument in five parts and closes with a decision framework. The figures are drawn from published research and internal modeling; sourcing tiers are noted so leaders can weigh evidence appropriately.

Part I Foundations: How the Transformer Won, and What It

Costs Before the Transformer (Vaswani et al., 2017), state-of-the-art sequence models were recurrent (RNNs, LSTMs, GRUs) or convolutional. Recurrent networks compute each hidden state as a function of the previous one, so training cannot be parallelized across the sequence, and long-range dependencies decay through vanishing gradients over

Your Data. Your AI. Your Way. Page 3

long temporal paths. These were not tuning problems; they were structural ceilings on how large and how fast a model could be trained. The Transformer removed recurrence entirely and replaced it with multi-head selfattention, computing direct pairwise interactions across all positions at once. That single move reduced the number of sequential operations per layer to a constant and bounded the signal path between any two tokens to a constant — which is precisely what let training saturate modern GPU and TPU clusters. The lesson for leadership is not the mathematics; it is that the Transformer's dominance is a hardware-efficiency story, and hardware-efficiency stories are exactly the kind that get overturned when the hardware and the workload change.

Why parallelism, not attention, was the breakthrough

Scaled dot-product attention maps queries, keys, and values to outputs through matrix products and a softmax, with a scaling factor of one over the square root of the key dimension to keep gradients stable at scale. Multi-head attention runs several of these projections in parallel subspaces, letting the network attend to syntactic, semantic, and positional relationships simultaneously without increasing overall arithmetic complexity. The engineering point is that all of this is dense linear algebra with no step-to-step dependency inside a layer — the shape modern accelerators are built to devour.

The table below is the part every infrastructure decision-maker should internalize: different layer types carry fundamentally different complexity and, more importantly, are bound by different hardware limits. Layer architecture FLOPs / layer Sequential ops Max path length Hardware bound

Self-Attention O(n² · d) O(1) O(1) Compute-bound

(training / prefill) Recurrent

(LSTM/GRU) O(n · d²) O(n) O(n) Memory-bandwidth

bound Convolutional (k×1) O(k · n · d²) O(1) O(logₖ n) Compute-bound

The efficiency lesson from the original benchmark

On the WMT English-German and English-French translation benchmarks, the Transformer did not merely edge out the recurrent and convolutional ensembles that preceded it — it beat them at a fraction of the training cost. The base model reached competitive quality after about twelve hours on eight GPUs; the prior state of the art measured its training in months.

Model BLEU (En-De) BLEU (En-Fr) Training cost

(FLOPs) Hardware & time

GNMT + RL ensemble 26.30 41.16 1.8 × 10²⁰ 8× K80 (months)

ConvS2S ensemble 26.36 41.29 7.7 × 10¹⁹ 8× P100

Transformer (Base) 27.30 38.10 3.3 × 10¹⁸ 8× P100 (12 hours)

Your Data. Your AI. Your Way. Page 4

Model BLEU (En-De) BLEU (En-Fr) Training cost

(FLOPs) Hardware & time

Transformer (Big) 28.40 41.80 2.3 × 10¹⁹ 8× P100 (3.5 days)

Leadership takeaway. Training is compute-bound; production inference is not. Once a model is trained, generating tokens is bound by memory capacity and memory bandwidth — not raw compute. That single shift is why the serving layer, examined next, is where enterprise AI economics are actually won or lost.

Your Data. Your AI. Your Way. Page 5

Part II The Serving Layer: Where Enterprise AI Economics

Actually Live

Autoregressive inference is memory-bound. As a model generates, it maintains a Key- Value (KV) cache that grows with every token produced. Early inference runtimes allocated that cache as one contiguous block sized for the theoretical maximum request length. In production, that strategy wasted 60% to 80% of GPU memory across three channels: internal fragmentation (space reserved for context lengths requests never reach), external fragmentation (allocation holes left by variable request durations), and redundant duplication (identical prompt prefixes — system instructions, few-shot examples, RAG contexts — copied per request instead of shared).

Wasted memory is not an abstraction. On an inference cluster, memory is the binding constraint on how many requests a GPU can serve at once, which sets throughput, which sets how many GPUs you must rent, which sets the bill. Waste memory and you are literally renting hardware to hold empty space.

PagedAttention: virtual memory for the KV cache

PagedAttention (Kwon et al., SOSP 2023), the technique behind the vLLM runtime, borrows the oldest idea in operating systems — paging. It partitions each sequence's KV cache into fixed-size blocks and allocates physical GPU memory blocks noncontiguously, on demand, through a block table that maps logical positions to physical locations. When several requests share a prompt prefix, their logical blocks point at the same physical block; duplication happens only on divergence, through copy-onwrite. In multi-tenant enterprise workloads, prefix sharing alone cuts memory use by over half.

The measured result is a step-change in how much a single accelerator can serve. Figure 1. Paged KV-cache memory raises serving throughput several-fold and drives memory waste below 4%. Source: PagedAttention / vLLM (SOSP 2023); representative 13B-on-A100 figures.

Your Data. Your AI. Your Way. Page 6

Serving system Throughput

(tok/s) Speedup KV memory

waste Max concurrency (13B/A100)

FasterTransformer ~450 1.0× (baseline) 60–80% 8 concurrent

Orca (iteration-level) ~620 1.38× 55–70% 12 concurrent

(PagedAttention) 1,700–2,200 2.0–4.3× < 4.0% 38–42 concurrent

The number that reaches the CFO

Model a representative enterprise workload — 100 million generated tokens per day on LLaMA-70B-class clusters of 8× NVIDIA H100 nodes at roughly $3.50 per GPU- hour. Moving from a legacy un-paged engine to a paged runtime raises per-node throughput by more than three times, which collapses the fleet needed to hold the same load, which collapses the operating bill.

Figure 2. Same workload, same GPUs, different runtime: paged serving reduces monthly OPEX and per-token cost by ~66.7%. Source: internal TCO model on published vLLM throughput; illustrative (Tier 2).

Metric Legacy un-paged PagedAttention (vLLM) Variance

Throughput / node 2,800 tok/s 9,500 tok/s +239.3%

Active GPU nodes 6 (48× H100) 2 (16× H100) −66.7% footprint

Monthly OPEX $120,960 $40,320 −$80,640 / month Cost / 1M tokens $0.0403 $0.0134 −66.7% unit cost What this means for the architecture decision. A two-thirds swing in unit cost came entirely from the runtime — the model, the hardware, and the workload were held constant. The runtime is the highest-leverage layer an enterprise controls.

And this analysis assumed one architecture. Part III shows why the next generation of models will not sit still inside a Transformer-shaped runtime at all.

Your Data. Your AI. Your Way. Page 7

Three moves technology leaders can make now Standardize the runtime. Move endpoints off unoptimized custom or default serving stacks onto paged-KV runtimes (vLLM or TensorRT-LLM with paged kernels). This is the single largest cost lever available without changing the model.

Engineer for prefix caching. Structure multi-turn agents and RAG systems around shared, standardized prompt prefixes so identical context is stored once — zero-copy

• and time-to-first-token stays flat under load.

Adopt continuous batching and chunked prefill. Use iteration-level schedulers that interleave prefill and decode so p99 latency holds under burst traffic instead of collapsing.

Your Data. Your AI. Your Way. Page 8

Part III The Model Layer Is Fragmenting: Architectures

Beyond the Transformer The Transformer remains dominant, but the assumption that every future model is a dense, feed-forward, layer-by-layer Transformer is no longer safe to build on. The architecture landscape is diversifying across at least six computational paradigms, each with a different inference-state shape and a different systems bottleneck. For an infrastructure owner this is the central risk of the next cycle: a runtime hard-wired to attention-and-KV-cache will serve the wrong shape for a growing fraction of the models worth running.

Mixture-of-Experts: capacity decoupled from compute

Mixture-of-Experts (MoE) increases total model capacity without making every parameter fire on every token. A router sends each token to a small number of experts

• say two of sixty-four — so a trillion-parameter model may activate only tens of

billions of parameters per token. That breaks a load-bearing assumption of Transformer serving: total parameters no longer equals active parameters. Figure 3. In MoE, model capacity and per-token compute are separate quantities. The runtime challenge moves from arithmetic to routing, placement, and communication.

The systems problem is no longer matrix multiplication; it is routing, expert placement across devices, load imbalance when many tokens choose the same expert, all-to-all interconnect traffic, and a memory hierarchy that keeps hot experts on the GPU, warm experts in host memory, and cold experts on NVMe. A runtime that optimizes only matmul underperforms badly here; MoE needs topology-aware, routing-aware scheduling and predictive expert prefetch.

State Space and recurrent models: state instead of cache

State Space Models (SSMs) and modern recurrent architectures carry a compact, continuously evolving hidden state rather than an explicit record of every past token. Where a Transformer's memory grows with sequence length, an SSM's memory is roughly a function of state dimension — effectively flat. Selective SSM variants let the state decide, per input, what to keep, what to forget, and how fast to evolve.

Your Data. Your AI. Your Way. Page 9

Figure 4. The inference-memory curve is a property of the architecture, not the runtime. A KV-cache assumption is actively wrong for state-based models. For long-running streams, edge devices, and persistent agents, this is decisive: a compact persistent state beats shuttling large KV caches or re-transmitting context to the cloud. But it demands a runtime that treats state lifecycle — initialize, update, checkpoint, restore, migrate, compress — as a first-class service, not an afterthought bolted onto a cache manager.

Dragon Hatchling, neuromorphic, and hybrid computation

Further out, three families break the dense-tensor mold more radically. Dragon Hatchling (BDH) is a biologically inspired, graph-based architecture in which working memory lives partly in dynamic synaptic state — the relationships between computational units strengthen and decay during inference, in a Hebbian pattern, rather than being frozen weights plus an append-only cache. Neuromorphic architectures are event-driven: computation happens only where meaningful state changes occur, so inactive regions consume little or no dynamic capacity, and the right efficiency metric becomes events per joule rather than tokens per second — the metric that matters for wearables, robotics, UAVs, IoT, and low-power persistent agents at the edge. Hybrid architectures combine several of these mechanisms in one model, assigning local attention to short-range dependencies, state space to long-range memory, sparse experts to specialized knowledge, and dynamic memory to persistent adaptation.

The consolidated view — what actually changes for the infrastructure owner across all seven families: Architecture Primary primitive Primary inference state Main systems challenge Transformer Attention KV cache Memory bandwidth & context growth Mixture-of- Experts Conditional expert compute Routing + expert residency Communication & load balancing

State Space

Model State transition Persistent state State scheduling & recurrence Recurrent Hidden-state evolution Hidden state Sequential dependency

Your Data. Your AI. Your Way. Page 10

Architecture Primary primitive Primary inference state Main systems challenge Dragon Hatchling Dynamic graph / state Synaptic + activation state Sparse graph, dynamic state mgmt Neuromorphic Events & spikes Neuron + synaptic + event state Sparse asynchronous scheduling

Hybrid Heterogeneous

compute Multiple state classes Cross-architecture orchestration

The two failure modes leadership must avoid

Failure Mode 1 — Transformer lock-in. Force every model into an attentioncentric runtime and you throw away the architectural efficiency of everything that is not a Transformer — you pay MoE's communication cost with none of its sparsity benefit, and you impose a growing KV cache on models designed to avoid one.

Failure Mode 2 — Architecture fragmentation. Overcorrect by standing up a separate runtime for each architecture — a Transformer stack, an MoE stack, an SSM stack, a neuromorphic stack — and you get duplicated infrastructure, incompatible tooling, and engineering effort spent maintaining silos instead of shipping capability.

Both failure modes have the same cure: one runtime substrate, many architecture adapters, architecture-specific optimization, and unified hardware orchestration.

Your Data. Your AI. Your Way. Page 11

Part IV The Synthesis: An Architecture-Agnostic, Self-

Instrumenting, Sovereign Runtime

The three threads of this paper converge on a single design. Foundations tell us inference is memory-bound. The serving layer tells us the runtime owns the cost curve. Diversification tells us the runtime can no longer assume one architecture. The platform that follows is not an inference engine for a model family; it is a generalized computation substrate. This is the platform Enclavia is building — and the three properties below are what make it a durable position rather than a feature.

One substrate, many adapters

The right architecture is not a separate engine per model. It is a single runtime that recognizes the computational architecture, translates it through an architecture adapter into a unified intermediate representation, applies architecture-specific optimization passes, and maps the result onto the most appropriate available hardware

• CPU, GPU, NPU, FPGA, or native neuromorphic silicon. State management is unified

but not flattened: token caches, recurrent state, SSM state, synaptic state, expert residency, and event queues are distinct managed classes under one coordinator, so no architecture's assumptions leak into the global runtime. The strategic principle is blunt: models will change, architectures will change, hardware will change — the runtime substrate should remain reusable.

Self-instrumenting: the runtime watches the model

Most agentic projects do not fail because the model is weak. They fail because no one can see inside the model when it drifts, hallucinates, or loops — and in a closed cloud API, none of that internal state is visible until the failure has already reached the user.

Enclavia's Shepherd-AI orchestrator instruments the model's internal state and corrects a failing worker before the user sees it. Each of the three failure modes that sink production agents is bound to a measured signal. Figure 5. The Shepherd-AI drift monitor binds each agent failure mode to a concrete, measurable internal signal

• and responds with a graduated re-prompt, reroute, or retrain rather than a silent failure.

The response is graduated, not binary: on a first trip the partial output and a drift value go back to the orchestrator, which re-prompts or reroutes to a more trusted model; a repeat offender is benched for retraining. Trust is earned and remembered,

Your Data. Your AI. Your Way. Page 12

so the system learns which models it can rely on for which tasks — a competence map no closed vendor can hand you. Crucially, this kind of instrumentation requires access to the residual stream and hidden states, which is exactly what a self-hosted, openweight runtime provides and a closed API does not.

Sovereign by construction: ownership, zero egress, continuous

authorization The final property is where the argument becomes non-negotiable for regulated and federal buyers. Because the runtime runs open-weight models on hardware the customer owns, there is no per-token meter and no data egress — the marginal cost of an added node approaches the cost of a software copy, and enterprises self-hosting open models report 60–80% savings on high-volume workloads. The edge binary contains no outbound network path by construction, which is verifiable by static binary analysis rather than promised by policy. And because the system runs inside the customer's perimeter, it can generate its own authorization evidence as it operates — control attestations mapped to NIST SP 800-53 r5, produced continuously, which is what makes a continuous-ATO posture realistic instead of an 18-to-36-month paperwork exercise.

The moat, stated plainly. Control requires measurement. Measurement requires the model's internal state. Access to internal state requires running the model yourself, inside your own boundary. Sovereignty is therefore not a compliance checkbox bolted onto the platform — it is the precondition for the reliability and the economics the rest of this paper describes. You cannot rent your way to it.

Why this is feasible now

This architecture was impractical eighteen months ago. Four things changed. Openweight models (Qwen, Mistral, Llama-class) closed the quality gap for the bounded, tool-using tasks that make up most enterprise and mission work — trailing frontier closed models by a narrowing margin rather than a chasm. High-memory accelerators put a full multi-agent stack inside a single-rack capital cost that competes with months of cloud spend. Local runtimes turned on-premise deployment into a configuration task rather than a research project. And sovereignty rules — the EU AI Act, data-residency regimes, federal on-premise mandates — increasingly require in-boundary deployment outright. The honest caveat: open-weight edge parity is real but not uniform across every benchmark, and the platform is engineered so that its deployable core does not depend on any single research-stage component maturing.

Part V A Decision Framework for Technology Leadership

The practical question is not whether to adopt AI — that decision is made — but where to place the bet so it compounds instead of decaying. The following framework maps the three layers of this paper to concrete leadership actions.

Your Data. Your AI. Your Way. Page 13

Layer The wrong default The durable move Why it compounds Model Standardize on one model / vendor Treat models as swappable behind a contract layer Insulates you from the fragmentation in Part III Runtime Default or custom serving stack Paged-KV, architectureagnostic runtime Owns the ~66.7% cost lever from Part II Reliability Trust model output; inspect on failure Instrument internal state; self-heal Catches drift before it reaches the user

Deployment Cloud API, per-token

meter Own the stack inside your boundary No egress, no token tax, an asset at the end Authorization 18–36 month ATO after the fact Continuous, self-generated evidence Compresses time-to-operate for regulated use The three questions to ask of any enterprise AI platform

• When a model drifts or hallucinates mid-task, can the platform see it happen —

from the model's internal state — and correct it before my user does? If the answer depends on a closed API, the honest answer is no.

• When the next architecture that matters is not a Transformer, does my runtime

serve it natively, or does it force the wrong shape and pay for the privilege?

• Does my sensitive data ever leave my boundary — and can I prove it doesn't by

inspection rather than by trusting a vendor's policy page? If an enterprise AI program needs to be in the fraction that survives the next two years, the architecture decision is the one to make now

• at the runtime layer, inside your own boundary.

About Enclavia.ai

Enclavia.ai (formerly IntraIntel.ai), based in Fairfax, VA, builds compliance-native, sovereign enterprise AI. The platform runs open-weight models inside the customer's own trust boundary — an architecture-agnostic runtime, a self-healing multi-agent orchestrator (Shepherd-AI), unified state and memory management, and an in-built continuous-authorization control plane mapped to NIST SP 800-53 r5. It is engineered for regulated and federal missions: clinical trials, MedTech, healthcare, financial services, and defense, aligned to HIPAA, SOC 2, FedRAMP, FDA SaMD practice, and the DoD Iron Bank / Platform One reciprocity path.

The ask. We are inviting a small number of design-partner programs — in regulated, IP-sensitive, or sovereignty-bound mission areas — to deploy an Enclavia runtime cell against a real internal workload and measure three things with us: data never leaving the perimeter, drift caught before it reaches the user, and the cost crossover in their own numbers.

Your Data. Your AI. Your Way. Page 14

Dev Roy · Founder & CEO, Enclavia.ai

dev@enclavia.ai · 703-984-9981 · https://enclavia.ai · Fairfax, VA Your Data. Your AI. Your Way. Sourcing & methodology note Evidence in this paper is drawn from three tiers, noted where figures appear. Tier 1 — peer-reviewed and published results: Transformer mechanics and WMT benchmarks (Vaswani et al., 2017); PagedAttention throughput and memory figures (Kwon et al., SOSP 2023). Tier 2 — internal modeling calibrated to published inputs: the TCO figures, which apply published vLLM throughput to a representative 100M-token/day workload and are illustrative rather than a quoted price. Tier 3 — forward-looking and product claims: continuous-ATO timelines, edge parity, and self-hosting savings, which are stated as targets or reported ranges. Architecturefamily descriptions consolidate the enterprise architecture briefings 'Foundations & Systems Optimization in LLMs' and 'Architecture Families Beyond the Conventional Transformer.' Open-weight edge parity is real but not uniform across every benchmark; the platform's deployable core does not depend on any single research-stage component maturing.

Your Data. Your AI. Your Way. Page 15