Layer 02 · Traffic
AI workload routing, designed like a route plan.
A routing table is a set of decisions written down: what this request is, what quality it requires, which paths are eligible, in what order, and what happens when a path fails. Most AI stacks send everything down one expensive path and call the invoice a surprise.
Class of service
Not every request deserves the same model.
Classification is the whole game. Once traffic has classes, routing, budgeting, evaluation and cost attribution all become tractable engineering problems.
High-Consequence
Outputs influencing money, contracts, safety, security, compliance, or material rights. Requires stronger evaluation and human acceptance controls.
Production
Customer or internal workflows with defined quality, latency, reliability, and cost objectives.
Mechanical
Extraction, classification, transformation, formatting, and bounded routing tasks. Smaller or local models may be sufficient after evaluation.
Exploratory
Isolated experimentation with capped budget and no automatic promotion into production.
Classes are a starting taxonomy, tailored per environment. The names matter less than the fact that a written budget exists for each one. Smaller or local models may be sufficient for evaluated mechanical workloads and can reduce marginal route cost, subject to quality, latency, capacity, hardware, energy, and operating requirements.
Test a class-of-service decision— publishing soon
Route eligibility
Eligibility first. Economics second.
A route that cannot satisfy the workload is not a cheap route — it is not a route at all.
The logic
Eligible Route = Capability ∩ Policy ∩ Privacy ∩ Authority ∩ Entitlement ∩ Context Fit ∩ Reliability ∩ Latency ∩ Locality ∩ Capacity Selected Route = Eligible path with the lowest expected verified total cost that clears acceptance, continuity, latency, and risk thresholds
The cheapest route is not eligible if it cannot satisfy the workload.
Eligibility constraints
- — Capability
- — Policy
- — Privacy
- — Authority
- — Entitlement
- — Context Fit
- — Reliability
- — Latency
- — Locality
- — Capacity
Verified total outcome cost can include
- · Direct model or provider cost
- · Retry cost
- · Human correction
- · Review
- · Failed execution
- · Continuity reconstruction
- · Escalation
- · Latency burden
- · Operational burden
This is deliberately not cheapest-model routing. A path that produces a cheap answer twice and a human correction once was never the low-cost path.
Architecture pattern
Cognitive Least Cost Routing
Cognitive Least Cost Routing is Hardpoint's architectural pattern for selecting an eligible AI execution path after capability, policy, privacy, authority, entitlement, context, reliability, latency, locality, and capacity requirements have been satisfied. Economic ranking occurs only among eligible paths.
CLCR is a design pattern Hardpoint applies inside a gateway or routing layer you own. It is not a Hardpoint software product, not a commercially supported platform, and not an industry standard — it is a named way of writing the decision down so it can be reviewed, argued with, and changed on evidence.
Methods and artifacts
Named patterns, stated for what they are.
Operating pattern
Model Concierge
Model Concierge is Hardpoint's human-facing operating pattern for coordinating hosted models, local resources, tools, specialist workers, and human review without forcing operators to manage every route manually.
Engagement artifact
Continuity Capsule
A structured handoff record carrying the objective, accepted decisions, current state, evidence, authorization boundaries, unresolved questions, exact next action, and required verification.
Engagement artifact
Routing Receipt
A per-request record showing workload class, eligible routes, exclusions, selected route, route reason, execution path, latency, direct cost, retries, evaluation, correction burden, fallback, and final disposition — the AI equivalent of an accountable execution record.
Evaluation method
Shadow Routing
Candidate paths are evaluated against a baseline before consequential production routing is changed. Nothing is promoted on the strength of a benchmark alone.
Methodology
Six steps, in order.
- 01
Inventory the traffic
Enumerate real AI requests by system, team and purpose. Volume, shape, sensitivity, latency tolerance.
- 02
Classify
Assign every request a class of service with written quality, latency and cost budgets.
- 03
Determine eligibility
For each class, establish which models, providers, tools, local resources and human paths actually satisfy every requirement.
- 04
Rank eligible routes
Order the eligible set by expected verified total cost per accepted outcome — economics apply only after eligibility.
- 05
Wire failover and continuity
Define fallback chains, timeouts, retry policy, escalation, and the handoff record that survives a route change.
- 06
Instrument, enforce & review
Emit a Routing Receipt per request, enforce policy in the gateway, and review the routing table on a schedule.
Interactive reference environment
Watch eligibility, failover and a receipt.
A synthetic routing decision, start to finish. No production systems are connected.
Synthetic reference environment
Knowledge source
01Knowledge source· done
- Salesforce
- Confluence
- Jira
- Git
- Runbooks
02Governed OKF concept· pending
concept: refund-authority owner: finance-ops sensitivity: internal freshness: 14d provenance: 3 sources
03Workload class· pending
High-Consequence
04Eligible routes· pending
- Local 8B · owned hardwareeligible
- Regional API · mid tiereligible
- Frontier API · primaryeligible
- Offshore API · lowest priceeligible
Exclusions are stated, not implied: Capability below acceptance; Locality policy excluded.
05Fallback· pending
Primary path degrades mid-request. The predetermined fallback takes the workload before the operator is paged.
06Routing Receipt· pending
class: high-consequence eligible: 2 of 4 excluded: capability, locality selected: frontier-api fallback: regional-api (engaged) retries: 1 latency: 2.4s evaluation: passed disposition: accepted
Interactive reference environment · Synthetic data · No production systems connected
Failure modes
What goes wrong without it.
Single path
One provider, no fallback. Their incident becomes your outage, and nobody has decided in advance what the product should do while it lasts.
No accounting
Spend is visible only as a monthly total. No team, workload or use case can be tied to it, so cost discussions become political rather than technical.
Unbudgeted quality
Nobody wrote down what good enough means, so every model change triggers an argument instead of a measurement.
Terminology
What this work is called.
Different organizations name this discipline differently. The engineering is the same: classify the workload, establish eligibility, rank routes, wire failover, measure everything.
Show us your traffic.
If you can export a month of AI usage, we can tell you where classification and routing would change the shape of your architecture.