Measure the system around the model.
The proposed study asks what changes operationally when an application moves from a conventional service to an AI-enabled system.
We define an “infrastructure tax” as the incremental operational burden relative to a declared baseline. This is a measurement question, not a claim that AI necessarily increases every cost—or that any added cost is unjustified.
The protocol separates deployment effort, runtime behavior, maintenance work, and application quality. It compares a conventional service, a managed-model integration, and a retrieval-augmented variant under documented workloads.
Compute differences separately for each metric. Do not combine engineer-hours, latency, and currency into an undefined aggregate score. Value and quality must be evaluated alongside overhead.
Three questions to test.
- Where does operational work accumulate? Separate one-time setup from recurring deployment, observability, evaluation, and incident tasks.
- What changes as architecture grows? Compare a managed-model integration with the same integration plus document ingestion and retrieval.
- What does the added complexity buy? Report task quality alongside latency and cost so that overhead is not interpreted without utility.
A controlled comparison, with visible assumptions.
Proposed task: answer questions over a fixed, versioned document corpus. The conventional baseline returns search results; AI variants generate answers. These outputs are not functionally identical, so usefulness will be scored separately against a shared task rubric.
| Configuration | Application path | Purpose |
|---|---|---|
| A · Baseline | Search API + document store | Reference operating footprint |
| B · Managed model | API + fixed context + hosted model | Model-integration overhead |
| C · Retrieval + model | API + retrieval + hosted model | Retrieval and ingestion overhead |
Proposed procedure
- Freeze the specification. Record the corpus hash, task set, scoring rubric, infrastructure, model identifier, prompts, region, load profile, timeout, and retry policy before data collection.
- Run a pilot. Estimate variability and select a repetition count and confidence-interval method before the confirmatory runs. Record changes to this draft in version control.
- Compare paired workloads. Use the same questions and concurrency levels; randomize configuration order. Separate warm-up from measurement and identify warm-cache and cold-cache runs.
- Measure operations. Log setup and maintenance time with a fixed task checklist. Exercise documented failure scenarios in an isolated test environment.
- Analyze transparently. Report failures, retries, sample counts, uncertainty, and raw per-run values. Compare A–B and B–C differences without treating every difference as a causal AI effect.
Latency, errors, traffic, and saturation provide the runtime measurement frame [1]. Proposed instrumentation uses traces, metrics, and logs [2].
| Dimension | Measure | Unit / reporting |
|---|---|---|
| Setup | Hands-on configuration and deployment effort | Engineer-hours; separate waiting time |
| Runtime | Latency, errors, retries, throughput | p50/p95 ms; counts; requests/s |
| Cost | All run costs, including failed requests | USD per run and per successful task |
| Operations | Maintenance effort and recovery duration | Engineer-hours; minutes by scenario |
| Quality | Task success under the frozen rubric | Successes / attempted tasks |
Make the measurement boundary explicit.
The diagram describes the proposed retrieval-augmented configuration. Baseline A omits model inference; B omits retrieval and uses a fixed context. Identity, logging, and deployment controls should be held constant where practical.
Fixed load profile
Request orchestration
Hosted model endpoint
Evidence comes next.
No empirical findings are available.
The protocol has not been executed. No cost multipliers, performance rankings, or claims about enterprise-wide impact can be inferred from this page.
X: USD per attempted task.
Y: task success rate.
One estimate per configuration, with uncertainty.
Setup and maintenance engineer-hours, shown separately by configuration and task category.
These are figure specifications, not charts of simulated or observed results. Data and figures will be added only after measurements exist.
A result should come with a route back to the evidence.
The repository currently contains this publication and an empty measurement template. A reproducible benchmark will require the following artifacts before findings are published:
- Environment: pinned dependencies, deployment manifests, service versions, and resource sizes.
- Inputs: corpus provenance and permissions, hashes, task set, prompts, and evaluation rubric.
- Execution: runner commands, seeds where supported, retries, timeouts, repetitions, and failure scenarios.
- Evidence: sanitized raw measurements, cost accounting, exclusions, and quality scores.
- Analysis: scripts that regenerate every table and figure from the released data.
How to use the measurement template
Create one row per measured run. Use configuration IDs A, B, or C. Record UTC timestamps and monetary values in USD, retaining the exchange-rate basis in the run manifest if needed. Keep the manifest and raw evidence in a versioned release. Blank values mean unavailable, never zero.
The file intentionally contains only a header. It is a starting schema, not an executable benchmark or a completed dataset.
What this study would—and would not—establish.
A single workload cannot characterize enterprise AI as a whole. Team familiarity, cloud region, service quotas, provider changes, cache state, and model quality can change the observed differences.
The search baseline and generative configurations offer different functionality. Their cost difference is descriptive; it is not automatically a measure of waste. Managed services also shift some work to providers rather than removing it.
Security and governance effort should be documented explicitly. NIST AI RMF 1.0 is background for structuring that discussion [3], not a certification or compliance claim. This protocol does not evaluate organizational or legal compliance.
Background & measurement foundations.
- Google. Monitoring Distributed Systems. Site Reliability Engineering. Background for runtime signals.
- OpenTelemetry. Signals. Documentation. Background for telemetry collection.
- NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0). 2023. DOI: 10.6028/NIST.AI.100-1. Background for risk documentation.
These references inform the proposed methodology; they are not evidence for study findings. References accessed 18 September 2026.