Taking machine learning from a prototype to a contractual SLA requires transforming the model into a reproducible, observable, secure, and operable service. MLOps connects data, code, infrastructure, and incident management so that availability, latency, predictive quality, and recovery time can be measured and contractually guaranteed.
Why a Prototype Is Not Ready for Production
A notebook can prove that a predictive signal exists, but it usually does not answer the questions required to operate the model: where the data will come from, how transformations will be repeated, which version is active, what happens when a dependency fails, and who is responsible for degradation.
The key difference is that a prototype optimizes an experimental metric, while production must balance at least five dimensions:
- Predictive quality: precision, recall, F1, AUC, MAE, or another metric appropriate to the problem.
- Operational reliability: availability, latency, error rate, and recovery.
- Data quality: completeness, validity, freshness, and distribution stability.
- Business impact: revenue, savings, productivity, safety, or risk reduction.
- Governance: traceability, access control, auditing, and rollback capability.
A model with a high AUC may be unfeasible if it takes five seconds per inference, depends on a variable that is unavailable in real time, or generates false negatives with an unacceptable cost. The production decision must consider the complete system, not only the serialized artifact.
What MLOps Must Control
MLOps is not merely a training pipeline. It is a set of processes and components for building, testing, deploying, monitoring, and replacing models in a controlled manner.
Versioning and Reproducibility
Each release must link, at a minimum:
- code commit;
- dataset version or the query that produced it;
- parameters and hyperparameters;
- runtime environment and dependencies;
- validation metrics;
- model artifact;
- person responsible for approval;
- deployment date and strategy.
Reproducibility does not mean obtaining numerically identical results on any hardware, which is not always possible. It means being able to reconstruct the experiment within defined tolerances and explain why a specific model was placed in production.
Testing Beyond Accuracy
A mature pipeline applies different testing layers:
- Unit tests: validate transformations, rules, and critical functions.
- Data contract tests: verify types, ranges, required fields, and accepted categories.
- Integration tests: cover databases, queues, APIs, feature stores, and external services.
- Model regression tests: compare the candidate against the current version.
- Load tests: measure latency, throughput, and resource consumption.
- Security tests: look for excessive permissions, exposed secrets, and malicious inputs.
- Resilience tests: simulate dependency outages and delayed data.
The approval criterion must be explicit. A candidate may need to outperform the current model in recall without exceeding a false-positive threshold, for example. To avoid opportunistic promotion, the comparison must also consider confidence intervals, stability by segment, and the evaluation period.
From ML Metrics to SLI, SLO, and SLA
The terms are not equivalent:
- SLI, or Service Level Indicator: an observed measurement, such as the percentage of requests completed within 300 milliseconds.
- SLO, or Service Level Objective: an internal target for the indicator, such as 99.7% of inferences within that limit.
- SLA, or Service Level Agreement: a contractual commitment with scope, exclusions, responsibilities, and consequences.
The SLA should be more conservative than observed capacity and leave room for maintenance and incidents. If the service has historically operated at 99.9% availability, contractually committing to the same number eliminates almost all operational margin.
Indicators That Can Be Included in the Contract
An online inference service can track:
- monthly API availability;
- latency at the p50, p95, and p99 percentiles;
- rate of 5xx responses and timeouts;
- maximum contracted volume;
- time to acknowledge and resolve incidents;
- feature freshness;
- percentage of predictions logged for auditing.
Predictive metrics require care. Actual quality may depend on labels that become available weeks later and on factors controlled by the client. In these cases, it is safer to define an internal quality SLO and establish the monitoring process, evaluation window, and minimum data conditions in the contract.
An example of measurable wording would be: “99.5% of valid requests will be processed monthly, excluding scheduled windows and the unavailability of sources under the contracting party’s responsibility.” The numbers are illustrative; they must be defined based on testing, criticality, and budget.
Architecture for Operating Models Securely
The architecture varies depending on whether inference is batch, online, or event-driven. In all cases, training, registry, and serving should be separated.
A typical architecture includes:
- data ingestion and validation;
- reproducible feature processing;
- automated training;
- evaluation against promotion criteria;
- model and metadata registration;
- deployment in a staging environment;
- testing and approval;
- gradual release to production;
- technical, statistical, and business monitoring.
For higher-risk deployments, strategies such as shadow, canary, and blue-green reduce impact. In shadow mode, the new model receives copies of the inputs without affecting decisions. In a canary deployment, it serves a small share of the traffic. Blue-green maintains two environments, making rollback easier.
Rollback must include the model, configuration, and data transformations. Reverting only the model file can break the service if the previous version expected different features.
Monitoring: A Healthy System Does Not Mean a Healthy Model
CPU, memory, and HTTP status codes show application health, but they do not detect all predictive degradation. Monitoring must combine four levels.
Operations and Infrastructure
Track availability, queues, percentile latency, errors, saturation, cost per inference, and dependency failures. Alerts should point to actionable symptoms, avoiding notifications for fluctuations with no impact.
Data
Measure missing values, duplicates, unknown categories, delays, invalid ranges, and distribution shifts. A statistical change does not prove performance degradation, but it signals a need for investigation.
Model
When labels are available, monitor metrics by period and segment. A global average can hide failures in specific units, products, age groups, or regions. Without immediate labels, use proxy signals, sample analysis, and comparisons of input and output distributions.
Business
Connect predictions to outcomes: conversion, time saved, losses avoided, or another validated metric. This level prevents the team from maintaining a technically sophisticated model that does not produce measurable value.
Incident Management and Contractual Responsibility
Every SLA needs a runbook. It should specify how to detect, classify, communicate, mitigate, and close incidents, as well as define owners and escalation channels.
A practical classification can distinguish between:
- Severity 1: service unavailable or critical risk, requiring immediate action.
- Severity 2: significant degradation, but with an operational alternative.
- Severity 3: limited failure without interruption to the main workflow.
- Severity 4: question, improvement, or cosmetic defect.
The contract must distinguish response time from resolution time. It must also clarify external dependencies, scheduled maintenance, contracting-party obligations, volume limits, log retention, security, privacy, and the availability calculation method.
Critical applications must have a fallback: a deterministic rule, previous model, manual processing, or safe response. If no alternative is acceptable, this limitation must be included in the risk analysis and SLA.
Production Readiness Checklist
Before signing a contractual commitment, verify that:
- [ ] the problem, target population, and supported decision are documented;
- [ ] the baseline and approval metrics have been defined;
- [ ] the data has contracts, owners, and automatic validation;
- [ ] training and inference use consistent transformations;
- [ ] the model, code, data, and environment are versioned;
- [ ] unit, integration, load, and regression tests are in place;
- [ ] secrets and access follow the principle of least privilege;
- [ ] gradual deployment and rollback have been tested;
- [ ] dashboards cover infrastructure, data, model, and business;
- [ ] alerts have an owner and a response procedure;
- [ ] labels and feedback return for evaluation;
- [ ] SLI, SLO, SLA, exclusions, and windows have been formalized;
- [ ] cost per prediction and maximum capacity are known;
- [ ] there is a contingency plan for model failures.
If several items depend on undocumented manual procedures, the service is still at the assisted operations stage, not at the stage of a robust SLA.
How Predictor Solutions Addresses This
Predictor Solutions implements custom software, applied artificial intelligence, data engineering, cloud/DevOps, and offensive security, integrating these disciplines into model operations. Its work includes reproducible pipelines, data validation, inference APIs, observability, gradual deployment, security testing, and the definition of indicators compatible with process risk.
In healthcare, the company develops Predictor Health, focused on dashboards and wearables, and Predictor AI Hospitals, for predicting sepsis, heart attacks, and pneumonia in intensive care units. These contexts require traceability, integration—including HL7 v2 and FHIR—and a clear separation between algorithmic support, clinical workflow, and human responsibility.
Predictor Solutions, headquartered in Lavras, Minas Gerais, has already served 9 medium-sized and large companies. Across its completed projects, it reports average savings of R$ 1.32 million per client per year, an average productivity increase of 70%, and profit growth of 43% in six months; however, each new project must validate its own indicators and measurement conditions.
Contact: contato@predictorsolutions.com / WhatsApp +55 31 98835-3246.