Putting machine learning into production requires more than publishing an API: data, code, models, infrastructure, metrics, and incident response must be managed as parts of a single system. The contractual SLA must reflect what the architecture and operation can measure and sustain, separating technical availability, latency, predictive quality, and model updates.
Why a Good Prototype Is Not Yet a Product
A notebook can demonstrate that there is a predictive signal in the data, but it usually does not answer essential production questions: how to reproduce training, validate new versions, detect anomalous data, roll back a deployment, or keep the service running during an external failure?
The transition from prototype to production changes the success criteria. During experimentation, metrics such as F1-score, precision, recall, MAE, or AUC usually dominate the analysis. In production, the model becomes part of a system that must also meet requirements such as:
- availability and latency;
- processing capacity;
- security and privacy;
- prediction traceability;
- cost per inference;
- failure recovery;
- data quality and freshness;
- a secure update process.
A model with excellent AUC may be unsuitable if it takes three seconds to respond to an operation that requires 200 milliseconds. Likewise, an API with 99.9% availability may deliver useless predictions if the data distribution has silently changed.
What MLOps Needs to Control
MLOps applies engineering, automation, and observability practices to the machine learning lifecycle. The goal is not to install a specific tool, but to make every version reproducible, testable, deployable, and auditable.
Complete Versioning
Versioning only the model file is not enough. At a minimum, a reproducible version must link:
- preparation and training code;
- dataset reference or version;
- parameters and hyperparameters;
- environment and dependencies;
- validation metrics;
- generated artifact;
- feature transformation rules;
- author, date, and run approval.
This link makes it possible to determine which code and data produced a prediction. For regulated use cases or high-impact decisions, it is also important to record the input, output, model version, and any subsequent rules applied—while complying with privacy and retention requirements.
Integration and Delivery Pipeline
A CI/CD pipeline for ML must test both software and statistical behavior. A practical sequence includes:
- unit tests for transformations;
- validation of data schemas, types, and ranges;
- detection of missing or unexpected features;
- training tests on a controlled sample;
- comparison with the reference model;
- vulnerability and dependency analysis;
- creation of an immutable image or package;
- deployment to a staging environment;
- load, integration, and security testing;
- automated or human approval before production.
A new version should not be promoted solely because it outperformed the previous model on one metric. It may increase false negatives, perform worse for a specific segment, or require more infrastructure. Promotion criteria must combine statistical thresholds, technical requirements, and business impact.
Inference Architecture: Online, Asynchronous, or Batch
The choice of inference mode directly affects cost, latency, and the SLA.
Online Inference
Online inference is appropriate when the response must be returned during the interaction, such as classifying a request or providing a real-time recommendation. It requires an available API, scalability, timeouts, circuit breakers, and a strategy for outages.
The latency budget must account for the entire path: authentication, feature retrieval, preprocessing, inference, post-processing, and network time. If the SLA specifies 500 ms, it is not safe to allocate all 500 ms to the model alone.
Asynchronous Processing
Asynchronous processing is appropriate when a request can enter a queue and be processed later. Queues decouple producers and consumers, absorb spikes, and facilitate retries. However, they require idempotency, duplicate-message control, a dead-letter queue, and delay metrics.
Batch Inference
Batch inference works well for periodic predictions, such as daily portfolio scoring. It is usually more cost-effective, but its operational commitment should be expressed as a completion window—for example, “95% of records processed by a specified time”—rather than as per-request latency.
From SLI to Contractual SLA
An SLA should not originate from a commercially selected percentage. First, measurable indicators are defined, followed by internal objectives and, finally, the contractual commitment.
- SLI: an observed indicator, such as the percentage of valid responses delivered within 500 ms.
- SLO: the internal operational objective for that indicator.
- SLA: the formal commitment, including scope, exclusions, measurement method, and consequences of noncompliance.
If the service promises 99.9% monthly availability, the theoretical downtime limit is approximately 43 minutes in a 30-day month. At 99.5%, it is approximately 3 hours and 36 minutes. This difference affects architecture, on-call coverage, redundancy, and cost.
Relevant SLIs for Machine Learning
A contract may combine different dimensions, but they should not be merged into a single number:
- Availability: proportion of eligible requests successfully served.
- Latency: p50, p95, and p99 percentiles, not only the average.
- Freshness: maximum age of data or features.
- Throughput: sustained volume per second or per window.
- Batch completion: percentage completed within the deadline.
- Error rate: technical failures, timeouts, and invalid responses.
- Recovery: RTO for service restoration and RPO for tolerable data loss.
Predictive quality deserves separate treatment. In many contexts, the actual label arrives days or months later, making an immediate accuracy guarantee impossible. In such cases, the contract can define the evaluation frequency, sampling window, metrics, analyzed population, and review triggers without promising permanent accuracy that changes in the real world make impossible to guarantee.
Observability for Data, Models, and Services
Monitoring CPU, memory, and HTTP errors covers only the infrastructure. An ML operation requires four layers of observability.
Service
Track availability, latency by percentile, error rate, saturation, queues, timeouts, and cost. Metrics should be segmented by model version and request type.
Data
Monitor schemas, missing values, cardinality, ranges, unknown categories, and update delays. A data contract violation should stop the pipeline or route records for handling instead of silently contaminating the model.
Model
Evaluate prediction distributions, confidence, feature drift, and output drift. When labels are available, calculate actual performance by period and relevant segments. Drift does not prove degradation, but it indicates the need for investigation.
Business
The technical metric must be connected to the operational outcome. A classifier may maintain its F1-score and still lose value if the cost of an action changes or if the team cannot handle the volume of alerts. Adoption rate, actionable alerts, and outcome per decision help reveal this difference.
Deployment and Rollback Strategies
Replacing 100% of traffic at once increases risk. The most commonly used strategies are:
- Shadow: the new version receives a copy of the traffic but does not influence the decision.
- Canary: a small share of requests uses the new model.
- Blue-green: two complete environments enable quick switching and rollback.
- Champion-challenger: the current model is continuously compared with candidate models.
Rollback must be tested, not merely documented. This includes restoring the previous artifact, compatible transformations, and routing configuration. When a schema changes, rollback may fail even if the old file still exists.
There must also be a degraded mode. Depending on the risk, an outage may trigger a deterministic rule, cached response, manual processing, or a queue for later execution. In critical decisions, returning a prediction without reliable data may be worse than declaring the service unavailable.
Checklist Before Committing to an SLA
Before signing a production commitment, verify that:
- [ ] peak load and seasonality have been tested;
- [ ] SLIs have defined formulas, sources, and measurement windows;
- [ ] external dependencies and contractual exclusions are explicit;
- [ ] alerts have assigned owners and response procedures;
- [ ] RTO and RPO are compatible with backups and architecture;
- [ ] rollback has been executed in a realistic environment;
- [ ] data, code, features, and models are versioned;
- [ ] logs make it possible to trace a prediction without improper exposure;
- [ ] drift and performance by segment are monitored;
- [ ] there is an error budget for planned changes;
- [ ] costs have been tested at the contracted volume;
- [ ] security, retention, and access controls match the data risk.
A sustainable SLA must be less than or equal to the system’s proven capacity, leaving a margin between the internal objective and the external commitment. If the internal SLO is identical to the SLA, any deviation eliminates the operational margin.
How Predictor Solutions Addresses This
Predictor Solutions structures machine learning projects as production systems, combining data engineering, applied artificial intelligence, cloud/DevOps, security, and custom development. Its work covers reproducible pipelines, APIs and batch processing, observability, gradual deployment, rollback, and the definition of SLIs, SLOs, and SLAs aligned with operational risk.
This experience is also reflected in its proprietary products: Predictor Health works with health dashboards and wearables, while Predictor AI Hospitals applies prediction to sepsis, heart attacks, and pneumonia in ICUs. In healthcare systems, the company also works with HL7 v2 and FHIR integrations, where traceability, data quality, and interoperability are part of model reliability.
Across its projects, Predictor Solutions has served 9 medium-sized and large companies, with reported results of R$ 1.32 million in average savings per client per year, an average productivity increase of 70%, and profit growth of 43% in six months. These figures do not replace the definition of specific metrics for each new project; they serve as an operational track record to guide verifiable goals.
Contact: contato@predictorsolutions.com / WhatsApp +55 31 98835-3246.