Machine Learning in Production: MLOps from Prototype to Contractual SLA

Putting machine learning into production requires transforming an experiment into a reproducible, observable, secure, and operable service under real-world conditions. MLOps connects data, models, infrastructure, and processes so that technical metrics—such as availability, latency, prediction quality, and recovery time—can be converted into measurable SLOs and, when necessary, a contractual SLA.

Why a Good Prototype Is Still Not a Production System

A notebook may demonstrate that there is a predictive signal in the data, but it usually does not address continuous ingestion, versioning, authentication, monitoring, or failure recovery. It may also depend on local files, manual steps, and libraries without pinned versions.

The transition to production changes the question. Instead of asking only “is the model accurate?”, the team needs to answer:

• Which version of the data generated each model?
• Will inference work with incomplete or delayed data?
• How long can the response take?
• How can degradation be detected without immediately knowing the actual outcome?
• Who receives the alert and decides whether to maintain, roll back, or retrain?
• Will the system continue to work if the model is unavailable?
• Which commitments can be made in a contract?

The operational unit is no longer just the model file. It now includes transformation code, the data contract, the runtime environment, infrastructure, security policies, telemetry, and incident procedures.

What MLOps Needs to Control

MLOps applies software engineering, data engineering, and operations principles to the machine learning lifecycle. A consistent implementation controls four types of artifacts.

Data and Features

Data must have its source, schema, period, and quality rules recorded. Seemingly minor changes—a unit of measurement, a new category, a renamed field, or a change in how a field is populated—can invalidate learned transformations.

The data contract should define at least:

• required and optional fields;
• types, units, and allowed values;
• handling of missing data;
• expected update frequency and delay;
• identification of personal or sensitive data;
• retention and disposal policy;
• the party responsible for the source of each dataset.

In healthcare, for example, an integration based on HL7 v2 or FHIR must distinguish the structural validity of the message from the clinical validity of its content. Receiving a valid FHIR resource does not guarantee that the values required for a prediction are present or temporally consistent.

Code, Environment, and Dependencies

Training and inference transformations should share the same implementation whenever possible. This reduces the risk of training-serving skew, a situation in which the model is trained with one logic and executed with another.

Source code, container images, dependencies, configurations, and infrastructure as code must be versioned. Rebuilding a version cannot depend on the memory of the person who ran the training process.

Model and Validation Evidence

The model registry should associate each version with:

• the dataset and its time range;
• hyperparameters and training code;
• metrics by relevant segment;
• technical approval and, when applicable, business approval;
• known limitations;
• the image or environment used for inference;
• promotion and rollback criteria.

The appropriate metric depends on the cost of an error. Accuracy can hide failures in rare classes; therefore, precision, recall, specificity, F1, area under the curve, and calibration should be evaluated as appropriate. For regression models, MAE, RMSE, and error by range can provide complementary perspectives.

Operational Service

The model must be packaged as a batch service, synchronous API, event consumer, or embedded component. The architecture should match the business process, not the format that was most convenient during prototyping.

An online API supports immediate decisions but requires availability and latency control. Batch processing tends to simplify costs and reprocessing, but it introduces delays. Streaming reduces the time between an event and a prediction at the cost of greater operational complexity.

Pipeline from Prototype to Production

An auditable workflow can be organized into the following stages:

1. Validate the problem: define the supported decision, affected population, cost of errors, and alternative without ML.
2. Create a baseline: compare the model with a simple rule, the current process, or a previous model.
3. Version the data: record the source, time range, transformations, and validations.
4. Automate training: remove manual steps and generate traceable artifacts.
5. Run tests: cover code, schema, transformations, integration, and model behavior.
6. Register and approve: store the model, metrics, limitations, and parties responsible for promotion.
7. Deploy gradually: use shadow deployment, canary, or blue-green according to risk.
8. Monitor: observe infrastructure, data, predictions, outcomes, and process impact.
9. Operate incidents: maintain actionable alerts, runbooks, rollback procedures, and designated owners.
10. Reevaluate: retrain or retire the model when its assumptions are no longer valid.

Shadow deployment runs the new model without using its response in the actual decision. Canary releases the version to a controlled portion of traffic. Blue-green maintains separate environments and allows switching between versions. The method should be selected based on the ability to observe outcomes, the impact of an incorrect prediction, and the cost of maintaining duplicate infrastructure.

How to Convert MLOps Metrics into SLOs and SLAs

An SLI is the measured indicator, such as availability or latency. An SLO is the internal target defined for that indicator. An SLA is the contractual commitment, usually accompanied by scope, exclusions, a measurement method, and consequences for noncompliance.

It is not safe to contractually promise the same target used internally. The SLO should preserve a margin for investigation, maintenance, and expected variations. Values must be calculated based on criticality, architecture, and actual measurements; generic numbers copied from another system do not represent the service’s capabilities.

Dimensions That Can Be Included in the SLA

• Availability: percentage of requests served within the agreed window.
• Latency: duration percentile, not just the average.
• Error rate: invalid technical responses, timeouts, or processing failures.
• Data freshness: maximum delay between the source, processing, and availability.
• Recovery: timeframe for restoring the service or activating a previous version.
• Support: channels, hours, severity levels, and response times.
• Traceability: ability to identify the model version, input, and outcome.

Predictive quality requires different treatment. The actual outcome may arrive days or months later, and the observed distribution may change. Instead of guaranteeing that every prediction will be correct, the contract can specify the evaluation method, eligible population, time window, handling of missing data, and procedure to follow when the metric falls below the agreed threshold.

What the Contract Should Clarify

An ML SLA needs to define boundaries of responsibility. Was the degradation caused by the service, by data submitted outside the contract, or by a change in the process that generates the labels? Without this distinction, the contractual discussion becomes subjective.

Include the following in the document:

• formula and source for each indicator;
• measurement window and time zone;
• external dependencies;
• maintenance periods;
• exclusion criteria;
• incident classification;
• parties responsible for communication and decisions;
• rollback policy;
• process for disputing measurements;
• consequences and corrective plan in the event of a violation.

Monitoring: Infrastructure, Data, Model, and Business

Monitoring only CPU and memory does not indicate whether the model remains useful. A mature operation combines four layers.

Infrastructure: availability, latency, queues, errors, resource consumption, and dependency failures.

Data: schema, completeness, ranges, unknown categories, delays, and feature distribution. Data drift indicates a statistical change in the inputs but does not prove a loss of quality.

Model: prediction distribution, confidence, calibration, metrics with available labels, and performance by segment. Concept drift occurs when the relationship between inputs and outcomes changes.

Business: adoption of recommendations, modified decisions, outcomes, and possible adverse effects. A technically stable model may fail to generate value if it is disconnected from the workflow.

Each alert should indicate severity, evidence, the responsible party, and the recommended action. Alerts without a runbook create noise; automatic retraining without criteria can promote a worse model. For critical applications, human approval and gradual deployment are usually safer controls.

Production Readiness Checklist

Before release, confirm:

• [ ] the problem, users, and decision are documented;
• [ ] there is a comparable baseline;
• [ ] data and transformations are versioned;
• [ ] training and inference are reproducible;
• [ ] automated tests are part of the pipeline;
• [ ] the model has a registry entry, an owner, and known limitations;
• [ ] secrets are not stored in the code or image;
• [ ] access follows the principle of least privilege;
• [ ] logs prevent improper exposure of personal data;
• [ ] metrics and alerts have designated owners;
• [ ] rollback has been tested;
• [ ] the SLI, SLO, and measurement method are defined;
• [ ] SLA dependencies and exclusions are documented;
• [ ] there is a plan for drift, retraining, and retirement.

How Predictor Solutions Addresses This

Predictor Solutions implements machine learning as part of operable systems, combining data engineering, custom software, applied artificial intelligence, cloud/DevOps, and offensive security. In healthcare, it works with HL7 v2 and FHIR integration and maintains products such as Predictor Health, focused on dashboards and wearables, and Predictor AI Hospitals, designed to predict sepsis, heart attacks, and pneumonia in ICUs.

The approach connects data contracts, reproducible pipelines, model registries, controlled deployment, observability, and the definition of indicators that can support SLOs and SLAs. The company, based in Lavras, Minas Gerais, has served 9 medium-sized and large organizations; across its projects, it reports average savings of R$ 1.32 million per client per year, an average productivity increase of 70%, and profit growth of 43% in six months.

Contact: contato@predictorsolutions.com / WhatsApp +55 31 98835-3246.