MLOps Guide: How to Take a Model from Notebook to Production
What MLOps really involves, how mature your setup needs to be, and a step-by-step path for getting machine learning models into production and keeping them healthy.
MLOps is the set of practices, tools and team habits that take a machine learning model from an experiment to a reliable production service, and keep it working afterwards. It covers versioned data and code, reproducible training, automated testing and deployment, monitoring for drift, and a clear process for retraining and rollback.
Many organisations can build a model that looks good in a notebook. Far fewer can run twenty of them in production, retrain them safely and explain last month's prediction to an auditor. That gap is what MLOps closes, and it is usually an engineering and process problem rather than a data science one.
Why models stall between notebook and production
In our experience, the reasons are remarkably consistent across industries:
- Training is not reproducible. Nobody can recreate the exact data snapshot, features and parameters behind the model that was approved.
- Features differ between training and serving. A feature computed in SQL for training is re-implemented in application code for inference, and the two quietly disagree.
- No owner after launch. Data scientists move to the next project, engineers do not understand the model, and performance decays unnoticed.
- Manual handoffs. A pickle file emailed to an engineer is not a deployment process.
- No definition of "good enough". Without agreed metrics and thresholds, every release becomes a debate.
The MLOps lifecycle
A useful way to think about MLOps is as a loop with six stages, each with its own controls.
| Stage | What happens | Key practices |
|---|---|---|
| Data management | Ingest, validate and version training data | Data contracts, schema and quality checks, snapshotting |
| Feature engineering | Build features consistently for training and serving | Shared feature pipelines or a feature store, point-in-time correctness |
| Experimentation | Train and compare candidate models | Experiment tracking, fixed random seeds, recorded data versions |
| Validation | Decide whether a model is fit to release | Holdout and backtest metrics, fairness and robustness checks, approval gates |
| Deployment | Serve predictions in batch or real time | Model registry, CI/CD, containerised serving, canary or shadow releases |
| Monitoring and retraining | Keep the model healthy | Data and prediction drift, business KPIs, alerting, scheduled or triggered retraining |
The loop matters. Monitoring feeds back into data and retraining, and without that feedback a model is a one-off project rather than a product.
MLOps maturity levels: how far do you need to go?
Not every organisation needs fully automated pipelines on day one. A pragmatic view of maturity:
| Level | Description | Suits |
|---|---|---|
| 0: Manual | Notebooks, manual training and hand-deployed artefacts | Proofs of concept only |
| 1: Repeatable | Training scripted and versioned, experiment tracking, model registry, manual promotion | First one to three production models |
| 2: Automated | Pipelines retrain on schedule or trigger, CI/CD for model and code, automated validation gates, monitoring | Several models with business-critical decisions |
| 3: Platform | Shared feature store, self-service templates, standard monitoring, governance across teams | Many teams and dozens of models |
A sensible target for most mid-size companies is level 1 for the first model and level 2 within a year. Jumping straight to a level 3 platform before you have three models in production usually wastes budget on tools nobody uses yet.
Choosing MLOps tooling
The tool landscape is crowded, so decide by capability first and product second.
- Experiment tracking and registry: MLflow, Weights & Biases, or the native services in SageMaker, Vertex AI and Azure Machine Learning.
- Orchestration: Airflow, Dagster, Prefect, Kubeflow Pipelines or cloud-native pipeline services.
- Feature management: a disciplined set of shared feature tables in your warehouse or lakehouse is often enough; Feast or a managed feature store helps once real-time features and many teams arrive.
- Serving: batch scoring jobs for most business use cases; containerised APIs (FastAPI, BentoML, KServe, managed endpoints) where you truly need real-time predictions.
- Monitoring: Evidently, WhyLabs, Arize or cloud-native model monitoring, plus your existing observability stack for latency and errors.
A managed cloud ML platform reduces operational burden but increases lock-in. Open-source components give flexibility but need engineers to run them. For a small team, managed services plus MLflow is often the best balance. Your data foundation matters too; our comparison of data lakehouse vs data warehouse covers how storage choices affect ML workloads.
A step-by-step path from model to production
This is the sequence we follow when taking a first model into production.
- Define the decision and the metric. Write down what the model will decide, how that maps to a business KPI, and the minimum acceptable performance. Agree it with the business owner.
- Decide batch or real time. Most use cases, such as forecasting, churn scoring or lead prioritisation, work perfectly well as daily or hourly batch jobs, which are far simpler to run.
- Put everything under version control. Code, pipeline definitions, configuration and environment files in Git; data snapshots and model artefacts versioned and linked to the commit that produced them.
- Script the training pipeline. Replace notebook cells with a pipeline that goes from raw data to registered model with one command, and runs the same way on a laptop and in the cloud.
- Use one feature definition for training and serving. Share code or tables so there is a single source of truth, and check point-in-time correctness to avoid leaking future data into training.
- Add automated tests. Unit tests for feature logic, data validation tests, and model tests that compare the candidate with the current production model on a fixed evaluation set.
- Register and gate. Push candidates to a model registry with metadata, metrics and lineage. Promotion to production requires passing thresholds and, for high-risk decisions, a named human approval.
- Deploy safely. Use CI/CD to package and deploy. Start with a shadow deployment or a small canary, compare outputs, then roll out fully. Keep the previous version one click away.
- Monitor three layers. System health (latency, errors, throughput), data and prediction drift, and the business outcome the model was meant to move.
- Automate retraining with guardrails. Retrain on a schedule or on drift triggers, but always pass the same validation gates before promotion.
- Document and hand over. A short model card covering purpose, data, limitations and owners, plus a runbook for alerts.
Many of the practices in steps 3, 6 and 8 are standard software engineering. Our guide to CI/CD best practices covers them in more depth.
Monitoring: what to watch after launch
Models degrade because the world changes. Watch for:
- Data drift: input distributions shift, for example a new customer segment or a pricing change.
- Concept drift: the relationship between inputs and outcome changes, as in fraud patterns adapting to your controls.
- Data quality failures: upstream schema changes, missing values or late-arriving data.
- Outcome performance: accuracy measured once true labels arrive, which may be days or months later for credit or churn.
Set alert thresholds that reflect business impact, route alerts to a named owner, and review model health in a monthly forum. Alerts that nobody owns are noise.
Governance and responsible ML
For regulated sectors such as lending, insurance and healthcare, MLOps doubles as your audit trail. Lineage from prediction back to model version, training data and approval record makes regulator questions answerable. Add bias and fairness checks to validation where decisions affect people, keep human review for high-impact cases, and document known limitations. If personal data is involved, align retention and access with the DPDP Act 2023 and sector rules; this is general information, not legal advice.
LLMs and GenAI: does MLOps still apply?
Yes, with adjustments often grouped under LLMOps. You may not train models, but you still version prompts, retrieval pipelines and model choices, evaluate outputs against test sets, monitor cost, latency and quality, and control releases. The discipline is the same: nothing reaches users without reproducible configuration, automated evaluation and a rollback path.
Frequently asked questions
What is the difference between MLOps and DevOps?
DevOps automates building, testing and releasing software. MLOps extends it to machine learning, adding data versioning, experiment tracking, model validation, drift monitoring and retraining, because model behaviour depends on data as well as code.
Which tools are used in MLOps?
Common choices include MLflow or Weights & Biases for tracking and registry, Airflow, Dagster or Kubeflow for orchestration, Feast for features, and Evidently or Arize for monitoring. Cloud platforms such as SageMaker, Vertex AI and Azure Machine Learning bundle many of these capabilities.
Do small teams need MLOps?
Yes, but a lightweight version. Versioned code and data, a scripted training pipeline, a model registry and basic monitoring are worth having even for one model. Full platforms and feature stores can wait until you run several models.
How long does it take to put an ML model into production?
It varies widely by data readiness, integration complexity and risk controls. As a rough guide, a batch model with clean data and a clear owner can go live in weeks, while real-time models embedded in core systems of a regulated business can take several months.
How often should machine learning models be retrained?
It depends on how quickly your data changes. Some models need weekly retraining, others hold up for a quarter or more. Let monitoring guide the schedule, and always validate retrained models before promotion.
How Sunday Labs can help
Sunday Labs helps teams build MLOps and data foundations sized to where they are, from a first production model to a shared platform across teams. Every engagement is led personally by our founder, with senior engineers who have run machine learning in production doing the hands-on work. If you have models stuck in notebooks or production models nobody is watching, start a conversation and we will help you plan a practical path forward.
