Return to page

Model Deployment and Operations


H2O Driverless AI offers model deployment, management and monitoring capabilities for the IT and DevOps teams. The platform makes it convenient for IT to deploy the winning model across a broad range of production environments. In addition, the data science and data engineering teams can monitor the performance of the model for any drifts in predictions and scores over time, as well as manage any re-training or tuning necessary at run-time.

Operationalizing Models

As enterprises “make their own AI”, new challenges emerge:

  • Maintaining reproducibility, traceability, and verifiability of machine learning models
  • Recording experiments, tracking insights, reproducibility of results
  • Searchability of models (or querying models)
  • Collaborating within the DS and ML teams
  • Visualizing model performance (drift, degradation, A/B testing)

Operationalizing models crosses functional boundaries

  • DevOps and IT teams are usually heavily involved
  • Model operations should require minimal changes to existing application workflows
  • Maintain data and model lineage in case of rollbacks, regulatory compliance
Orange files and folders network Orange files and folders network

Model Deployment Options with Driverless AI

Driverless AI offers the following options for deploying machine learning (ML)

models, depending on where the AI application is running:

  • 1. As a cloud service
  • 2. As a local REST endpoint
  • 3. As a Java object file

The model could be directly deployed in a cloud service. Driverless AI offers the ability to export the model directly in AWS Lambda or Sagemaker. Ideal for building and running your AI applications in AWS. Configuration details can be seen here.

The model could be configured to run on a local REST server. Driverless AI offers the ability to deploy the scoring pipeline on a local server. Ideal for AI workloads in on-premises environments. Configuration details can be seen here.

The model could be abstracted into a Java object as a standalone model scoring engine. Driverless AI allows downloading the model as a Plain Old Java Object (POJO) or Model Object Optimized (MOJO) file. POJO and MOJO files are standalone scoring engines. Ideal for running AI applications in low-latency environments such as edge devices or on-premises. Configuration details can be seen here.

Model Operations

As AI applications enter production:

  • Data scientists and data engineers need to manage the transition of ML models
  • Development → Staging → Production
  • Model accuracy can drift over time. Data scientists need alerts if drift exceeds certain thresholds.
  • DevOps/IT need a central store for models, model artifacts, related inference, etc.
  • ML engineers may need to re-calibrate or re-tune production models
  • Seamless collaboration between data science, DevOps and IT teams becomes important
Downloads_Hero_Image Downloads_Hero_Image

Key MLOps Capabilities in Driverless AI Today

Model Management

(Data Science Teams)

  • Projects Workspace
  • Model store
  • Collaboration

Model Deployment


  • Deploy to different environments – in the cloud, on-premises

Model Monitoring

(Business/Data Science)

  • Specify metrics or parameters to monitor
mlops mlops

MLOps with H2O Driverless AI

Model Monitoring and Management Done Right

Driverless AI includes new capabilities for model administration, monitoring and management. These capabilities allow:

  • DevOps teams to monitor the models for system health checks
  • Data science teams to monitor metrics around drift detection, model degradation, A/B testing
  • Provides alerts for recalibration and retraining

Telemetry and Alerts

Driverless AI can monitor models for drift, anomalies, model metrics and residuals, and provide alerts on a dashboard for potential re-tuning or re-training of models.

Model comparison

The MLOps offers capabilities to compare multiple models by looking at their confusion matrices. Moreover, our A/B testing functionality helps to run and compare multiple models in production before they are deployed in production.

Full model traceability

Data scientists can track back a prediction on a specific model and investigate the report to understand how it was created. This also includes the ability to frequently retrain and publish updated models to the runtime environment.

Model Governance

It is crucial for IT and DevOps to implement access rules and decision rights across the enterprises as AI goes into production. MLOps provides important capabilities such as role-based access controls for models as well as tracking who built the model and who deployed it.

yntIZGMH4z2inNT6CAXUBxaR-Xz1QI09M6YF5UIx9QYlB1Aw3qvcUd5MV4fF3YEy6qz3k3vA2FgL9dE_ccpiescm8mciRpCSK1OYBBxiDNo3W3VGo7YoWIB-31v4H7rizBR6VJJivjE yntIZGMH4z2inNT6CAXUBxaR-Xz1QI09M6YF5UIx9QYlB1Aw3qvcUd5MV4fF3YEy6qz3k3vA2FgL9dE_ccpiescm8mciRpCSK1OYBBxiDNo3W3VGo7YoWIB-31v4H7rizBR6VJJivjE

Deploy Model as a REST Endpoint

Pasted Graphic 27 Pasted Graphic 27