Return to page

BLOG

Building a Manufacturing Product Defect Classification Model and Application using H2O Hydrogen Torch, H2O MLOps, and H2O Wave

 headshot

By Shivam Bansal | minute read | May 15, 2023

Blog decorative banner image

Primary Authors:  Nishaanthini Gnanavel and Genevieve Richards

Effective product quality control is of utmost importance in the manufacturing industry. The presence of defective components can have adverse effects on various aspects, including escalating production costs, compromising product quality, diminishing product longevity, and leading to substantial resource wastage. However, by promptly identifying defects and implementing suitable measures, significant reductions in operational expenses and quality-related expenditures can be achieved. 

To mitigate these expenses, one possible approach is to conduct manual inspections during the production process. However, this method is time-consuming, expensive, and can lead to production delays, ultimately impacting market responsiveness. To address this challenge, the implementation of a machine vision defect detection system becomes imperative. Such a system has the potential to decrease production costs by minimizing the quantity of rejected and reprocessed faulty materials. It also aids in minimizing wastage of labor, materials, and machine time, reducing energy consumption, and facilitating the identification of underlying causes for defects. 

Manufacturing Product Defect Classification Solution using H2O AI Cloud

Our customer data science team, recently developed a solution to detect different types of defects on manufacturing products (steel sheets) using deep learning techniques and deployed the model in MLOps. The solution is then packaged in an AI application which can be accessed by different stakeholders. 

Snapshot of the Product Defect Classification Application
Snapshot of the Product Defect Classification Application

H2O’s AI cloud offers a diverse range of components that facilitate the complete lifecycle of developing, deploying, and utilizing machine learning or deep learning models. In this particular solution, we have leveraged three crucial components: 

  • AI Engine – H2O Hydrogen Torch: The H2O Hydrogen Torch application empowers data scientists of all levels, whether novice or experienced, to develop intricate deep learning models to address diverse problem domains like computer vision, natural language processing, and audio processing. Notably, this can be accomplished without requiring extensive coding knowledge. Utilizing the capabilities of H2O Hydrogen Torch, we successfully constructed a sophisticated multi-class deep learning model. The primary objective of this model was to accurately predict the presence and specific type of defects on micro thin steel sheets.
hydrogen-torch-marketecture-1.png hydrogen-torch-marketecture-1.png
  • Deployment: H2O MLOps: Upon conducting model tuning and identifying the optimal model using Hydrogen Torch, we proceeded to deploy it utilizing H2O MLOps. This advanced product facilitates the seamless export of Hydrogen Torch models to a collaborative MLOps project. Within this environment, users gain access to comprehensive features for efficient management, deployment, and monitoring of the deployed model.
mlops-screenshot-300x118.png mlops-screenshot-300x118.png
  • Application: H2O Wave: The development of the application was accomplished utilizing H2O Wave, an open-source Python development framework designed to streamline the creation of real-time interactive AI applications. With its user-friendly interface, H2O Wave empowers data scientists, machine learning engineers, and software developers to efficiently develop cutting-edge applications.

Model Details

Dataset:  For model training purposes, a dataset comprising surface images of hot-rolled steel sheets was utilized. To prepare the dataset, the images were converted to grayscale and subsequently labeled into six distinct categories. 

Model:  By employing the H2O Hydrogen Torch model, numerous image classification models were trained, incorporating diverse backbones like efficientnets, resnets, mobilenets, and VGG architectures. These models were integrated together to enhance overall performance. The selected loss function for training was CrossEntropy, accompanied by the Adam W optimization algorithm. 

Snapshot of Hydrogen Torch Model Scoring and Interpretation
Snapshot of Hydrogen Torch Model Scoring and Interpretation

Deployment:  Through the utilization of H2O MLOps, the scoring pipeline generated from the Hydrogen Torch AI engine was deployed as a standalone model. This deployment offered a REST endpoint and allowed inference calls to be made using CURL request parameters. The REST endpoint was seamlessly integrated into the AI application, providing configurability. This implies that in the future, if there is a need to modify the deployment options from single to batch, or single to champion/challenger model, or even enable AB testing mode, all such options can be supported effortlessly. 

Application:  The entire solution, model, and the deployment were wrapped in a Wave Application to view the results, set alerts, monitor the predictions etc. The wave app has a range of functionalities including: 

  • Bring your own data –  Upload two types of files: a zip file of images or a video file as an input.
  • Bring your own Model – Use the existing model or can create and deploy your own model in MLOps and use its endpoint URL in the app.
  • Email Alerts – Receive notifications and alert emails about the detected defects during the scoring process. These can be sent based on your configurations in the configuration tab. There are two types of alert emails. The first one is instance defect alert emails where alert emails are sent during the scoring process when a defect is detected with prediction score above the alert threshold set by the user. The second one is the batch defect summary alert email which is sent after the scoring process to give a summary about the defect classification process.
  • Global Monitoring View – This view showcases the results of the defect classification process after the scoring process and helps to analyze the defect classification process image by image.
graph-ss-manufacturing-300x159.png graph-ss-manufacturing-300x159.png

The dashboard features various useful visualizations relevant to the scoring results. The Defect Classification Analysis by image names section of the dashboard displays both the raw input image of the hot rolled steel strips and a spectral image that highlights the area of the defect more clearly to the user. The spectral image is generated using OpenCV Saliency, which helps the user analyze the defect in that strip more effectively.  

The Product Defect Classification app provides a reliable and efficient solution to the problem of product quality control in the manufacturing industry. With the ability to detect defects, classify them and provide timely and accurate results, this application helps in reducing operational and quality related costs by minimizing rejected and reprocessed defective materials. Ultimately, our Product Defect Classification app is a valuable tool for the manufacturing industry to improve product quality, reduce costs, and increase overall efficiency. 

 

For any queries, please reach out at cloud-feedback@h2o.ai Contact us if you are interested in discussing this use-case or any other use case further. 

 headshot

Shivam Bansal , Director, Field Technology

Shivam is the 3x Kaggle Grandmaster, 5 times winner of Kaggle’s Analytics / Data Science for Good Competition, and the winner of several other offline and online competitions. He holds a master's degree from the National University of Singapore and was a Valedictorian. He has extensive cross-industry and hands-on experience in building data science products and applications. He brings a strong blend of technical and business skills with a practical and solution-driven approach. He supports various functions within the company which include - engineering, pre-sales, and customer success. His LinkedIn profile can be found here.

 headshot

Genevieve Richards

Genevieve (Gen) is a data science practitioner with over 5 years experience in AI use cases across different domains in particular Financial services. She has two Bachelor Degree in Arts (Linguistics & Psychology) & IT (Computer Science) at the University of Queensland and Queensland University of Technology respectively.  She is passionate about AI democratization, Responsible AI and AI for Social Good.  Her LinkedIn profile can be found here.

 headshot

Nishaanthini Gnanavel

Nishaanthini Gnanavel is a Machine Learning Engineer Intern at H2O.ai. With a keen interest in the field of artificial intelligence and data science, Nishaanthini is committed to leveraging her skills and knowledge to contribute to innovative solutions in the industry. Connect with her on LinkedIn