AI Automated Microstructure Analysis for Intelligent Manufacturing

Additive Manufacturing (AM) is a rapidly evolving robust technology that constructs intricately detailed and complex structures layer-by-layer. Unlike traditional manufacturing techniques, AM offers a revolutionary approach to manufacture parts innovatively and efficaciously, which makes it an exciting area for development and a desirable technology for Industry 4.0. The flexibility of fabrication and the enhancement of applicability of AM enable its use across numerous fields, such as medical, aeronautics, and transport industries. Laser Powder Bed Fusion (LPBF) is one of the frequently used metal 3D printing technologies in AM. It utilizes high- powered lasers to selectively melt and fuse metal powder, enabling rapid production of printed parts through a layer-by-layer approach. The quality of printing is pivotal as it directly influences the reliability and safety of final products [1]. Laser-based AM involves numerous complex physical processes that lead to the evolution of microstructure and various defects in a printed part. To ensure the production of high-quality printed parts, it is essential to develop effective approaches for monitoring part geometry and identifying defects in the multi- track, multi-layer printing process using LPBF in AM.

Microstructure analysis plays a vital role in AM processes, providing valuable insights into material properties and print quality. Detecting melt pool boundaries and porosity defects within the microstructure samples presents a complex challenge due to inherent image processing and data analysis complexities. To tackle this issue, we propose a deep learning-based approach that leverages state-of-the-art deep learning models with convolutional neural network architectures. This enables automatic segmentation and detection of melt pools and porosity in AM microstructure images.

Recent advancements in Artificial Intelligence (AI), Deep Learning (DL), a subset of Machine Learning (ML) within the domain of Computer Vision (CV), has revolutionized the detection of melt pool and porosity defects in AM by using automating image segmentation tasks in the field of manufacturing. This AI-driven solution not only reduces the risk of human errors but also lowers costs and enhances time efficiency. Computer vision’s primary goal is to interpret visual information from images. Traditional digital image processing algorithms often struggle with various conditions, such as lighting, clutter, occlusion, and noise. Deep Learning emerged as a dominant force in the field of computer vision, notably enhancing performance in tasks like image classification [2], object detection [3], and segmentation [4]. Convolutional neural networks (CNNs) have proven to be efficient at learning and extracting features for processing 2D or 3D images. For various tasks, DL models are applied, including classifying microstructure images to identify and distinguish defects [5]. Detection tasks involve locating objects [6], while segmentation enables precise inspection of the AM process by recognizing target objects at the pixel level.

Detection and inspection of defects

Defects in AM can intricately impact the mechanical properties of printed parts. Various types of defects, including lack of fusion defects, cracking, and porosity, can arise. Addressing and minimizing defects is essential to ensure the integrity and quality of printed parts. Several machine learning algorithms have been used in optimizing defect detection processes in laser-based metal AM processes including Artificial Neural Networks, Support Vector Machines, K-Nearest Neighbors, Tree algorithms, and Deep Belief Networks, among others. Presently, these methods demonstrate proficiency in detecting simpler defects but exhibit reduced effectiveness when confronted with more complex scenarios involving multi-layer and multi-tracking printing samples. To address this challenge, the need for more advanced models becomes evident [7].

In response, we propose an automatic defect detection method utilizing cutting-edge deep learning techniques. Our approach leverages an encoder-decoder-based convolution neural network architecture. Our experiments have demonstrated that the integration of the Feature Pyramid Network (FPN), known for its effectiveness in extracting multi-scale information, with the Dense Net 201 backbone network architecture yields superior performance and accuracy.

The melt pool appears as a concave-shaped region when viewed from the side. These melt pools are formed during the material melting process initiated by the movement of a laser beam across the surface of the metal powder in each layer. Different materials exhibit diverse thermophysical properties, leading to significant variations in melt pool shapes and sizes. While traditional mechanistic models can assist with understanding the mechanism behind defect formation, the increasing complexity introduced by diverse materials, multi-layer configurations, and physical processes requires the development of individual models for each scenario. In contrast, AI-aided segmentation technology offers an end-to-end solution, providing a universal method for detecting melt pool boundaries and facilitating comprehensive analysis. This approach addresses the challenges posed by diverse materials and complex configurations.

In one of our projects focused on melt pool segmentation in multi-layer, multi-track laser powder bed processes, we have achieved outstanding performance using a deep learning network (U-Net) with Efficient-Net as the backbone, leveraging transfer learning on a limited dataset. Recognizing melt pools in images can be challenging due to their subtle visibility, often attributed to low contrast. To address this issue, we have applied additional data optimization techniques, such as data denoising and filter application, to enhance the visual clarity of the original microscopy images.

AI has played a pivotal role in automating AM processes. Microstructure analysis occupies a crucial position in ensuring the quality control of AM, given the variations in phase composition, solidification, and occurrence of defects. As we progress, the significance of control strategies in AM processes becomes even more pronounced. AI makes substantial contributions to the automation of AM across various domains, including material design, in-situ monitoring, predictive maintenance, simulation, and more. Prominent areas of focus include the optimization of process parameters, defect reduction, and quantitative analyses. From an AI perspective, challenges such as the scarcity of diverse datasets for robust model training and the need for model optimization emerge as critical considerations for future advancements.

Authors acknowledge the infrastructure and support of Center for Agile and Adaptive and Additive Manufacturing (CAAAM) funded through State of Texas Appropriation: 190405-105-805008-220. This work has also been supported in part by the U.S. NSF grants CNS-2231519, CNS-1852134, OAC-2017564, and DUE-2225229.

No conflict of interest.

1. Yanzhou Fu, Austin RJ Downey, Lang Yuan, Tianyu Zhang, Avery Pratt, et al. (2022) Machine learning algorithms for defect detection in metal laser-based additive manufacturing: A review. Journal of Manufacturing Processes 75: 693-710.
2. Alex Krizhevsky, Ilya Sutskever, Geoffrey E Hinton (2017) ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6): 84-90.
3. Christian Gobert, Edward W Reutzel, Jan Petrich, Abdalla R Nassar, Shashi Phoha, et al. (2018) Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high-resolution imaging. Additive Manufacturing 21: 517-528.
4. Evan Shelhamer, Jonathan Long, Trevor Darrell (2015) Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, Pp. 3431-3440.
5. Hao Wen, Chang Huang, Shengmin Guo (2021) The Application of Convolutional Neural Networks (CNNs) to Recognize Defects in 3D-Printed Parts. Materials (Basel) 14(10): 2575.
6. Yingjie Zhang, Geok Soon Hong, Dongsen Ye, Kunpeng Zhu, Jerry YH Fuh, et al. (2018) Extraction and evaluation of melt pool, plume and spatter information for powder-bed fusion AM process monitoring. Materials & Design 156: 458-469.
7. Zhaochen Gu, Shashank Sharma, Daniel A Riley, Sameehan S Joshi, Song Fu, et al. (2023) A universal predictor-based machine learning model for optimal process maps in laser powder bed fusion process. J Intell Manuf 34: 3341-3363.