A methodology of steel microstructure recognition using SEM images by machine learning based on textural analysis

• Published in: Materials today communications 2020-12, Vol.25, p.101514, Article 101514
• Main Authors: Tsutsui, Kazumasa, Terasaki, Hidenori, Uto, Kyohei, Maemura, Tatsuya, Hiramatsu, Shogo, Hayashi, Kotaro, Moriguchi, Koji, Morito, Shigekazu
• Format: Article
• Language: English
• Subjects: Bainite; Gray-level co-occurrence matrix; Image processing; Low-carbon steel; Machine learning; Martensite; SEM image
• ISSN: 2352-4928; 2352-4928
• DOI: 10.1016/j.mtcomm.2020.101514

•Textural-based recognitions of SEM images of low-carbon steels are demonstrated.•Combination with tree-based algorithms and majority voting successfully works.•Two types of SEM sources of FE and W are used and compared with their differences.•Homogeneity is an important feature to the present recognition for both sources.•Classifiers trained only using one of the sources cannot classify the other images. We attempt image-based recognition of scanning electron microscopy (SEM) of low-carbon steels using machine learning (ML) methodology utilizing feature extraction with textural analysis. Specimens are subjected to eight different heat treatments to generate microstructures of martensite, upper bainite, lower bainite, and these mixed structures are selected as candidates for the proposed recognition method. Additionally, two types of SEM sources, i.e. field emission (FE) and tungsten (W), are used, and the images individually apply to the classifications. To extract features, a textural analysis based on gray-level co-occurrence matrix (GLCM) is adopted based on the captured SEM images. For evaluating GLCM features, the original SEM images are cropped into 16 mini-images, and the following three recognition schemes are then applied: recognize mini-images individually, recognize original image categories by majority vote on predictions of mini-images, and recognize original image by using feature averages of the mini-images for training. We also adopt two decision tree-based machine learning models of random forest (RF) and gradient boosting machine (GBM). The voting schemes for both models accomplish satisfactory accuracies of about 85% for eight steel microstructures. Especially, GBM is found to exhibit more stable performance than that of RF, as differences in accuracies between types of SEM sources are lower. In addition, differences between the SEM sources are discussed by focusing on important features estimated by these ML models, and the features from FE-SEM tend to be distributed more broadly than those of W-SEM. Although a textural value called homogeneity is commonly selected as an important feature, regardless of sources and models, the other important GLCM features exhibit different trends for both SEM sources. Models trained by FE- and W-SEM tend to prefer indices of high sensitivity with regards to local brightness differences and smoothness, respectively. Due to these differences in GLCM features and their importance, models trained with ima