How Deep Learning and Generative AI Empower Advanced Welding Manufacturing Processes

Acentral challenge in automating and advancing complex welding manufacturing processes is that key quality metrics are often not directly measurable in real time. This article presents my perspective on why deep learning has become a game changer in addressing this challenge. However, whether the measured process signals contain adequate raw information to predict these hidden outputs is not easily known. To address this, the article introduces the use of generative artificial intelligence (AI) to test information adequacy, an idea illustrated through recent work at the University of Kentucky. This approach offers a new foundation for building smarter, more adaptive welding systems.

From Control Engineering to Welding Automation

Welding is an intricate physical process where high heat and rapid material transformation combine in a dynamic and challenging environment. As a control engineer by training, I approached welding initially through the lens of precision and regulation. My journey into welding began with developing a power source for robotic welding — a task that, at the time, had no off-the-shelf solutions.

The system I built controlled the short-circuiting gas metal arc welding process under carbon dioxide shielding. It was based on converting three-phase alternating current power into direct current using silicon-controlled rectifiers, where the firing angle controlled the output. A microcontroller read the arc voltage and adjusted this to maintain a stable process, producing a smooth, consistent “sha” sound with an auditory signature of fine, repeatable droplet transfer with minimal spatter.

This early work illustrates what I wish to call direct monitoring and control: The system directly measured an output (voltage), compared it to a target, and adjusted the control (firing angle) accordingly.

When Direct Monitoring Falls Short

However, not all critical outputs in welding are directly measurable in real time. A typical example is the state of weld joint penetration. It is essential for weld integrity and strength but occurs beneath the workpiece surface, making it invisible during welding.

Engineers often rely on indirect indicators, such as the visible weld pool, to infer joint penetration. This transforms the challenge into an indirect monitoring problem: Using observable phenomena to estimate a hidden but critical outcome.

In indirect monitoring, the signals we can measure are not what we care most about. Instead, we look for patterns or correlations to infer the unobservable (Ref. 1). Success depends on whether our chosen signals carry sufficient information about the hidden outcome. That determination requires both domain knowledge and careful empirical testing.

The Iterative Path to Indirect Monitoring

Because the relationship between observed signals and the hidden output is usually complex, we can’t just write down a simple formula. Instead, we need to learn this relationship through data. Traditionally, this process involves extracting features from sensor readings, then building a model that maps those features to the desired output.

Finding the right features and model often takes repeated trial and error. Each trial involves the following:
 

1. Selecting a new way to extract features from the sensor data.
2. Proposing a model structure (e.g., linear, polynomial, neural network).
3. Optimizing the model’s parameters to match the observed outputs best.

These three steps — feature selection, model choice, and parameter tuning — form nested loops of iteration. Inner loops adjust the model parameters. Middle loops refine the model type. Outer loops re-evaluate the feature selection. The process is time-consuming and traditionally driven by human reasoning, intuition, and trial.

The central problem is this: You can’t tell how good a feature set is unless you’ve found the best possible model to use with it. And you can’t judge a model structure without testing it across the best possible parameter sets. That interdependence is why indirect monitoring is difficult and labor intensive.

Automating the Search with Deep Learning

Deep learning offers a breakthrough by replacing human-led exploration with a data-driven search over vast spaces of possible models and feature extractors. This is made possible by the structure of these neural networks:

• A convolutional neural network can automatically extract relevant features from raw sensor data, such as images.
• A fully connected network can then map those features to the target output.

Combining these into a single pipeline allows the system to be trained end-to-end: Raw input goes in, the desired output comes out, and the system automatically adjusts its internal parameters to improve accuracy.

The critical insight is that deep learning models can represent various traditional models within a unified structure. Rather than manually trying dozens of feature sets and model types, we can automatically train a deep model to find the best combination.

This doesn’t eliminate all challenges but vastly increases the speed and breadth of exploration. Also, with sufficient data and computing power, it can match or exceed what humans can do through manual iteration.

The Hidden Pitfall: Irrelevant Information

While deep learning is powerful, it also has a weakness: It will exploit any pattern in the data to reduce prediction error, even spurious or irrelevant ones.

Let’s say we’re trying to predict a weld quality metric, such as the back-side weld bead width, that only varies between 3 and 6 mm. Because the range is so narrow, many irrelevant inputs might seem to help reduce prediction error during training. But this improvement doesn’t generalize; when conditions change, those irrelevant cues no longer work, and the model fails.

This is why the adequacy of the raw input data (Ref. 2) is so important. The data must fundamentally contain the information needed to determine the desired output. Otherwise, no model — no matter how advanced — can make accurate predictions across different conditions.

Unfortunately, neither human experts nor deep learning alone can guarantee that the chosen data is adequate. We need better tools to test this.

Generative AI for Testing Information Adequacy

One promising approach is to reverse the modeling task. Instead of predicting a simple output from complex inputs, we try to predict complex sensor data using the simpler outputs as inputs.

For instance, we can attempt to reconstruct a high-resolution weld pool image using only the back-side weld width as input. If successful, this indicates a deep structural connection between the two. If not, it suggests the simple output doesn’t fully capture the complexity of the input, and the irrelevant information in the complex input may have helped reduce the training error.

Generative adversarial networks (GANs) are ideal for this task. GANs learn to generate realistic data samples by pitting two networks against each other: One that tries to create fake samples, and one that tries to detect them. This setup encourages the system to learn true patterns in the data rather than shortcuts.

We’ve applied this approach to weld automation at the University of Kentucky. We built a conditional GAN that uses a current weld pool image to predict the appearance of the back side of the workpiece, an indirect but richer view of joint penetration quality (Refs. 3, 4).

Even more interestingly, we extended this to use sequences of weld pool images (Refs. 3, 4). Our hypothesis was that, like human welders, models need to observe how the weld pool evolves to understand what’s happening beneath the surface. Indeed, including the image sequence dramatically improved prediction accuracy. Hence, a single weld pool image may lack the adequacy for the needed raw information to determine the weld joint penetration, but dynamic sequential weld pool images improve the adequacy.

Developing Collaborative and Predictive Welding

Deep learning also enables new modes of human-robot collaboration in welding. In one project, we developed a remote-controlled welding system where a human operator observes the weld pool and adjusts parameters in real time.

Based on the current settings, we explored whether the system could predict how the weld pool would evolve over the next few frames. To answer this, we trained another conditional GAN to generate future weld pool images from the current welding parameters (Ref. 5).

This predictive capability gives operators a more powerful tool — not just a live view of the weld, but a preview of what will happen if they maintain their current actions.

Conclusion: Deep Learning as an Enabler of Smart Welding

Welding process automation has long been limited by the inability to measure or control key quality indicators in real time. Though necessary, indirect monitoring was traditionally too slow and unreliable for widespread use.

Deep learning and generative AI have changed the game. They allow us to turn raw, high-dimensional data into actionable insights even when the underlying relationships are complex or hidden.

By automating the discovery of meaningful patterns, these technologies help close the loop between observation, inference, and control. They enable smarter, more adaptive welding systems that approach or in some cases exceed human capabilities.

The future of welding isn’t just mechanized. It’s intelligent. And that future is arriving faster than we think.

References

Zhang, Y. M., Wang, Q. Y., and Liu, Y. K. 2021. Adaptive intelligent welding manufacturing. Welding Journal 100(2): 63-s to 83-s.

Yu, R., Cao, Y., Chen, H., Ye, Q., and Zhang, Y. 2023. Deep learning based real-time and in-situ monitoring of weld penetration: Where we are and what are needed revolutionary solutions? Journal of Manufacturing Processes 93: 15–46.

Mucllari, E., Yu, R., Cao, Y., Ye, Q., and Zhang, Y. 2023. Do we need a new foundation to use deep learning to monitor weld penetration? IEEE Robotics and Automation Letters 8(6): 3669–3676.

Mucllari, E., Cao, Y., Ye, Q., and Zhang, Y. 2024. Modeling imaged welding process dynamic behaviors using generative adversarial network (GAN) for a new foundation to monitor weld penetration using deep learning. Journal of Manufacturing Processes 124: 187–195.

Li, T., Cao, Y., Ye, Q., and Zhang, Y. 2025. Generative adversarial networks (GAN) model for dynamically adjusted weld pool image toward human-based model predictive control (MPC). Journal of Manufacturing Processes 141: 210–221.

YUMING ZHANG (yuming.zhang@uky.edu) is with the Department of Electrical and Computer Engineering & Institute for Sustainable Manufacturing, University of Kentucky, Lexington, Ky.