Weather Guard Lightning Tech

An Expert’s Insight on Root Cause Analysis

This week, Allen and Joel talk to Jonathan Zalar of IWTG Consulting about the complicated RCA process. With 20+ years of experience, Zalar details OEM investigations like analyzing turbine data, assessing damage on-site, and convening engineering teams to determine causes. By understanding the inner workings of the OEM process, operators can get their turbines back up and running faster with less of a struggle.

Website: https://www.iwtgconsulting.com/

LinkedIn: https://www.linkedin.com/in/jonzalar/

Sign up now for Uptime Tech News, our weekly email update on all things wind technology. This episode is sponsored by Weather Guard Lightning Tech. Learn more about Weather Guard’s StrikeTape Wind Turbine LPS retrofit. Follow the show on Facebook, YouTube, Twitter, LinkedIn and visit Weather Guard on the web. And subscribe to Rosemary Barnes’ YouTube channel here. Have a question we can answer on the show? Email us!

Pardalote Consulting – https://www.pardaloteconsulting.com
Weather Guard Lightning Tech – www.weatherguardwind.com
Intelstor – https://www.intelstor.com

Allen Hall: Welcome to the special edition of the Uptime Wind Energy Podcast. I’m your host, Allen Hall, along with my co host, Joel Saxum. Our guest is Jonathan Zalar managing partner of IWTG Consulting, and IWTG is based in South Carolina. In the United States, Jonathan has a long career in the wind industry, working for 22 years with GE 13 years with GE Vernova specifically, Jonathan has a wealth of knowledge from both his work in the field and in the engineering offices.

He’s a mechanical engineering major and also holds an MBA. So I put you in a very select class, Jonathan, which we’re going to tap on here. Jonathan, welcome to the program.

Jonathan Zalar: Thanks for having me. I appreciate you guys taking the time.

Allen Hall: There’s not a lot of engineers that go after their MBA and then stick to engineering.

They tend to go to MBA and they go into the business world and have a nice comfy office and you took the other route.

Jonathan Zalar: Yeah, I went right into my MBA after my undergrad. Guess I want to stay in college a little longer.

Joel Saxum: Yeah, that doesn’t make you a bad person, okay? I wish I was still there.

Allen Hall: Jonathan, you have a really a wealth of knowledge here on what happens in the field because you’re out there doing it and interacting with the engineering groups that were doing the design work and support work at their offices.

And one of the issues that Joel and I get wrapped into a lot is RCA’s. And people ask us about this all the time. And we were just at an insurance symposium a week or two ago, Joel and I were, and everybody has a different perspective of what actually happens and what an OEM does behind the scenes, because there’s a lot of things that happen behind the curtain that unless you really are on the inside, you just don’t know.

But there’s a lot of good positive things that an OEM is doing during an RCA. So I just like to walk through what happens during an RCA. If you had a blade issue out in the field and you call the OEM, what typically, what typical things happen there? And maybe you can just walk us through what that process is.

Jonathan Zalar: When something like that happens, it’s like a major event and GE and other OEMs have protocols in place, first of all, to ensure safety, right? Is everybody okay? And then, then it’s like, all right, now it’s time to put your CSI hat on and go investigate.

Joel Saxum: Horatio Zalar, is that what it is?

Jonathan Zalar: While this is all happening, while you’re working with the customer, it’d be like, hey, can we come here? We’re going to send, these experts out there to go look at whatever it is, a blade, for example. The teams are also looking at the data because when a, event happens, there’s data collected on the turbine.

There’s engineers looking at that data, trying to understand, what happened from a data perspective while you’re mobilizing people to go there and, being able to access the turbine can take weeks. Sometimes months, depending on time of year, and how the way is oriented. It does feel like it takes a long time, just to get started

sometimes.

Joel Saxum: I know just the pictures that you see online when there’s a failure, and this could, this is blades, lightning strikes, nacelle fires, whatever it may be, but a lot of times we just looked at one the other day, Allen, where it was like a it looked like spaghetti, right? The blades were ripped up and the tower was bent over itself.

And to look at that and say Oh, we’d like to go do an investigation on that. Like you said, securing the site and being safe first. Is one thing because you can’t expect to bring anybody in there until that is because it’s if a wind gust came one way, the whole thing could come down or something of that sort.

Jonathan Zalar: There is a procedures in place to go analyze the way the turbines currently situated what the wind is to go make sure something like that doesn’t happen. Yeah, safety is like the number one thing for sure.

Joel Saxum: Yeah, but and then the I guess on the outside of that is being say this is a. It doesn’t matter, OEMX Turbine, the people from OEMX will have access to data behind the scenes, usually, right?

If it’s a pretty new turbine, most of the time they’re connected somehow to the SCADA system or the controllers, so they can start their investigation even before something happens in the field, is what you’re saying.

Jonathan Zalar: They’re doing, trying to do two things. One, look at the data to understand if they have a good idea, if it’s already an issue that they’re working on.

And if not, then they’re collecting as much data as they can to, when the investigation actually starts.

Allen Hall: So is part of the issue that I’ve seen on some of these investigations is the the investigation starts too late, right? By the time they get everybody spooled up, particularly on blades, a lot of times if a blade’s been hanging, they’re damaged and the turbine’s shut down, you lose a lot of that fine detail that you have.

I think it’s, it does seem to be important that an operator contacts the OEM quickly to get something started, get the process started.

Jonathan Zalar: Especially with a blade that banana peel, for instance, like if it’s hanging out there for a week, it’s just rubbing and it’s rubbing away evidence. Basically with drones that have come into play now that helps for sure.

Because you can send a drone out there and it falls on the drone. Not a big deal. So you can do something there pretty quick, but to go get a sample, send it to a lab. That takes time, and you can lose some of the critical evidence.

Allen Hall: Yeah, Joel and I see that in lightning all the time. It rains, so it washes away evidence.

Joel Saxum: It’s like fire investigations, too, right? Fire, like I’ve talked to some fire experts before, and they’re like, The biggest thing is like, nobody touches, it has to be an unmolested because otherwise the evidence, like if someone goes in there and starts to like move, just even opens a door the wrong way, they can remove evidence that needs to be seen for that investigation to have basically to have efficacy, right?

To make sure that they have all the data and everything can be learned from it. Let’s go let’s walk through this step by step. We’ve got, we said we had something come down, whether it’s a turbine tower, gearbox failure, blade, whatever it is, or asset owner calls. The OEM, I’m sure at the same time they’re probably calling their internal risk group and getting a hold of their insurance or whatever.

But they, but what we want to concentrate on here is the OEM process. So they call their OEM, OEM makes sure, in conjunction with the asset owner, site is safe, site is secure. We’ve got, we were, we’re good to go. The OEM then starts looking, if they have access to the data, which they usually do, starts looking at data in the background to figure out their things.

Then what’s the next step?

Jonathan Zalar: The OEM will send out some of their like experts. GE had really good people that would go out, especially on blades, and they would know what to look for. They would use the drone photos too to maybe tell them where to put the blade, what orientation when they bring it down, and they’ll go out there tons of pictures, and then depending on the issue or not, they’ll probably take some samples, cut up a couple pieces, put it in a trailer, ship it to a lab.

Basically collecting as much evidence as they can as quickly as they can before, winter rain and everything else washes some of that away.

Allen Hall: The on site investigation that happens from the OEM’s perspective is, those people are experts in the technology that they’re looking at, generically, but they’re not the people that design the equipment, they’re not the people that design the blades or the gearbox or Typically, or maybe they did come out of there.

Jonathan Zalar: They may came out of that team, but they are, there are definitely more field oriented people for these type of issues, but sometimes they will bring out a designer if they need to it all depends on the case.

Allen Hall: This sounds a lot like the airplane business that I’ve been in for a long time.

When we, there’s an accident investigation, the aircraft manufacturer sends out experts. Now, those experts are knowledgeable people about the airplane product, but they’re not specific. Like they didn’t design the elevator, but they didn’t design the propeller, right? But they have a pretty good understanding of what the systems are.

So when those people are out on site photos, images, samples. Now the sample piece. What is, what’s happening with those sample pieces that they may collect? Where are those headed off to?

Jonathan Zalar: In, in GE’s case, they had to think of one or two labs, so it’ll be set to one of those, and then they’ll do microscopy on it, so cut it down to little pieces, get a high end microscope.

I’m not an expert on this, I was more of a person who looked at the results. And then as someone who’s like leading it from a engineering perspective, they’ll spend a lot of time with the person who cut the sample and whatever conclusions they have for adhesion, for example, or number of layers, stuff like that.

So they’re getting really detailed about whatever samples they collect. You need somebody very knowledgeable deciding which samples to take, and then somebody extremely knowledgeable on, how to go look at that sample. So

Joel Saxum: being in an OEM, too, you have, they’re the largest people, right?

So you’ve got access to all kinds of different engineers. So if you’re sitting there and you go, alright, here’s the specialty RCA team. We’ve got the investigation going. We’ve got some samples taken. We’re at this stage where we’re trying to figure out what’s really going on. Alright, we’ve got microscopy done.

Great. Let’s get a hold of that may be the glue expert or it may be the fiberglass expert or the carbon fiber expert and you have access to all those or you would have had access to all of those people, right? So you have, you can send an email to someone down the hallway or walk down there and say Hey, let’s look at this.

Let’s look at this. And that’s one of the advantages that the OEM has. Is it not only do you have access to data, but you have access to experts within each of those kind of … We’re going to say, maybe call them sub silos, right? Because it’s not just you’re a composites person. You’re actually that really toned down one in there.

And so you would go then to those people to get their insights on what may have happened.

Jonathan Zalar: Yeah, and there might be someone in the middle who’s more of a Blade expert of everything, right? Knows all of it, and they might say, Oh, let’s go talk to This person, because they know a little bit more about this, or they designed this part of the blade, and y’all, you go huddle up over there and guys, try to walk through what’s going on.

Joel Saxum: So does that, in an OEM case, and I know this is a very generalized statement, but does those RCA investigations, when you’re dealing with or trying to engage those individuals, do those take precedent? Or is that kind of yeah, we’ll get to that when we get to it. We’re working on a new blade design.

Jonathan Zalar: It’s like number two under safety.

Allen Hall: Wow. Way up the chain.

Joel Saxum: Okay. So now we’re at the, now we’re at the stage where we’re in, we’ve got data, we’ve got some samples, possibly we’re back in the office and we’re engaging the experts. What does that look like?

Allen Hall: Yeah. Who’s at the table there?

Jonathan Zalar: Depends on the issue, but yeah, you’re getting the people that

probably have the most knowledge in the subjects, right? The people who’ve been around long as to, or maybe even designed the blade or designed a similar blade. They’re in the room, you’re looking at data, trying to understand, hey, is this a one off or is this something we need to worry about? That’s a very key question.

And that’s kind of part of the second or third part, which is like containment. Like, how do you stop the bleeding? Is this a fleet issue or not? That’s one of the biggest questions you’re trying to answer as quick as possible.

Joel Saxum: Yeah. That’s where you get into that, that not pseudo gray area of engineering and business, right?

Because at some level you have to worry about. What could financially be impacting the rest of the larger OEM scale as well?

Jonathan Zalar: Yeah. I felt like at least at GE like that was isolated from the business part and it’s more, Hey, technical people, you go figure this out, work with the customer resolution people and y’all built They’ll help with the customer information part.

We were we were let go to go figure out what was going on.

Allen Hall: So does some part of that involve looking at all the SCADA data from the turbine? Because I would think some of the issues that are happening on blades in particular may be related to the operational aspects of the turbine.

There’s some tweaking going on.

Jonathan Zalar: There was like two sets of data. There’s like a high speed data that you can look at the time of the event. You can actually almost tell which blade did what, when, do a rotation. And I, from there, there are probably some 10 minute data points that could indicate something if you’re looking at it for a fleet issue, and then you can start cutting the deck with 10 minute data if you need to, all depends on the issue.

Joel Saxum: Yeah, off air we were talking about one thing that was cool being in an OEM is that if, of course, if you’re connected to the controllers, you guys have, you have the possibility of going hey, if this is a.

I don’t know, GE 1 5 problem. Give me all of the data for the GE 1 5s that looks like this and you had access to that data of possibly saying, give me 5, 000 turbines of data so we can start looking at something statistically.

Jonathan Zalar: Correct. Yes. And I had some very good data people on the team, very knowledgeable in turbines too, that could go take a look at 10, 000 turbines in two hours, come back with an answer.

Joel Saxum: The rest of the industry just doesn’t have that, right? That’s the thing we’re always talking about, like sharing data. I wish we had more of this. Wish we had more of that. But the OEMs are the ones that actually hold the, they hold the cards there.

Jonathan Zalar: Yeah. Especially if you’re yes, they do.

Allen Hall: But they have also the engineers that designed it, which is the key to this.

So you can have all the data in the world or you want it, but unless you have the knowledge behind what that data means, it’s pretty much pointless. It takes a long time. Oh, Jonathan can attest to this. It takes a long time to become an expert in a particular aspect of a wind turbine. That’s what Rosemary is about in our program.

Like she’s been around a long time and she’s very knowledgeable about those things that she knows about. But unless you have those people on your staff that just live and breathe that, I don’t know if they make heads or tails of what’s going on data wise. And I think that’s why the OEM, getting OEM involved is really critical here.

Joel Saxum: Jonathan, I want to ask you one more question about, this is a, it would be more of an internal thing. So at what stage, or how are they treated different if you go If you get to a split in sitting in the conference room with everybody, all the technical prowess in there, and you say, all right, guys, this looks like a one off issue, or this looks like a serial defect.

How do you treat those differently?

Jonathan Zalar: First you need to verify that. You get to the point where you’re like, hey, I think this is a one off, and how do you know? You get chief engineers involved, and a lot of people have signed off to say, hey, freak event, one off. Whatever it was, if it’s something that you’re concerned about is more widespread, then that’s when you probably bring in more people.

You’re looking at the data, potentially instrumenting turbines going to go to field inspection samples. Then that process starts taking a longer time, for sure.

Joel Saxum: I know this is a weird thing to bring up here, but I was just reading the news. This is literally yesterday, I saw in the news, SGRE laying off all of the engineers that designed the 4X and 5X.

So now in SGRE, if they’re going to continue this investigation into what’s actually happened here. All of those people are gone.

Jonathan Zalar: Yeah, I’m not sure did they lay off leadership or did they lay off design engineers.

Allen Hall: It said engineers.

Jonathan Zalar: A little too soon in my opinion, unless they know what’s wrong.

Joel Saxum: That’s that’s my same thought as well.

Allen Hall: Let’s go back, I want to go step back to the chief engineer aspect of this, because I think maybe GE’s a little bit unique in that they still have chief engineers. A lot of the engineering industry has pulled back from that over time.

But when I worked for GE years ago, chief engineers were a real key to making the operations work. Within GE, and I, this doesn’t have to be specific to GE, but I just curious here. They still have chief engineers that are responsible for a particular product line that really know the ins and outs, and that’s a very unique person to be able to do that, but there is a focal point on the engineering side, right?

Jonathan Zalar: For you mean like a 1. 5 SLE, or do you mean like a blade?

Allen Hall: A turbine model, right? Is it turbine model or is it blade model? Maybe it’s by blade.

Jonathan Zalar: There are there were, I’m not sure now, but there were chief engineers for the major component areas, but there’s also a overall like system engineering chief and like the system engineers are more responsible for the whole product.

Allen Hall: Okay. So they even have chief engineers lower down into the main components then, which is, that’s the way I would do it. Yeah. Okay. That’s the way I would do it. So that’s a, is that your really first touch point? Hey, Chief Engineer, this happened, I just, heads up, this is coming. Is that sort of your focal point if you’re out in the field and doing the RCA work?

Is that your key contact and then the Chief Engineer is grabbing the people to bring into the conference room? Is that how it flows?

Jonathan Zalar: No, it’s, it was more of the systems team was leading the effort, at least for these major issues. And then they would be reviewing that with the Chief Engineer.

And you definitely bring the person in there so they know what’s going on, especially with a big issue, like they’re going to be involved. But they’re not leading, they’re not leading the investigation, but they are approving at the end. They’re asking questions, asking to go back and go look at something else because you want to be, you want to be right.

Allen Hall: You want another set of eyes on whatever the quote unquote answer is, right? And someone who’s knowledgeable about the product on a deep level to go, yes, that makes sense to me. Okay. That’s. A good approach.

Jonathan Zalar: And the big issues, there’s more than two sets of eyes looking at it, for sure.

Allen Hall: So you’re all at the table, you fleshed out, it’s either a sort of a system wide issue or a one off.

You make that decision, and then from there, what’s the support role look like? What are you going back to the operator with? How much information is brought to them? What do they need to know to get to the next step of repairing, replacing, whatever the, what are the answer is there.

Jonathan Zalar: Like part of that safety review initially about like, how to go approach that kind of helps with the, how do you get the turbine back up and running?

What do you need to do? And there’s teams probably separate from the RCA that will like, help with the foundation analysis. Can you go put another turbine up on there? Stuff like that. The RCA is going to, the RCA team is going to stay focused on. Hey, what happened? What’s the root cause?

How do we correct this and then prevent it?

Joel Saxum: That’s more of the client success people then, right? Like the client interaction, they’ll then take load of what happens next. So if they have to deal with the asset owner, their insurance company, their consultants, or something of that sort, that’s a different team.

Jonathan Zalar: Yes, and I’m happy I was a different team because that would just slow it down even more.

Joel Saxum: So that someone there in the customer success reign takes the kind of the control of the, all of the externals per se, and then navigates that mess.

Jonathan Zalar: Multiple times, like I would be presenting to customers on like where we were, like RCA updates throughout the process.

There were definitely touch points and stuff like that, but the hey, how does it get this turbine back out there running? That wasn’t really.

Allen Hall: Okay. That’s a good, that’s a good way to run it. I’ve seen just from mostly outside the United States where it does seem to be a linear process, but nothing happens until the turbine to the RCA is done.

And then there’s a team that then figures out what the next step is. That doesn’t make any sense to me. In the United States, what you explained makes sense to me. Hey, let’s get the operations up and running again. Let’s get that done. We’ll figure out the problem on the side. It’s got a parallel effort going on instead of a linear effort.

That makes complete sense to me. So then, in that parallel effort, then, is there a lot of crosstalk between the two teams or is it? Engineering RCA. Hey, we had an engineering issue where we really need to hone in on this. The customer side is getting the customer back up and running again. So they’re productive.

But in the meantime, you’re paralleling an engineering approach. Okay.

Jonathan Zalar: Yeah, I mean, the customer, most of the customers, the bigger ones I dealt with, they have, they also have two teams. One’s I need to make power. The other one’s I need to worry about my other turbines like so there’s multiple teams pushing on multiple teams.

Joel Saxum: These bigger operators to you know in the United States in the United States if you’re dealing with those, your NextEra your EDFs, your RWEs they’ve got their own engineering teams as well.

So more than likely, they’re running a parallel process of you guys. Maybe grabbing some data from you, if if it’s available or something like that, but they’re using their own data and they’re running their own analysis to come to their own conclusions at the same time, because they’ve got to, they’ve got to safeguard themselves, right?

That’s part of doing business.

Jonathan Zalar: Yeah. And there’s, having a good relationship is also key too, because there were many times where I had to call up one of the really big customers and be like, Hey, can we go instrument a turban or can we go collect some samples? Can we go do this? And that relationship is really important, but also that speeds up the process especially if it’s not a one off.

Allen Hall: Okay, that makes sense then. Yeah, because if it’s engineering collaboration, things tend to go faster. Because it’s an understanding we’re all trying to solve a problem. Let engineers alone solve a problem, they’ll pretty much go off and do it relatively quickly. People, a bunch of insurance people or management people on top of that, it tends to slow it down.

Jonathan Zalar: Every time.

Allen Hall: Yeah, and that’s a good point. If you’re an operator, that’s one of the things to remember is look, you’re paying these people for a reason, let them go do their job, just make sure it’s moving. And that’s it right there. You want to get to the fastest answer that way, instead of trying to interject a bunch of politics into it.

Joel Saxum: If you’re an operator and you run into one of these issues and you’re not sure where to turn, call IWTG Consulting. They’ve got the expertise, Jonathan?

Jonathan Zalar: It’s a fun thing to do. You really are an investigator, right? You get to go solve a problem. It’s one of the funnier jobs I’ve had.

I really enjoy doing it.

Joel Saxum: You’re doing puzzles, but at a grand scale.

Jonathan Zalar: If you get happy when something breaks at home, I’m that type of person. I’m like, yes, something broke.

Allen Hall: So what does the end product look like after all this? Is it just a report? Is it a meeting? Is it a report and a series of meetings?

What happens at the end? When you say, we know what this is it’s this one off turbine issue. Here’s what happened. Here’s how we do to prevent it going forward. What is that? What does that closeout look like?

Jonathan Zalar: The closeout with the customer, it, it’s going to be a meeting, it’s going to be, a presentation of some sort, and sometimes it’s a report depending on who they are, but, and it’s going to be like, does it, more does everyone agree this is what it was, and nine times out of ten, by the time you’ve through everything, everyone agrees.

Allen Hall: You hear from the field that there’s complaints that the OEM and the operator just don’t agree. Really? I think politically they may not agree, but engineering wise they probably do agree. Usually the financial part they never agree on. Engineering wise though, it does seem like there’s an agreement.

Typically, right?

Jonathan Zalar: Not always day one, but usually by, midway through to the end mostly both teams have a good idea where it’s heading, right? There’s gonna be some one offs that it’s just no one knows. Like that, that will happen, but that’d be really rare. But I don’t remember many that says I know it’s this.

And someone else says, I know it’s that usually there’s enough smart people that have looked at enough data that they can convince one or the other that they’re more likely.

Joel Saxum: Yeah. And like you said, Allen, that’s on the engineering side, right? What ends up muddying the water there is when you get an insurance company and then they grab a lawyer and then there’s arbitration and there’s all this, and people are sitting there but who’s on the hook for the four million bucks and that kind of stuff, right? That’s where it gets lost sometimes.

Allen Hall: That’s good though, that the OEMs are trying to keep the engineers separate from that discussion, because you, at the end of the day It’s all about operating and producing power if you get tangled up in all the money part of it up front, it’ll never come to a solution and I’m glad that at least, GE side that they’re working the engineering solution, which makes sense that GE is an engineering company and that all makes sense to me. And this is where IWTG comes into the play, right?

That if you’re an OEM You have your team. If you’re an operator, you don’t have everybody you probably need, and especially some of those mid tier operators. Even the large operators don’t have all the experience with a particular turbine type because there’s just things you just don’t know, right?

You don’t dig deep into a system architecture as an owner of a turbine. You know how to operate it, but if there’s things, complex things happening, you may not have those details. This is where IWTG comes in, you call Jonathan up and say, Hey, how do we go debug this? What’s going on? And what, how do we, how do we flow through this to get to the right engineering answer?

Jonathan Zalar: Yeah. I think it helps just also like explaining to some of the, smaller customers of what the process is help them, support the OEM with whatever process are going through. It’s been pretty helpful with a couple of customers I’ve worked with.

Joel Saxum: Yeah, I would say one of the, one of the biggest hurdles there is navigating the whole thing, but it’s understanding who to talk to and what data that needs to happen and all these different things because that, that’s a big problem in the industry too, is like people you throw it at a site manager. That site manager is so dang busy with just the everyday stuff they have going on, they just go man, hey, I got this I’ll deal with that later, right? So if they if if an asset owner wants to actually get moving on this thing, having someone that knows what they’re doing is a good help.

Allen Hall: It’s a necessity today.

Yeah, there’s definitely a place for those experts. And in the United States, there’s not a lot of people walking around that have that sort of expertise. You have to spend your time in the trenches. And Jonathan has done that. So this is why it’s so good to talk to him because he’s been there and he’s lived through it.

Jonathan, how do people get a hold of IWTG and how do they get a hold of you directly?

Jonathan Zalar: My website’s www. iwtgconsulting. com. They can reach out to me there. I’m also on LinkedIn.

Allen Hall: And Jonathan, I really appreciate you coming on the program and we want to have you back because there are a wealth of knowledge and it’s good to get that knowledge out into the industry.

Jonathan Zalar: Thanks for having me. I really appreciate it.

An Expert’s Insight on Root Cause Analysis