NextGen Possibilities

30 Years of Chasing Break-downs: Why We Approach System Failures In Futility

> "If everyone is thinking alike, then someone isn't thinking" - George S. Patton

There is an interesting feature in the way we approach problems. It is very urgency driven. As in, unless we feel urgency little action begins. More about this in a bit....

Over the past two decades I have been in hundreds of plants, seen all the various ways maintenance and upkeep is approached. The way I would summarize what I've seen is not a positive thing.

I would note the following characteristics:

• Little imagination
• Little craftsmanship
• Degradation from OEM delivered conditions
• OEM delivered conditions pre-degraded
• Near zero vision and subsequent strategy and planning
• Uninformed and personnel with limited ability

With putting this out I am not deriding these folks. Many work their tails off to keep things running; management, engineers and technicians.

What Has Driven How We Do Things?

Any effort cannot fully develop without a vision, leader, plan and good communication.

These days and for the several before, rapid change has been a prominent theme. People change out often, including management. Operations that used to have engineers then don't. Managers used to have sufficient budgets then don't. Equipment that used to be engineered for maintenance and longevity aren't now.

This is the result of forces in our economy. It may be a sign of the health in how we do things in a capital driven economy.

Where Does An Answer Exist?

While these things are not good and can reduce the quality of automated systems rapidly as they age or are modified, an opportunity exists in it.

We have to learn how to do better with less money, resources and formal roles. In other words, who does what needs to change. Those left with the reigns need to be tooled up. Of those, them who want to rise up need to step through significant resistance and believe in themselves.

Urgency needs to be redefined in terms of what controls when we feel it. If we felt a sense of urgency when a 1 year plan is going off track, it would reduce the frequency of things we need to react to that create truly urgent things that stop production.

The model of a backyard mechanic is what we use to support multi-million dollar plant automation systems. Not that this is bad, many of these people are brilliant. Not boardroom brilliant, make-use-of-anything-you-can-find-and-make-it-work brilliant. I have learned a great deal from people with this kind of genius.

Everything Where You Can Find It

When I first started working inside a multi-national, truly world class manufacturer, I discovered a very interesting thing.

All the PLC's, HMI's and most of the I/O was interconnected on ethernet, controller and device level networks. All ethernet nodes were accessible on a multiple access point WIFI network. All controllers were accessible from a laptop in 90% of the locations needed to observe the process.

This kind of design doesn't happen without intention. It has to be concepted, built and paid for. Imagine this kind of forward thinking in how maintenance planning and execution is done. In how information was controlled and developed.....How operators are trained.

Why Do We Underpower Our Approaches?

Because sometimes, thinking is hard.....

"Thinking is the hardest work there is, which is probably the reason why so few engage in it." - Henry Ford

What do you do when you get some new object? Whether it is a USB drive, business card or manual. If there is a drawer, it may go in there. If there is a flat surface with room on it, it may go there. On a shelf?

There is a huge problem in this. First of all, the reason we do it is because it takes time and energy to figure out the best place to put it. It may be simple or complex but still it takes time and energy. Over time, this puts things everywhere making them difficult to find.

All that was needed was a system for it to go into, and more importantly, some thinking on how to break it apart and put it in the right place in this system.

Despite how it can feel, there are limited types of things that come in. Putting thought into where each type of thing could come from allows us to create a place for it ahead of time.

Business cards - put the info into a contact management system.....have a contact management system

Manuals - have a place on a shelf for them....organized in a logical way

USB drives - for the love of everything sensical, put those files into a file management system.....

New Possibilities

Imagine the possibilities. Build a vision in your mind of a factory that runs and you have a high degree of confidence that it will at speeds and with reliability you dream of. That it won't wake you up at night. It won't interrupt your vacation, your kids' birthday party or your anniversary dinner.

It's quite common that this is not true when you are responsible for automated assets. They break, and sometimes, most of the time, not at good times. This is because we naturally fill our day. So, when an emergency event occurs, it's never a good time.

What if there was a way to care for these assets in such a way that they become very durable? In other words, they like to run.

What if your systems were reliable and robust? What if they had a bunch of uptime and hardly any downtime? And when down, it was because you brought them down.

All of this is possible. Depending on the existing state of your system, you may have a long road. But working a smart plan with a good team, you can certainly get there.

Reshore Possibilities  

There is a lot of talk about reshoring. What does that mean?

Over the past two decades manufacturing has been moving out of the U.S. The cost to manufacture in the U.S. vs. Elsewhere made the move necessary in the eyes of many corporations.

Now the tide is turning, as it almost always does. Depending on political conditions, this pace could drastically pick up.

If it does, there will be a boom like we haven't seen in forty years. And it will be awesome! But we need to be ready.

Because we know how to do this, the ramp up would likely be very rapid. Let's be ready. For this, but also to better manage what we have now.

Lifecycle Protocol (LCP)

LCP  [=]  Legacy Factory Converted To Modernized  [+]  New Factory State Maintained

In the past and even until now, it has been common to only address what is currently causing, or soon to cause, us pain or failure. This allows the systems to deteriorate all the while being undetected. Then when this deterioration does start to show up, the volume of issues become overwhelming. The Lifecycle Protocol (LCP) approach seeks to avoid this.

LCP looks at everything in the infrastructure, including the processes and people involved. It considers what things need to look like 10 years down the road. It uses the resources you currently have. It blends them with a strategy, that if followed, will apply the right resources....at the right time, in the right amounts....in the best way possible with what is available to you. The activity involved with implementing this will also create a profile of what isn't being done. This will help make the case for various aspects of capital funds planning.

100% Readiness

It is possible to always be ready for anything that could come up:

• VFD won't start....or maybe it starts but the speed reference is not followed
• An HMI button won't do the commanded function
• Flagging a PE doesn't have the expected result
• A servo drive won't do anything
• One conveyor section is acting different that it used to

Things will fail, go wrong or act in ways that are confusing. To wait for that to happen like it's a strange thing, is a strange thing.

We can work like dogs ahead of time so that weird things don't happen as often. When they do we instantly know how to find out why and how to resolve it.

No more multi-hour or multi-day downs for a PLC card, missing signal or broken wire.

Problem Solving

Problems Can't Be Solved Reliably Without A Plan

When dealing with problems that bring a machine or system to a stop, it is tempting to think of them as 'Problems'. All day long though, we in fact deal with problems.

Generally, once we know what is wrong, the path to fixing it becomes much clearer.

Inside the DMAIC methodology, one of the most important parts of solving complex problems is defining them. To create a problem statement.

Before addressing a problem, first determine what is actually wrong. And the extents to which it is 'wrong':

• What is happening/not happening?
• When is this problem happening?
• How long does it happen?
• What else affects the occurrence?

This is also a good time to build in reliability with weaknesses found in the process of finding and solving the problem. If this attitude and practice is institutionalized, constant improvements will occur in a way that has almost zero perceived cost to capital.

Productivity Systems

Personal Productivity Systems To Get Things Done

Somewhere along the way, it has been determined that floor level personnel don't need a system by which to keep track of what they need to do.

Personally, I have seen several attempts to deal with this. With systems such as CMMS or other maintenance planning & execution systems. These systems serve the group and directives from the management tier. But they do little to help individuals keep their own local world under control.

The combination of three systems can go a long way to deal with this:

• Calendar
• Notes
• Tasks

In my opinion, I strongly believe this should be left to individuals to figure out what specific tools will work best for them.

Electronic tools work best for me but many still prefer paper.

The system I use is a blend of tools carefully chosen and modified over the years, all inside Nextcloud:

1. CALENDAR
2. NOTES
3. TASKS

This works very, very well. It does require constant cloud access, syncing and use across two laptops an iPhone, iPad and Microsoft Surface Pro. There are many other combinations and configurations that can work well.

The IT infrastructure that assigns, tracks and manages maintenance tasks is very useful for controlling what needs to be done on a more macro level. At the micro level, people need to put together a system that is tightly integrated with how they think, what they are used to and comfortable with.

Simply sit with them, show them the value of managing their own local world. Then, work with them to make agreements on what can be done to help them with the tools they want. It's worth small investments in devices and policy changes (if possible) to enhance what they can do with these tools.

The Like New State.....Always

Few of us have had the opportunity to begin working inside a completely new, green-field site. Those of us who have, know what it is like when everything is new, crisp and in good order.

This doesn't last long though. Soon, poor practices, laziness and sloppy modifications will begin degrading the condition of electrical cabinets, code files, drawing accuracy and cleanliness.

Pushing multiple disciplines that resist this is worth the effort. It may at times be an uphill climb though. Technicians and mechanics will resist, operations managers will push to limit downtime and plant managers will resist spending extra money.

This has to be sold to these people. They have to be led in understanding the value of keeping things in excellent condition.

Managers can be shown how a well ordered system is faster to repair, cheaper to modify.

Technicians and mechanics can be shown that working inside a well ordered system makes finding wires, understanding drawings and knowing how systems are connected will make their days smoother. And their nights at the plant less frequent.

Slowing down, thinking towards the future and dealing with your own anxiety when under the gun will bring you good things.

Secured And Future-Proof

We have skimmed very rapidly many aspects of how to maintain an excellent plant automated system. At the end I will point to other resources to dive into this and more, with more depth and clarity.

It's very easy to see the whole of the system as one big thing. Really though, it is a collection of many small things. Each of these small things are the place where high performance is determined or limited.

View each of these small things as something to be secured. Take quick looks, make good and efficient plans, follow up on them.

Hire and retain people that will follow you in this. If they don't, coach them on this path. If they are still resisting after you have worked to develop them to your system, start a discipline process that will help them understand this is how things will be done.

Your goal should be to make the system future-proof. Able to withstand time and able to resist poor practices. View your system as something that needs to retain high value and that it is your job to protect it.

CA's Contribution

I Can Help......How Controlling Accomplishment Can Help You

There are so many possibilities, so many ways to improve the systems we maintain, improve and operate.

What are some examples of what could be different? :

• Instant Information- Instead of looking for information, knowing instantly where it is and that it is sufficient. Drawings, OEM manuals, device manuals.
• No More Lost Code- Having full confidence that PLC, HMI and other controller code is up-to-date, reliable and protected. Ready to recover from a failure. Ready to be used to make a modification.
• Controllers interconnected on ethernet, devices networked to controllers- Less need for pipe and wire for changes. Plus, more flexibility for moving and adding sensors and actuators
• Preventive work done before it's critical- intuitive decision making by technicians and mechanics
• Understanding Each Other- Mechanics understand Controls Techs, Controls Techs understand Mechanics
• Methodology for solving problems- true understanding of what a problem is and how to solve it
• Personal Systems- How to properly make a plan, a personal system for managing the tasks and activities of that plan
• Performance information available to the operation- the right kind, in the right way
• A plan to modernize that is built-in- Knowing years ahead of time what money needs to be spent; when and why
• Real Knowledge- Expertology on technology and the application of it
• Spares ready to go- The right spare parts on hand, a system to know which ones are right and that you have them
• Perfect Panels- Electrical panels that are not rats nests, are sealed, well lit and rapid understanding of what is what and where it goes
• Fully documented systems- Explanations, Diagrams, Drawings of what makes up machines, the system they are in and the full plant.
• Rapid Shakedown- Visualization systems that tell everything, Alerts for exceptions to what is 'normal'
• High Powered change control- Basic and embedded Project Management methods. Change is always occurring, projects are how that is done. Plus, a system that propagates information when changes occur....so it is ready for the next situation.
• Partner Protocol- OEM integration in a way that retains their confidence. Service provider integration like they are a part of the team

This is a short snapshot of what can be done. You can have these things.

Controlling Accomplishment spends more time developing people than other courses of study. I strongly believe many initiatives fail because we demand people be better but do nothing to enable it.

Controlling Accomplishment is very comprehensive. In short, it covers the play between people, processes and systems. All aspects support the other. And while it is possible to cover deficiencies in these areas and where they interplay, it doesn't stay hidden forever. Some of you are experiencing the rising up of what predecessors were hiding, some are experiencing your own.

With Controlling Accomplishment I will take you through the following progression to optimize your plant :

Assessment & Your Model @ Start

1. Operator Conversion
2. Process Conversion
3. System Conversion
4. Output Process Conversion
5. Information Systems
6. Conversion Assembly