Rapid Response Panel

Thought I'd share with you fine folks a panel that was put together for a project I'm currently working on. It's called a Rapid Response Panel. When things go wrong, this panel is a one stop shop so an operator can bring any system in this facility to a safe state.

My involvement with this contraption was to communicate what needs to happen when things go wrong. For instance, when "X" happens, these valves need to close, these valves need to open, but these over here can only close once "y" happens, this motor trips, and on and on. I know nothing of how to design the wires and button to make all this happen. I just know what needs to happen to keep from blowing the place up.

In case you're curious, this is for an LPG export terminal. We've got four refrigeration compressors at 8500 horsepower each and loading pumps that will put propane on a ship at 15,000 gallons per minute. I'm trying to temper any language that sounds braggadocios, but I must say that I am very proud of this project.

I'll be at the FAT for this stuff in January, so if any of you are in the Phoenix area please give me some recommendations of things I need to go/see/do.

Does red stand for Oh ****?

iijakii said:

Does red stand for Oh ****?

Click to expand...

Ha! Actually, I would classify this as an Oh **** panel. The red kills motors. Blues are isolation valves.

Not to be cruel, but this looks like a human factors nightmare, and a safety risk in itself. The last thing I want to have to deal with, when the s**t hits the fan and panic sets in, is a panel with ~130 lights and switches that have to be dealt with in different specific sequences, depending on what the fault is, in order to prevent my a$$ and the neighborhood from being blown to bits. The potential for mistakes in the heat of the moment just seems enormous. The intense and continuous training required to keep the person(s) responsible for responding current in procedures is almost a full time job. Given the propensity of corporations to cut corners, especially on training and level of personnel ...

How many people at the plant have the intimate level of knowledge that the OP does about what needs to happen when? Are there at least two on duty 24/7/365 who are never in the same place at the same time?

Brew on

Is there a button on there that makes less buttons?

Nice panel! I wired up simular panels when I was an electrician. Now I'm a control room operator at a power plant. All of my control stations are through DCS controls. Panels like this are convienent when s*** hits the fan. All start stop station in front of you instead of scattered out though multiple pages on a computer screen. We are trained to handle the heat of the moment situations in highly complex facility. That's part of operations. I'm lucky that my company and union require continuous on the job training for such situations. So I don't think it looks like a nightmare of switches and lights. Good job!

TexasWine said:

I'll be at the FAT for this stuff in January, so if any of you are in the Phoenix area please give me some recommendations of things I need to go/see/do.

Click to expand...

Learn Spanish, if you aren't fluent already.

Psylocide said:

Learn Spanish, if you aren't fluent already.

Click to expand...

In Houston, so it's pretty much a requirement already!

doug293cz said:

Not to be cruel, but this looks like a human factors nightmare, and a safety risk in itself. The last thing I want to have to deal with, when the s**t hits the fan and panic sets in, is a panel with ~130 lights and switches that have to be dealt with in different specific sequences, depending on what the fault is, in order to prevent my a$$ and the neighborhood from being blown to bits. The potential for mistakes in the heat of the moment just seems enormous. The intense and continuous training required to keep the person(s) responsible for responding current in procedures is almost a full time job. Given the propensity of corporations to cut corners, especially on training and level of personnel ...

How many people at the plant have the intimate level of knowledge that the OP does about what needs to happen when? Are there at least two on duty 24/7/365 who are never in the same place at the same time?

Brew on

Click to expand...

X2...as an chief engineer of a large organization, that panel worries me....big time!.....

Actually, when I first opened the thread and looked at the picture, I thought it was a joke and the thread would follow the lines of "don't ever build your brew control panel to look like this!" ......then I read the write up and my thought was "they can't be serious" ....

I'm skeptical about the rapid part. If things are controlled with a DCS or PLC there are ways of using a stand alone PLC to do a sequence to shut down properly. This is what's done in the nuclear world.

Wait, I've seen that panel before...you got your inspiration from this, didn't you?

As someone who has been living around the effects of failed SIS implementations on multiple units in the same facility...I think the giant wall of buttons is a must-have until the industry has gotten to point that automated shutdown actually works.

It should stand as the second redundant system in a triple redundancy. First, hit the "scram" button and watch the monitors to see if the automated shutdown works. Anything starts to vary from the design shutdown tolerances (temperature, pressure, etc), start hitting appropriate buttons while simultaneously deploying wrenches into the affected area to turn valves where the actuators fail as well. Step three is start manually turning valves in the correct order to avoid an upset.

The problem with fully automated facilities for shutdown is they cannot be deployed across the same unit in multiple facilities or multiple units in the same facility (exceptions do exist). A pipe still at one refinery is massively different from the until that does the same work 30 miles down the road. Even two pipe stills on separate trains in the same facility are usually different.

Finally, real world testing of emergency shutdown of a lot of facilities or units is just not feasible and certainly not safe so everyone is depending on the model until the first emergency presents itself.

the panel may look 'crazy' but for trained operators, it isn't that big a deal. i don't believe it is designed as a true 'oh ****' panel where you press one button and everything shuts down.

Onkel_Udo said:

As someone who has been living around the effects of failed SIS implementations on multiple units in the same facility...I think the giant wall of buttons is a must-have until the industry has gotten to point that automated shutdown actually works.



It should stand as the second redundant system in a triple redundancy. First, hit the "scram" button and watch the monitors to see if the automated shutdown works. Anything starts to vary from the design shutdown tolerances (temperature, pressure, etc), start hitting appropriate buttons while simultaneously deploying wrenches into the affected area to turn valves where the actuators fail as well. Step three is start manually turning valves in the correct order to avoid an upset.



The problem with fully automated facilities for shutdown is they cannot be deployed across the same unit in multiple facilities or multiple units in the same facility (exceptions do exist). A pipe still at one refinery is massively different from the until that does the same work 30 miles down the road. Even two pipe stills on separate trains in the same facility are usually different.



Finally, real world testing of emergency shutdown of a lot of facilities or units is just not feasible and certainly not safe so everyone is depending on the model until the first emergency presents itself.

Click to expand...

Are we talking Japan?

trimixdiver1 said:

This is what's done in the nuclear world.

Click to expand...

Not at the plant I worked at, it was all analog with wires and relays.

trimixdiver1 said:

Are we talking Japan?

Click to expand...

Nope...in this case a local Midwest refinery but I can list cases in Lousiana, Texas, Missouri...just off the top of my head. When things start going pear-shaped:

You try the automated system

If the system is new enough new (never tested under fire) or the potential for disaster large enough you start sending Wrench Monkeys (Junior Operators and/maint. personnel) toward the most critical valves

If automation does not work, you start tripping valves (closing some, opening RV's), shutting down compressors and pumps from the panel

When (not if) not every actuator works as designed, a wrench monkey turns it by hand, wrench or chain wheel.

For the most part, what stands between a minor instability and a major upset is the skill of the operators. As a data junky, I do not believe this is how it should be but I have watched two systems, newly installed, fully tested and professionally designed have to be deployed...neither worked 100%.

In one case the minor issue was detected, appropriate "non-automated" action was taken but when a valve jammed they sent operators out to spin manually while hitting the big red button for a controlled shutdown...the logic could not execute the alternate shutdown process so just froze halfway through at a most inappropriate step. Operators attempted to finish the sequence from the control rooms as the first wrench monkey closed the original valve with chainwheel. All was shut down safely except for the fact the unit who could not be brought up for a couple of weeks and a lot of expense.

Root cause...actuator was design for normal loads plus an appropriate safety factor but in a partial shutdown (throttleback) situation it saw loads beyond "normal" but below the unit's max...and above the actuator's max. Engineering bust. The software issue is still under investigation.

In the power gen world we have multiple layers of automated protection. If the primary fails then there's the backup, and both will typically protect prior to the operator taking action. That being said there is still an oh $h;+ button because things happen, but Ive only seen that used once when there was a problem upstream of the unit.

Sounds like your jobs are more customized, which makes it a more time consuming task to walk through all the failure modes and ensure the requirements and automation were defined correctly. In addition it's not uncommon for mech eng's to size components for given reqs but don't understand plant operability, so the automation is used as a bandaid for sub optimal system layout. Also the operators Ive worked with have ranged from very knowledgable to ones that would have trouble competing with my 4 year old daughter. That panel does look like something from a museum but I could understand if you want something to be used as the last line of defense.

ingchr1 said:

Not at the plant I worked at, it was all analog with wires and relays.

Click to expand...

Yeah when I was a young engineer i was amazed me to see this. I think that it's a disservice to the nuclear industry that there are so many hurdles to upgrading automation that would lead to safer overall plant operation. We can do better than 1960's tech.

BigBlock said:

Yeah when I was a young engineer i was amazed me to see this. I think that it's a disservice to the nuclear industry that there are so many hurdles to upgrading automation that would lead to safer overall plant operation. We can do better than 1960's tech.

Click to expand...

once the nuke industry hit the brakes back in the 80s, there was no market to push new products. the cost to certify new products is insane. once things got rolling again in the late 2000's (pre-fukushima), there was an effort for more modern products but i recall a lot of old timers pushing back. the manufacturers pushed back as well and basically said they would only offer the legacy product as a replacement for existing (i'm speaking about protective relaying here). the plant would need to send them the old one as 'proof' they were replacing something. then fukushima happened and the brakes were back on. i haven't been involved with the industry for five years now, no idea where the technology is at...