Blackout: Shocked Into Action

 

TDworld

http://www.osisoft.com/osisoft/downloads/TDWorld_Sep2006.pdf

 

THE BLACKOUT ON AUGUST 14, 2003, WAS THE LARGEST IN U.S. HISTORY. It started with a line going out in Ohio. It affected eight U.S. states and one Canadian province. It covered more than 9000 sq miles (23,310 sq km). It disrupted over 61,000 MW of power. It caused three deaths. It closed 12 major airports. It touched more than 50 million people — one-third of the Canadian population and one-seventh of the U.S. population. It cost more than US$4 billion in lost economic activity.

As part of the postmortem to this event, the joint Department of Energy (DOE) and Federal Energy Regulatory Commission (FERC) report to Congress in February 2006 stated that the blackout was “caused in part by deficiencies by the system that monitors the electric grid and a lack of awareness of deteriorating conditions by the operators who monitor the system.”

Among the report recommendations are that:

North American Electric Reliability Council (NERC) should evaluate and adopt better real-time tools for operators

NERC, FERC and Canadian authorities should require the use of time-synchronized data recorders

FERC and Canadian authorities should establish requirements for the collection and reporting of data needed for post-blackout analysis.

The blackout focused attention on the need for real-time wide-area visibility and situational awareness. None of this can be done without monitoring across the national grid.

THE RESPONSE

In October 2003, just two months after the blackout, the DOE gathered together a small group of utilities, vendors and universities. This group has become the nucleus of the Eastern Interconnect Phasor Project (EIPP). Entergy, New York Power Authority (NYPA), Tennessee Valley Authority (TVA) and American Electric Power (AEP) are the founding utilities of EIPP.

The purpose of the 2003 meeting was to begin defining new methods to mitigate effects of future system disturbances. The vision of EIPP is to improve power-system reliability through wide-area measurement, monitoring and control.

Almost three years later, EIPP now has more than 100 utility and nonutility participants. During the past three years, the EIPP task teams have taken the Eastern grid from a place with virtually no phasor measurement units (PMUs) to a grid with more than 75 planned or implemented PMUs. Now that EIPP has the infrastructure in place, over the next several years it will focus on the areas of visualization tools, planning, communications infrastructure, protection and switching, and grid monitoring.

BENEFITS OF PMU DATA

A combination of trend analysis and post-disturbance analysis will broaden the overall understanding of the Eastern Interconnect power-system dynamics. PMU data can be used to calculate the frequency-response characteristics for a generator trip, the oscillatory modes and associated damping, identification and analysis of voltage swings. Trending analysis can be done in terms of correlating with time of day, season, peak, load and so forth. The knowledge gained from this analysis can be used for developing transmission-operating procedures and reinforcement plans.

One of the major lessons learned from the 2003 blackout was the importance of having time-synchronized system-data recorders. In the absence of these, it was very laborious to put together the events and sequences that occurred across the Northeast and at several utilities that led to the blackout. Hence, recommendation 28 of the joint U.S.-Canada task force investigating the blackout was: Require use of time-synchronized data recorders.

A great advantage of PMU data is that because it is time-synchronized and has a high sampling rate, it can be used for measuring the phase-angle difference between two buses. This improves a utility's ability to monitor grid dynamics and stability for reliability management. This level of information was never available before, and the potential to monitor and manage the system was not possible before. All of these benefits will be significant steps in enhancing the reliability of the grid.

ENTERGY PHASOR MEASUREMENT SYSTEM

At the time of the August 2003 blackout, Entergy, like most utilities in the Eastern Interconnect, did not possess any PMUs on its transmission grid. Today, Entergy maintains one of the newest and largest PMU systems in the Eastern Interconnect. The Entergy PMU system is dispersed across four states and is recording on the 115-kV/138-kV, 230-kV and 500-kV transmission levels.

Entergy's 15 current PMU sites were selected based on the need to monitor important interfaces and possible instability areas of the Entergy system. Site selection also depended on availability of Entergy-owned fiber and bandwidth. None of the existing sites use leased fiber. Several sites were eliminated as possible candidates because fiber and bandwidth were not available or too costly to employ. Entergy currently has a siting study underway to select future sites based on the need for redundancy, monitoring and control of the Entergy grid.

COMMUNICATIONS

Entergy's internal communication design for the Entergy phasor system used expertise from various Entergy departments, SAIC and OSIsoft. The idea was that a dual phasor data collector (PDC) design could deliver all internal requirements, while also satisfying the external requirements, all in a secure fashion. An internal server on Entergy's corporate network would poll PMUs and also service internal requests via OSI's Processbook client application. A second external server would sit in Entergy's demilitarized zone (DMZ) to synchronize data with the internal server, and also send and receive data with TVA via the virtual private network (VPN). Open protocol communication (OPC) was chosen to share phasor data with TVA. While it is based on Microsoft's Distributed COM (DCOM) standard, it enables open standards communication without having to resort to a proprietary solution.

This design provided a server to share data externally without having to allow that server access to the internal network to poll the meters. Furthermore, as there is also an internal server, Entergy still has access to its own phasor data in the event of external server loss. Because of how OSIsoft's Pi synchronizes data, there are no issues about how the external server receives data from the internal server. All data synchronization occurs via the Pi-to-Pi interface and is initiated internally.

In January 2006, Entergy's PMU stations, previously on the corporate network, were migrated to a newly designed, private, secure, Entergy-owned fiber network used solely for critical data. Phasor data is viewed as controls-type, critical data. As such, this private network allows for more stringent access privileges to the PMU data, including restricted access privileges to both the servers and PMUs. The figure above is a high-level view of how the new communications network will deliver phasor data.

PHASOR DATA COLLECTOR

The Arbiter meters use software in the form of the Arbiter Interface from OSIsoft to pass data from the substation to the PI UDS system or PDC. The Pi system then stores the data on servers for multiple days. The PMUs, as well as the Pi interface, now support 37.118 communications protocol, which requires a minimum data-sampling rate of 30 samples/sec.

From Entergy's internal PDC, all data points are exported via multiple instances of the Pi-to-Pi interface to an external PDC, which resides on the Entergy DMZ. OPC Server software is installed on the DMZ server to allow TVA access to the data. A software package from Matrikon, called the OPC Tunneller, is used to automatically configure DCOM settings and enable reliable OPC communications over differing domains or work groups.

FIELD IMPLEMENTATION

The project used Entergy personnel to perform the field installations of the PMUs. In addition to the communications considerations for a potential PMU site, several other steps preceded PMU installation. For instance, after Entergy's System Planning and Operations identified the transmission lines to be monitored, Substation Design identified a potential transformer and current transformer from which to collect PMU data. From various station drawings, System Planning verified the selection of input locations. Since many of the PTs and CTs selected for monitoring have been in service for years, each device was field inspected to verify that they are both in service, appear physically sound and match drawings. Since all of our substations are in constant service, a hands-on inspection was not always possible. Also, the project coordinator, Entergy field personnel and relay design made a site visit to each location. Information verified included: adequate physical room in control house, availability of rack space for Arbiter meter, visual of selected PT and CT, and availability of dc power.

ACKNOWLEDGEMENT

The Entergy phasor system reflects the most current, state-of-the-art understanding of phasor measurements. This system is a reality because of the dedication, efforts, hard work and courage of the Entergy Phasor Project Team. I am honored to work with such an incredible group of individuals.

-------------------------------------------------------------------------------


Floyd Galvan, a licensed professional engineer in Texas, is a senior project manager for Research & Development at Entergy Corp. His areas of specialization include phasor measurement and control, grid visualization, applications of phasor measurements, long-term planning and regional energy pricing. Galvan has worked throughout the industry in various areas including system planning, fuels procurement and wholesale energy forecasting. He has held leadership positions within the DOE EIPP and CEATI, and has served on numerous panels and committees at the NSF. He received a BSEE degree from Texas A&M University-Kingsville and a master's degree in liberal arts with an art history focus from Southern Methodist University. FGalvan@entergy.com

PMU Locations in the Entergy System ★ Existing PMU locations ★ New PMU locations
10 sites are operational
5 sites came on-line summer of 2006
PMU — Arbiter 1133A, Power Sentinal
PDC — OSIsoft PI Historian


EASTERN INTERCONNECT PHASOR PROJECT PARTICIPANTS Manufacturer participants in the EIPP include:
ABB General Electric Patrick Engineering
Ametek Gemscape Power Measurement
Arbiter Global Energy PowerWorld
AREVA InStep Psymetrix
Bigwoods Systems Macrodyne Qualitrol
Ciber MCGWare SEL
Concurrent Technologies Corp. Mehta Tech Siemens
Cyberpower NxtPhase SISCO
Doble OSIsoft V&R Energy
Researchers and organizations supporting EIPP efforts include:
Arizona State University EPRI NYSERDA
AWEA Georgia Tech Rensselaer Polytechnic Institute
CEA KEMA Texas A&M University
CEC Michigan Tech University of Wyoming
CERTS Montana Tech Virginia Tech
Cornell University Navigant Washington State University
Department of Energy NIST West Virginia University


Ongoing Work in Phasor Measurements at Entergy and in the Industry Projects Participants
Integrating phasor measurements into state estimation, Phase II AREVA, TVA, Entergy, PG&E, Manitoba Hydro, Idaho Power, ORNL, Northeastern University, Washington State University, First Energy
Transmission line dynamic ratings using PMUs Entergy, NEETRAC/Georgia Tech, TVA
Detection, prevention and mitigation of cascading events using PMUs prototype implementation at TVA and Entergy PSERC, Entergy, TVA, AREVA, ATC, EPRI, Korean EPRI, Texas A&M, Washington State University, Arizona State University
Distributed state estimation with application to alarm processing CTC, Georgia Tech, Entergy, METC, NYPA
Placement and benefits of phasor measurement units for monitoring and control of the Entergy grid Northeastern University, Entergy
System security using phase-angle differences Entergy, Washington State University
Wide-area real-time frequency visualization and location of disturbance EPRI, Entergy, TVA, Virginia Tech


WHAT IS PHASOR DATA?


Phasors, as used in this article, refer to high-speed, very accurately time-tagged data in general. In fact, this phasor data are actually “phasor quantities,” which refer to the graphic means used to represent quantities having both magnitude and angular position. Thus, phasors are used to represent the sine waves in terms of magnitude and phase angle, and explicitly account for real and reactive circuit elements. Voltage and current magnitude and phase angle for all three phases and frequency are the fundamental phasor data collected. Real, reactive and apparent power for each phase and the total are also collected along with power factor. Finally, the zero, positive and negative sequence components of the voltage and current quantities are measured.

One of the best ways to think of phasor data is in comparison to SCADA data. Phasor data typically have a refresh rate of 30 samples/sec, while SCADA data have a refresh rate of 2 sec to 5 sec. Phasor data are very accurately time-tagged so that state estimator, operator display or planning study can easily “line up” or resolve what is happening and when. With SCADA information, which is not time-tagged, some data are newer than others, but there is no way to tell the difference. Phasor data collection makes use of modern communication technology and enables action in response to system dynamics. SCADA data uses legacy communications and was never meant to respond to rapidly changing system transient events. Ultimately, phasor data will provide a platform to automatic switching schemes for wide-area transmission grid control.

The actual collection of the phasor data involves two generic pieces of equipment: a phasor measurement unit (PMU) and a phasor data collector (PDC). The PMU is installed at the substation or switchyard and makes high-bandwidth, high-frequency, GPS time-stamped, synchronized measurements. The PDC polls all the PMUs 30 times per second and stores the recorded measurements. Entergy uses the Arbiter 1133A Power Sentinel as its PMU and OSIsoft's Pi Historian as its PDC.

THE PEOPLE


When Warren Buffett recently willed his fortune to the Bill and Melinda Gates Foundation, the world took notice. In this single act, one of the largest foundations in the fight against world disease and human strife was created. On a larger scale, this act is an illustration of how people can make an incredible difference when we focus our collective energies.

It is from our collective energies that real, lasting change happens. Some of that energy is at work today in our electric utility industry, within the Eastern Interconnect Phasor Project (EIPP) and at Entergy in particular. There are many men and women from many companies, agencies and universities who are hard at work in moving the electric utility industry into the next paradigm.

EIPP


The Leadership Committee of EIPP is comprised of the chairs from the six task teams along with representatives from NERC, Pacific Northwest National Labs, CERTS and the Department of Energy:


Equipment Placement — Mike Ingram, TVA
Data Management — Paul Myrda, METC
Business Management — Floyd Galvan, Entergy
Performance Requirements — Damir Novosel, InfraSource
Off-Line Applications — Navin Bhatt, AEP
Real-Time Applications — Terry Bilke, MISO

There have been some significant accomplishments by individuals of EIPP who deserve mention:


Matt Donnelly, formerly of PNNL, was the first project manager for EIPP. He directed and molded the project from its inception to its current form. Donnelly provided the leadership and vision required to move the project forward.

Mike Ingram, Ritchie Carroll and Lisa Beard from TVA were instrumental in developing the Super-phasor data collector (PDC) at TVA. The Super-PDC is the vehicle that allows PMU information to be collected, stored and rebroadcast to the entire project. Carroll is the lead architect for the Super-PDC.

Floyd Galvan, author of this article, led the Business Management Task Team that put in place the contracts with EIPP utilities to share phasor information. This gave EIPP participants access to high-speed GPS-synchronized, time-tagged data across the Eastern United States.

ENTERGY

The Entergy Phasor Project Team consists of:

Floyd Galvan, Project Manager
Ian Barras, Assistant Project Manager
Sujit Mandal, Planning & Phasor Applications
Leonard Chamberlin, Communications Infrastructure Architect
Steve Winsett, Pi Applications and Visualization
Jim Varnado, Field Installation, Construction & Maintenance
Jerry Berndsen, Communications & Testing
Ryan Prejean, Entergy Real-Time Operations

In the future, you will hear more about EIPP as it morphs into the North American Phasor Project. Right now, the EIPP website can give you up-to-date information on activities: http://phasors.pnl.gov/.