REVIEWER: Robbie Cook, technical manager of databases at Australian Energy Market Operator.
BACKGROUND: The Australian Energy Market Operator is headquartered in Melbourne and provides an array of functions for gas and electricity delivery, including operations, development and planning. It manages the National Electricity Market (NEM), as well as the retail and wholesale gas markets of eastern and southern Australia.
PLATFORMS: Dell/Intel servers on the Windows Server operating system.
PROBLEM SOLVED: AEMO’s primary responsibility is to balance the demand and supply of electricity, conducting wholesale trading of electricity in a spot market where supply and demand are instantaneously matched. AEMO calculates the financial liability of all market participants on a daily basis and settles transactions for all trade in the NEM every week. We manage the market and power system from two control centers in different states, both of which are equipped with identical communication and IT systems and operate around the clock. The entire NEM, or individual regions within it, can be operated from either or both centers, ensuring continuous supply and giving AEMO the flexibility to respond quickly to dramatic changes in the market or power system. The systems must be highly available, and any system failure can have substantial financial implications. If our systems experience an outage of more than 20 minutes, we may be forced to suspend electricity trading in the spot market. Our database systems support electricity price calculations, general market data, power dispatch information, reporting, settlements and billing information for both internal and external users, and can be accessed by multiple applications at any time. They must be available 24x7 with no outages, not even for system maintenance. System upgrades and rollout of new software and hardware are performed regularly, but these changes increase the possibility of system disruptions, so we needed a solution that could replicate servers incrementally and provide rapid site failover if necessary.
PRODUCT FUNCTIONALITY: We use Quest’s SharePlex for Oracle to support our database replication needs. SharePlex keeps both data centers in sync, ensures high availability of data and mitigates risks associated with managing large volumes of mission-critical data, so we can perform site failover procedures without disruption to the market.
STRENGTHS: The key strength of SharePlex is that it keeps our two data centers constantly in sync (replication latency averages approximately one second to replicate between databases in Sydney and Brisbane), so that failover is seamless. The main technology advantage SharePlex offers is that it does not have to wait for a redo log switch before sending the transactions to the replicated database; this means that replication latency is very low and data losses during a failure scenario are minimal. Most importantly, if we experience a systems failure, SharePlex replication lets us quickly recover by transferring the active systems to the co-primary site, knowing that the data is already up to date. SharePlex was the only asynchronous replication architecture that could meet AEMO’s requirement to support site switchover without interruption to the market. SharePlex is a mature and robust technology that has proven to be very reliable, no matter what the circumstances.
WEAKNESSES: We are currently using an older version of SharePlex (6.0), and the configuration of the product is a more manual process than we’d like. In speaking with product management, however, I learned that improvements to ease setup and sync processes have been made in several subsequent SharePlex releases.
SELECTION CRITERIA: We chose Quest solutions to meet our unique requirements because they are aligned with our business activities and continue to provide reliable, easy-to-use technology. Most importantly, the two-way communication with Quest developers means we can share ideas and issues directly with the product experts.
DELIVERABLES: SharePlex allows AEMO to maintain co-primary database availability with minimal replication latency across two data centers that are more than 1,000 kilometers (621 miles) apart. The average delay in replicating data between sites is sub one second, with peak latency under 10 seconds. This ensures that if the active database suddenly becomes unavailable, the Australian Wholesale Electricity Market can be switched immediately to the other site without the delays normally associated with standby architectures.
VENDOR SUPPORT: AEMO engaged Quest consulting to assist with the initial design and implementation of SharePlex. The Quest team’s advice and support was of the highest order, and the subsequent support from their service team and developers has been the same. Quest’s willingness to allow customers direct access to product managers and developers is outstanding – a shining example that other IT providers would do well to follow.
DOCUMENTATION: The product documentation provided by Quest is of a high standard and would enable most customers to establish effective replication. When a user’s requirements are complex or exceptional (as in our case), it is prudent to obtain additional help from the Quest support or consulting teams.