Challenges Difficulties transferring and leveraging mass volumes of SCADA data for operational insights without real-time application was leading to unnecessary downtime internally and in the field.

  • Difficulties transferring and leveraging mass volumes of SCADA data for operational insights without real-time application was leading to unnecessary downtime internally and in the field
  • Kepware solutions were implemented for their ability to extract and share real-time data, allowing seamless interconnectivity among industrial devices and applications like predictive maintenance and machine learning
  • Real-time analysis of existing data and new inflows increased forecast accuracy and project speed while minimizing equipment downtime through actionable insights

The Challenge

The company has employed the CygNet supervisory control and data acquisition (SCADA) system for the past seven years to acquire and manage huge amounts of data across all operations and practices for visualization, alarming and reporting requirements. The data is extremely diverse, from data associated with oil and gas production to data on maintenance of wells and equipment. With volumes of operational data coming in daily, the company wanted to leverage this data for advanced analytics projects that could increase production, predict equipment failures, reduce shut-in wells and streamline analysis of historical data.

However, collecting volumes of data into their data warehouse, powered by Cloudera, was no simple task, as it was historically difficult to transfer data out of their CygNet SCADA system. Gathering data using ETL (Extract, Transform, Load) from just one CygNet server was difficult enough with each server running on up to 15 databases. With the company’s nine CygNet servers needed for their wide-spread production operations, they were running hundreds of jobs each day to handle the massive volume of data. The SCADA system didn’t allow for real-time streaming of data flows, so ETL micro-batches running in 15-minute intervals had to factor in missed updates outside of those windows, which was extremely time consuming. In addition, requests for data beyond their two-year history retention period called for archived databases to be spun-up into new environments and converted to the latest version of their SCADA system’s historian. The laborious process of extracting data was an administrative burden and took time away from performing the actual analysis of the data.

These inefficiencies in collecting, organizing and analyzing data were also preventing the company from creating efficiencies in the field. For example, water haulers transporting waste fluids away from production sites didn’t have the necessary insights from real-time data. These insights have the ability to reduce downtime and costs caused by remote route locations, severe weather and communication lags between job completions. The company also wanted to reduce downtime on electrical submersible pumps (ESP), one of their most costly issues. With the existing system, days could pass before failures and root causes were identified, and the company was at the mercy of service providers dealing with numerous other clients.

The Solution

The company selected Kepware industrial connectivity solutions from PTC to unlock big data opportunities and optimize its operations. Critical to the selection was the ability to easily extract data from existing systems and migrate it between platforms in real-time, while establishing interconnectivity and open communications between industrial devices and applications through a single source. This was accomplished by deploying the KEPServerEX OPC Client driver to pull the real-time data out of the CygNet server and the IoT Gateway Plug-in to send the collected data up to their Cloudera data warehouse via MQTT. The company would now leverage MQTT’s publish and subscribe method of transmitting data in real-time, as opposed to constantly requesting and pulling data into the system via batching models. Their archived data, which had been previously spun-up and loaded into the data warehouse, along with new real-time inflows of production data, would now be readily available for analysis.

The Benefits

Providing the company’s Cloudera data lake with centralized connectivity between all well sites, equipment and hard-to-reach data, Kepware’s solutions now provides a single service that handles all data, instead of having different individual processes on each SCADA server. Kepware solutions streamline communication with the data warehouse to pull data in real-time from numerous servers and databases, resulting in timelier reporting and analyses. Every change in data is now available immediately and pulled directly into the data warehouse through streaming capabilities, preventing missed updates associated with prior micro-batch models. This increases reporting and forecast accuracy by capturing real-time data with higher frequencies of collections. Having Kepware solutions connect to the data warehouse unlocks data previously trapped in the archives. If engineering departments want historical well data beyond two years, it can now be pulled straight from the warehouse and aggregated directly into analysis tools. Being able to do this without spinning up archived databases is a huge administrative benefit.

The company can provide its field operations teams with predictive actions through real-time data communications and insights. ESP analysis enables the company to predict when there will be a failure and the trends that lead to failures. Drawing actionable conclusions from trends in failures reduces downtime and connects the company to service providers more quickly, so they can efficiently schedule repairs to keep wells online and minimize downtime. Water haulers now receive the added benefits of estimated times before their tank batteries are full, so they can better plan their jobs servicing wells to keep production running.


By adding Kepware solutions into their operations, the leading US E&P company was able to gain invaluable, real-time insights into their data streams, unlocking efficiencies across devices, applications and field equipment. Kepware solutions allowed for seamless integration with zero disruptions to the existing SCADA architecture or operations, providing the company with stellar results they set out for.

PTC Solutions for Manufacturing

Ready to experience your own industrial transformation success story? Learn more about how we can help.