COVID-19 continues to disrupt normal life and has resulted in substantial changes in the policies, priorities, and activities of individuals, organizations, and governments. What we’ve seen though is these changes are proving to be a catalyst for technology and innovation.

Specifically for industrial enterprises, the need for digital transformation has accelerated (see here and here). One of the areas of increased investment is creating digital twins for products and implementing digital thread solutions. With the boom in the number of connected IIoT devices, there is an ongoing need to manage the large volumes of data coming through these devices. The new use cases for smart factories to solve for operational efficiencies and better service products have led to time-series data being produced at a large scale. As a result, it has become more important than ever for modern IIoT solutions to be able to ingest and manage large volumes of high-speed data.

This post will look at how different storage solutions in ThingWorx can help you with your IIoT data management.

First, let’s talk about the types of IoT data that need to be managed by the ThingWorx platform:

Figure 1: Diagram showing different types of IoT data. 

1. Data coming from the connected devices and systems, such as: 

• Time-stamped high-speed telemetry data that send sensor readings like temperature and pressure
• Remote access byte streams that are transferred when you open a tunneling session with the connected device to remotely access and control the connected smart devices
• Files or packages that are transferred back and forth for remote software content or device management use cases
• Data created from sales systems, ERP, MES, and accessed by the platform

2. Data generated within the platform, such as:

• Application and configuration data
• Digital twin model data, metadata, and others

Let's dive deeper into time-series data:

Time-series data is any data that is timestamped. It is a sequence of data taken at successive spaced points in time. In the case of an IIoT implementation, capturing time-series data is critical to analyze the historical data, monitor the current data, and be able to predict the future behavior of the systems and machines.

Figure 2: Graphical representation showing a typical time-series data. 

There are several examples for time-series data such as:

• 100,000 smart connected pumps are being monitored within a geographical area for leakages and vibrations where each pump reports 20 property values to ThingWorx every five minutes.
•  A smart factory with 250 monitored machines each sending 60 property updates to ThingWorx per second allowing to capture Overall Equipment Effectiveness (OEE), performance, and manufacturing productivity.
• Millions of devices in a smart city sending intermittent data about various city traffic and parking lights.
• Weather data coming out from different weather recording sensors based on environmental changes or triggers.

How does ThingWorx manage time-series data?

In ThingWorx, IIoT data is stored in a persistence provider that enables the connection to a data store and provides an ability to perform a CRUD operation on that data. ThingWorx includes several persistence provider implementations that enable you to choose a database option that best fits your use case. ThingWorx has the ability to leverage multiple data stores, which provides immense flexibility that best meets your IT preferences and investments.

Figure 3: Architectural diagram showing different supported persistence providers with ThingWorx platform. 

ThingWorx stores its model data, system and metadata, and other relational data in a Model provider such as H2, PostgreSQL, Microsoft SQL Server, and Azure SQL Database and its telemetry data (called as Data Provider) in a dedicated time-series database – InfluxDB. Please note that while RDBMS databases can be used as both model and data provider, for large amounts of time-series data management, PTC recommends using a dedicated Data provider. Learn more about how to store data with ThingWorx and the criteria to select a data store.

PTC has partnered with InfluxData to leverage InfluxDB as our only purpose-built time-series database supported with ThingWorx. It allows for the ability to ingest large amounts of IoT time-series data at high speed and perform complex and time-bound queries to extract meaningful insights from the data. In addition, it provides high data compression and downsampling of data to allow for saving cost of storing large amounts of timeseries data.

Below is the graph showing ThingWorx scale testing results with different persistence providers (To learn more, refer to the ThingWorx Sizing guide).

Figure 4: Graph showing property writes per second and HTTP Ops rate for different persistence provider solutions with ThingWorx platform. 

Together with InfluxData, PTC has strong roadmap plans to further leverage the capabilities of a new query language called “Flux” introduced with InfluxDB 2.X to provide additional time-series capabilities natively with the ThingWorx platform.