Article - CS342359

Active-Active HA ThingWorx instance does not work properly occasionally with Application Log error "ChangeWatcher failed to sync"

Modified: 07-Oct-2025   


Applies To

  • ThingWorx Platform 9.0 to 10.0
  • Microsoft SQL Server
  • Azure SQL Server

Description

  • Active-Active HA (High Availability) ThingWorx instance does not work properly occasionally with below Application Log error
    • ChangeWatcher failed to sync
    • Uable to login to ThingWorx
    • ThingWorx Platform nodes in the cluster shut down unexpectedly
    • LDAP users could not login to ThingWorx
    • Model change on node A is not reflected to other nodes such as permission change
  • Does ThingWorx Platform automatically connects to database when Microsoft SQL Server is configured as HA instance database?
  • Errors in Application Log
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset by peer: socket write error 
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset by peer: socket write error
[L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
[L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners!
[L: ERROR] [O: c.t.p.ThingworxDefaultTransaction] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back...
[L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
[L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners!
[L: ERROR] [O: c.t.p.PersistenceProviderTransactionManager] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Error occurred during commit for the ThingworxPersistenceProvider persistence provider.
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to syn
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] ChangeWatcher failed to sync
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Timeout. Cannot synchronize model during xxx ms
[L: ERROR] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Shutting down server..
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Model sync unable to update database with model changes after xx attempts due to exception An SQLException was provoked by the following failure: java.lang.InterruptedException
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception An SQLException was provoked by the following failure: java.lang.InterruptedException
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception class org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed to perform cache operation (cache is stopped): default
  • Errors in Error Log
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2023)
    ... 19 common frames omitted
Wrapped by: com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:2924)
    at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2031)
    at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6418)
    at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7581)
    at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7544)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:3019)
    at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2979)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:3024)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.setAutoCommit(SQLServerConnection.java:3157)
    at com.mchange.v2.c3p0.impl.NewProxyConnection.setAutoCommit(NewProxyConnection.java:1059)
    at com.thingworx.persistence.ThingworxDefaultTransaction.getTransactionObject(ThingworxDefaultTransaction.java:193)
    at com.thingworx.system.ChangeWatcher$Watcher.changeProcessing(ChangeWatcher.java:349)
    at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:271)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset 
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset 
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back... com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
    at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:1090)
    at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:3195)
    at com.mchange.v2.c3p0.impl.NewProxyConnection.rollback(NewProxyConnection.java:1033)
    at com.thingworx.persistence.ThingworxDefaultTransaction.commit(ThingworxDefaultTransaction.java:106)
    at com.thingworx.persistence.PersistenceProviderTransactionManager.handleCommitTransaction(PersistenceProviderTransactionManager.java:172)
    at com.thingworx.persistence.PersistenceProviderTransactionManager.lambda$endTransactionRequired$1(PersistenceProviderTransactionManager.java:123)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at com.thingworx.persistence.PersistenceProviderTransactionManager.endTransactionRequired(PersistenceProviderTransactionManager.java:123)
    at com.thingworx.persistence.TransactionFactory.endTransactionRequired(TransactionFactory.java:120)
    at com.thingworx.system.ChangeWatcher$Watcher.finallyProcessing(ChangeWatcher.java:396)
    at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:279)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
This is a printer-friendly version of Article 342359 and may be out of date. For the latest version click CS342359