Article - CS342359
Active-Active HA ThingWorx instance sometimes does not work properly and returns error "ChangeWatcher failed to sync" in Application Log
Modified: 13-Feb-2026
Applies To
- ThingWorx Platform 9.0 to 10.0
- Microsoft SQL Server
- Azure SQL Server
Description
- Active-Active HA (High Availability) ThingWorx instance occasionally returns error
ChangeWatcher failed to syncin ThingworxStorage\logs\ApplicationLog.log and fails to work properly
-
- Uable to login to ThingWorx
- ThingWorx Platform nodes in the cluster shut down unexpectedly
- LDAP users unable to login to ThingWorx
- Model change on node is not reflected to other nodes such as permission change
- Does ThingWorx Platform automatically connect to database when Microsoft SQL Server is configured as HA instance database?
- Errors in ThingworxStorage\logs\ApplicationLog.log:
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset by peer: socket write error [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset by peer: socket write error [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use! [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners! [L: ERROR] [O: c.t.p.ThingworxDefaultTransaction] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back... [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use! [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners! [L: ERROR] [O: c.t.p.PersistenceProviderTransactionManager] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Error occurred during commit for the ThingworxPersistenceProvider persistence provider. [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to syn
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] ChangeWatcher failed to sync
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Timeout. Cannot synchronize model during xxx ms
[L: ERROR] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Shutting down server..
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Model sync unable to update database with model changes after xx attempts due to exception An SQLException was provoked by the following failure: java.lang.InterruptedException
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception An SQLException was provoked by the following failure: java.lang.InterruptedException
[L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception class org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed to perform cache operation (cache is stopped): default
-
[L: ERROR] [O: o.a.i.s.c.t.TcpCommunicationSpi] [I: ] [U: XXX] [S: ] [P: XXX] [T: pool-12-thread-1] Failed to send message to remote node [node=TcpDiscoveryNode [id=XXX, consistentId=XXX, addrs=ArrayList [XXX], sockAddrs=null, discPort=47500, order=1, intOrder=1, lastExchangeTime=XXX, loc=false, ver=2.14.0#20220929-sha1:951e8deb, isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, msg=GridNearSingleGetRequest [futId=XXX, key=UserKeyCacheObjectImpl [part=22, val=com.thingworx.things.events.EntityEventProcessor_lastEntityEventChangeLogId, hasValBytes=true], flags=1, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], taskNameHash=0, createTtl=-1, accessTtl=-1, txLbl=null, mvccSnapshot=null]]] [L: ERROR] [O: o.a.i.s.d.t.TcpDiscoverySpi] [I: ] [U: ] [S: ] [P: XXX] [T: tcp-client-disco-sock-writer-#2%twx-core-server%-#40%twx-core-server%] Failed to send message: null [L: ERROR] [O: c.t.s.s.w.p.WSExecutionInstance] [I: ] [U: XXX] [S: ] [P: XXX] [T: WSExecutionProcessor-86] error executing APIRequest Message: Unable to dispatch [ uri = /Things/XXX/Services/UpdateSubscribedPropertyValues/]: Unable to Invoke Service UpdateSubscribedPropertyValues on XXX : class org.apache.ignite.IgniteClientDisconnectedException: Operation has been cancelled (client node disconnected)., sending ERROR ResponseMessage to caller! [L: ERROR] [O: c.t.c.s.e.RoutingGroupServiceImpl] [I: ] [U: ] [S: ] [P: XXX] [T: pool-26-thread-1] Error executing syncRoutingGroups - Errors in ThingworxStorage\logs\ErrorLog.log:
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2023)
... 19 common frames omitted
Wrapped by: com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:2924)
at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2031)
at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6418)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7581)
at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7544)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:3019)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2979)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:3024)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.setAutoCommit(SQLServerConnection.java:3157)
at com.mchange.v2.c3p0.impl.NewProxyConnection.setAutoCommit(NewProxyConnection.java:1059)
at com.thingworx.persistence.ThingworxDefaultTransaction.getTransactionObject(ThingworxDefaultTransaction.java:193)
at com.thingworx.system.ChangeWatcher$Watcher.changeProcessing(ChangeWatcher.java:349)
at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:271)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset
[L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back... com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:1090)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:3195)
at com.mchange.v2.c3p0.impl.NewProxyConnection.rollback(NewProxyConnection.java:1033)
at com.thingworx.persistence.ThingworxDefaultTransaction.commit(ThingworxDefaultTransaction.java:106)
at com.thingworx.persistence.PersistenceProviderTransactionManager.handleCommitTransaction(PersistenceProviderTransactionManager.java:172)
at com.thingworx.persistence.PersistenceProviderTransactionManager.lambda$endTransactionRequired$1(PersistenceProviderTransactionManager.java:123)
at java.util.HashMap.forEach(HashMap.java:1289)
at com.thingworx.persistence.PersistenceProviderTransactionManager.endTransactionRequired(PersistenceProviderTransactionManager.java:123)
at com.thingworx.persistence.TransactionFactory.endTransactionRequired(TransactionFactory.java:120)
at com.thingworx.system.ChangeWatcher$Watcher.finallyProcessing(ChangeWatcher.java:396)
at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:279)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
This is a printer-friendly version of Article 342359 and may be out of date. For the latest version click CS342359