Article - CS342359

Active-Active HA ThingWorx instance sometimes does not work properly and returns error "ChangeWatcher failed to sync" in Application Log

Modified: 13-Feb-2026   


Applies To

  • ThingWorx Platform 9.0 to 10.0
  • Microsoft SQL Server
  • Azure SQL Server

Description

  • Active-Active HA (High Availability) ThingWorx instance occasionally returns error ChangeWatcher failed to sync in ThingworxStorage\logs\ApplicationLog.log and fails to work properly
      • Uable to login to ThingWorx
      • ThingWorx Platform nodes in the cluster shut down unexpectedly
      • LDAP users unable to login to ThingWorx
      • Model change on node is not reflected to other nodes such as permission change

    • Does ThingWorx Platform automatically connect to database when Microsoft SQL Server is configured as HA instance database?
    • Errors in ThingworxStorage\logs\ApplicationLog.log:
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset by peer: socket write error 
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset by peer: socket write error
    [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
    [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners!
    [L: ERROR] [O: c.t.p.ThingworxDefaultTransaction] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back...
    [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] A PooledConnection that has already signalled a Connection error is still in use!
    [L: WARN] [O: c.m.v.c.i.NewPooledConnection] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] [c3p0] Another error has occurred [ com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. ] which will not be reported to listeners!
    [L: ERROR] [O: c.t.p.PersistenceProviderTransactionManager] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Error occurred during commit for the ThingworxPersistenceProvider persistence provider.
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to syn
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] ChangeWatcher failed to sync
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Timeout. Cannot synchronize model during xxx ms
    [L: ERROR] [O: c.t.s.ThingWorxServer] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Shutting down server..
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Model sync unable to update database with model changes after xx attempts due to exception An SQLException was provoked by the following failure: java.lang.InterruptedException
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception An SQLException was provoked by the following failure: java.lang.InterruptedException
    [L: ERROR] [O: c.t.s.ChangeWatcher] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-9-thread-1] Unable to update cache with model changes after xx attempts: encountered exception class org.apache.ignite.internal.processors.cache.CacheStoppedException: Failed to perform cache operation (cache is stopped): default
    • [L: ERROR] [O: o.a.i.s.c.t.TcpCommunicationSpi] [I: ] [U: XXX] [S: ] [P: XXX] [T: pool-12-thread-1] Failed to send message to remote node [node=TcpDiscoveryNode [id=XXX, consistentId=XXX, addrs=ArrayList [XXX], sockAddrs=null, discPort=47500, order=1, intOrder=1, lastExchangeTime=XXX, loc=false, ver=2.14.0#20220929-sha1:951e8deb, isClient=false], msg=GridIoMessage [plc=2, topic=TOPIC_CACHE, topicOrd=8, ordered=false, timeout=0, skipOnTimeout=false, msg=GridNearSingleGetRequest [futId=XXX, key=UserKeyCacheObjectImpl [part=22, val=com.thingworx.things.events.EntityEventProcessor_lastEntityEventChangeLogId, hasValBytes=true], flags=1, topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], taskNameHash=0, createTtl=-1, accessTtl=-1, txLbl=null, mvccSnapshot=null]]]
      [L: ERROR] [O: o.a.i.s.d.t.TcpDiscoverySpi] [I: ] [U: ] [S: ] [P: XXX] [T: tcp-client-disco-sock-writer-#2%twx-core-server%-#40%twx-core-server%] Failed to send message: null
      [L: ERROR] [O: c.t.s.s.w.p.WSExecutionInstance] [I: ] [U: XXX] [S: ] [P: XXX] [T: WSExecutionProcessor-86] error executing APIRequest Message: Unable to dispatch [ uri = /Things/XXX/Services/UpdateSubscribedPropertyValues/]: Unable to Invoke Service UpdateSubscribedPropertyValues on XXX : class org.apache.ignite.IgniteClientDisconnectedException: Operation has been cancelled (client node disconnected)., sending ERROR ResponseMessage to caller!
      [L: ERROR] [O: c.t.c.s.e.RoutingGroupServiceImpl] [I: ] [U: ] [S: ] [P: XXX] [T: pool-26-thread-1] Error executing syncRoutingGroups
    • Errors in ThingworxStorage\logs\ErrorLog.log:
    [L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] ChangeWatcher failed to sync java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:210)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2023)
        ... 19 common frames omitted
    Wrapped by: com.microsoft.sqlserver.jdbc.SQLServerException: Connection reset
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:2924)
        at com.microsoft.sqlserver.jdbc.TDSChannel.read(IOBuffer.java:2031)
        at com.microsoft.sqlserver.jdbc.TDSReader.readPacket(IOBuffer.java:6418)
        at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7581)
        at com.microsoft.sqlserver.jdbc.TDSCommand.startResponse(IOBuffer.java:7544)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection$1ConnectionCommand.doExecute(SQLServerConnection.java:3019)
        at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7194)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2979)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(SQLServerConnection.java:3024)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.setAutoCommit(SQLServerConnection.java:3157)
        at com.mchange.v2.c3p0.impl.NewProxyConnection.setAutoCommit(NewProxyConnection.java:1059)
        at com.thingworx.persistence.ThingworxDefaultTransaction.getTransactionObject(ThingworxDefaultTransaction.java:193)
        at com.thingworx.system.ChangeWatcher$Watcher.changeProcessing(ChangeWatcher.java:349)
        at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:271)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    [L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Model sync unable to update database with model changes after 1 attempts due to exception Connection reset 
    [L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Unable to update cache with model changes after 1 attempts: encountered exception Connection reset 
    [L: ERROR] [O: E.c.q.l.c.Logger] [I: ] [U: SuperUser] [S: ] [P: xxx] [T: pool-10-thread-1] Could not commit the transaction. Rolling back... com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
        at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:1090)
        at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:3195)
        at com.mchange.v2.c3p0.impl.NewProxyConnection.rollback(NewProxyConnection.java:1033)
        at com.thingworx.persistence.ThingworxDefaultTransaction.commit(ThingworxDefaultTransaction.java:106)
        at com.thingworx.persistence.PersistenceProviderTransactionManager.handleCommitTransaction(PersistenceProviderTransactionManager.java:172)
        at com.thingworx.persistence.PersistenceProviderTransactionManager.lambda$endTransactionRequired$1(PersistenceProviderTransactionManager.java:123)
        at java.util.HashMap.forEach(HashMap.java:1289)
        at com.thingworx.persistence.PersistenceProviderTransactionManager.endTransactionRequired(PersistenceProviderTransactionManager.java:123)
        at com.thingworx.persistence.TransactionFactory.endTransactionRequired(TransactionFactory.java:120)
        at com.thingworx.system.ChangeWatcher$Watcher.finallyProcessing(ChangeWatcher.java:396)
        at com.thingworx.system.ChangeWatcher$Watcher.run(ChangeWatcher.java:279)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
    This is a printer-friendly version of Article 342359 and may be out of date. For the latest version click CS342359