Article - CS321821
Updating lastConnectionTime property of Remote Things can create a deadlock scenario if the associated queue is full on ThingWorx Platform
Modified: 22-Nov-2022
Applies To
- ThingWorx Platform 8.1 to 8.5
Description
- When many Remote Things connect at once the ThingWorx Platform becomes unstable
- lastConnectionTime property updates seem related to ThingWorx performance
- ThingWorx Platform performance is poor
- ThingWorx Platform has stopped working and Composer is no longer accessible
- Thread dumps show numerous thread waiting on java.util.concurrent.locks.ReentrantLock$NonfairSync:
- Blocked Thread
"https-jsse-nio-8443-exec-85" tid=0x7746d in WAITINGBlocked: 2696[-1ms], Waited: 879194[-1ms] User CPU: 35m26s - synchronizer <0x143fa594> (a java.util.concurrent.ThreadPoolExecutor$Worker) at sun.misc.Unsafe.park(Native Method) - waiting on <0x37c62d09> (a java.util.concurrent.locks.ReentrantLock$NonfairSync), held by https-jsse-nio-8443-exec-84 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) at com.thingworx.communications.platform.modules.DispatchingServerCommunicationModule.processBindMessage(DispatchingServerCommunicationModule.java:471) at com.thingworx.communications.platform.modules.DispatchingServerCommunicationModule.processCompletedBindMessage(DispatchingServerCommunicationModule.java:348) at com.thingworx.communications.platform.modules.DispatchingServerCommunicationModule.processBindRequestMessage(DispatchingServerCommunicationModule.java:267) at com.thingworx.communications.platform.modules.DispatchingServerCommunicationModule.process(DispatchingServerCommunicationModule.java:146) at com.thingworx.communications.server.endpoints.DispatchingServerEndpoint.messageReceived(DispatchingServerEndpoint.java:116) at com.thingworx.communications.server.connection.jsr356.Jsr356ServerConnection.onMessage(Jsr356ServerConnection.java:338) at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43 - Locking Thread:
"https-jsse-nio-8443-exec-84" tid=0x77467 in WAITINGBlocked: 2755[-1ms], Waited: 881187[-1ms] User CPU: 38m20s - synchronizer <0x37c62d09> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) - synchronizer <0x3f476c55> (a java.util.concurrent.ThreadPoolExecutor$Worker) at sun.misc.Unsafe.park(Native Method) - waiting on <0x6f66f64> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingDeque.putLast(LinkedBlockingDeque.java:396) at java.util.concurrent.LinkedBlockingDeque.put(LinkedBlockingDeque.java:649) at com.thingworx.persistence.provider.StreamEntryProcessor.queueEntry(StreamEntryProcessor.java:142) at com.thingworx.system.managers.ThingManager$PersistentPropertyCache.put(ThingManager.java:796) at com.thingworx.things.Thing.rawSetPropertyVTQ(Thing.java:4103) at com.thingworx.things.Thing.rawSetPropertyVTQ(Thing.java:4063) at com.thingworx.things.connected.RemoteThing.updateLastConnection(RemoteThing.java:322) at com.thingworx.things.connected.RemoteThing.setConnected(RemoteThing.java:291) at com.thingworx.things.connected.RemoteThing.setCommunicationEndpoint(RemoteThing.java:359) at com.thingworx.things.connected.RemoteThing.bindEndpoint(RemoteThing.java:419) ...
- Blocked Thread
This is a printer-friendly version of Article 321821 and may be out of date. For the latest version click CS321821