Zxchat Suddenly not...
 
Notifications
Clear all

Zxchat Suddenly not running and buddy chat lost on all mailbox servers

1 Posts
1 Users
0 Likes
298 Views
(@dhirmansyah)
Posts: 3
Topic starter
 

Hi All,

I just wondering, maybe some of you have experienced with this issue, today our zxchat suddenly not running and all buddy list chat disappear on all our mailbox servers, and this is 2 times already.

we found temporary workaround for this issue is : we restarted mailbox service on one of our mailbox server and then zxchat service normal and buddy list appear on all mailboxs servers.

Below our log err :

2017-08-21 05:15:42,735 INFO [qtp1684106402-2415795: http://groupmail.example.com/service/soap/ZxChatRequest ] [] extensions - Hazelcast: java.util.concurrent.TimeoutExce
ption: Call BasicInvocation{ serviceName='hz:impl:mapService', op=com.hazelcast.spi.impl.IsStillExecutingOperation{serviceName='hz:impl:mapService', callId=830865, invo
cationTime=1503267315588, waitTimeout=-1, callTimeout=5000}, partitionId=-1, replicaIndex=0, tryCount=0, tryPauseMillis=0, invokeCount=1, callTimeout=5000, target=Addre
ss[zimb-mailbox3.example.com]:8736, backupsExpected=0, backupsCompleted=0} encountered a timeout
at com.hazelcast.spi.impl.BasicInvocationFuture.resolveApplicationResponse (BasicInvocationFuture.java:352)
at com.hazelcast.spi.impl.BasicInvocationFuture.resolveApplicationResponseOrThrowException (BasicInvocationFuture.java:320)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:194)
at com.hazelcast.spi.impl.BasicInvocationFuture.isOperationExecuting (BasicInvocationFuture.java:421)
at com.hazelcast.spi.impl.BasicInvocationFuture.waitForResponse (BasicInvocationFuture.java:244)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:193)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:173)
at com.hazelcast.map.impl.proxy.MapProxySupport.containsKeyInternal (MapProxySupport.java:566)
at com.hazelcast.map.impl.proxy.MapProxyImpl.containsKey (MapProxyImpl.java:237)
at com.zextras.lib.hazelcast.HazelcastInstanceAccessor.contains (HazelcastInstanceAccessor.java:107)
at com.zextras.lib.hazelcast.HazelcastServiceAccessor.selectStoreAccessor (HazelcastServiceAccessor.java:86)
at com.zextras.lib.hazelcast.HazelcastServiceAccessor.getNullable (HazelcastServiceAccessor.java:175)
at com.zextras.lib.provisioning.HazelcastObjectAccessor.getEntry (HazelcastObjectAccessor.java:89)
at com.zextras.lib.provisioning.AttributeReaderAggregator.getEntry (AttributeReaderAggregator.java:27)
at com.zextras.lib.provisioning.AttributeReaderAggregator.getEntry (AttributeReaderAggregator.java:27)
at com.zextras.lib.provisioning.Attribute.read (Attribute.java:108)
at com.zextras.lib.provisioning.AccountConfig.getValue (AccountConfig.java:72)
at com.zextras.lib.provisioning.AccountConfig.isHistoryEnabled (AccountConfig.java:124)
at com.zextras.newchat.events.EventSoapSessionRegistred.isHistoryEnabled (EventSoapSessionRegistred.java:63)
at com.zextras.newchat.soap.encoders.EventSoapSessionRegistredEncoder.encode (EventSoapSessionRegistredEncoder.java:27)
at com.zextras.newchat.operations.RegisterSoapSession.exec (RegisterSoapSession.java:92)
at com.zextras.newchat.events.EventManager.execOperations (EventManager.java:54)
at com.zextras.newchat.soap.InitialSoapRequestHandler.handleRequest (InitialSoapRequestHandler.java:139)
at com.zextras.newchat.soap.ChatSoapHandler.handleRequest (ChatSoapHandler.java:65)
at com.zextras.newchat.soap.ChatSoapHandler.handleRequest (ChatSoapHandler.java:65)
at org.openzal.zal.soap.InternalDocumentHelper.handle (InternalDocumentHelper.java:75)
at org.openzal.zal.soap.InternalDocumentHandler.handle (InternalDocumentHandler.java:42)
at com.zimbra.soap.SoapEngine.dispatchRequest (SoapEngine.java:605)
at com.zimbra.soap.SoapEngine.dispatch (SoapEngine.java:458)
at com.zimbra.soap.SoapEngine.dispatch (SoapEngine.java:271)
at com.zimbra.soap.SoapServlet.doWork (SoapServlet.java:303)
at com.zimbra.soap.SoapServlet.doPost (SoapServlet.java:213)
at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)
at com.zimbra.cs.servlet.ZimbraServlet.service (ZimbraServlet.java:206)
at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:821)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1685)
at com.zimbra.cs.servlet.CsrfFilter.doFilter (CsrfFilter.java:158)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.RequestStringFilter.doFilter (RequestStringFilter.java:54)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.SetHeaderFilter.doFilter (SetHeaderFilter.java:59)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter (ETagHeaderFilter.java:47)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter (ContextPathBasedThreadPoolBalancerFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter (ZimbraQoSFilter.java:116)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter (ZimbraInvalidLoginFilter.java:123)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at org.eclipse.jetty.servlets.DoSFilter.doFilterChain (DoSFilter.java:473)
at org.eclipse.jetty.servlets.DoSFilter.doFilter (DoSFilter.java:318)
at org.eclipse.jetty.servlets.DoSFilter.doFilter (DoSFilter.java:288)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle (SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle (SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1158)
at org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope (SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1090)
at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle (HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:119)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle (RewriteHandler.java:318)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle (GzipHandler.java:437)
at org.eclipse.jetty.server.handler.DebugHandler.handle (DebugHandler.java:84)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:119)
at org.eclipse.jetty.server.Server.handle (Server.java:517)
at org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:306)
at org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:242)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:261)
at org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run (SelectChannelEndPoint.java:75)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run (QueuedThreadPool.java:572)
at java.lang.Thread.run (Thread.java:745)
2017-08-21 05:19:32,575 INFO [hz.mailbox04.example.com.response] [] extensions - Hazelcast: [mailbox04.example.com]:8736 [dev] [3.4.2] While processing response...
2017-08-21 05:19:32,575 INFO [hz.mailbox04.example.com.response] [] extensions - Hazelcast: com.hazelcast.core.HazelcastException: No invocation for response: NormalResponse{callId=830865, urgent=true, value=false, backupCount=0}
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.notifyRemoteCall (BasicOperationService.java:618)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.handle (BasicOperationService.java:602)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.access$1300 (BasicOperationService.java:595)
at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch (BasicOperationService.java:580)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.process (BasicOperationScheduler.java:532)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.doRun (BasicOperationScheduler.java:524)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.run (BasicOperationScheduler.java:501)

2017-08-21 05:15:42,735 INFO [qtp1684106402-2415795: http://groupmail.example.com/service/soap/ZxChatRequest ] [] extensions - Hazelcast: java.util.concurrent.TimeoutExce
ption: Call BasicInvocation{ serviceName='hz:impl:mapService', op=com.hazelcast.spi.impl.IsStillExecutingOperation{serviceName='hz:impl:mapService', callId=830865, invo
cationTime=1503267315588, waitTimeout=-1, callTimeout=5000}, partitionId=-1, replicaIndex=0, tryCount=0, tryPauseMillis=0, invokeCount=1, callTimeout=5000, target=Addre
ss[zimb-mailbox3.example.com]:8736, backupsExpected=0, backupsCompleted=0} encountered a timeout
at com.hazelcast.spi.impl.BasicInvocationFuture.resolveApplicationResponse (BasicInvocationFuture.java:352)
at com.hazelcast.spi.impl.BasicInvocationFuture.resolveApplicationResponseOrThrowException (BasicInvocationFuture.java:320)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:194)
at com.hazelcast.spi.impl.BasicInvocationFuture.isOperationExecuting (BasicInvocationFuture.java:421)
at com.hazelcast.spi.impl.BasicInvocationFuture.waitForResponse (BasicInvocationFuture.java:244)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:193)
at com.hazelcast.spi.impl.BasicInvocationFuture.get (BasicInvocationFuture.java:173)
at com.hazelcast.map.impl.proxy.MapProxySupport.containsKeyInternal (MapProxySupport.java:566)
at com.hazelcast.map.impl.proxy.MapProxyImpl.containsKey (MapProxyImpl.java:237)
at com.zextras.lib.hazelcast.HazelcastInstanceAccessor.contains (HazelcastInstanceAccessor.java:107)
at com.zextras.lib.hazelcast.HazelcastServiceAccessor.selectStoreAccessor (HazelcastServiceAccessor.java:86)
at com.zextras.lib.hazelcast.HazelcastServiceAccessor.getNullable (HazelcastServiceAccessor.java:175)
at com.zextras.lib.provisioning.HazelcastObjectAccessor.getEntry (HazelcastObjectAccessor.java:89)
at com.zextras.lib.provisioning.AttributeReaderAggregator.getEntry (AttributeReaderAggregator.java:27)
at com.zextras.lib.provisioning.AttributeReaderAggregator.getEntry (AttributeReaderAggregator.java:27)
at com.zextras.lib.provisioning.Attribute.read (Attribute.java:108)
at com.zextras.lib.provisioning.AccountConfig.getValue (AccountConfig.java:72)
at com.zextras.lib.provisioning.AccountConfig.isHistoryEnabled (AccountConfig.java:124)
at com.zextras.newchat.events.EventSoapSessionRegistred.isHistoryEnabled (EventSoapSessionRegistred.java:63)
at com.zextras.newchat.soap.encoders.EventSoapSessionRegistredEncoder.encode (EventSoapSessionRegistredEncoder.java:27)
at com.zextras.newchat.operations.RegisterSoapSession.exec (RegisterSoapSession.java:92)
at com.zextras.newchat.events.EventManager.execOperations (EventManager.java:54)
at com.zextras.newchat.soap.InitialSoapRequestHandler.handleRequest (InitialSoapRequestHandler.java:139)
at com.zextras.newchat.soap.ChatSoapHandler.handleRequest (ChatSoapHandler.java:65)
at com.zextras.newchat.soap.ChatSoapHandler.handleRequest (ChatSoapHandler.java:65)
at org.openzal.zal.soap.InternalDocumentHelper.handle (InternalDocumentHelper.java:75)
at org.openzal.zal.soap.InternalDocumentHandler.handle (InternalDocumentHandler.java:42)
at com.zimbra.soap.SoapEngine.dispatchRequest (SoapEngine.java:605)
at com.zimbra.soap.SoapEngine.dispatch (SoapEngine.java:458)
at com.zimbra.soap.SoapEngine.dispatch (SoapEngine.java:271)
at com.zimbra.soap.SoapServlet.doWork (SoapServlet.java:303)
at com.zimbra.soap.SoapServlet.doPost (SoapServlet.java:213)
at javax.servlet.http.HttpServlet.service (HttpServlet.java:707)
at com.zimbra.cs.servlet.ZimbraServlet.service (ZimbraServlet.java:206)
at javax.servlet.http.HttpServlet.service (HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle (ServletHolder.java:821)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1685)
at com.zimbra.cs.servlet.CsrfFilter.doFilter (CsrfFilter.java:158)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.RequestStringFilter.doFilter (RequestStringFilter.java:54)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.SetHeaderFilter.doFilter (SetHeaderFilter.java:59)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ETagHeaderFilter.doFilter (ETagHeaderFilter.java:47)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ContextPathBasedThreadPoolBalancerFilter.doFilter (ContextPathBasedThreadPoolBalancerFilter.java:107)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ZimbraQoSFilter.doFilter (ZimbraQoSFilter.java:116)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at com.zimbra.cs.servlet.ZimbraInvalidLoginFilter.doFilter (ZimbraInvalidLoginFilter.java:123)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at org.eclipse.jetty.servlets.DoSFilter.doFilterChain (DoSFilter.java:473)
at org.eclipse.jetty.servlets.DoSFilter.doFilter (DoSFilter.java:318)
at org.eclipse.jetty.servlets.DoSFilter.doFilter (DoSFilter.java:288)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1668)
at org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:581)
at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle (SecurityHandler.java:524)
at org.eclipse.jetty.server.session.SessionHandler.doHandle (SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle (ContextHandler.java:1158)
at org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope (SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope (ContextHandler.java:1090)
at org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle (ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle (HandlerCollection.java:109)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:119)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle (RewriteHandler.java:318)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle (GzipHandler.java:437)
at org.eclipse.jetty.server.handler.DebugHandler.handle (DebugHandler.java:84)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle (HandlerWrapper.java:119)
at org.eclipse.jetty.server.Server.handle (Server.java:517)
at org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:306)
at org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:242)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded (AbstractConnection.java:261)
at org.eclipse.jetty.io.FillInterest.fillable (FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run (SelectChannelEndPoint.java:75)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob (QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run (QueuedThreadPool.java:572)
at java.lang.Thread.run (Thread.java:745)
2017-08-21 05:19:32,575 INFO [hz.mailbox04.example.com.response] [] extensions - Hazelcast: [mailbox04.example.com]:8736 [dev] [3.4.2] While processing response...
2017-08-21 05:19:32,575 INFO [hz.mailbox04.example.com.response] [] extensions - Hazelcast: com.hazelcast.core.HazelcastException: No invocation for response: NormalResponse{callId=830865, urgent=true, value=false, backupCount=0}
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.notifyRemoteCall (BasicOperationService.java:618)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.handle (BasicOperationService.java:602)
at com.hazelcast.spi.impl.BasicOperationService$ResponsePacketHandler.access$1300 (BasicOperationService.java:595)
at com.hazelcast.spi.impl.BasicOperationService$BasicDispatcherImpl.dispatch (BasicOperationService.java:580)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.process (BasicOperationScheduler.java:532)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.doRun (BasicOperationScheduler.java:524)
at com.hazelcast.spi.impl.BasicOperationScheduler$ResponseThread.run (BasicOperationScheduler.java:501)

 
Posted : 08/21/2017 05:05