cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

CONTROLLER Cannot Login

Tadeu.Alves
Creator

Hi there,

 

Today my Controller got stuck with high CPU usage, i restarted it and now for some weird reason no one can login i can only acess the /controller/admin.jsp

 

Inside the server log i got a lot of entries like this:

 

[#|2021-12-06T22:39:37.411-0300|INFO|glassfish 4.1|javax.enterprise.system.core.security|_ThreadID=69;_ThreadName=http-listener-1(13);_TimeMillis=1638841177411;_LevelValue=800;_MessageID=NCLS-SECURITY-05046;|Audit: Authentication refused for [singularity-agent@customer1].|#]

[#|2021-12-06T22:39:37.412-0300|INFO|glassfish 4.1|javax.enterprise.system.core.security.com.sun.enterprise.security.jmac.callback|_ThreadID=69;_ThreadName=http-listener-1(13);_TimeMillis=1638841177412;_LevelValue=800;|jmac.loginfail|#]

 

I even tried to create a new Account but now when i log in everything is blank.. in the controller did i break something?

Is it possible to recover what i "destroyed"

Any help is appreciated.

2 REPLIES 2

Tadeu.Alves
Creator

Guys the server is "comming back" i did nothing and all of sudden some parts of the configuration are showing, but i don't see nothing of the instrumentation that i did and when i log in i get this error: 

Error
500 Internal Server Error

Exception Id:b13eec1b-bc08-463d-b77f-19f881a04c67

 

Looking inside the server.log i see many SEVERE errors, all of them are:

 

Unable to decrypt AccountAccess Key.
com.singularity.ee.util.security.credentialstore.exception.CryptoOperationFailureException: Failed to decrypt: Input length must be multiple of 16 when decrypting with padded cipher


Here is one of the many stacks that show the same error only different threads:

 

com.singularity.ee.controller.entity.AccountEntity|_ThreadID=1010;_ThreadName=config-caches-refresh-54;_TimeMillis=1638886343390;_LevelValue=1000;|Unable to decrypt AccountAccess Key.
com.singularity.ee.util.security.credentialstore.exception.CryptoOperationFailureException: Failed to decrypt: Input length must be multiple of 16 when decrypting with padded cipher
at com.singularity.ee.util.security.credentialstore.KeyStoreBasedEncryptionWrapper.decryptCipherTextString(KeyStoreBasedEncryptionWrapper.java:97)
at com.singularity.ee.util.scs.SCSDecryptionUtilityManager.decryptCipherText(SCSDecryptionUtilityManager.java:86)
at com.singularity.ee.controller.entity.AccountEntity.scsOperationsDoToDTO(AccountEntity.java:716)
at com.singularity.ee.controller.entity.AccountEntity.postLoadLifeCycleEvent(AccountEntity.java:705)
at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.hibernate.ejb.event.BeanCallback.invoke(BeanCallback.java:37)
at org.hibernate.ejb.event.EntityCallbackHandler.callback(EntityCallbackHandler.java:94)
at org.hibernate.ejb.event.EntityCallbackHandler.postLoad(EntityCallbackHandler.java:87)
at org.hibernate.ejb.event.EJB3PostLoadEventListener.onPostLoad(EJB3PostLoadEventListener.java:47)
at org.hibernate.event.def.DefaultLoadEventListener.assembleCacheEntry(DefaultLoadEventListener.java:700)
at org.hibernate.event.def.DefaultLoadEventListener.loadFromSecondLevelCache(DefaultLoadEventListener.java:605)
at org.hibernate.event.def.DefaultLoadEventListener.doLoad(DefaultLoadEventListener.java:459)
at org.hibernate.event.def.DefaultLoadEventListener.load(DefaultLoadEventListener.java:227)
at org.hibernate.event.def.DefaultLoadEventListener.onLoad(DefaultLoadEventListener.java:147)
at org.hibernate.impl.SessionImpl.fireLoad(SessionImpl.java:1090)
at org.hibernate.impl.SessionImpl.immediateLoad(SessionImpl.java:1026)
at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:176)
at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:215)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:190)
at com.singularity.ee.controller.entity.AccountEntity_$$_jvst12b_108.isMarkedAsDeleted(AccountEntity_$$_jvst12b_108.java)
at com.appdynamics.platform.apm.service.backend.BackendConfigCacheServiceImpl.refreshCache(BackendConfigCacheServiceImpl.java:380)
at com.appdynamics.platform.apm.service.backend.BackendConfigCacheServiceImpl$$EnhancerByGuice$$2ddbec59.CGLIB$refreshCache$1(<generated>)
at com.appdynamics.platform.apm.service.backend.BackendConfigCacheServiceImpl$$EnhancerByGuice$$2ddbec59$$FastClassByGuice$$8ce8ba83.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:75)
at com.appdynamics.platform.persistence.TransactionInterceptor.lambda$invoke$0(TransactionInterceptor.java:37)
at com.singularity.ee.controller.beans.model.EJBManagerBean.runWithinSupportsTransaction(EJBManagerBean.java:23)
at sun.reflect.GeneratedMethodAccessor116.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1081)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1153)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4786)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:656)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:824)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.appdynamics.platform.persistence.EntityManagerInterceptor.call(EntityManagerInterceptor.java:45)
at com.appdynamics.platform.persistence.GlassfishEntityManagerInterceptor.manageEntityManagerLifecycle(GlassfishEntityManagerInterceptor.java:25)
at sun.reflect.GeneratedMethodAccessor94.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.appdynamics.controller.persistence.ControllerExceptionHandlingInterceptor.call(ControllerExceptionHandlingInterceptor.java:71)
at com.appdynamics.controller.persistence.GlassfishControllerExceptionHandlingInterceptor.handleExceptions(GlassfishControllerExceptionHandlingInterceptor.java:18)
at sun.reflect.GeneratedMethodAccessor93.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:608)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at sun.reflect.GeneratedMethodAccessor92.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:883)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:823)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:369)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4758)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4746)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at com.sun.proxy.$Proxy340.runWithinSupportsTransaction(Unknown Source)
at com.appdynamics.platform.persistence.GlassfishPersistenceManager.runWithinTransactionAttributeBlock(GlassfishPersistenceManager.java:35)
at com.appdynamics.platform.persistence.TransactionInterceptor.invoke(TransactionInterceptor.java:33)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:75)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:54)
at com.appdynamics.platform.apm.service.backend.BackendConfigCacheServiceImpl$$EnhancerByGuice$$2ddbec59.refreshCache(<generated>)
at com.singularity.ee.controller.beans.ControllerManager.lambda$getCacheRefreshForAppTasks$8(ControllerManager.java:2657)
at com.appdynamics.platform.services.threads.ExecutorRegistry$ReportMetricsCallable.call(ExecutorRegistry.java:474)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.crypto.IllegalBlockSizeException: Input length must be multiple of 16 when decrypting with padded cipher
at com.sun.crypto.provider.CipherCore.prepareInputBuffer(CipherCore.java:1005)
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:848)
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)
at javax.crypto.Cipher.doFinal(Cipher.java:2168)
at com.singularity.ee.util.security.credentialstore.impl.EncryptionSchemeLegacy.decrypt(EncryptionSchemeLegacy.java:68)
at com.singularity.ee.util.security.credentialstore.EncryptionCommon.decryptCipherTextString(EncryptionCommon.java:102)
at com.singularity.ee.util.security.credentialstore.KeyStoreBasedEncryptionWrapper.decryptCipherTextString(KeyStoreBasedEncryptionWrapper.java:94)
... 77 more

 

Hi @Tadeu.Alves,

 

This seems like a critical issue, I would contact Support for more immediate help. If you do and support helps you, can you please share what you learned here?

 

Knowledge sharing is what drives this community forward. 

 

 


Thanks,

Ryan, Cisco AppDynamics Community Manager




Found something helpful? Click the Accept as Solution button to help others find answers faster.

Liked something? Click the Thumbs Up button.



Check out Observabiity in Action

new deep dive videos weekly in the Knowledge Base.