HOAB

History of a bug

Springboot OAUTH & PKCS failed & log level of the filter

Rédigé par gorki Aucun commentaire

Problem :

When using Spring Oauth2 resource server, it checks the received token, to do so it must retrieve token validaty or get certificates to validate token. These two calls are usually made in HTTPS for obvious security reason.

When the certificate is not known by the Spring Oauth2 resource server JVM, it fails as SSL handshake can not complete. It fails. Without a single message :) 

Solution :

Put the following log level on : 

org.springframework.security.oauth2.server.resource.web.BearerTokenAuthenticationFilter: DEBUG

I see that the exception is propagate to authenticationFailureHandler I see that it should be handle by authenticationFailureHandler but somewhere in the chain, it's not traced… Could search deeper next time.

It will activate this log : 

            try {
                AuthenticationManager authenticationManager = this.authenticationManagerResolver.resolve(request);
                Authentication authenticationResult = authenticationManager.authenticate(authenticationRequest);
                ...
            } catch (AuthenticationException var11) {
                SecurityContextHolder.clearContext();
                if (debug) {// 134
                    this.logger.debug("Authentication request for failed!", var11);
                }
                this.authenticationFailureHandler.onAuthenticationFailure(request, response, var11);
            }

Keycloak admin stuck on HTTP 204 because of port 443

Rédigé par gorki Aucun commentaire

Problem :

I use Keycloak 19.0.1 behind a proxy (nginx) and wasn't able to connect to the admin part of keycloak.

With a reverse proxy nginx and keycloak, login in admin console lead to be blocked on :

/realms/master/protocol/openid-connect/login-status-iframe.html/init?client_id=security-admin-console ....

With a 204 return code and no other errors. 

Solution :

I had to explore keycloak source code to find the cause ; This test failed in keycloak.js : if ((event.origin !== loginIframe.iframeOrigin) in keycloak.js

After a (lot of) time of search, it appears that it compares : https://mydomain/keycloak and https://mydomain:443/keycloak because I'd setup hostname-port to 443 in keycloak.config.

My keycloak configuration : 

hostname=mydomain  
proxy=reencrypt  
hostname-strict=false  
hostname-port=443  
hostname-path=keycloak  
http-relative-path=keycloak  
hostname-admin-url=https://mydomain/keycloak

So keycloak build his URL as follow : https://mydomain:443/

And the browser send : https://mydomain/ as 443 is a default port and not displayed in the URL.

By removing the port, it works perfectly : 

#hostname-port=443

I open a discussion to improve documentation here

 

Introscope & SAML (saml.jsp)

Rédigé par gorki Aucun commentaire

Problem :

I tried to connect Broadcom Introscope 10.7 and SAML given by Keycloak.

Based on these documents : 

Well not enough to make it works.

Solution :

Thanks to remote debug mode, the key is that the callback URL is : 

https://<webview url>/saml.jsp

Search for saml.jsp + introscope on google. Good luck.

Here are the steps (assuming that you already have a keycloak realm up and ready) : 

Step 1 : IntroscopeEnteprise.properties

introscope.saml.enable=true
introscope.saml.request.binding=POST
introscope.saml.idpUrl=<URL_KEYCLOAK>/realms/<your realm>/protocol/saml
introscope.saml.issuer=com.ca.apm.webview.serviceprovider
introscope.saml.webstart.issuer=com.ca.apm.webstart.serviceprovider
introscope.saml.em.issuer=com.ca.apm.em.serviceprovider
introscope.saml.principalAttributeName=principalName
introscope.saml.groupsAttributeName=groups
introscope.saml.webstart.tokenTimeoutInSeconds=60
introscope.saml.internalIdp.enable=false
# introscope.saml.internalIdpUrl=http://localhost:8080/idp/profile/SAML2/POST/SSO

Step 2 : Keycloak configuration

  1. Create a client named as introscope.saml.issuer so in our case :  com.ca.apm.webstart.serviceprovider
  2. Enter the callback URL in Master SAML Processing URL : https://<webview url>/saml.jsp

Step 3 : Certificates

You should secure you communication between Introscope and Keycloak : 

  1. Provide HTTPS for Keycloak
  2. Provide HTTPS for Introscope
  3. Sign information in Keycloak client
  4. Import Keycloak key in a JKS truststore for Java (Webview part). Keycloak client certificate are in the client definition, tab “Keys”.
    1. Follow Official guide to create the JKS
  5. Point to this truststore (example : spprivatekey.jks) - next steps.

Step 4 : IntroscopeWebview.properties:

apm.webview.saml.sp.truststore=/path/to/spprivatekey.jks

Introscope & Unsupported major.minor version 52.0

Rédigé par gorki Aucun commentaire

Problem :

Launching a JVM with Introscope agent on a old JVM 1.7 result in : 

A problem occurred while attempting to create the delegate agent 
[IntroscopeAgent] Agent Unavailable 

Well, as the agent is heavily customized, removing customization was the first step. It starts. 
OK, a few lambda removing alter, I recompiled everything with Java 1.7 target.. And still the same message. 

Solution : 

After a hours ... 

Step 1 : 

  • Decompile
    • com.wily.introscope.agent.AgentShim
  • Add more logs, it confirms this is a class loading problem with major/minor version. I finally get the class name 

Step 2 : 

  • After checking X times my maven settings, I finally used : *
    • javap -v | grep version
    • to check if it was ok or not. 
  • and surprisingly, the generated class was OK ! 
  • but I used also assembly plugin 

Step 3 : 

  • The cause was that I recompile some classes of an old jar and rebuild it 
  • I tracked the faulty classes version in the different repositories  The root cause was that I installed in a local repository the old jar without the modified classes with the command : 
plaintext mvn install:install-file -DcreateChecksum=true -Dfile=./agent/wily/Agent.jar -DgroupId=com.ca.agent -DartifactId=javaagent -Dversion=10.0.7SP3 -Dpackaging=jar -DlocalRepositoryPath=local-maven-repo

which install it also after build in the global repository... 

And assembly plugin use in priority the global repository. 

Well I didn't take time to understand why maven do not use only my local repository for this jar. 

I just add : rm -rf ~/.m2/repository/com/ca/agent/sqlagent in the beginning of my install script  ! 

Well a few hours lost again here...

Cannot create GC thread but a lot of memory

Rédigé par gorki Aucun commentaire

Problem :

Launching a JVM I have the message : "Cannot create GC thread. Out of system resources"

  • Enough memory
  • Enough swap
  • Enough ulimit
  • Enough threads-max
  • Enough CPU

Event extend the PID limit...

Important (at the end) : debian version = 10.11

Solution :

After a hours of googling, I found :

But none of these solutions works and none was matching the number I had :

  • number of open files < ulimit -n
  • maximum process/tasks < ulimit -u

But in a thread, I found something that was working : UserTasksMax.
I'm running SystemD, I have around 10805 task running for my user.
And from : https://manpages.debian.org/stretch/systemd/logind.conf.5.en.html

UserTasksMax=

Sets the maximum number of OS tasks each user may run concurrently. This controls the TasksMax= setting of the per-user slice unit, see systemd.resource-control(5) for details. If assigned the special value "infinity", no tasks limit is applied. Defaults to 33%, which equals 10813 with the kernel's defaults on the host, but might be smaller in OS containers.

For my suspect PID (a lot of files) :

  • cat /proc/21890/status | grep Thread => 1 thread
  • ls /proc/21890/task | wc
  • confirmed by the usual command : ps -eLf | grep calrisk | wc

I have around 10805 threads running for a given JVM very close to the limit.

Complete guide :

https://www.journaldufreenaute.fr/nombre-maximal-de-threads-par-processus-sous-linux/

Parameters not present in all man page, it could grown up to 12288 on latest version.

To be check !

 

 

Fil RSS des articles de cette catégorie