javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hello all,

I am using Jetty 9.4.14 on multiple servers. On one of my servers I get
heaps of SSLHandshakeException errors. They occur with different domain
names for which I do have valid certificates in my keystore. I am using
Jetty SNI and have dozens of certificates in my keystore. I use the same
keystore (JKS format) on all my servers but only one server shows this
behaviour. Strangely enough, these errors only occur with requests that
are sent from Java applications, either from the server process itself
or from one of my other servers.
This started occurring about a week ago, long after I upgraded to Jetty
9.4.14. The only thing that changed in the meantime is the SSL-keystore
that has grown.

Does this ring any bells? Has anyone experienced similar problems? I
have tried restarting the process, server etc. but that only helps for a
short while.

Any pointers would be welcome.

Kind regards,

Silvio
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Greg Wilkins

Silvio,

no bell ringing.   
Any chance of a stack trace... it may be obvious, but as a guide to walk through the code it may make us see something.
Of course if you could run with -Djavax.net.debug=ssl (or a filtered version of that) would be even more helpful.

regards



On Mon, 17 Dec 2018 at 22:17, Silvio Bierman <[hidden email]> wrote:
Hello all,

I am using Jetty 9.4.14 on multiple servers. On one of my servers I get
heaps of SSLHandshakeException errors. They occur with different domain
names for which I do have valid certificates in my keystore. I am using
Jetty SNI and have dozens of certificates in my keystore. I use the same
keystore (JKS format) on all my servers but only one server shows this
behaviour. Strangely enough, these errors only occur with requests that
are sent from Java applications, either from the server process itself
or from one of my other servers.
This started occurring about a week ago, long after I upgraded to Jetty
9.4.14. The only thing that changed in the meantime is the SSL-keystore
that has grown.

Does this ring any bells? Has anyone experienced similar problems? I
have tried restarting the process, server etc. but that only helps for a
short while.

Any pointers would be welcome.

Kind regards,

Silvio
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hi Greg,

I have done some extensive testing, logging etc. to try and figure out what is going wrong here. I managed to get quite some stack traces but only to find out that the error occurs on the client side, which is in all cases a Java11 process using the standard HttpURLConnection to evaluate a URL that targets a Jetty 9.4.14 server that is embedded inside again a Java11 process. In many cases they are the same process, ie. the client code runs as part of a web-application that sometimes communicates with itself. But in also quite some cases they are separate processes running on different servers. Since the error does not show up on the server side I have no stack traces that contain anything familiar to you, so I will fully understand if you can not help me here.

Invariably this is what it looks like:

javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching xxx.yyy.com found.
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1329)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1204)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1151)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    ...
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching tangram.jambo-mobile.com found.
    at java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:207)
    at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:98)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:459)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:434)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:233)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1313)
    ... 100 more

I am a grateful user of Jetty SNI support with about 300 certificates (not counting intermediates) in my keystore, which is currently still in JKS format.

What dazzles me is that everything works flawlessly most of the time. But then something triggers this on one of our servers and for a period of time it will keep reoccurring for one of the domain names. During that period people are using the web-application through their browser without any issues unless they try to do something that depends on a self-served request (like having the system send out an email which lets JavaMail evaluate a URL that lands in the same application) which then fails as subscribed. This makes me suspect my server goes into some kind of state that makes it hand out the wrong certificates, incomplete chains or even no certificate at all. But then again, I am just guessing here. Perhaps a have some rotten certificate in my store that messes up the SNI process.

Jetty version: jetty-distribution-9.4.14.v20181114, Java version: OpenJDK 64-Bit Server VM (build 11.0.1+13-Ubuntu-2ubuntu1, mixed mode, sharing), OS: Ubuntu 18.10 server (for both client and server processes).

If anyone can give me a pointer I would be very grateful.

Kind regards,

Silvio


On 17-12-18 21:48, Greg Wilkins wrote:

Silvio,

no bell ringing.   
Any chance of a stack trace... it may be obvious, but as a guide to walk through the code it may make us see something.
Of course if you could run with -Djavax.net.debug=ssl (or a filtered version of that) would be even more helpful.

regards



On Mon, 17 Dec 2018 at 22:17, Silvio Bierman <[hidden email]> wrote:
Hello all,

I am using Jetty 9.4.14 on multiple servers. On one of my servers I get
heaps of SSLHandshakeException errors. They occur with different domain
names for which I do have valid certificates in my keystore. I am using
Jetty SNI and have dozens of certificates in my keystore. I use the same
keystore (JKS format) on all my servers but only one server shows this
behaviour. Strangely enough, these errors only occur with requests that
are sent from Java applications, either from the server process itself
or from one of my other servers.
This started occurring about a week ago, long after I upgraded to Jetty
9.4.14. The only thing that changed in the meantime is the SSL-keystore
that has grown.

Does this ring any bells? Has anyone experienced similar problems? I
have tried restarting the process, server etc. but that only helps for a
short while.

Any pointers would be welcome.

Kind regards,

Silvio
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hi all,

One addition: this morning I replaced the keystore file on one of the servers because some almost-expired certificates had been updated and subsequently triggered a SslContextFactory.reload through the application. Within 15 minutes the logging showed about two dozen failed requests. Then it silently went away. May be a coincidence of course.

Silvio


On 16-01-19 12:30, Silvio Bierman wrote:
Hi Greg,

I have done some extensive testing, logging etc. to try and figure out what is going wrong here. I managed to get quite some stack traces but only to find out that the error occurs on the client side, which is in all cases a Java11 process using the standard HttpURLConnection to evaluate a URL that targets a Jetty 9.4.14 server that is embedded inside again a Java11 process. In many cases they are the same process, ie. the client code runs as part of a web-application that sometimes communicates with itself. But in also quite some cases they are separate processes running on different servers. Since the error does not show up on the server side I have no stack traces that contain anything familiar to you, so I will fully understand if you can not help me here.

Invariably this is what it looks like:

javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching xxx.yyy.com found.
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1329)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1204)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1151)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    ...
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching tangram.jambo-mobile.com found.
    at java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:207)
    at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:98)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:459)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:434)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:233)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1313)
    ... 100 more

I am a grateful user of Jetty SNI support with about 300 certificates (not counting intermediates) in my keystore, which is currently still in JKS format.

What dazzles me is that everything works flawlessly most of the time. But then something triggers this on one of our servers and for a period of time it will keep reoccurring for one of the domain names. During that period people are using the web-application through their browser without any issues unless they try to do something that depends on a self-served request (like having the system send out an email which lets JavaMail evaluate a URL that lands in the same application) which then fails as subscribed. This makes me suspect my server goes into some kind of state that makes it hand out the wrong certificates, incomplete chains or even no certificate at all. But then again, I am just guessing here. Perhaps a have some rotten certificate in my store that messes up the SNI process.

Jetty version: jetty-distribution-9.4.14.v20181114, Java version: OpenJDK 64-Bit Server VM (build 11.0.1+13-Ubuntu-2ubuntu1, mixed mode, sharing), OS: Ubuntu 18.10 server (for both client and server processes).

If anyone can give me a pointer I would be very grateful.

Kind regards,

Silvio


On 17-12-18 21:48, Greg Wilkins wrote:

Silvio,

no bell ringing.   
Any chance of a stack trace... it may be obvious, but as a guide to walk through the code it may make us see something.
Of course if you could run with -Djavax.net.debug=ssl (or a filtered version of that) would be even more helpful.

regards



On Mon, 17 Dec 2018 at 22:17, Silvio Bierman <[hidden email]> wrote:
Hello all,

I am using Jetty 9.4.14 on multiple servers. On one of my servers I get
heaps of SSLHandshakeException errors. They occur with different domain
names for which I do have valid certificates in my keystore. I am using
Jetty SNI and have dozens of certificates in my keystore. I use the same
keystore (JKS format) on all my servers but only one server shows this
behaviour. Strangely enough, these errors only occur with requests that
are sent from Java applications, either from the server process itself
or from one of my other servers.
This started occurring about a week ago, long after I upgraded to Jetty
9.4.14. The only thing that changed in the meantime is the SSL-keystore
that has grown.

Does this ring any bells? Has anyone experienced similar problems? I
have tried restarting the process, server etc. but that only helps for a
short while.

Any pointers would be welcome.

Kind regards,

Silvio
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hi all,

Sorry for spamming but I have another observation after further log file analysis: the problem only seems to happen with hostnames that are covered by wildcard certificates. I noticed some comments about a change to Jetty regarding SNI behavior and wildcard certs shortly before the release of 9.4.14. I know I am grasping at straws here but this all started for us after switching to 9.4.14 (and JDK11 at almost the same time...) but to narrow things down I will:

- run one server on JDK9 and Jetty 9.4.14
- run another with JDK11 and Jetty 9.4.13

and see for a while if either of those does not cause problems. If so I will post the results here.

For now I have disabled client side certificate validation for all requests that are sent to the application itself. This definitely suppresses the issue, which is not surprising of course. For now this is probably harmless.

Cheers,

Silvio


On 16-01-19 12:53, Silvio Bierman wrote:
Hi all,

One addition: this morning I replaced the keystore file on one of the servers because some almost-expired certificates had been updated and subsequently triggered a SslContextFactory.reload through the application. Within 15 minutes the logging showed about two dozen failed requests. Then it silently went away. May be a coincidence of course.

Silvio


On 16-01-19 12:30, Silvio Bierman wrote:
Hi Greg,

I have done some extensive testing, logging etc. to try and figure out what is going wrong here. I managed to get quite some stack traces but only to find out that the error occurs on the client side, which is in all cases a Java11 process using the standard HttpURLConnection to evaluate a URL that targets a Jetty 9.4.14 server that is embedded inside again a Java11 process. In many cases they are the same process, ie. the client code runs as part of a web-application that sometimes communicates with itself. But in also quite some cases they are separate processes running on different servers. Since the error does not show up on the server side I have no stack traces that contain anything familiar to you, so I will fully understand if you can not help me here.

Invariably this is what it looks like:

javax.net.ssl.SSLHandshakeException: No subject alternative DNS name matching xxx.yyy.com found.
    at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:128)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:259)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1329)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1204)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1151)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    ...
Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching tangram.jambo-mobile.com found.
    at java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:207)
    at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:98)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:459)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:434)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:233)
    at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129)
    at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1313)
    ... 100 more

I am a grateful user of Jetty SNI support with about 300 certificates (not counting intermediates) in my keystore, which is currently still in JKS format.

What dazzles me is that everything works flawlessly most of the time. But then something triggers this on one of our servers and for a period of time it will keep reoccurring for one of the domain names. During that period people are using the web-application through their browser without any issues unless they try to do something that depends on a self-served request (like having the system send out an email which lets JavaMail evaluate a URL that lands in the same application) which then fails as subscribed. This makes me suspect my server goes into some kind of state that makes it hand out the wrong certificates, incomplete chains or even no certificate at all. But then again, I am just guessing here. Perhaps a have some rotten certificate in my store that messes up the SNI process.

Jetty version: jetty-distribution-9.4.14.v20181114, Java version: OpenJDK 64-Bit Server VM (build 11.0.1+13-Ubuntu-2ubuntu1, mixed mode, sharing), OS: Ubuntu 18.10 server (for both client and server processes).

If anyone can give me a pointer I would be very grateful.

Kind regards,

Silvio


On 17-12-18 21:48, Greg Wilkins wrote:

Silvio,

no bell ringing.   
Any chance of a stack trace... it may be obvious, but as a guide to walk through the code it may make us see something.
Of course if you could run with -Djavax.net.debug=ssl (or a filtered version of that) would be even more helpful.

regards



On Mon, 17 Dec 2018 at 22:17, Silvio Bierman <[hidden email]> wrote:
Hello all,

I am using Jetty 9.4.14 on multiple servers. On one of my servers I get
heaps of SSLHandshakeException errors. They occur with different domain
names for which I do have valid certificates in my keystore. I am using
Jetty SNI and have dozens of certificates in my keystore. I use the same
keystore (JKS format) on all my servers but only one server shows this
behaviour. Strangely enough, these errors only occur with requests that
are sent from Java applications, either from the server process itself
or from one of my other servers.
This started occurring about a week ago, long after I upgraded to Jetty
9.4.14. The only thing that changed in the meantime is the SSL-keystore
that has grown.

Does this ring any bells? Has anyone experienced similar problems? I
have tried restarting the process, server etc. but that only helps for a
short while.

Any pointers would be welcome.

Kind regards,

Silvio
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Greg Wilkins
Silvio,

I am reading your emails... but so far I've had no idea pop into my head.

The only thing I can think of is perhaps replacing the SslContextFactory with exactly the code from 9.4.12 (I think 13 was a bad release for other reasons) and see if that makes any difference.  If it works, then you could probably bisect the commits (only about 8 done last year).

cheers




On Tue, 29 Jan 2019 at 11:51, Silvio Bierman <[hidden email]> wrote:
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hi Greg,

I would love to do some experimenting with the code but have never built my own Jetty before. I merely download the releases when they become available. I have little to no experience with Git and none at all with Maven (I am ashamed to admit that we still use SVN and Ant). Perhaps a good time to get started. I will look into the documentation and see where that takes me,

Thanks so far.

Kind regards,

Silvio

On 29-01-19 08:30, Greg Wilkins wrote:
Silvio,

I am reading your emails... but so far I've had no idea pop into my head.

The only thing I can think of is perhaps replacing the SslContextFactory with exactly the code from 9.4.12 (I think 13 was a bad release for other reasons) and see if that makes any difference.  If it works, then you could probably bisect the commits (only about 8 done last year).

cheers




On Tue, 29 Jan 2019 at 11:51, Silvio Bierman <[hidden email]> wrote:
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
In reply to this post by Greg Wilkins
Hi Greg,

I just want to let you all know that I have reproduced the same issue with a server running on Jetty 9.4.11.v20180605. I triggered an SslContextFactory.reload on the server and immediately got the same error on the client.

Since this version by far predates the first occurrence of the issue on our servers I am now convinced this is something that was introduced with our move from JDK10 (actually JDK11 early builds on Ubuntu 18.04) to JDK11 (and Ubuntu 18.10) which coincided with our move from Jetty 9.4.13 to 9.4.14. Now we know the Jetty version is clearly not causing this it must be something related to JDK11.

Since it is triggered by manipulating the server I am still looking in that direction. My next step will be testing against a server (with Jetty 9.4.14) running on JDK8. I will keep you posted.

Kind regards,

Silvio

On 29-01-19 08:30, Greg Wilkins wrote:
Silvio,

I am reading your emails... but so far I've had no idea pop into my head.

The only thing I can think of is perhaps replacing the SslContextFactory with exactly the code from 9.4.12 (I think 13 was a bad release for other reasons) and see if that makes any difference.  If it works, then you could probably bisect the commits (only about 8 done last year).

cheers




On Tue, 29 Jan 2019 at 11:51, Silvio Bierman <[hidden email]> wrote:
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hello all,

Another follow up on the same issue:

I have not yet set up a server on JDK8 but I have tried some other variation to further narrow things down. Using -Djdk.tls.client.protocols=TLSv1.2 on the client side of the connection forces the use of TLSv1.2 (as opposed to 1.3 which is the JDK11 default) and in that case everything works fine. The server still uses 1.3 if available and therefore still fails for 1.3 clients at the same time. But the 1.2 clients work without issues.

My next try will be running the server with TLSv1.2 but I expect that will simply suppress the issue for all clients.

I do not know if this is useful information at all. I have read somewhere that the 1.3 code in JDK11 results in a different ordering of some callbacks to the code managing server sockets but I have too little experience there to make any educated guesses.

I will return soon with more information.

Kind regards,

Silvio


On 30-01-19 00:48, Silvio Bierman wrote:
Hi Greg,

I just want to let you all know that I have reproduced the same issue with a server running on Jetty 9.4.11.v20180605. I triggered an SslContextFactory.reload on the server and immediately got the same error on the client.

Since this version by far predates the first occurrence of the issue on our servers I am now convinced this is something that was introduced with our move from JDK10 (actually JDK11 early builds on Ubuntu 18.04) to JDK11 (and Ubuntu 18.10) which coincided with our move from Jetty 9.4.13 to 9.4.14. Now we know the Jetty version is clearly not causing this it must be something related to JDK11.

Since it is triggered by manipulating the server I am still looking in that direction. My next step will be testing against a server (with Jetty 9.4.14) running on JDK8. I will keep you posted.

Kind regards,

Silvio

On 29-01-19 08:30, Greg Wilkins wrote:
Silvio,

I am reading your emails... but so far I've had no idea pop into my head.

The only thing I can think of is perhaps replacing the SslContextFactory with exactly the code from 9.4.12 (I think 13 was a bad release for other reasons) and see if that makes any difference.  If it works, then you could probably bisect the commits (only about 8 done last year).

cheers




On Tue, 29 Jan 2019 at 11:51, Silvio Bierman <[hidden email]> wrote:
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: javax.net.ssl.SSLHandshakeException No subject alternative DNS name matching xxx found.

Silvio Bierman
Hello all,

I finally tested this with a server (Jetty 9.4.14) running on JDK8 and running on JDK11 with -Djdk.tls.server.protocols=TLSv1.2. With both configurations I was unable to trigger this issue. For sake of completeness I also tried JDK11 with default TLS settings and a non-wildcard certificate. Same result there.

From all this I conclude that a JDK11 client with TLSv1.3 enabled (which is the default) intermittently fails to complete a HTTPS request to a Jetty server running on JD11 also with TLSv1.3 enabled if the used domain name is covered by a wildcard certificate. Forcing a SslContext#reload on the server immediately triggers the issue after which it sometimes disappears within a couple of minutes but I have also observed the issue to persist for multiple hours. The issue may also occur spontaneously and in that case it also takes up to several hours to go away. The issue has been reproduced with Jetty versions 9.4.11, 9.4.13 and 9.4.14.

Running the server with TLSv1.3 disabled seems to be the best way to suppress the issue for now.

Kind regards,

Silvio


On 01-02-19 16:32, Silvio Bierman wrote:
Hello all,

Another follow up on the same issue:

I have not yet set up a server on JDK8 but I have tried some other variation to further narrow things down. Using -Djdk.tls.client.protocols=TLSv1.2 on the client side of the connection forces the use of TLSv1.2 (as opposed to 1.3 which is the JDK11 default) and in that case everything works fine. The server still uses 1.3 if available and therefore still fails for 1.3 clients at the same time. But the 1.2 clients work without issues.

My next try will be running the server with TLSv1.2 but I expect that will simply suppress the issue for all clients.

I do not know if this is useful information at all. I have read somewhere that the 1.3 code in JDK11 results in a different ordering of some callbacks to the code managing server sockets but I have too little experience there to make any educated guesses.

I will return soon with more information.

Kind regards,

Silvio


On 30-01-19 00:48, Silvio Bierman wrote:
Hi Greg,

I just want to let you all know that I have reproduced the same issue with a server running on Jetty 9.4.11.v20180605. I triggered an SslContextFactory.reload on the server and immediately got the same error on the client.

Since this version by far predates the first occurrence of the issue on our servers I am now convinced this is something that was introduced with our move from JDK10 (actually JDK11 early builds on Ubuntu 18.04) to JDK11 (and Ubuntu 18.10) which coincided with our move from Jetty 9.4.13 to 9.4.14. Now we know the Jetty version is clearly not causing this it must be something related to JDK11.

Since it is triggered by manipulating the server I am still looking in that direction. My next step will be testing against a server (with Jetty 9.4.14) running on JDK8. I will keep you posted.

Kind regards,

Silvio

On 29-01-19 08:30, Greg Wilkins wrote:
Silvio,

I am reading your emails... but so far I've had no idea pop into my head.

The only thing I can think of is perhaps replacing the SslContextFactory with exactly the code from 9.4.12 (I think 13 was a bad release for other reasons) and see if that makes any difference.  If it works, then you could probably bisect the commits (only about 8 done last year).

cheers




On Tue, 29 Jan 2019 at 11:51, Silvio Bierman <[hidden email]> wrote:
Hello all,

Another followup on the same topic: triggering a
SslContextFactory.reload on the server consistently and immediately
triggers the problem on the client side, restarting the server is close
to 100% (seems timing related). I was still leaning toward something
fishy in the client code or even the JDK11 SSL client socket code but
now I am almost certain this is going awry on the server side.

Still JDK11 on both client and server side and Jetty 9.4.14.v20181114
server, using domain names that are covered by wildcard certificates.

I am busy setting up a server with 9.4.11 and JDK8 to see what happens
there but since I am packed it may take another week or so to get results.

I will keep you posted.



>> One addition: this morning I replaced the keystore file on one of the
>> servers because some almost-expired certificates had been updated and
>> subsequently triggered a SslContextFactory.reload through the
>> application. Within 15 minutes the logging showed about two dozen
>> failed requests. Then it silently went away. May be a coincidence of
>> course.
>>
>> Silvio
>>

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users