502 Bad gateway in jetty +dropwizard

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

502 Bad gateway in jetty +dropwizard

satish koritala
one of our production box is started giving 502 Bad Gateway error and below is the error we are getting 


WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,8<=1024<=1024,i=0,q=1024} rejected org.eclipse.jetty.io.AbstractConnection$2@b88f1d0

WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.io.SelectorManager: Could not process key for channel java.nio.channels.SocketChannel[connected local=/172.xx.xx.xx:8080 remote=/172.xx.xx.xx:13361]


We are using jetty 9.2.13 jar. 





_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: 502 Bad gateway in jetty +dropwizard

Greg Wilkins
Satish,

Jetty 9.2 is end of life, plus this is a production issue, so it is difficult for us to give open source support. 
I would suggest either updating to a more recent version of jetty or seeking commercial support for the old version you have in production.

The warnings are not familiar to me, so it is not immediately a known problem.  The "rejected" warning is a bit of a worry?  Have you configured a limited job queue on the QueuedThreadPool?  If so, we  recommend not to do that and to just use the default queue, which is unbounded.

regards



On 9 June 2018 at 18:24, satish koritala <[hidden email]> wrote:
one of our production box is started giving 502 Bad Gateway error and below is the error we are getting 


WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,8<=1024<=1024,i=0,q=1024} rejected org.eclipse.jetty.io.AbstractConnection$2@b88f1d0

WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.io.SelectorManager: Could not process key for channel java.nio.channels.SocketChannel[connected local=/172.xx.xx.xx:8080 remote=/172.xx.xx.xx:13361]


We are using jetty 9.2.13 jar. 





_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users



--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: 502 Bad gateway in jetty +dropwizard

satish koritala
Hello
We are using default values of jetty/dropwizard and not configured any custom param.

On Sat, Jun 9, 2018 at 10:23 PM, Greg Wilkins <[hidden email]> wrote:
Satish,

Jetty 9.2 is end of life, plus this is a production issue, so it is difficult for us to give open source support. 
I would suggest either updating to a more recent version of jetty or seeking commercial support for the old version you have in production.

The warnings are not familiar to me, so it is not immediately a known problem.  The "rejected" warning is a bit of a worry?  Have you configured a limited job queue on the QueuedThreadPool?  If so, we  recommend not to do that and to just use the default queue, which is unbounded.

regards



On 9 June 2018 at 18:24, satish koritala <[hidden email]> wrote:
one of our production box is started giving 502 Bad Gateway error and below is the error we are getting 


WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,8<=1024<=1024,i=0,q=1024} rejected org.eclipse.jetty.io.AbstractConnection$2@b88f1d0

WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.io.SelectorManager: Could not process key for channel java.nio.channels.SocketChannel[connected local=/172.xx.xx.xx:8080 remote=/172.xx.xx.xx:13361]


We are using jetty 9.2.13 jar. 





_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users



--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: 502 Bad gateway in jetty +dropwizard

Greg Wilkins

Satish,

the jetty code that produces that warning from the QueuedThreadPool class in 9.2 is:

        if (!isRunning() || !_jobs.offer(job))
        {
            LOG.warn("{} rejected {}", this, job);
            throw new RejectedExecutionException(job.toString());
        }
        

So either the threadpool has been stopped (that would be a big problem) OR the job queue has rejected the offered job!
By default, jetty uses an unbounded queue.  However, we don't know how dropwizard configures the thread pool and they may not use the defaults?  

So our recommendations remain:
  • upgrade to a recent jetty version
  • don't override the queue type in QueuedThreadPool (or stop DropWizard doing it or who ever is doing it).
  • don't stop the server while running requests!
cheers






On 9 June 2018 at 19:07, satish koritala <[hidden email]> wrote:
Hello
We are using default values of jetty/dropwizard and not configured any custom param.


On Sat, Jun 9, 2018 at 10:23 PM, Greg Wilkins <[hidden email]> wrote:
Satish,

Jetty 9.2 is end of life, plus this is a production issue, so it is difficult for us to give open source support. 
I would suggest either updating to a more recent version of jetty or seeking commercial support for the old version you have in production.

The warnings are not familiar to me, so it is not immediately a known problem.  The "rejected" warning is a bit of a worry?  Have you configured a limited job queue on the QueuedThreadPool?  If so, we  recommend not to do that and to just use the default queue, which is unbounded.

regards



On 9 June 2018 at 18:24, satish koritala <[hidden email]> wrote:
one of our production box is started giving 502 Bad Gateway error and below is the error we are getting 


WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.util.thread.QueuedThreadPool: dw{STARTED,8<=1024<=1024,i=0,q=1024} rejected org.eclipse.jetty.io.AbstractConnection$2@b88f1d0

WARN  [2018-06-09 13:40:32,374] org.eclipse.jetty.io.SelectorManager: Could not process key for channel java.nio.channels.SocketChannel[connected local=/172.xx.xx.xx:8080 remote=/172.xx.xx.xx:13361]


We are using jetty 9.2.13 jar. 





_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users



--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users



--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users