Jetty Refusing HTTP Connections

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Jetty Refusing HTTP Connections

Alastair McMullan
Hi.

I've integrated a customised Jetty HTTP Listener with the JMeter performance
testing tool from Apache to listen for forwarded HTTP messages leaving the
system being tested.

The test environment is:
JMeter is used as a performance test harness submitting test to a "Middleware
Server".
The Middleware Server then forwards the request to its intended recipient, Jetty
intercepts this forward and routes it back into Jmeter to complete the test and
measure performance of the Middleware Server.

For some reason Jetty appears to be randomly rejecting connection requests from
the system under test.

Port snooping utilities provide no further information.
I've checked and confirmed resource levels:
    Maximum available threads = 256,
    Minimum available threads = 10,
    Average number of threads being used is 4
There has never been a demand for more than 10

Has anyone experienced anything similar, or can anyone suggest othere avenues of
investigation.

Thanks in advance
Ali


-------------------------------------------------------
This SF.Net email is sponsored by:
Power Architecture Resource Center: Free content, downloads, discussions,
and more. http://solutions.newsforge.com/ibmarch.tmpl
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support
Reply | Threaded
Open this post in threaded view
|

Re: Jetty Refusing HTTP Connections

Greg Wilkins-5

this sounds very strange?

Perhaps it is the OS that is rejecting the connections if they all come in burst?

Try setting the AcceptQueueSize on the listener.  This controls how many connections
can be in the unaccepted state... ie waiting for Jetty to accept them.

If this is low, then the OS may reject a connection before Jetty get's a chance
to accept it.

cheers



Alastair McMullan wrote:

> Hi.
>
> I've integrated a customised Jetty HTTP Listener with the JMeter performance
> testing tool from Apache to listen for forwarded HTTP messages leaving the
> system being tested.
>
> The test environment is:
> JMeter is used as a performance test harness submitting test to a "Middleware
> Server".
> The Middleware Server then forwards the request to its intended recipient, Jetty
> intercepts this forward and routes it back into Jmeter to complete the test and
> measure performance of the Middleware Server.
>
> For some reason Jetty appears to be randomly rejecting connection requests from
> the system under test.
>
> Port snooping utilities provide no further information.
> I've checked and confirmed resource levels:
>     Maximum available threads = 256,
>     Minimum available threads = 10,
>     Average number of threads being used is 4
> There has never been a demand for more than 10
>
> Has anyone experienced anything similar, or can anyone suggest othere avenues of
> investigation.
>
> Thanks in advance
> Ali
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by:
> Power Architecture Resource Center: Free content, downloads, discussions,
> and more. http://solutions.newsforge.com/ibmarch.tmpl
> _______________________________________________
> Jetty-support mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jetty-support
>



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support
Reply | Threaded
Open this post in threaded view
|

Re: Jetty Refusing HTTP Connections

Alastair McMullan
Thanks Gerg

It sounds like a real posibility. It will be a few days before I get an
opportunity to try this out, i'll post a reply to the group when i try it.

Thanks Again
Ali




Quoting Greg Wilkins <[hidden email]>:

>
> this sounds very strange?
>
> Perhaps it is the OS that is rejecting the connections if they all come
> in burst?
>
> Try setting the AcceptQueueSize on the listener.  This controls how many
> connections
> can be in the unaccepted state... ie waiting for Jetty to accept them.
>
> If this is low, then the OS may reject a connection before Jetty get's a
> chance
> to accept it.
>
> cheers
>
>
>
> Alastair McMullan wrote:
> > Hi.
> >
> > I've integrated a customised Jetty HTTP Listener with the JMeter
> performance
> > testing tool from Apache to listen for forwarded HTTP messages leaving
> the
> > system being tested.
> >
> > The test environment is:
> > JMeter is used as a performance test harness submitting test to a
> "Middleware
> > Server".
> > The Middleware Server then forwards the request to its intended
> recipient, Jetty
> > intercepts this forward and routes it back into Jmeter to complete the
> test and
> > measure performance of the Middleware Server.
> >
> > For some reason Jetty appears to be randomly rejecting connection
> requests from
> > the system under test.
> >
> > Port snooping utilities provide no further information.
> > I've checked and confirmed resource levels:
> >     Maximum available threads = 256,
> >     Minimum available threads = 10,
> >     Average number of threads being used is 4
> > There has never been a demand for more than 10
> >
> > Has anyone experienced anything similar, or can anyone suggest othere
> avenues of
> > investigation.
> >
> > Thanks in advance
> > Ali
> >
> >
> > -------------------------------------------------------
> > This SF.Net email is sponsored by:
> > Power Architecture Resource Center: Free content, downloads,
> discussions,
> > and more. http://solutions.newsforge.com/ibmarch.tmpl
> > _______________________________________________
> > Jetty-support mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/jetty-support
> >
>
>


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.
Get Certified Today * Register for a JBoss Training Course
Free Certification Exam for All Training Attendees Through End of 2005
Visit http://www.jboss.com/services/certification for more information
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support
Reply | Threaded
Open this post in threaded view
|

TCP socket is Resetting Connections

Alastair McMullan
In reply to this post by Greg Wilkins-5
Hi.

I am running Jetty as part of a performance testing environment to Capture what
are Asynchronous server responses.
For some reason and at random intervals when the Server attempts to establish
connection with Jetty's listening port the TCP responds with a Reset message.

This may not be the correct forum for this issue as I dont think Jetty is
causing the problem, but I'm hoping someone may have had a similar experience.


Jetty and the Server under test are on seperate hardware.

When the machine the server is running on is rebooted and server restarted the
problem disappears for a time, but netstat shows no unusual build up of port usage.
Both Jetty and the Server are running on RedHat Active server 3

When Jetty is run against an installation of the server on the same hardware it
operates successfully although i dont have the opportunity to apply the same
volume of testing.

Thanks in advance

Ali



-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support
Reply | Threaded
Open this post in threaded view
|

Re: TCP socket is Resetting Connections

Martin Roos
some ideas

a) maybe you are exceeding your file descriptor limit

b) see your netstat how many dead connections are on linger, maybe decrese
so_longer value or set it off at all

c) backlist on server port too short ?

d) if nothing above helps, make a quick summary about what do you see on
your netstat output concering the jetty sockets (count with grep alive
sockets, dead sockets etc.) and send it back to the list.

e) is there anything else running on the machine ? something that can
exhaust the socket limit ?

martin



> Hi.
>
> I am running Jetty as part of a performance testing environment to Capture
> what
> are Asynchronous server responses.
> For some reason and at random intervals when the Server attempts to
> establish
> connection with Jetty's listening port the TCP responds with a Reset
> message.
>
> This may not be the correct forum for this issue as I dont think Jetty is
> causing the problem, but I'm hoping someone may have had a similar
> experience.
>
>
> Jetty and the Server under test are on seperate hardware.
>
> When the machine the server is running on is rebooted and server restarted
> the
> problem disappears for a time, but netstat shows no unusual build up of
> port usage.
> Both Jetty and the Server are running on RedHat Active server 3
>
> When Jetty is run against an installation of the server on the same
> hardware it
> operates successfully although i dont have the opportunity to apply the
> same
> volume of testing.
>
> Thanks in advance
>
> Ali
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
> Register for a JBoss Training Course.  Free Certification Exam
> for All Training Attendees Through End of 2005. For more info visit:
> http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
> _______________________________________________
> Jetty-support mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jetty-support
>




-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_id=7628&alloc_id=16845&op=click
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support
Reply | Threaded
Open this post in threaded view
|

Am I stopping Jetty correctly

Alastair McMullan-2
In reply to this post by Alastair McMullan
Hi.

I've posted a couple of requests in this last week or so regarding Jetty
refusing Connections by sending a TCP reset packed on receiving a
Connection request.

Investigations (and suggestions) has lead me down the route that I may
not be closing connections and stopping Jetty correctly.

I am running Jetty as part of a Test application measuring the
performance of an asynchronous Middleware service. Basically Jetty is
used to listen for the asynchronous activity.

I've checked in the FAQ and Tutorials for Jetty, they give specific
examples as to how to start Jetty as part of an application but not for
stopping it.

I've attached my code for starting and stopping Jetty, Am I doing
anything obvious which would cause either Connections to be left open or
Jetty process not to be stopped.

STARTING JETTY

            listeningServer = new HttpServer();
           
            // Create and add a port listener
            SocketListener listener=new SocketListener();
            listener.setPort(Integer.parseInt(this.getResponsePort()));
            listeningServer.addListener(listener);
           
            // Create a context
            HttpContext context = new
AsyncReplyHandler(asyncResponsePool, getRequestProtocol());
            context.setContextPath("/");
            listeningServer.addContext(context);
           
            // Start the server
            listeningServer.start ();
             
            // give the server time to start
            while(!listeningServer.isStarted()){}
            serverStarted = true;

REQUEST HANDLING CODE AND CONNECTION CLOSURE

        public void handle(String pathInContext, String pathParams,
            HttpRequest request, HttpResponse response) {
        try{
            processAsyncReply(request);
            response.setStatus(HttpResponse.__200_OK);
            PrintWriter out = new PrintWriter(response.getOutputStream());
           
            out.print(ACKNOWLEDGEMENT_RESPONSE);
           
            out.flush();
            out.close();
            response.getOutputStream().flush();
            response.getOutputStream().close();
        }
        catch (IllegalMonitorStateException imse){
...
...


STOPPING JETTY

        try{
            listeningServer.stop(true);
        }catch(InterruptedException ie){
            //I'm Stopping the server dont care about interrupting threads.
        }
       
        while(listeningServer.isStarted()){}
       
        listeningServer.destroy();
        serverStarted = false;

Thanks in advance
Alastair





-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support