[jetty-dev] Less Jetty Throughput

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[jetty-dev] Less Jetty Throughput

Mohamed Razeed
Hi,

We are using Jetty 9.4.5 

Tired to benchmark the blank end point (which will just return hello world message) and got the Throughput 11000 requests per second.

But when tried to benchmark our real end point where our business logic exists in same environment, achieved the Throughput 2300 requests per second. To achieve more throughout, we tried to increase queue size but no use. Please help me on this.

Each request taking 50 to 100 ms to process

Our application running on VM: 6 CPU and 8gb ram allocated to our application.

Regards
Razeed

_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev
Reply | Threaded
Open this post in threaded view
|

Re: [jetty-dev] Less Jetty Throughput

Joakim Erdfelt-8
Things to check ...

* Are you using 100% of the CPU? (common bottleneck)
* Are you using 100% of the network capacity? (common bottleneck)
* Are you testing with multiple http clients (potentially on different machines on the same network)?
* Have you tweaked any defaults? (leave everything at default, test, test again, get numbers, THEN change the defaults to suit your needs, and keep testing and gathering numbers even in production so you can anticipate changes)
* Are you logging your GC behavior/activity? (this should be done to see if your performance hit is from excessive memory usage and bad GC behavior) 

Considering that you only hit 11,000 requests a second on a black endpoint I think you don't have enough clients performing your testing.
Localhost testing (to avoid bottleneck that network testing introduces) on a laptop from 2012 can hit 80,000 requests a second doing testing against a jetty instance running cometd with configured endpoints.

We have a large range of users that can easily saturate their network with Jetty (can't go any faster then what the network can handle).

From Original HTC G1 Android device (500Mhz CPU with 128MB of RAM) serving over wifi at 54Mbit/s
Through an Old Raspberry Pi's (1.6Ghz quad core ARM CPU on 1GB of RAM) serving over usb/wifi at 110Mbit/s
To commodity hardware intel i5 CPUs with 8GB of RAM serving at 1Gbit/s speeds
To modern hardware 64 core machines with 128GB of RAM serving at 10Gbit/s speeds
All the way to specialty Compute Engines with 200+ cores and 1TB of RAM serving at Gen 6/7 FibreChannel speeds of 50Gbit/s (not sure if this is saturation or not, but it's darned fast)

People use Jetty for a large and wide spectrum uses, from the small and barely used to large production installations at Fortune 500 companies and even to places where even we are surprised (NASA, high speed trading groups, various well respected international research facilities, dynamic testing infrastructures, etc)

Joakim Erdfelt / [hidden email]


On Thu, Feb 28, 2019 at 8:32 AM Mohamed Razeed <[hidden email]> wrote:
Hi,

We are using Jetty 9.4.5 

Tired to benchmark the blank end point (which will just return hello world message) and got the Throughput 11000 requests per second.

But when tried to benchmark our real end point where our business logic exists in same environment, achieved the Throughput 2300 requests per second. To achieve more throughout, we tried to increase queue size but no use. Please help me on this.

Each request taking 50 to 100 ms to process

Our application running on VM: 6 CPU and 8gb ram allocated to our application.

Regards
Razeed
_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev

_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev
Reply | Threaded
Open this post in threaded view
|

Re: [jetty-dev] Less Jetty Throughput

Guillaume Maillard
In reply to this post by Mohamed Razeed
Hi,

In order to help you, we need some data, please use a tool like JProfiler to provide us a complete state of the CPU and memory usage.

Regards,

Guillaume

Le jeu. 28 févr. 2019 à 14:32, Mohamed Razeed <[hidden email]> a écrit :
Hi,

We are using Jetty 9.4.5 

Tired to benchmark the blank end point (which will just return hello world message) and got the Throughput 11000 requests per second.

But when tried to benchmark our real end point where our business logic exists in same environment, achieved the Throughput 2300 requests per second. To achieve more throughout, we tried to increase queue size but no use. Please help me on this.

Each request taking 50 to 100 ms to process

Our application running on VM: 6 CPU and 8gb ram allocated to our application.

Regards
Razeed
_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev

_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev
Reply | Threaded
Open this post in threaded view
|

Re: [jetty-dev] Less Jetty Throughput

Greg Wilkins

Note also that benchmarks that return requests per second are seldom actually benchmarking what you think.

Consider a server that can handle 1000 parallel connections, but each request takes 10ms.  The max throughput of that server is 100,000 requests per second.   However if you open 1 connection and run it as fast as possible, you will only measure 100 requests per second.  If you open 100 connections and run them as fast as possible, you will still only measure 10,000 request per second.

On the other hand, I server that can only handle 10 parallel connections, but each request takes 1ms has a max throughput of 10,000, but will measure 10x better than the previous server in small numbers of connections.    You are measuring latency not throughput!

Finally a server that can server 100,000,000,000 request per second, but each request takes 10 seconds is pretty useless!

Rather than offering as much load as possible (which often stresses your client more than the server) and measuring throughput, it is a far better test to offer load at a fixed rate and measure latency (ave, medium and max).  If you want to know the max through put, then keep increasing load offered while the measured latency is within an acceptable quality of service.  



On Fri, 1 Mar 2019 at 04:04, Guillaume Maillard <[hidden email]> wrote:
Hi,

In order to help you, we need some data, please use a tool like JProfiler to provide us a complete state of the CPU and memory usage.

Regards,

Guillaume

Le jeu. 28 févr. 2019 à 14:32, Mohamed Razeed <[hidden email]> a écrit :
Hi,

We are using Jetty 9.4.5 

Tired to benchmark the blank end point (which will just return hello world message) and got the Throughput 11000 requests per second.

But when tried to benchmark our real end point where our business logic exists in same environment, achieved the Throughput 2300 requests per second. To achieve more throughout, we tried to increase queue size but no use. Please help me on this.

Each request taking 50 to 100 ms to process

Our application running on VM: 6 CPU and 8gb ram allocated to our application.

Regards
Razeed
_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev
_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev


--

_______________________________________________
jetty-dev mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-dev