Re: IOException: 11/invalid_priority_frame_rate

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: IOException: 11/invalid_priority_frame_rate

Greg Wilkins

Those exceptions are due to a new DOS protection feature introduced as the result of CVE-2019-9512 and associated CVEs.

HTTP2 now has a new jetty.http2.rateControl.maxEventsPerSecond parameter that defaults to 20 per connection for all pings, bad frames, settings frames, priority changes etc.     It may be that 20 is too low for you or that you are under attack?

These are not really ignorable as the connection over which they come is closed, which can be disruptive if these are false positives.

regards





On Wed, 30 Oct 2019 at 21:37, Óscar Frías Barranco <[hidden email]> wrote:
Hi again.

We are also seeing a similar exception which I copy below, any feedback about it too?

java.io.IOException: 11/invalid_ping_frame_rate
        at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:513)
        at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:508)
        at org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:414)
        at org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:384)
        at org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:223)
        at org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:215)
        at org.eclipse.jetty.http2.parser.PingBodyParser.onPing(PingBodyParser.java:99)
        at org.eclipse.jetty.http2.parser.PingBodyParser.parse(PingBodyParser.java:69)
        at org.eclipse.jetty.http2.parser.Parser.parseBody(Parser.java:198)
        at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:127)
        at org.eclipse.jetty.http2.parser.ServerParser.parse(ServerParser.java:115)
        at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:248)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
        at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
        at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.base/java.lang.Thread.run(Thread.java:834)
        Suppressed: java.lang.Throwable: HttpInput failure
                at org.eclipse.jetty.server.HttpInput.failed(HttpInput.java:823)
                at org.eclipse.jetty.http2.server.HttpChannelOverHTTP2.onFailure(HttpChannelOverHTTP2.java:323)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:221)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onSessionFailure(HTTP2ServerConnection.java:261)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:128)
                at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1156)
                ... 29 more

Thanks!
Óscar



On Wed, Oct 30, 2019 at 11:22 AM Óscar Frías Barranco <[hidden email]> wrote:
Hello.

We are randomly seeing this error on some of the requests to our server after we migrated from Jetty 9.4.20 to 9.4.22

What does it mean?  Can we ignore it?

java.io.IOException: 11/invalid_priority_frame_rate
        at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:513)
        at org.eclipse.jetty.http2.HTTP2Session.onConnectionFailure(HTTP2Session.java:508)
        at org.eclipse.jetty.http2.parser.Parser$Listener$Wrapper.onConnectionFailure(Parser.java:414)
        at org.eclipse.jetty.http2.HTTP2Connection$ParserListener.onConnectionFailure(HTTP2Connection.java:384)
        at org.eclipse.jetty.http2.parser.BodyParser.notifyConnectionFailure(BodyParser.java:223)
        at org.eclipse.jetty.http2.parser.BodyParser.connectionFailure(BodyParser.java:215)
        at org.eclipse.jetty.http2.parser.PriorityBodyParser.onPriority(PriorityBodyParser.java:121)
        at org.eclipse.jetty.http2.parser.PriorityBodyParser.parse(PriorityBodyParser.java:106)
        at org.eclipse.jetty.http2.parser.Parser.parseBody(Parser.java:198)
        at org.eclipse.jetty.http2.parser.Parser.parse(Parser.java:127)
        at org.eclipse.jetty.http2.parser.ServerParser.parse(ServerParser.java:115)
        at org.eclipse.jetty.http2.HTTP2Connection$HTTP2Producer.produce(HTTP2Connection.java:248)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
        at org.eclipse.jetty.http2.HTTP2Connection.produce(HTTP2Connection.java:170)
        at org.eclipse.jetty.http2.HTTP2Connection.onFillable(HTTP2Connection.java:125)
        at org.eclipse.jetty.http2.HTTP2Connection$FillableCallback.succeeded(HTTP2Connection.java:348)
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
        at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
        at java.base/java.lang.Thread.run(Thread.java:834)
        Suppressed: java.lang.Throwable: HttpInput failure
                at org.eclipse.jetty.server.HttpInput.failed(HttpInput.java:823)
                at org.eclipse.jetty.http2.server.HttpChannelOverHTTP2.onFailure(HttpChannelOverHTTP2.java:323)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:221)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onSessionFailure(HTTP2ServerConnection.java:261)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:128)
                at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1156)
                ... 29 more
        Suppressed: java.lang.Throwable: HttpInput failure
                at org.eclipse.jetty.server.HttpInput.failed(HttpInput.java:823)
                at org.eclipse.jetty.http2.server.HttpChannelOverHTTP2.onFailure(HttpChannelOverHTTP2.java:323)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:221)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onSessionFailure(HTTP2ServerConnection.java:261)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:128)
                at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1156)
                ... 29 more
        Suppressed: java.lang.Throwable: HttpInput failure
                at org.eclipse.jetty.server.HttpInput.failed(HttpInput.java:823)
                at org.eclipse.jetty.http2.server.HttpChannelOverHTTP2.onFailure(HttpChannelOverHTTP2.java:323)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:221)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onSessionFailure(HTTP2ServerConnection.java:261)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:128)
                at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1156)
                ... 29 more
        Suppressed: java.lang.Throwable: HttpInput failure
                at org.eclipse.jetty.server.HttpInput.failed(HttpInput.java:823)
                at org.eclipse.jetty.http2.server.HttpChannelOverHTTP2.onFailure(HttpChannelOverHTTP2.java:323)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onStreamFailure(HTTP2ServerConnection.java:221)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnection.onSessionFailure(HTTP2ServerConnection.java:261)
                at org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory$HTTPServerSessionListener.onFailure(HTTP2ServerConnectionFactory.java:128)
                at org.eclipse.jetty.http2.HTTP2Session.notifyFailure(HTTP2Session.java:1156)
                ... 29 more

Thanks for your help!
Óscar

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users


--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: IOException: 11/invalid_priority_frame_rate

Greg Wilkins

We are looking at the default... however I think the bigger problem is that we added this limit as per connector rather than per connection.  We will fix that in the next release so the rate limit will scale with bigger servers.

regards



--

_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/jetty-users