[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

JIRA jira@codehaus.org
Issue Type: Bug Bug
Affects Versions: 6.1.26
Assignee: Unassigned
Created: 20/Sep/12 11:13 PM
Description:

Similar to the bug seen in JETTY-1264, we saw a single instance of a deadlock between SelectorManager.doStop and Timeout.tick. The issue is that the Timeout object is trying to lock the SelectorManager, but the SelectorManager is spinning waiting for the Selector thread to die while holding onto the lock.

Project: Jetty
Priority: Major Major
Reporter: Todd Lipcon
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Reply | Threaded
Open this post in threaded view
|

[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

JIRA jira@codehaus.org
Todd Lipcon commented on Bug JETTY-1546

Here is the trace:

"main" prio=10 tid=0x0000000040a99000 nid=0x318b runnable [0x00007f02d30ac000]
java.lang.Thread.State: RUNNABLE
at org.mortbay.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:929)
locked <0x000000050de2de48> (a org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.io.nio.SelectorManager.doStop(SelectorManager.java:241)
at org.mortbay.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:76)
locked <0x000000050def8488> (a java.lang.Object)
at org.mortbay.jetty.nio.SelectChannelConnector.close(SelectChannelConnector.java:136)
locked <0x000000050de2df38> (a org.mortbay.jetty.nio.SelectChannelConnector)
at org.apache.hadoop.http.HttpServer.stop(HttpServer.java:751)
"1987744292@qtp-664112322-1 - Acceptor0 SelectChannelConnector@0.0.0.0:60931" prio=10 tid=0x00000000413d2800 nid=0x322b waiting for monitor entry [0x00007f02c3052000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.mortbay.thread.Timeout.tick(Timeout.java:132)
waiting to lock <0x000000050de2de48> (a org.mortbay.io.nio.SelectorManager$SelectSet)
at org.mortbay.thread.Timeout.tick(Timeout.java:153)
at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:784)
at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:193)
at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

The line numbers won't match 6.1.26 exactly, since this is a branch based on some code that Greg W provided a while back. The actual branch is here: https://github.com/cloudera/jetty-hadoop-fix/

The race is exceedingly rare - I'd estimate about one in a million. I saw it during some overnight stress tests which happen to start and stop hundreds of Jetty servers per node, on 100 nodes in parallel.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Reply | Threaded
Open this post in threaded view
|

[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org
Greg Wilkins commented on Bug JETTY-1546

Thanks for raising this, but as has been discussed before we are unlikely to address this issue. It looks to only be rarely when stopping the server and jetty-6 is long depreciated.

Jetty-6 is only going to be updated now with commercial support. We do want to help out other open source projects, but just don't have the resources to work on a long deprecated release.

Jetty-9 is out soon and is smaller, lighter, faster, etc etc. Once it is stable, I'd be keen to work with the hadoop project to chase down all your transitive dependencies and work with them all to upgrade. But note that jetty-6 is in an entirely different package space, so 6 can cohabit with 7, 8, or 9

cheers

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Reply | Threaded
Open this post in threaded view
|

[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org
Todd Lipcon commented on Bug JETTY-1546

Thanks, Greg. I'll continue to ignore the race, then - never seen it in production, since in practice we never stop our servers except in test cases

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email
Reply | Threaded
Open this post in threaded view
|

[jira] (JETTY-1546) Deadlock between Timeout.tick and SelectorManager.stop

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org
Jan Bartel closed Bug JETTY-1546 as Won't Fix

Seems like both parties are happy if we close this, at least for now, with a won't fix. If this bug reappears in jetty-9, please do just reopen it.

Jan

Change By: Jan Bartel (15/Oct/12 1:21 AM)
Resolution: Won't Fix
Status: Open Closed
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators.
For more information on JIRA, see: http://www.atlassian.com/software/jira
--------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email