SelectChannelConnector and maxIdleTimeout

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

SelectChannelConnector and maxIdleTimeout

thebig1
Hello,

I am using Jetty as a standalone web server, serving a combination of .jsp and file resources.  I found a problem when downloading large files from Jetty which take over 30 seconds: the file download quits (looking as if it was complete), although the file is cut off at whatever point it got to.  Using a download tool like curl allows you to finish the download from that spot.

Doing some troubleshooting brought me to find that the maxIdleTimeout value in SelectChannelConnector is what causes this.  By default it is set to 30000ms (30 seconds).  If any download exceeds this time, the connection is broken.  By reducing this value to 3000 (3 seconds) I can reproduce the same problem in that time span.

While I don't know if this is a bug or not, it is definitely unexpected.  I assume that once the download has started, Jetty does not count ongoing data transfer as 'active' time, however, this is counter intuitive.  An obvious workaround is to increase the timeout value to some arbitrarily high value, like 10 minutes, but that will prevent true idle connections from timing out.  On a high traffic site, this is obviously not ideal.

Does anyone have a suggestion which I may have missed, or some other configuration option I can try?

FYI, I have included my jetty.xml file below.  It is based off of the default one, with sections which I don't need removed, and a few extra items added.

Cheers


-----Begin file attachment-----
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="Server" class="org.mortbay.jetty.Server">

   
   
   
    <Set name="ThreadPool">
     
      <New class="org.mortbay.thread.QueuedThreadPool">
        <Set name="minThreads">10</Set>
        <Set name="maxThreads">50</Set>
        <Set name="lowThreads">5</Set>
        <Set name="SpawnOrShrinkAt">2</Set>
      </New>
    </Set>



   
   
   
   
   

       
    <Call name="addConnector">
      <Arg>
          <New class="org.mortbay.jetty.nio.SelectChannelConnector">
            <Set name="host"></Set>
            <Set name="port">8080</Set>
            <Set name="maxIdleTime">600000</Set>
            <Set name="Acceptors">2</Set>
            <Set name="statsOn">false</Set>
            <Set name="confidentialPort">8443</Set>
            <Set name="lowResourcesConnections">5000</Set>
            <Set name="lowResourcesMaxIdleTime">5000</Set>
          </New>
      </Arg>
    </Call>


   
     
   
    <Set name="handler">
      <New id="Handlers" class="org.mortbay.jetty.handler.HandlerCollection">
        <Set name="handlers">
         <Array type="org.mortbay.jetty.Handler">
           <Item>
             <New id="Contexts" class="org.mortbay.jetty.handler.ContextHandlerCollection"/>
           </Item>
           <Item>
             <New id="DefaultHandler" class="org.mortbay.jetty.handler.DefaultHandler"/>
           </Item>
           <Item>
             <New id="RequestLog" class="org.mortbay.jetty.handler.RequestLogHandler"/>
           </Item>
         </Array>
        </Set>
      </New>
    </Set>
   
   
   
   
   
   
   
   
   
   
   
   
    <Call name="addLifeCycle">
      <Arg>
        <New class="org.mortbay.jetty.deployer.ContextDeployer">
          <Set name="contexts"><Ref id="Contexts"/></Set>
          <Set name="configurationDir">./www</Set>
          <Set name="scanInterval">1</Set>
        </New>
      </Arg>
    </Call>

   
   
   
   
   
   
   
    <Ref id="RequestLog">
        <Set name="requestLog">
         <New id="RequestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
           <Set name="filename">./logs/general-yyyy_mm_dd.request.log</Set>
                <Set name="filenameDateFormat">yyyy_MM_dd</Set>
                <Set name="retainDays">14</Set>
                <Set name="append">true</Set>
                <Set name="extended">true</Set>
           <Set name="logCookies">false</Set>
            </New>
        </Set>
    </Ref>
   
               
               
               
               
               
               
    <New id="StdErr" class="java.io.PrintStream">
      <Arg>
        <New class="org.mortbay.util.RolloverFileOutputStream">
          <Arg>./logs/stderr-yyyy_mm_dd.log</Arg>
          <Arg type="boolean">true</Arg>
          <Arg type="int">14</Arg>
          <Get id="ServerLogName" name="datedFilename"/>
        </New>
      </Arg>
    </New>
    <New id="StdOut" class="java.io.PrintStream">
      <Arg>
        <New class="org.mortbay.util.RolloverFileOutputStream">
          <Arg>./logs/stdout-yyyy_mm_dd.log</Arg>
          <Arg type="boolean">true</Arg>
          <Arg type="int">14</Arg>
          <Get id="ServerLogName" name="datedFilename"/>
        </New>
      </Arg>
    </New>   

    <Call class="org.mortbay.log.Log" name="info"><Arg>Redirecting stderr/stdout to disk</Arg></Call>
    <Call class="java.lang.System" name="setErr"><Arg><Ref id="StdErr"/></Arg></Call>
    <Call class="java.lang.System" name="setOut"><Arg><Ref id="StdOut"/></Arg></Call>   

   
   
   
    <Set name="stopAtShutdown">true</Set>
    <Set name="sendServerVersion">true</Set>
    <Set name="sendDateHeader">true</Set>
    <Set name="gracefulShutdown">1000</Set>

</Configure>

-----End file attachment-----
Reply | Threaded
Open this post in threaded view
|

Re: SelectChannelConnector and maxIdleTimeout

valli nanduri
which version of jetty are you using? and what is the expected load to your server per day?


thebig1 wrote
Hello,

I am using Jetty as a standalone web server, serving a combination of .jsp and file resources.  I found a problem when downloading large files from Jetty which take over 30 seconds: the file download quits (looking as if it was complete), although the file is cut off at whatever point it got to.  Using a download tool like curl allows you to finish the download from that spot.

Doing some troubleshooting brought me to find that the maxIdleTimeout value in SelectChannelConnector is what causes this.  By default it is set to 30000ms (30 seconds).  If any download exceeds this time, the connection is broken.  By reducing this value to 3000 (3 seconds) I can reproduce the same problem in that time span.

While I don't know if this is a bug or not, it is definitely unexpected.  I assume that once the download has started, Jetty does not count ongoing data transfer as 'active' time, however, this is counter intuitive.  An obvious workaround is to increase the timeout value to some arbitrarily high value, like 10 minutes, but that will prevent true idle connections from timing out.  On a high traffic site, this is obviously not ideal.

Does anyone have a suggestion which I may have missed, or some other configuration option I can try?

FYI, I have included my jetty.xml file below.  It is based off of the default one, with sections which I don't need removed, and a few extra items added.

Cheers


-----Begin file attachment-----
<?xml version="1.0"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">

<Configure id="Server" class="org.mortbay.jetty.Server">

   
   
   
    <Set name="ThreadPool">
     
      <New class="org.mortbay.thread.QueuedThreadPool">
        <Set name="minThreads">10</Set>
        <Set name="maxThreads">50</Set>
        <Set name="lowThreads">5</Set>
        <Set name="SpawnOrShrinkAt">2</Set>
      </New>
    </Set>



   
   
   
   
   

       
    <Call name="addConnector">
      <Arg>
          <New class="org.mortbay.jetty.nio.SelectChannelConnector">
            <Set name="host"></Set>
            <Set name="port">8080</Set>
            <Set name="maxIdleTime">600000</Set>
            <Set name="Acceptors">2</Set>
            <Set name="statsOn">false</Set>
            <Set name="confidentialPort">8443</Set>
            <Set name="lowResourcesConnections">5000</Set>
            <Set name="lowResourcesMaxIdleTime">5000</Set>
          </New>
      </Arg>
    </Call>


   
     
   
    <Set name="handler">
      <New id="Handlers" class="org.mortbay.jetty.handler.HandlerCollection">
        <Set name="handlers">
         <Array type="org.mortbay.jetty.Handler">
           <Item>
             <New id="Contexts" class="org.mortbay.jetty.handler.ContextHandlerCollection"/>
           </Item>
           <Item>
             <New id="DefaultHandler" class="org.mortbay.jetty.handler.DefaultHandler"/>
           </Item>
           <Item>
             <New id="RequestLog" class="org.mortbay.jetty.handler.RequestLogHandler"/>
           </Item>
         </Array>
        </Set>
      </New>
    </Set>
   
   
   
   
   
   
   
   
   
   
   
   
    <Call name="addLifeCycle">
      <Arg>
        <New class="org.mortbay.jetty.deployer.ContextDeployer">
          <Set name="contexts"><Ref id="Contexts"/></Set>
          <Set name="configurationDir">./www</Set>
          <Set name="scanInterval">1</Set>
        </New>
      </Arg>
    </Call>

   
   
   
   
   
   
   
    <Ref id="RequestLog">
        <Set name="requestLog">
         <New id="RequestLogImpl" class="org.mortbay.jetty.NCSARequestLog">
           <Set name="filename">./logs/general-yyyy_mm_dd.request.log</Set>
                <Set name="filenameDateFormat">yyyy_MM_dd</Set>
                <Set name="retainDays">14</Set>
                <Set name="append">true</Set>
                <Set name="extended">true</Set>
           <Set name="logCookies">false</Set>
            </New>
        </Set>
    </Ref>
   
               
               
               
               
               
               
    <New id="StdErr" class="java.io.PrintStream">
      <Arg>
        <New class="org.mortbay.util.RolloverFileOutputStream">
          <Arg>./logs/stderr-yyyy_mm_dd.log</Arg>
          <Arg type="boolean">true</Arg>
          <Arg type="int">14</Arg>
          <Get id="ServerLogName" name="datedFilename"/>
        </New>
      </Arg>
    </New>
    <New id="StdOut" class="java.io.PrintStream">
      <Arg>
        <New class="org.mortbay.util.RolloverFileOutputStream">
          <Arg>./logs/stdout-yyyy_mm_dd.log</Arg>
          <Arg type="boolean">true</Arg>
          <Arg type="int">14</Arg>
          <Get id="ServerLogName" name="datedFilename"/>
        </New>
      </Arg>
    </New>   

    <Call class="org.mortbay.log.Log" name="info"><Arg>Redirecting stderr/stdout to disk</Arg></Call>
    <Call class="java.lang.System" name="setErr"><Arg><Ref id="StdErr"/></Arg></Call>
    <Call class="java.lang.System" name="setOut"><Arg><Ref id="StdOut"/></Arg></Call>   

   
   
   
    <Set name="stopAtShutdown">true</Set>
    <Set name="sendServerVersion">true</Set>
    <Set name="sendDateHeader">true</Set>
    <Set name="gracefulShutdown">1000</Set>

</Configure>

-----End file attachment-----