out of memory error using GZIPOutputStream

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

out of memory error using GZIPOutputStream

jian chen
Hi,

This strange problem has been bugging me for 2 days. I have the
following GZIPOutputStream on top of the servlet response
outputstrream to render the html to the browser.

      String encodings = req.getHeader("Accept-Encoding");    
      if (encodings != null && encodings.indexOf("gzip") != -1) // GZIP
      {
         resp.setHeader("Content-Encoding", "gzip");
         GZIPOutputStream gos = new GZIPOutputStream(resp.getOutputStream());
         OutputStreamWriter out = new OutputStreamWriter(gos, "UTF8");
         evaluate(s, ctx, out);
         out.flush();
         gos.finish();
      }
      else // No compression
      {
         resp.setCharacterEncoding("UTF8");
         evaluate(s, ctx, resp.getWriter());
      }

The evaluate(...) method is for Velocity template engine to generate the html.

So, when the velocity template page is huge, then, I get a out of memory error.

Any ideas?

Thanks,

Jian


-------------------------------------------------------
SF.Net email is sponsored by: Tell us your software development plans!
Take this survey and enter to win a one-year sub to SourceForge.net
Plus IDC's 2005 look-ahead and a copy of this survey
Click here to start!  <a href="http://www.idcswdc.com/cgi-bin/survey?id5hix">http://www.idcswdc.com/cgi-bin/survey?id5hix
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: out of memory error using GZIPOutputStream

Russell Howe
jian chen wrote:
> Hi,
>
> This strange problem has been bugging me for 2 days. I have the
> following GZIPOutputStream on top of the servlet response

[jian]

> So, when the velocity template page is huge, then, I get a out of memory error.
>
> Any ideas?

Increase the amount of memory available to Java? I guess
GZIPOutputStream isn't capable of compressing on the fly, and buffers
the whole input before gzipping it?

--
Russell Howe
[hidden email]

Today's Nemi: http://www.metro.co.uk/img/pix/nemi_apr27.jpg


-------------------------------------------------------
SF.Net email is sponsored by: Tell us your software development plans!
Take this survey and enter to win a one-year sub to SourceForge.net
Plus IDC's 2005 look-ahead and a copy of this survey
Click here to start!  http://www.idcswdc.com/cgi-bin/survey?id=105hix
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: out of memory error using GZIPOutputStream

jian chen
Thanks for your suggestions. I looked at the jdk source code for
GZIPOutputStream implementation, and it seems to me that they also use
a buffer internally. And as I recall, GZIP works in zipping data in
chunks. So, it should send out a chunk once it is done rather than
buffering them all.

Jian

On 4/27/05, Russell Howe <[hidden email]> wrote:

> jian chen wrote:
> > Hi,
> >
> > This strange problem has been bugging me for 2 days. I have the
> > following GZIPOutputStream on top of the servlet response
>
> [jian]
>
> > So, when the velocity template page is huge, then, I get a out of memory error.
> >
> > Any ideas?
>
> Increase the amount of memory available to Java? I guess
> GZIPOutputStream isn't capable of compressing on the fly, and buffers
> the whole input before gzipping it?
>
> --
> Russell Howe
> [hidden email]
>
> Today's Nemi: http://www.metro.co.uk/img/pix/nemi_apr27.jpg
>
> -------------------------------------------------------
> SF.Net email is sponsored by: Tell us your software development plans!
> Take this survey and enter to win a one-year sub to SourceForge.net
> Plus IDC's 2005 look-ahead and a copy of this survey
> Click here to start!  http://www.idcswdc.com/cgi-bin/survey?id=105hix
> _______________________________________________
> jetty-discuss mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/jetty-discuss
>


-------------------------------------------------------
SF.Net email is sponsored by: Tell us your software development plans!
Take this survey and enter to win a one-year sub to SourceForge.net
Plus IDC's 2005 look-ahead and a copy of this survey
Click here to start!  <a href="http://www.idcswdc.com/cgi-bin/survey?id5hix">http://www.idcswdc.com/cgi-bin/survey?id5hix
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: out of memory error using GZIPOutputStream

jian chen
Hi,

I got more information regarding the memory issue. The stack trace is:

java.lang.OutOfMemoryError: (class: org/mortbay/http/EOFException)
        at org.mortbay.http.BufferedOutputStream.flush(BufferedOutputStream.java:206)
        at org.mortbay.util.ByteBufferOutputStream.ensureSpareCapacity(ByteBufferOutputStream.java:357)
        at org.mortbay.util.ByteBufferOutputStream.write(ByteBufferOutputStream.java:197)
        at org.mortbay.http.BufferedOutputStream.write(BufferedOutputStream.java:146)
        at org.mortbay.http.HttpOutputStream.write(HttpOutputStream.java:423)
        at org.mortbay.jetty.servlet.ServletOut.write(ServletOut.java:54)
        at java.util.zip.DeflaterOutputStream.deflate(Unknown Source)
        at java.util.zip.DeflaterOutputStream.write(Unknown Source)
        at java.util.zip.GZIPOutputStream.write(Unknown Source)
        at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(Unknown Source)
        at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(Unknown Source)
        at sun.nio.cs.StreamEncoder.write(Unknown Source)

So, could it be a jetty issue?

Thanks,

Jian


On 4/27/05, jian chen <[hidden email]> wrote:

> Thanks for your suggestions. I looked at the jdk source code for
> GZIPOutputStream implementation, and it seems to me that they also use
> a buffer internally. And as I recall, GZIP works in zipping data in
> chunks. So, it should send out a chunk once it is done rather than
> buffering them all.
>
> Jian
>
> On 4/27/05, Russell Howe <[hidden email]> wrote:
> > jian chen wrote:
> > > Hi,
> > >
> > > This strange problem has been bugging me for 2 days. I have the
> > > following GZIPOutputStream on top of the servlet response
> >
> > [jian]
> >
> > > So, when the velocity template page is huge, then, I get a out of memory error.
> > >
> > > Any ideas?
> >
> > Increase the amount of memory available to Java? I guess
> > GZIPOutputStream isn't capable of compressing on the fly, and buffers
> > the whole input before gzipping it?
> >
> > --
> > Russell Howe
> > [hidden email]
> >
> > Today's Nemi: http://www.metro.co.uk/img/pix/nemi_apr27.jpg
> >
> > -------------------------------------------------------
> > SF.Net email is sponsored by: Tell us your software development plans!
> > Take this survey and enter to win a one-year sub to SourceForge.net
> > Plus IDC's 2005 look-ahead and a copy of this survey
> > Click here to start!  http://www.idcswdc.com/cgi-bin/survey?id=105hix
> > _______________________________________________
> > jetty-discuss mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/jetty-discuss
> >
>


-------------------------------------------------------
SF.Net email is sponsored by: Tell us your software development plans!
Take this survey and enter to win a one-year sub to SourceForge.net
Plus IDC's 2005 look-ahead and a copy of this survey
Click here to start!  <a href="http://www.idcswdc.com/cgi-bin/survey?id5hix">http://www.idcswdc.com/cgi-bin/survey?id5hix
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: out of memory error using GZIPOutputStream

jian chen
Hi,

I figured out the issue. It is due to my in-proper use of the Velocity
template engine. I need to turn on template caching for the Velocity
engine.

I apologize if my question created any confusion.

Jetty works extremely well.

Thanks,

Jian


On 4/27/05, jian chen <[hidden email]> wrote:

> Hi,
>
> I got more information regarding the memory issue. The stack trace is:
>
> java.lang.OutOfMemoryError: (class: org/mortbay/http/EOFException)
>         at org.mortbay.http.BufferedOutputStream.flush(BufferedOutputStream.java:206)
>         at org.mortbay.util.ByteBufferOutputStream.ensureSpareCapacity(ByteBufferOutputStream.java:357)
>         at org.mortbay.util.ByteBufferOutputStream.write(ByteBufferOutputStream.java:197)
>         at org.mortbay.http.BufferedOutputStream.write(BufferedOutputStream.java:146)
>         at org.mortbay.http.HttpOutputStream.write(HttpOutputStream.java:423)
>         at org.mortbay.jetty.servlet.ServletOut.write(ServletOut.java:54)
>         at java.util.zip.DeflaterOutputStream.deflate(Unknown Source)
>         at java.util.zip.DeflaterOutputStream.write(Unknown Source)
>         at java.util.zip.GZIPOutputStream.write(Unknown Source)
>         at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(Unknown Source)
>         at sun.nio.cs.StreamEncoder$CharsetSE.implWrite(Unknown Source)
>         at sun.nio.cs.StreamEncoder.write(Unknown Source)
>
> So, could it be a jetty issue?
>
> Thanks,
>
> Jian
>
>
> On 4/27/05, jian chen <[hidden email]> wrote:
> > Thanks for your suggestions. I looked at the jdk source code for
> > GZIPOutputStream implementation, and it seems to me that they also use
> > a buffer internally. And as I recall, GZIP works in zipping data in
> > chunks. So, it should send out a chunk once it is done rather than
> > buffering them all.
> >
> > Jian
> >
> > On 4/27/05, Russell Howe <[hidden email]> wrote:
> > > jian chen wrote:
> > > > Hi,
> > > >
> > > > This strange problem has been bugging me for 2 days. I have the
> > > > following GZIPOutputStream on top of the servlet response
> > >
> > > [jian]
> > >
> > > > So, when the velocity template page is huge, then, I get a out of memory error.
> > > >
> > > > Any ideas?
> > >
> > > Increase the amount of memory available to Java? I guess
> > > GZIPOutputStream isn't capable of compressing on the fly, and buffers
> > > the whole input before gzipping it?
> > >
> > > --
> > > Russell Howe
> > > [hidden email]
> > >
> > > Today's Nemi: http://www.metro.co.uk/img/pix/nemi_apr27.jpg
> > >
> > > -------------------------------------------------------
> > > SF.Net email is sponsored by: Tell us your software development plans!
> > > Take this survey and enter to win a one-year sub to SourceForge.net
> > > Plus IDC's 2005 look-ahead and a copy of this survey
> > > Click here to start!  http://www.idcswdc.com/cgi-bin/survey?id=105hix
> > > _______________________________________________
> > > jetty-discuss mailing list
> > > [hidden email]
> > > https://lists.sourceforge.net/lists/listinfo/jetty-discuss
> > >
> >
>


-------------------------------------------------------
This SF.Net email is sponsored by: NEC IT Guy Games.
Get your fingers limbered up and give it your best shot. 4 great events, 4
opportunities to win big! Highest score wins.NEC IT Guy Games. Play to
win an NEC 61 plasma display. Visit http://www.necitguy.com/?r 
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss