Avoiding file-locking when developing with embedded Jetty

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Avoiding file-locking when developing with embedded Jetty

Johannes Brodwall
Hi

I would like to hear if someone could suggest how to scratch this itch I often have when using Jetty embedded in applications.

I am often using Jetty during development. I'd like for static files that are normally packaged into a war or jar file to be easy to edit when I am developing (that is, when the files are not packaged). At the same time, I want my code to be as similar as possible during development and in production.

My best attempt so far has been to place the static content under src/main/resources/webapp and package it into the Jar-file.

In order to avoid locking the files when I'm running the server in the debugger, I've implemented the following:

    public static WebAppContext createApplicationContext() {
        WebAppContext webapp = new WebAppContext("/webapp", "/app");

        if (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file")) {
            // Avoid locking static content when running exploded
            webapp.setWar("src/main/resources/webapp");
            webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer", "false");
        }
        return webapp;
    }

This runs in a main method like so:

public class SimpleServer {

    public static void main(String[] args) throws Exception {
        HandlerList handlers = new HandlerList();
        handlers.addHandler(new ShutdownHandler("randomtoken", false, true));
        handlers.addHandler(createApplicationContext());

        Server server = new Server(5000);
        server.setHandler(handlers);
        server.start();

        System.out.println("Started " + server.getURI());
    }
}

As the rest of the code is extremely simple, the magic replacement of the target file with the source file and the setting of the very poorly documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer" parameter both feel really frustrating. The code is magic enough that I've ended up creating a "framework" to run it which is clearly not what I want.


1. Are there currently better ways of doing this?
2. Is there any way something that accomplishes the same could be added to Jetty itself?


~Johannes



_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Jan Bartel-3
Hi Johannes,

Have you considered using the jetty-runner? Here's the doc:
http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner

You could point it at your src/main/resources/webapp dir as the webapp
to deploy. However, as it seems you're running on windows, you'll need
to use a context xml file instead to set the context param to turn off
fileMappedBuffers, and to point to your webapp and context path.

OTOH, your main is a pretty simple class ... it could be made more
generic by using system properties to set the context path, webapp to
deploy and the fileMappedBuffers property.

cheers
Jan

On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]> wrote:

> Hi
>
> I would like to hear if someone could suggest how to scratch this itch I
> often have when using Jetty embedded in applications.
>
> I am often using Jetty during development. I'd like for static files that
> are normally packaged into a war or jar file to be easy to edit when I am
> developing (that is, when the files are not packaged). At the same time, I
> want my code to be as similar as possible during development and in
> production.
>
> My best attempt so far has been to place the static content under
> src/main/resources/webapp and package it into the Jar-file.
>
> In order to avoid locking the files when I'm running the server in the
> debugger, I've implemented the following:
>
>     public static WebAppContext createApplicationContext() {
>         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>
>         if
> (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
> {
>             // Avoid locking static content when running exploded
>             webapp.setWar("src/main/resources/webapp");
>
> webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
> "false");
>         }
>         return webapp;
>     }
>
> This runs in a main method like so:
>
> public class SimpleServer {
>
>     public static void main(String[] args) throws Exception {
>         HandlerList handlers = new HandlerList();
>         handlers.addHandler(new ShutdownHandler("randomtoken", false,
> true));
>         handlers.addHandler(createApplicationContext());
>
>         Server server = new Server(5000);
>         server.setHandler(handlers);
>         server.start();
>
>         System.out.println("Started " + server.getURI());
>     }
> }
>
> As the rest of the code is extremely simple, the magic replacement of the
> target file with the source file and the setting of the very poorly
> documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer" parameter
> both feel really frustrating. The code is magic enough that I've ended up
> creating a "framework" to run it which is clearly not what I want.
>
>
> 1. Are there currently better ways of doing this?
> 2. Is there any way something that accomplishes the same could be added to
> Jetty itself?
>
>
> ~Johannes
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Johannes Brodwall
Hi Jan

What I am trying to achieve is to easily start building new applications with embedded Jetty without having to use external programs (like jetty-runner), XML files (like context XML) or frameworks (like DropWizard). I can copy and paste this main class, but I would rather avoid the magic parts so it doesn't require special knowledge (especially the trick with setting the init parameter).

I am indeed running on Windows (is that the only platform that supports fileMappedBuffers)? Setting the init-parameter like this actually does work (I use it every day) and requires less code than what's needed to have a custom context XML. But like I said: It's undocumented and a bit magic.

I am still trying to figure out what change I would like to see in Jetty for this.

I agree that in this case, the src/main/resources/webapp path is specific to the way this application is packaged and something that makes sense to configure.

I guess that ideally for me would be if the default value of fileMappedBuffers would depend on whether the value in setWar is a directory or a JAR file. Does this distinction make sense to you?


~Johannes


On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
Hi Johannes,

Have you considered using the jetty-runner? Here's the doc:
http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner

You could point it at your src/main/resources/webapp dir as the webapp
to deploy. However, as it seems you're running on windows, you'll need
to use a context xml file instead to set the context param to turn off
fileMappedBuffers, and to point to your webapp and context path.

OTOH, your main is a pretty simple class ... it could be made more
generic by using system properties to set the context path, webapp to
deploy and the fileMappedBuffers property.

cheers
Jan

On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]> wrote:
> Hi
>
> I would like to hear if someone could suggest how to scratch this itch I
> often have when using Jetty embedded in applications.
>
> I am often using Jetty during development. I'd like for static files that
> are normally packaged into a war or jar file to be easy to edit when I am
> developing (that is, when the files are not packaged). At the same time, I
> want my code to be as similar as possible during development and in
> production.
>
> My best attempt so far has been to place the static content under
> src/main/resources/webapp and package it into the Jar-file.
>
> In order to avoid locking the files when I'm running the server in the
> debugger, I've implemented the following:
>
>     public static WebAppContext createApplicationContext() {
>         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>
>         if
> (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
> {
>             // Avoid locking static content when running exploded
>             webapp.setWar("src/main/resources/webapp");
>
> webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
> "false");
>         }
>         return webapp;
>     }
>
> This runs in a main method like so:
>
> public class SimpleServer {
>
>     public static void main(String[] args) throws Exception {
>         HandlerList handlers = new HandlerList();
>         handlers.addHandler(new ShutdownHandler("randomtoken", false,
> true));
>         handlers.addHandler(createApplicationContext());
>
>         Server server = new Server(5000);
>         server.setHandler(handlers);
>         server.start();
>
>         System.out.println("Started " + server.getURI());
>     }
> }
>
> As the rest of the code is extremely simple, the magic replacement of the
> target file with the source file and the setting of the very poorly
> documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer" parameter
> both feel really frustrating. The code is magic enough that I've ended up
> creating a "framework" to run it which is clearly not what I want.
>
>
> 1. Are there currently better ways of doing this?
> 2. Is there any way something that accomplishes the same could be added to
> Jetty itself?
>
>
> ~Johannes
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Jan Bartel-3
Hi Johannes,

On 5 January 2015 at 09:40, Johannes Brodwall <[hidden email]> wrote:
> Hi Jan
>
> What I am trying to achieve is to easily start building new applications
> with embedded Jetty without having to use external programs (like
> jetty-runner), XML files (like context XML) or frameworks (like DropWizard).

Sheesh, we already give you so many ways to start jetty, and its still
not enough :) :) :)

> I can copy and paste this main class, but I would rather avoid the magic
> parts so it doesn't require special knowledge (especially the trick with
> setting the init parameter).

The main class is really teensy weensy - you're configuring and
initiating an entire servlet container in just a couple of lines :)
The only "magic" part I can see is the init param to turn off
fileMappedBuffers - and that is only necessary iff you're running on a
windows platform && likely to be replacing the contents of the file.
If you're not going to replace the contents of the file then you can
leave that set to to true.

BTW, useFileMappedBuffers is set to false by default in the
DefaultServlet. We use the webdefault.xml file which defines the
DefaultServlet to set it to true. So your other approaches would be:
* use your own webdefault.xml file (but you don't like using xml files)
* redefine the DefaultServlet in your own web.xml or web-override.xml
with the param set to false (but again, you don't like to use xml
files)

Really, I think the best solution is to use a system property on the
command line for when you're running in dev mode (useFileMappedBuffers
false) vs running in prod mode (maybe useFileMappedBuffers true?).

>
> I am indeed running on Windows (is that the only platform that supports
> fileMappedBuffers)?

Only parameter that DOESN"T support fileMappedBuffers properly.

>  Setting the init-parameter like this actually does work
> (I use it every day) and requires less code than what's needed to have a
> custom context XML. But like I said: It's undocumented and a bit magic.

Well, to be fair, its not *entirely* undocumented:
http://www.eclipse.org/jetty/documentation/current/troubleshooting-locked-files-on-windows.html

> I am still trying to figure out what change I would like to see in Jetty for
> this.

Me too ;)

> I agree that in this case, the src/main/resources/webapp path is specific to
> the way this application is packaged and something that makes sense to
> configure.
>
> I guess that ideally for me would be if the default value of
> fileMappedBuffers would depend on whether the value in setWar is a directory
> or a JAR file. Does this distinction make sense to you?

Not exactly. As I said above, the value of useFileMappedBuffers
depends on whether the same file is likely to change when running on
windows. If it changes, then you have to set useFileMappedBuffers
false. Eg, if you're editing the files in dev mode, then files will
change . If you're redeploying an unpacked webapp then the same file
will change.

You can also look at the other solutions listed on that
troubleshooting page, which mostly involves copying parts of the weapp
to a tmp dir so that it is never the same file that will change (not
such a great option when running in dev mode and you want to see the
changes in your file reflected).

cheers
Jan


>
>
> ~Johannes
>
>
> On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
>>
>> Hi Johannes,
>>
>> Have you considered using the jetty-runner? Here's the doc:
>>
>> http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner
>>
>> You could point it at your src/main/resources/webapp dir as the webapp
>> to deploy. However, as it seems you're running on windows, you'll need
>> to use a context xml file instead to set the context param to turn off
>> fileMappedBuffers, and to point to your webapp and context path.
>>
>> OTOH, your main is a pretty simple class ... it could be made more
>> generic by using system properties to set the context path, webapp to
>> deploy and the fileMappedBuffers property.
>>
>> cheers
>> Jan
>>
>> On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]>
>> wrote:
>> > Hi
>> >
>> > I would like to hear if someone could suggest how to scratch this itch I
>> > often have when using Jetty embedded in applications.
>> >
>> > I am often using Jetty during development. I'd like for static files
>> > that
>> > are normally packaged into a war or jar file to be easy to edit when I
>> > am
>> > developing (that is, when the files are not packaged). At the same time,
>> > I
>> > want my code to be as similar as possible during development and in
>> > production.
>> >
>> > My best attempt so far has been to place the static content under
>> > src/main/resources/webapp and package it into the Jar-file.
>> >
>> > In order to avoid locking the files when I'm running the server in the
>> > debugger, I've implemented the following:
>> >
>> >     public static WebAppContext createApplicationContext() {
>> >         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>> >
>> >         if
>> >
>> > (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
>> > {
>> >             // Avoid locking static content when running exploded
>> >             webapp.setWar("src/main/resources/webapp");
>> >
>> >
>> > webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
>> > "false");
>> >         }
>> >         return webapp;
>> >     }
>> >
>> > This runs in a main method like so:
>> >
>> > public class SimpleServer {
>> >
>> >     public static void main(String[] args) throws Exception {
>> >         HandlerList handlers = new HandlerList();
>> >         handlers.addHandler(new ShutdownHandler("randomtoken", false,
>> > true));
>> >         handlers.addHandler(createApplicationContext());
>> >
>> >         Server server = new Server(5000);
>> >         server.setHandler(handlers);
>> >         server.start();
>> >
>> >         System.out.println("Started " + server.getURI());
>> >     }
>> > }
>> >
>> > As the rest of the code is extremely simple, the magic replacement of
>> > the
>> > target file with the source file and the setting of the very poorly
>> > documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer"
>> > parameter
>> > both feel really frustrating. The code is magic enough that I've ended
>> > up
>> > creating a "framework" to run it which is clearly not what I want.
>> >
>> >
>> > 1. Are there currently better ways of doing this?
>> > 2. Is there any way something that accomplishes the same could be added
>> > to
>> > Jetty itself?
>> >
>> >
>> > ~Johannes
>> >
>> >
>> >
>> > _______________________________________________
>> > jetty-users mailing list
>> > [hidden email]
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>>
>>
>> --
>> Jan Bartel <[hidden email]>
>> www.webtide.com
>> 'Expert Jetty/CometD developer,production,operations advice'
>> _______________________________________________
>> jetty-users mailing list
>> [hidden email]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Johannes Brodwall
Hi Jan

Thanks for the reply and sorry about my "thinking out loud" emails. Things are getting clearer as we speak and I now actually know what change I want (to contribute). I write this mail to hear if you're interested in a patch for this or if I would be wasting my time.

On Thu, Jan 8, 2015 at 2:06 PM, Jan Bartel <[hidden email]> wrote:

Sheesh, we already give you so many ways to start jetty, and its still
not enough :) :) :)

Actually, you're right - there are enough ways. The approach that I am using is the same as the one from the Heroku sample application. It is also the way described under "Setting a Web Application Context" under https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty. I personally thing these are the best ways of using Jetty.

The example code from Heroku is similar to the one I showed: https://github.com/heroku/java-getting-started/. However, I also need static resources like HTML, CSS and JavaScript. It can look like this (provided that the resources are included in the jar-file):

WebAppContext webapp = new WebAppContext("/webapp", "/app");

This approach works great on the server, but it is very cumbersome during development if the project also contains static resources (HTML. CSS, JS). If I start Jetty with a WebAppContext in the IDE I will encounter a problem:
  • If I change a static file (HTML, CSS, JS) and save, the IDE will try and copy this file to the output directory. Since the file has been opened with useFileMappedBuffers, this copy will fail. In Eclipse, the project will get an error and I have to stop Jetty and cleanly build it to continue.
You can change this behavior by changing the line above to use the source folder instead. If I'm using Maven, it will probably look like this:

WebAppContext webapp = new WebAppContext("/webapp", "src/main/resources/app");

This avoids leaving the project in an invalid state, but it leads to two other problems:
What I really want is to be able to say something to the effect of the first line:

WebAppContext webapp = new WebAppContext("/webapp", "/app");

This can probably not be made to be backwards compatible, but I can do something like this:

WebAppContext webapp = new WebAppContext("/webapp", Resource.newProjectResource("/app", new String[] { "src/main/resource" }));

What this would do:
  • Look for /app in the classpath. If it is in a directory (and not a JAR), this means we're running in exploded mode. In that case, try to substitute the source directories (in this case src/main/resource). If that works, make sure file mapped buffers are not used.
  • If /app is in a jar-file or not in the source folders, it works exactly as Resource.newResource("/app")
On more thing: org.eclipse.jetty.annotations.AnnotationConfiguration doesn't currently work with classpath resources, only with WAR-files.

I have made an experimental implementation of the above code. In order to make it work, I had to fight org.eclipse.jetty.server.ResourceCache.getDirectBuffer pretty hard. I can fool this code "if (_useFileMappedBuffer && resource.getFile()!=null)" by making my resource return null for a file, but it would probably be more meaningful to create a new method org.eclipse.jetty.util.resource.Resource.isSupportingDirectBuffer.

The resulting code would look something like this:

public class SimpleServer {
    public static void main(String[] args) throws Exception {
        HandlerList handlers = new HandlerList();
        handlers.addHandler(new ShutdownHandler("randomtoken", false, true));
        handlers.addHandler(new WebApplication("/app", Resource.newProjectResource("/app")));
        Server server = new Server(5000);
        server.setHandler(handlers);
        server.start();
        System.out.println("Started " + server.getURI());
    }
}

Again: This is the same way of starting the Jetty as Heroku and on the Emdedding Jetty documentation pages, with additional ease of use during development. So it is not a new method, but an improvement to what I personally feel is the current best method. It also addresses the section "Setting a Web Application Context" under https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty, where there are currently two examples. This approach unifies them. (This section seems to be missing from the new documentation at https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html)

The changes I would contribute are as follows:
  • ResourceBuffer.getDirectBuffer() would use the new method Resource.isSupportingDirectBuffer() instead of checking for Resource.getFile() == null
  • A new method Resource.newProjectResource would check if the classpath resource should be treated as a source resource and return a subclass of Resource which does not support direct buffer if so
  • A new constructor to WebAppContext would take contextPath and baseResource
In addition, I would like to contribute a subclass of WebAppContext (ClasspathWebAppContext) with a new method scanForAnnotations which mimics AnnotationConfiguration when you're not dealing with a WAR-file.

Would you be interested in patches to this effect? If so, I can create a bug report with the modifications to Resource, WebAppContext and ResourceBuffer.


~Johannes


> On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
>>
>> Hi Johannes,
>>
>> Have you considered using the jetty-runner? Here's the doc:
>>
>> http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner
>>
>> You could point it at your src/main/resources/webapp dir as the webapp
>> to deploy. However, as it seems you're running on windows, you'll need
>> to use a context xml file instead to set the context param to turn off
>> fileMappedBuffers, and to point to your webapp and context path.
>>
>> OTOH, your main is a pretty simple class ... it could be made more
>> generic by using system properties to set the context path, webapp to
>> deploy and the fileMappedBuffers property.
>>
>> cheers
>> Jan
>>
>> On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]>
>> wrote:
>> > Hi
>> >
>> > I would like to hear if someone could suggest how to scratch this itch I
>> > often have when using Jetty embedded in applications.
>> >
>> > I am often using Jetty during development. I'd like for static files
>> > that
>> > are normally packaged into a war or jar file to be easy to edit when I
>> > am
>> > developing (that is, when the files are not packaged). At the same time,
>> > I
>> > want my code to be as similar as possible during development and in
>> > production.
>> >
>> > My best attempt so far has been to place the static content under
>> > src/main/resources/webapp and package it into the Jar-file.
>> >
>> > In order to avoid locking the files when I'm running the server in the
>> > debugger, I've implemented the following:
>> >
>> >     public static WebAppContext createApplicationContext() {
>> >         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>> >
>> >         if
>> >
>> > (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
>> > {
>> >             // Avoid locking static content when running exploded
>> >             webapp.setWar("src/main/resources/webapp");
>> >
>> >
>> > webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
>> > "false");
>> >         }
>> >         return webapp;
>> >     }
>> >
>> > This runs in a main method like so:
>> >
>> > public class SimpleServer {
>> >
>> >     public static void main(String[] args) throws Exception {
>> >         HandlerList handlers = new HandlerList();
>> >         handlers.addHandler(new ShutdownHandler("randomtoken", false,
>> > true));
>> >         handlers.addHandler(createApplicationContext());
>> >
>> >         Server server = new Server(5000);
>> >         server.setHandler(handlers);
>> >         server.start();
>> >
>> >         System.out.println("Started " + server.getURI());
>> >     }
>> > }
>> >
>> > As the rest of the code is extremely simple, the magic replacement of
>> > the
>> > target file with the source file and the setting of the very poorly
>> > documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer"
>> > parameter
>> > both feel really frustrating. The code is magic enough that I've ended
>> > up
>> > creating a "framework" to run it which is clearly not what I want.
>> >
>> >
>> > 1. Are there currently better ways of doing this?
>> > 2. Is there any way something that accomplishes the same could be added
>> > to
>> > Jetty itself?
>> >
>> >
>> > ~Johannes
>> >
>> >
>> >
>> > _______________________________________________
>> > jetty-users mailing list
>> > [hidden email]
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>>
>>
>> --
>> Jan Bartel <[hidden email]>
>> www.webtide.com
>> 'Expert Jetty/CometD developer,production,operations advice'
>> _______________________________________________
>> jetty-users mailing list
>> [hidden email]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Jan Bartel-3
Hi Johannes,

On 14 January 2015 at 09:04, Johannes Brodwall <[hidden email]> wrote:
> Hi Jan
>
> Thanks for the reply and sorry about my "thinking out loud" emails.

No problem.


> Things
> are getting clearer as we speak and I now actually know what change I want
> (to contribute). I write this mail to hear if you're interested in a patch
> for this or if I would be wasting my time.

See  below.

>
> On Thu, Jan 8, 2015 at 2:06 PM, Jan Bartel <[hidden email]> wrote:
>>
>>
>> Sheesh, we already give you so many ways to start jetty, and its still
>> not enough :) :) :)
>
>
> Actually, you're right - there are enough ways. The approach that I am using
> is the same as the one from the Heroku sample application. It is also the
> way described under "Setting a Web Application Context" under
> https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty. I personally thing
> these are the best ways of using Jetty.

You are referring to old doco, the eclipse wiki is no longer the
canonical repository for jetty documentation. You should look at:
http://www.eclipse.org/jetty/documentation/. If you're looking at the
old wiki because you're using jetty-7 or jetty-8, then  they're
end-of-life, and the best way forward is to upgrade to jetty-9 ;)



>
> The example code from Heroku is similar to the one I showed:
> https://github.com/heroku/java-getting-started/. However, I also need static
> resources like HTML, CSS and JavaScript. It can look like this (provided
> that the resources are included in the jar-file):
>
> WebAppContext webapp = new WebAppContext("/webapp", "/app");
>
>
> This approach works great on the server, but it is very cumbersome during
> development if the project also contains static resources (HTML. CSS, JS).
> If I start Jetty with a WebAppContext in the IDE I will encounter a problem:
>
> If I change a static file (HTML, CSS, JS) and save, the IDE will try and
> copy this file to the output directory. Since the file has been opened with
> useFileMappedBuffers, this copy will fail. In Eclipse, the project will get
> an error and I have to stop Jetty and cleanly build it to continue.
>
> You can change this behavior by changing the line above to use the source
> folder instead. If I'm using Maven, it will probably look like this:
>
> WebAppContext webapp = new WebAppContext("/webapp",
> "src/main/resources/app");
>
>
> This avoids leaving the project in an invalid state, but it leads to two
> other problems:
>
> It makes the code to use during development different from the production
> code as described here
> https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty#Setting_a_Web_Application_Context.
> I've experienced this as the seed that leads to a large amount of
> development-only code in more than one project.
> The useFileMappedBuffers is still in effect and if I try and modify a static
> file, the IDE will reject saving it (after it has been served)
>
> What I really want is to be able to say something to the effect of the first
> line:
>
> WebAppContext webapp = new WebAppContext("/webapp", "/app");
>
>
> This can probably not be made to be backwards compatible, but I can do
> something like this:
>
> WebAppContext webapp = new WebAppContext("/webapp",
> Resource.newProjectResource("/app", new String[] { "src/main/resource" }));
>
>
> What this would do:
>
> Look for /app in the classpath. If it is in a directory (and not a JAR),
> this means we're running in exploded mode. In that case, try to substitute
> the source directories (in this case src/main/resource). If that works, make
> sure file mapped buffers are not used.
> If /app is in a jar-file or not in the source folders, it works exactly as
> Resource.newResource("/app")
>
> On more thing: org.eclipse.jetty.annotations.AnnotationConfiguration doesn't
> currently work with classpath resources, only with WAR-files.

Hmmn no, it actually does work with directories of classes and jar files.

>
> I have made an experimental implementation of the above code. In order to
> make it work, I had to fight
> org.eclipse.jetty.server.ResourceCache.getDirectBuffer pretty hard. I can
> fool this code "if (_useFileMappedBuffer && resource.getFile()!=null)" by
> making my resource return null for a file, but it would probably be more
> meaningful to create a new method
> org.eclipse.jetty.util.resource.Resource.isSupportingDirectBuffer.
>
> The resulting code would look something like this:
>
> public class SimpleServer {
>     public static void main(String[] args) throws Exception {
>         HandlerList handlers = new HandlerList();
>         handlers.addHandler(new ShutdownHandler("randomtoken", false,
> true));
>         handlers.addHandler(new WebApplication("/app",
> Resource.newProjectResource("/app")));
>         Server server = new Server(5000);
>         server.setHandler(handlers);
>         server.start();
>         System.out.println("Started " + server.getURI());
>     }
> }
>
> Again: This is the same way of starting the Jetty as Heroku and on the
> Emdedding Jetty documentation pages, with additional ease of use during
> development. So it is not a new method, but an improvement to what I
> personally feel is the current best method. It also addresses the section
> "Setting a Web Application Context" under
> https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty, where there are
> currently two examples. This approach unifies them. (This section seems to
> be missing from the new documentation at
> https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html)
>
> The changes I would contribute are as follows:
>
> ResourceBuffer.getDirectBuffer() would use the new method
> Resource.isSupportingDirectBuffer() instead of checking for
> Resource.getFile() == null
> A new method Resource.newProjectResource would check if the classpath
> resource should be treated as a source resource and return a subclass of
> Resource which does not support direct buffer if so
> A new constructor to WebAppContext would take contextPath and baseResource
>
> In addition, I would like to contribute a subclass of WebAppContext
> (ClasspathWebAppContext) with a new method scanForAnnotations which mimics
> AnnotationConfiguration when you're not dealing with a WAR-file.
>
> Would you be interested in patches to this effect? If so, I can create a bug
> report with the modifications to Resource, WebAppContext and ResourceBuffer.

OK, so the approach we have taken to integrate jetty with other
environments, is to subclass WebInfConfiguration. See the
MavenWebInfConfiguration for an example
(https://github.com/eclipse/jetty.project/blob/master/jetty-maven-plugin/src/main/java/org/eclipse/jetty/maven/plugin/MavenWebInfConfiguration.java).
This class is responsible for unpacking/copying a webapp (if
necessary) and establishing all of the container and webapp
classpaths, resource bases etc. We use this approach not only in
maven, but also ant and osgi where the webapp may be in different
forms (an unpacked war, or unassembled src dirs or bundles etc etc),
and the classpaths may be special (eg osgi). With regard to maven,
once the MavenWebInfConfiguration has done its work, the
AnnotationConfiguration (and all the other XXXConfigurations) is able
to work without modification. Our approach has also been to avoid
sublcassing WebAppContext so that as much of the tooling that we
already have just works and all of the adaptation is done in the
XXXConfiguration classes, which are designed to be pluggable.  So
ideally I'd like any solution to follow this pattern.

However, I'd have to say that I'm not sure that doing any of that
would be a great saving ... All you're really wanting to do is to set
2 different variables: 1.  the location of the webapp (a packed war or
unassembled dev dir) and 2. turn on or off useFileMappedBuffers. So it
seems an easy solution to me if you don't want to change the main
class code at all to simply pass these in either in a .properties file
or environment variables or on the command line. No?

I'm happy to keep discussing this further to see if there is something
that we can distill out that will make dev/prod with a main class
easier ... I don't think we've nailed it yet, but might be on the
track of something.

cheers
Jan

>
>
> ~Johannes
>
>>
>> > On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
>> >>
>> >> Hi Johannes,
>> >>
>> >> Have you considered using the jetty-runner? Here's the doc:
>> >>
>> >>
>> >> http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner
>> >>
>> >> You could point it at your src/main/resources/webapp dir as the webapp
>> >> to deploy. However, as it seems you're running on windows, you'll need
>> >> to use a context xml file instead to set the context param to turn off
>> >> fileMappedBuffers, and to point to your webapp and context path.
>> >>
>> >> OTOH, your main is a pretty simple class ... it could be made more
>> >> generic by using system properties to set the context path, webapp to
>> >> deploy and the fileMappedBuffers property.
>> >>
>> >> cheers
>> >> Jan
>> >>
>> >> On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]>
>> >> wrote:
>> >> > Hi
>> >> >
>> >> > I would like to hear if someone could suggest how to scratch this
>> >> > itch I
>> >> > often have when using Jetty embedded in applications.
>> >> >
>> >> > I am often using Jetty during development. I'd like for static files
>> >> > that
>> >> > are normally packaged into a war or jar file to be easy to edit when
>> >> > I
>> >> > am
>> >> > developing (that is, when the files are not packaged). At the same
>> >> > time,
>> >> > I
>> >> > want my code to be as similar as possible during development and in
>> >> > production.
>> >> >
>> >> > My best attempt so far has been to place the static content under
>> >> > src/main/resources/webapp and package it into the Jar-file.
>> >> >
>> >> > In order to avoid locking the files when I'm running the server in
>> >> > the
>> >> > debugger, I've implemented the following:
>> >> >
>> >> >     public static WebAppContext createApplicationContext() {
>> >> >         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>> >> >
>> >> >         if
>> >> >
>> >> >
>> >> > (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
>> >> > {
>> >> >             // Avoid locking static content when running exploded
>> >> >             webapp.setWar("src/main/resources/webapp");
>> >> >
>> >> >
>> >> >
>> >> > webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
>> >> > "false");
>> >> >         }
>> >> >         return webapp;
>> >> >     }
>> >> >
>> >> > This runs in a main method like so:
>> >> >
>> >> > public class SimpleServer {
>> >> >
>> >> >     public static void main(String[] args) throws Exception {
>> >> >         HandlerList handlers = new HandlerList();
>> >> >         handlers.addHandler(new ShutdownHandler("randomtoken", false,
>> >> > true));
>> >> >         handlers.addHandler(createApplicationContext());
>> >> >
>> >> >         Server server = new Server(5000);
>> >> >         server.setHandler(handlers);
>> >> >         server.start();
>> >> >
>> >> >         System.out.println("Started " + server.getURI());
>> >> >     }
>> >> > }
>> >> >
>> >> > As the rest of the code is extremely simple, the magic replacement of
>> >> > the
>> >> > target file with the source file and the setting of the very poorly
>> >> > documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer"
>> >> > parameter
>> >> > both feel really frustrating. The code is magic enough that I've
>> >> > ended
>> >> > up
>> >> > creating a "framework" to run it which is clearly not what I want.
>> >> >
>> >> >
>> >> > 1. Are there currently better ways of doing this?
>> >> > 2. Is there any way something that accomplishes the same could be
>> >> > added
>> >> > to
>> >> > Jetty itself?
>> >> >
>> >> >
>> >> > ~Johannes
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > jetty-users mailing list
>> >> > [hidden email]
>> >> > To change your delivery options, retrieve your password, or
>> >> > unsubscribe
>> >> > from
>> >> > this list, visit
>> >> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >>
>> >>
>> >>
>> >> --
>> >> Jan Bartel <[hidden email]>
>> >> www.webtide.com
>> >> 'Expert Jetty/CometD developer,production,operations advice'
>> >> _______________________________________________
>> >> jetty-users mailing list
>> >> [hidden email]
>> >> To change your delivery options, retrieve your password, or unsubscribe
>> >> from this list, visit
>> >> https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >
>> >
>> >
>> > _______________________________________________
>> > jetty-users mailing list
>> > [hidden email]
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>>
>>
>> --
>> Jan Bartel <[hidden email]>
>> www.webtide.com
>> 'Expert Jetty/CometD developer,production,operations advice'
>> _______________________________________________
>> jetty-users mailing list
>> [hidden email]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Johannes Brodwall

On Fri, Jan 16, 2015 at 12:39 PM, Jan Bartel <[hidden email]> wrote:

>
> Actually, you're right - there are enough ways. The approach that I am using
> is the same as the one from the Heroku sample application. It is also the
> way described under "Setting a Web Application Context" under
> https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty. I personally thing
> these are the best ways of using Jetty.

You are referring to old doco, the eclipse wiki is no longer the
canonical repository for jetty documentation. You should look at:
http://www.eclipse.org/jetty/documentation/. If you're looking at the
old wiki because you're using jetty-7 or jetty-8, then  they're
end-of-life, and the best way forward is to upgrade to jetty-9 ;)


I'm am luckily using Jetty 9. :-) But the corresponding page in the new documentation doesn't seem to cover this as well. The scenario of the exploded webapp is missing at https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html

To be honest, I think that the old documentation with the simple non-commented code examples is more encouraging than the new with lots of embedded comments.
 

>
> I have made an experimental implementation of the above code. In order to
> make it work, I had to fight
> org.eclipse.jetty.server.ResourceCache.getDirectBuffer pretty hard. I can
> fool this code "if (_useFileMappedBuffer && resource.getFile()!=null)" by
> making my resource return null for a file, but it would probably be more
> meaningful to create a new method
> org.eclipse.jetty.util.resource.Resource.isSupportingDirectBuffer.
>
> The resulting code would look something like this:
>
> public class SimpleServer {
>     public static void main(String[] args) throws Exception {
>         HandlerList handlers = new HandlerList();
>         handlers.addHandler(new ShutdownHandler("randomtoken", false,
> true));
>         handlers.addHandler(new WebApplication("/app",
> Resource.newProjectResource("/app")));
>         Server server = new Server(5000);
>         server.setHandler(handlers);
>         server.start();
>         System.out.println("Started " + server.getURI());
>     }
> }
>
> Again: This is the same way of starting the Jetty as Heroku and on the
> Emdedding Jetty documentation pages, with additional ease of use during
> development. So it is not a new method, but an improvement to what I
> personally feel is the current best method. It also addresses the section
> "Setting a Web Application Context" under
> https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty, where there are
> currently two examples. This approach unifies them. (This section seems to
> be missing from the new documentation at
> https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html)
>
> The changes I would contribute are as follows:
>
> ResourceBuffer.getDirectBuffer() would use the new method
> Resource.isSupportingDirectBuffer() instead of checking for
> Resource.getFile() == null
> A new method Resource.newProjectResource would check if the classpath
> resource should be treated as a source resource and return a subclass of
> Resource which does not support direct buffer if so
> A new constructor to WebAppContext would take contextPath and baseResource
>
> In addition, I would like to contribute a subclass of WebAppContext
> (ClasspathWebAppContext) with a new method scanForAnnotations which mimics
> AnnotationConfiguration when you're not dealing with a WAR-file.
>
> Would you be interested in patches to this effect? If so, I can create a bug
> report with the modifications to Resource, WebAppContext and ResourceBuffer.

OK, so the approach we have taken to integrate jetty with other
environments, is to subclass WebInfConfiguration. See the
MavenWebInfConfiguration for an example
(https://github.com/eclipse/jetty.project/blob/master/jetty-maven-plugin/src/main/java/org/eclipse/jetty/maven/plugin/MavenWebInfConfiguration.java).
This class is responsible for unpacking/copying a webapp (if
necessary) and establishing all of the container and webapp
classpaths, resource bases etc. We use this approach not only in
maven, but also ant and osgi where the webapp may be in different
forms (an unpacked war, or unassembled src dirs or bundles etc etc),
and the classpaths may be special (eg osgi). With regard to maven,
once the MavenWebInfConfiguration has done its work, the
AnnotationConfiguration (and all the other XXXConfigurations) is able
to work without modification. Our approach has also been to avoid
sublcassing WebAppContext so that as much of the tooling that we
already have just works and all of the adaptation is done in the
XXXConfiguration classes, which are designed to be pluggable.  So
ideally I'd like any solution to follow this pattern.


It's an option that I haven't looked into much. Adding it to my study list. :-)
 
However, I'd have to say that I'm not sure that doing any of that
would be a great saving ... All you're really wanting to do is to set
2 different variables: 1.  the location of the webapp (a packed war or
unassembled dev dir) and 2. turn on or off useFileMappedBuffers. So it
seems an easy solution to me if you don't want to change the main
class code at all to simply pass these in either in a .properties file
or environment variables or on the command line. No?


Yes, but...

My main concern is about the teachability of embedded Jetty, not so much solving my own problem in a specific case. 1. I would like to do the switch automatically - after all, why do we have computers except to find out "the right thing" for us? 2. All the ways of avoiding filemapped buffers are difficult to come by and feel like black magic.

I think the location of the web app is a smaller problem. I think that I can find a satisfying way of doing this (especially if the annotation scanning can be made to work).

The filemapped buffers is something that I am more concerned about.

Again, I'm mainly concerned about teachability. I have several times had people express "wow, this was really easy to get started with" and then a few hours laters start swearing about the locked files, only to throw Jetty away in disgust.

To be honest, I think that you got the useFileMappedBuffer wrong. Perhaps the default should have been false? Perhaps the default should have been false if the resource was in the classpath?


~Johannes

 
>
>
> ~Johannes
>
>>
>> > On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
>> >>
>> >> Hi Johannes,
>> >>
>> >> Have you considered using the jetty-runner? Here's the doc:
>> >>
>> >>
>> >> http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner
>> >>
>> >> You could point it at your src/main/resources/webapp dir as the webapp
>> >> to deploy. However, as it seems you're running on windows, you'll need
>> >> to use a context xml file instead to set the context param to turn off
>> >> fileMappedBuffers, and to point to your webapp and context path.
>> >>
>> >> OTOH, your main is a pretty simple class ... it could be made more
>> >> generic by using system properties to set the context path, webapp to
>> >> deploy and the fileMappedBuffers property.
>> >>
>> >> cheers
>> >> Jan
>> >>
>> >> On 2 January 2015 at 16:15, Johannes Brodwall <[hidden email]>
>> >> wrote:
>> >> > Hi
>> >> >
>> >> > I would like to hear if someone could suggest how to scratch this
>> >> > itch I
>> >> > often have when using Jetty embedded in applications.
>> >> >
>> >> > I am often using Jetty during development. I'd like for static files
>> >> > that
>> >> > are normally packaged into a war or jar file to be easy to edit when
>> >> > I
>> >> > am
>> >> > developing (that is, when the files are not packaged). At the same
>> >> > time,
>> >> > I
>> >> > want my code to be as similar as possible during development and in
>> >> > production.
>> >> >
>> >> > My best attempt so far has been to place the static content under
>> >> > src/main/resources/webapp and package it into the Jar-file.
>> >> >
>> >> > In order to avoid locking the files when I'm running the server in
>> >> > the
>> >> > debugger, I've implemented the following:
>> >> >
>> >> >     public static WebAppContext createApplicationContext() {
>> >> >         WebAppContext webapp = new WebAppContext("/webapp", "/app");
>> >> >
>> >> >         if
>> >> >
>> >> >
>> >> > (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
>> >> > {
>> >> >             // Avoid locking static content when running exploded
>> >> >             webapp.setWar("src/main/resources/webapp");
>> >> >
>> >> >
>> >> >
>> >> > webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
>> >> > "false");
>> >> >         }
>> >> >         return webapp;
>> >> >     }
>> >> >
>> >> > This runs in a main method like so:
>> >> >
>> >> > public class SimpleServer {
>> >> >
>> >> >     public static void main(String[] args) throws Exception {
>> >> >         HandlerList handlers = new HandlerList();
>> >> >         handlers.addHandler(new ShutdownHandler("randomtoken", false,
>> >> > true));
>> >> >         handlers.addHandler(createApplicationContext());
>> >> >
>> >> >         Server server = new Server(5000);
>> >> >         server.setHandler(handlers);
>> >> >         server.start();
>> >> >
>> >> >         System.out.println("Started " + server.getURI());
>> >> >     }
>> >> > }
>> >> >
>> >> > As the rest of the code is extremely simple, the magic replacement of
>> >> > the
>> >> > target file with the source file and the setting of the very poorly
>> >> > documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer"
>> >> > parameter
>> >> > both feel really frustrating. The code is magic enough that I've
>> >> > ended
>> >> > up
>> >> > creating a "framework" to run it which is clearly not what I want.
>> >> >
>> >> >
>> >> > 1. Are there currently better ways of doing this?
>> >> > 2. Is there any way something that accomplishes the same could be
>> >> > added
>> >> > to
>> >> > Jetty itself?
>> >> >
>> >> >
>> >> > ~Johannes
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > jetty-users mailing list
>> >> > [hidden email]
>> >> > To change your delivery options, retrieve your password, or
>> >> > unsubscribe
>> >> > from
>> >> > this list, visit
>> >> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >>
>> >>
>> >>
>> >> --
>> >> Jan Bartel <[hidden email]>
>> >> www.webtide.com
>> >> 'Expert Jetty/CometD developer,production,operations advice'
>> >> _______________________________________________
>> >> jetty-users mailing list
>> >> [hidden email]
>> >> To change your delivery options, retrieve your password, or unsubscribe
>> >> from this list, visit
>> >> https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >
>> >
>> >
>> > _______________________________________________
>> > jetty-users mailing list
>> > [hidden email]
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>>
>>
>> --
>> Jan Bartel <[hidden email]>
>> www.webtide.com
>> 'Expert Jetty/CometD developer,production,operations advice'
>> _______________________________________________
>> jetty-users mailing list
>> [hidden email]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Jan Bartel-3
Hi Johannes,

On 22 January 2015 at 11:02, Johannes Brodwall <[hidden email]> wrote:

>
> On Fri, Jan 16, 2015 at 12:39 PM, Jan Bartel <[hidden email]> wrote:
>>
>>
>> >
>> > Actually, you're right - there are enough ways. The approach that I am
>> > using
>> > is the same as the one from the Heroku sample application. It is also
>> > the
>> > way described under "Setting a Web Application Context" under
>> > https://wiki.eclipse.org/Jetty/Tutorial/Embedding_Jetty. I personally
>> > thing
>> > these are the best ways of using Jetty.
>>
>> You are referring to old doco, the eclipse wiki is no longer the
>> canonical repository for jetty documentation. You should look at:
>> http://www.eclipse.org/jetty/documentation/. If you're looking at the
>> old wiki because you're using jetty-7 or jetty-8, then  they're
>> end-of-life, and the best way forward is to upgrade to jetty-9 ;)
>>
>
> I'm am luckily using Jetty 9. :-)

Excellent! :)

> But the corresponding page in the new
> documentation doesn't seem to cover this as well. The scenario of the
> exploded webapp is missing at
> https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html
>
> To be honest, I think that the old documentation with the simple
> non-commented code examples is more encouraging than the new with lots of
> embedded comments.

That's a bit disappointing considering the effort we've gone to to
improve the doco :(  Maybe might be a good idea to make a contribution
to the doco to put in the info that you really want to see:

  https://github.com/jetty-project/jetty-documentation

[snip]

> Again, I'm mainly concerned about teachability. I have several times had
> people express "wow, this was really easy to get started with" and then a
> few hours laters start swearing about the locked files, only to throw Jetty
> away in disgust.
>
> To be honest, I think that you got the useFileMappedBuffer wrong. Perhaps
> the default should have been false? Perhaps the default should have been
> false if the resource was in the classpath?

Mmmn, well as we discussed, it is false in the code, and overridden to
be true by the webdefault.xml (which is of course replaceable). The
thing is, having it set to "true" is the right thing to do in both of
the following circumstances: 1. when running in production, 2. when
running on *nix. Now, most production servers are *nix machines, so
the setting makes a lot of sense. The only combination that is
problematic is Windows in development.

In the past, we've had this set around the other way, and we got
yelled at by all the *nix users because performance was worse. Now its
back to true, we get yelled at by the Windows users :).  We figure
that either one community or the other is going to have to change the
setting to suit, which is why we give you a number of different ways
to change it.

Perhaps this can be resolved simply by more (obvious) warnings in the
documentation?

Jan

>
>
> ~Johannes
>
>
>>
>> >
>> >
>> > ~Johannes
>> >
>> >>
>> >> > On Fri, Jan 2, 2015 at 6:04 PM, Jan Bartel <[hidden email]> wrote:
>> >> >>
>> >> >> Hi Johannes,
>> >> >>
>> >> >> Have you considered using the jetty-runner? Here's the doc:
>> >> >>
>> >> >>
>> >> >>
>> >> >> http://www.eclipse.org/jetty/documentation/current/runner.html#jetty-runner
>> >> >>
>> >> >> You could point it at your src/main/resources/webapp dir as the
>> >> >> webapp
>> >> >> to deploy. However, as it seems you're running on windows, you'll
>> >> >> need
>> >> >> to use a context xml file instead to set the context param to turn
>> >> >> off
>> >> >> fileMappedBuffers, and to point to your webapp and context path.
>> >> >>
>> >> >> OTOH, your main is a pretty simple class ... it could be made more
>> >> >> generic by using system properties to set the context path, webapp
>> >> >> to
>> >> >> deploy and the fileMappedBuffers property.
>> >> >>
>> >> >> cheers
>> >> >> Jan
>> >> >>
>> >> >> On 2 January 2015 at 16:15, Johannes Brodwall
>> >> >> <[hidden email]>
>> >> >> wrote:
>> >> >> > Hi
>> >> >> >
>> >> >> > I would like to hear if someone could suggest how to scratch this
>> >> >> > itch I
>> >> >> > often have when using Jetty embedded in applications.
>> >> >> >
>> >> >> > I am often using Jetty during development. I'd like for static
>> >> >> > files
>> >> >> > that
>> >> >> > are normally packaged into a war or jar file to be easy to edit
>> >> >> > when
>> >> >> > I
>> >> >> > am
>> >> >> > developing (that is, when the files are not packaged). At the same
>> >> >> > time,
>> >> >> > I
>> >> >> > want my code to be as similar as possible during development and
>> >> >> > in
>> >> >> > production.
>> >> >> >
>> >> >> > My best attempt so far has been to place the static content under
>> >> >> > src/main/resources/webapp and package it into the Jar-file.
>> >> >> >
>> >> >> > In order to avoid locking the files when I'm running the server in
>> >> >> > the
>> >> >> > debugger, I've implemented the following:
>> >> >> >
>> >> >> >     public static WebAppContext createApplicationContext() {
>> >> >> >         WebAppContext webapp = new WebAppContext("/webapp",
>> >> >> > "/app");
>> >> >> >
>> >> >> >         if
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > (SimpleServer.class.getResource(webapp.getWar()).getProtocol().equals("file"))
>> >> >> > {
>> >> >> >             // Avoid locking static content when running exploded
>> >> >> >             webapp.setWar("src/main/resources/webapp");
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > webapp.setInitParameter("org.eclipse.jetty.servlet.Default.useFileMappedBuffer",
>> >> >> > "false");
>> >> >> >         }
>> >> >> >         return webapp;
>> >> >> >     }
>> >> >> >
>> >> >> > This runs in a main method like so:
>> >> >> >
>> >> >> > public class SimpleServer {
>> >> >> >
>> >> >> >     public static void main(String[] args) throws Exception {
>> >> >> >         HandlerList handlers = new HandlerList();
>> >> >> >         handlers.addHandler(new ShutdownHandler("randomtoken",
>> >> >> > false,
>> >> >> > true));
>> >> >> >         handlers.addHandler(createApplicationContext());
>> >> >> >
>> >> >> >         Server server = new Server(5000);
>> >> >> >         server.setHandler(handlers);
>> >> >> >         server.start();
>> >> >> >
>> >> >> >         System.out.println("Started " + server.getURI());
>> >> >> >     }
>> >> >> > }
>> >> >> >
>> >> >> > As the rest of the code is extremely simple, the magic replacement
>> >> >> > of
>> >> >> > the
>> >> >> > target file with the source file and the setting of the very
>> >> >> > poorly
>> >> >> > documented "org.eclipse.jetty.servlet.Default.useFileMappedBuffer"
>> >> >> > parameter
>> >> >> > both feel really frustrating. The code is magic enough that I've
>> >> >> > ended
>> >> >> > up
>> >> >> > creating a "framework" to run it which is clearly not what I want.
>> >> >> >
>> >> >> >
>> >> >> > 1. Are there currently better ways of doing this?
>> >> >> > 2. Is there any way something that accomplishes the same could be
>> >> >> > added
>> >> >> > to
>> >> >> > Jetty itself?
>> >> >> >
>> >> >> >
>> >> >> > ~Johannes
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > jetty-users mailing list
>> >> >> > [hidden email]
>> >> >> > To change your delivery options, retrieve your password, or
>> >> >> > unsubscribe
>> >> >> > from
>> >> >> > this list, visit
>> >> >> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Jan Bartel <[hidden email]>
>> >> >> www.webtide.com
>> >> >> 'Expert Jetty/CometD developer,production,operations advice'
>> >> >> _______________________________________________
>> >> >> jetty-users mailing list
>> >> >> [hidden email]
>> >> >> To change your delivery options, retrieve your password, or
>> >> >> unsubscribe
>> >> >> from this list, visit
>> >> >> https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > jetty-users mailing list
>> >> > [hidden email]
>> >> > To change your delivery options, retrieve your password, or
>> >> > unsubscribe
>> >> > from
>> >> > this list, visit
>> >> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >>
>> >>
>> >>
>> >> --
>> >> Jan Bartel <[hidden email]>
>> >> www.webtide.com
>> >> 'Expert Jetty/CometD developer,production,operations advice'
>> >> _______________________________________________
>> >> jetty-users mailing list
>> >> [hidden email]
>> >> To change your delivery options, retrieve your password, or unsubscribe
>> >> from this list, visit
>> >> https://dev.eclipse.org/mailman/listinfo/jetty-users
>> >
>> >
>> >
>> > _______________________________________________
>> > jetty-users mailing list
>> > [hidden email]
>> > To change your delivery options, retrieve your password, or unsubscribe
>> > from
>> > this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/jetty-users
>>
>>
>>
>> --
>> Jan Bartel <[hidden email]>
>> www.webtide.com
>> 'Expert Jetty/CometD developer,production,operations advice'
>> _______________________________________________
>> jetty-users mailing list
>> [hidden email]
>> To change your delivery options, retrieve your password, or unsubscribe
>> from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/jetty-users
>
>
>
> _______________________________________________
> jetty-users mailing list
> [hidden email]
> To change your delivery options, retrieve your password, or unsubscribe from
> this list, visit
> https://dev.eclipse.org/mailman/listinfo/jetty-users



--
Jan Bartel <[hidden email]>
www.webtide.com
'Expert Jetty/CometD developer,production,operations advice'
_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding file-locking when developing with embedded Jetty

Johannes Brodwall

On Thu, Jan 22, 2015 at 5:29 PM, Jan Bartel <[hidden email]> wrote:
> But the corresponding page in the new
> documentation doesn't seem to cover this as well. The scenario of the
> exploded webapp is missing at
> https://www.eclipse.org/jetty/documentation/current/embedding-jetty.html
>
> To be honest, I think that the old documentation with the simple
> non-commented code examples is more encouraging than the new with lots of
> embedded comments.

That's a bit disappointing considering the effort we've gone to to
improve the doco :(  Maybe might be a good idea to make a contribution
to the doco to put in the info that you really want to see:

  https://github.com/jetty-project/jetty-documentation


Good point. I will look into that.


> Again, I'm mainly concerned about teachability. I have several times had
> people express "wow, this was really easy to get started with" and then a
> few hours laters start swearing about the locked files, only to throw Jetty
> away in disgust.
>
> To be honest, I think that you got the useFileMappedBuffer wrong. Perhaps
> the default should have been false? Perhaps the default should have been
> false if the resource was in the classpath?

Mmmn, well as we discussed, it is false in the code, and overridden to
be true by the webdefault.xml (which is of course replaceable). The
thing is, having it set to "true" is the right thing to do in both of
the following circumstances: 1. when running in production, 2. when
running on *nix. Now, most production servers are *nix machines, so
the setting makes a lot of sense. The only combination that is
problematic is Windows in development.

Getting something that works for everybody sounds hard.

What about disabling file mapped buffers for Windows?

Most people (hopefully) don't use Windows in production and even if they did, they are likely to use packaged WAR file rather than exploded resources on disk. Alternatively, I think I can make due if file mapped buffers are never used for classpath resources.

In the past, we've had this set around the other way, and we got
yelled at by all the *nix users because performance was worse. Now its
back to true, we get yelled at by the Windows users :).  We figure
that either one community or the other is going to have to change the
setting to suit, which is why we give you a number of different ways
to change it.

Perhaps this can be resolved simply by more (obvious) warnings in the
documentation?

I will look into that after I return from my vacation next week. :-)


~Johannes


_______________________________________________
jetty-users mailing list
[hidden email]
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/jetty-users