Re: Yet More Expect-Continue

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Yet More Expect-Continue

Greg Wilkins-5
Anthony Cook wrote:


Wow - marathon effort there!!!   But I think I see your line of reasoning....
A couple of points:

The servlet spec originally ( and still a little bit) tries to pretend
that it is not tied to HTTP and that other protocols could be supported.
But this is simply not true - the API for a base servlet is so broken
that you could not possibly write any non trivial application using
that API.  There is NO HOPE of every writting a servlet that will work
in both HTTP and some other non-HTTP protocol (excluding things such as
mod_jk which have HTTP semantics).

The question of who should handle a HTTP header is very very difficult
and no clear in the spec:

  + the container can handle content-length
    in a request, but there is no support for content-type other than form
    parameters, so the servlet must handle it.

  + the container can send a 401 for unauthenticated if a constraint is
    failed. But then a servlet can also respond with a 401.

  + If a servlet does not consume a request body, the container must
    read it if persistent sessions are to work.


So the HTTP spec is implemented as a cooperation between container and
servlets.

In the case of the Expect header:

 + the container can send 417 if it does not know about the expected value.
 + the container can send a 100 if it needs to read the body for form authentication.

But after that, the container cannot decide if it is "efficient or appropriate" for
the body to be sent.   But the servlet can signal to the container that it intends to
handle the body - by reading from the input stream!

So my proposal is to use that signal to trigger the sending of a 100-continues.

I think it is in the spirit of the blurded responsibilities between container and
servlet.   I don't think it puts any burden on the servlet writer to even know
what is happening and it gives a meaningful result so that clients can perform
a handshake to avoid sending big or inappropriate request bodies.

All in all - I'd have been happier if HTTP did not have Expect Continues - as I
see plenty of other ways to implement this.  But as it does have it, I think
Jetty's implementation now is reasonable?


cheers



> Hi Greg,
>
> Your points are understood.  Naturally, I have a "slightly" different
> point of view. :) ...
>
> Greg Wilkins wrote:
> | Which goes to the real root of the problem!  What is a servlet?  Is it
> | a protocol handler or a web application component?
> |
>
> "A /servlet/ is a.../web component/" that is "managed by a container".
> So starts section SRV.1.1, "What is a Servlet?", of versions 2.4 & 2.3
> of the servlet specification.  Section 1.1 of the 2.2 spec. concurs.
>
> [[ ''Our ignorance of history causes us to [confuse the present].'' --
> Gustave Flaubert (para.) ]]
>
> In the 2.1 spec. -- before the dawn of J2EE, the "web-app", and the JCP
> -- we were informed that "servlets.../extend/ the functionality of /Web
> servers/".  Since then the specification, along with the paradigm, has
> evolved such that now it is the "servlet container" (aka "servlet
> engine" -- 2.1), that is an /extension/ to the Web server, through which
> "[s]ervlets interact with Web clients via a request/response paradigm
> implemented by the servlet container".
>
> HTTP is certainly one such "request/response paradigm" and all versions
> of the specification make gratuitous references to it, but does that
> mean that servlet containers (or "engines" -- 2.1) must "implement" the
> HTTP protocol, or merely support parts of it for proper functioning?  In
> the 2.1 spec. we were informed that "[t]he request-response paradigm is
> /modeled on the behaviour of/ HTTP", not "is an implementation of HTTP",
> so it would seem that the latter meaning is actually the correct one.
> However, let's consider it further to be sure...
>
> While the current specification refers to the container "implementing"
> the HTTP specifications, in the same paragraph's opening sentence, we
> are informed first that servlet containers "must /support/ HTTP as a
> protocol for requests and responses" (and, as well, may also support
> "other request/response based protocols").  I think we would be doing
> the spec. writers a disservice if we assume that their grasp of English
> grammer and usage is lacking, and English grammer rules tell us that the
> first sentence of a paragraph should convey the paragraph's idea (as
> does the 'overview' or 'summary' section of a document in general).  So
> the idea we have being conveyed is that servlet containers "/support/",
> not "implement", HTTP for request/response handling.
>
> Further, "grammer rules" aside, if we say that the container must
> "implement" the protocol -- as in its entirety -- then the container
> becomes not just an "extension to the Web server" but a Web server in
> its own right and the need for a distinct Web server is dispensed with
> entirely.  However, in answering the question of "[w]hat is a Servlet
> Container" (SRV.1.2 -- 2.3+), the spec. tells us that the servlet
> container is "/a part of/ a _Web server or application server that
> provides the network services over which requests and responses are
> sent_".  The servlet container is a /part of the server/ that handles
> the network communications, rather than it being /the server itself/ and
> ~ doing so.  Looking again to history we can get a better idea of just
> what exactly this was intended to mean:
>
> "In functionality, servlets lie somewhere between [CGI] programs and
> proprietary server extensions such as Netscape Server API (NSAPI).
> Unlike CGI programs and NSAPI modules, [servlets are not specific] to
> either a platform or a server." -- "About Java Servlets", spec. 2.1
>
> This fundamental idea remains persistent and in section SRV.1.4,
> "Comparing Servlets with Other Technologies", of the current
> specification the first half of the preceding is repeated verbatim (but
> adds also "Apache modules" to the notion of "proprietary server
> extensions").  The rest of this section then lists the "advantages
> [servlets have] over [those] other /server extension mechanisms/".  So
> again, we run into the overriding theme that servlets -- and their
> containers/engines -- /extend/ Web servers and enhance their
> functionality (e.g. such as with adding HTTP Sessions), rather than
> being HTTP "protocol handlers", or aka "Web servers".  They were/are not
> meant to replace Web servers nor to provide complete implementations of
> HTTP "handling".  That function still rests squarely with the Web
> server, or application server, to which the servlet container is attached.
>
> | Lots of the API indicates it is a protocol handler, as headers are
> | available and the developer must handle them correctly according to
> |
>
> The specification indicates that the container must "support" HTTP to
> enable servlet functioning.  That the container must "implement", to
> some degree, the HTTP specification in order to "support" it is a
> natural consequence, though ultimately it is the "servlet" that provides
> the Web server's extended functionality.  For the container that means
> being able to /parse/ the HTTP request-response streams in order to
> expose their properties to the servlet and it's client respectively
> (though, again, it is the Web server that handles the receiving and
> transmitting of those, not the servlet or its container).
>
> Those properties the container exposes to the servlet through the API to
> facilitate the servlet's potential need for access to them.  Does that,
> therefore, make the servlet an HTTP "protocol handler"?  Does that make
> the /servlet writer/ one as well?  To be semantically correct we should
> say that, actually, it makes them both HTTP "request/response _content
> handlers_" and that's really a different distinction -- which also
> neverminds that, API wise, a "servlet" does not even have to be an
> /HttpServlet/ in order to provide the Web server with its "extended
> functioning".
>
> | the RFC and it is not possible for the container to correct developer
> | protocol errors.
> |
>
> Which now brings us back to the original issue. :)  By "developer
> protocol errors" I presume you are refering to the /servlet developer/
> (for whom the API is given).
>
> So, is the servlet developer, therefore, responsible for responding to
> the 'Expect' request header?  If so, that seems to invalidate any
> expectation of handling the request header before the server/container
> accepts the request body (also neverminding that the servlet is really a
> content handler -- i.e. "message body" -- and not a protocol handler)?
>
> It also overlooks the fact that the servlet container, not the
> developer, deals with the 401 issue -- adding more blur to the
> distinction of who does what.  So, now the situation is that the
> container should first return a '401' response before passing the
> 'Expect' header to the servlet.  Nevermind that doing so creates the
> need for 3 request/response cycles to complete the transaction, rather
> than just 2.  Reasonable enough on its face, and seems to fit with the
> specification's description of 4xx class codes being sometimes
> appropriate for the 'Expect' header response -- except that, as pointed
> out previously, the 'Expect' header is about specifying requirement for
> certain "server behaviours", not about asking "am I authorized".
>
> So what exactly is the "behaviour" being specified?  That authentication
> is required before passing up the message body??  I still do not believe
> this is what the W3C had in mind for this header, especially since
> authentication handling is specified in an /OPTIONAL/ adjunct to the
> HTTP protocol specification and not included within it directly.  This
> again brings up the distinction between "HTTP protocol handler" and
> "servlet handler", and their respective specifications, in this issue.
> Specification for the latter says it must "support/implement" the
> specification for the former, but the HTTP specification makes
> authentication handling an /OPTIONAL/ feature, while the servlet spec.
> since version 2.2 makes security, in general, a mandatory container
> feature.  This is an important consideration in this case since, before
> servlet spec. 2.4, support for only HTTP 1.0 is required and the
> 'Expect' header was not introduced until HTTP 1.1 -- and then only in
> RFC 2616, not RFC 2068.
>
> In fact, where a request is not authorized, the server already returns a
> 401 response without consideration of the message body!  An 'Expect'
> header is not required for this and where it is desirable to not send
> the body unless authorized, a simple HEAD request before the POST, along
> with a modicum of extra client programming will accomplish the same
> thing.  Though this may be akin to the "work around" which Tony
> mentions, and for whatever reason finds undesirable, it is still cleaner
> and more efficient than expecting the 'server' to respond with each code
> separately (assuming it has to respond to either), since it keeps the
> r/r cycle count at 2 -- and if authentication may be expected in the
> first place, why not just send the credentials immediately and keep the
> r/r cycle at 1?  So, ultimately, perhaps Tony's issue is really one of
> /application design/ rather than of server or container protocol
> handling...
>
> Regards,
>
> Anthony

-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
Jetty-support mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-support




-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Yet More Expect-Continue

Anthony Cook
Greg Wilkins wrote:
> Wow - marathon effort there!!!   But I think I see your line of reasoning....
 >

Yes.  My colleages refer to me as "the man of many words", especially
since I have a penchant for being prosaic... ':)

> The servlet spec originally ( and still a little bit) tries to pretend
> that it is not tied to HTTP and that other protocols could be supported.
 >

Yes, in the early days (say, pre-2.0), Sun [marketing] promoted a notion
that servlet technology could be used to extend any type of server with
added functioning and that Web servers were to be seen as a "reference
platform" -- as well, of course, as being the most convenient way to
have their new technology be adopted.

[[Admittedly, in those days I scratched my head quite a bit trying to
envision what additional functioning servlet technology could bring to
FTP or POP servers, for exaample. :-p]]

This, too, has a historical significance as servlet technology was also
seen as the server-side equivalent to HotJava which was promoted as both
a "proof of concept" for Java on the desktop and also as a testbed for
applet technology.

In any case, 'servlet technology' was spawned out of frustration with
common CGI issues and the technology developers sought initially to
create a "better CGI".  Sun also used Netscape Server and many issues
surrounding NSAPI gave servlet technology developers a
two-birds-with-one-throw shot.  The wounds at Sun caused by extensive
reliance on these earlier technologies are still healing, which probably
has more to do with why they retain "honorable mention" status as
comparison targets to this day than for any other reason (e.g. what
percentage of readers even know what NSAPI was, despite that it still
"survives" in iPlanet, let alone remember what was bad about it? :-p).
Ultimately, it was a marketing guffaw that cast servlet's into an
unrealistic, and technically unintended, light and this has steadily
been corrected beginning with version 2.0 of the spec.

Additionally, those early days saw the beginnings of the "application
server" paradigm and market with introduction of proprietary server
extension APIs like NSAPI and ISAPI.  Like 'servlet technology',
'application servers' were seen as not being tied to HTTP and, indeed,
many early products offered their own proprietary communication
frameworks, like BEA's t3.  When the spec. says that servlets can
support "other request/response protocols" it is really refering to
these "other server types", as were the technology developers from whom
the marketing folks got their notions.  Of course, now EJB and other
distributed object frameworks properly fill that gap and servlets are
left inextricably tied to HTTP.

> But this is simply not true - the API for a base servlet is so broken
> that you could not possibly write any non trivial application using
> that API.  There is NO HOPE of every writting a servlet that will work
> in both HTTP and some other non-HTTP protocol (excluding things such as
> mod_jk which have HTTP semantics).
>

In the end, HTTP won the day and, notwithstanding terminal service-style
solutions, there's hardly an "application server" available that does
not at least support HTTP.  The vast majority of those, as well, are
Java based and that, of course, means at least servlets are supported.
Even Wikipedia has a tough time coming up with any non-Java app servers,
save Zope and .Net, though AOLserver could also be listed.

As for the API, even the Servlet interface javadoc (J2EE 1.4)
distinguishes "servlets" as running within "a Web server".  While Sun's
''network'' may not yet be ''the computer'', the world of information
has come to rest solidly upon the child of CERN.  Regardless of what
original "notions" may have been for servlet technology, nevermind
application servers in general, it's pretty clear that it has since
grown out of its infantic notions of being a panacea for general server
extensions and come to accept HTTP as KING.

In any case, I do not believe the technology intends for /any servlet/
to support multiple communication protocols.  Yes, we have
GenericServlet, which can be extended and commonly is so, especially for
purely stream-based application operation.  Yet, stream-based operation
implies a "rich" or "thick" client is at the other end and such is also
typically developed with a particular connection protocol in mind.  With
that in mind, even GenericServlet suggests not extending it directly,
but rather a protocol-specific subclass.

> The question of who should handle a HTTP header is very very difficult
> and no clear in the spec:
>

''If it was easy, then everyone would be doing it.''  Implementing a
specification as complex as the current HTTP and Servlet specifications
is certainly an equally complex and difficult task.  That, of course, is
why there are few commercial and open-source implementations to choose
from and why fewer still are worth mass appeal. ;)

Naturally, as an extension to the web server, there are some headers
which are appropriate for a servlet to "handle", as may be dictated by
application requirements, and accordingly the API makes them available.
  Of course, as the first point of contact the web server or application
server may choose to handle some headers before passing them through.
Likewise, the container may also do so and, indeed, is even dictated by
the spec to do so in certain contexts (e.g. language encodings, security
constraints, etc.).

So, yes, the real question is who should handle which headers and when
and the spec does indeed provide little in the way of guidance on this.
  Ultimately, the question must be phrased within the contexts which
various headers may apply and that, in turn, requires a solid grounding
in their meanings within the HTTP spec. /and/ the historical intention
behind their additions.

What the spec. does indicate is that the majority of headers,
particularly _content_ related ones, should be left to the servlet.
Others, it makes no sense at all for a servlet to handle (e.g.
Connection), makes little if any sense to handle in light of container
requirements (e.g. Authorization) or, at least, puts a burden on the
servlet writer to handle correctly (e.g. Expect).

>   + the container can handle content-length
>     in a request, but there is no support for content-type other than form
>     parameters, so the servlet must handle it.
 >

Agreed, though content-length should also fall to the servlet, as is
also indicated by the spec. -- assuming it passes through the Web
server, which may already be configured to disallow requests based on
their length (e.g. Apache's LimitRequestBody).  Having the container do
so takes it outside the scope of its contract.

>   + the container can send a 401 for unauthenticated if a constraint is
>     failed. But then a servlet can also respond with a 401.
>

Yes, a servlet may send back any response code it so chooses.  Yet,
whether it /should/ send certain ones must, again. be considered in
context and I suspect the spec. writers expect their readers to be
technically versed enough to instinctively understand this.

In this case, the spec. provides for four types of declarative
authentication to be configured.  Within context of any of these the
servlet will never be called upon to handle the matter.  That leaves
'programmatic authentication' for the servlet -- a heavily frowned upon
practice in web-app development in general, and one for which the spec.
does not even make concessions.  Be that as it may, HTTP provides for
two scenarios when a '401' response is appropriate:

1. when authentication is required but an Authorization header is not
present in the request; and

2. when an Authorization header is present but authentication fails.

Assuming the first case, does it make sense for the servlet to send a
'401' response?  Perhaps it does if the client is a 'rich' or 'thick'
type, but then a serious question of application design is brought to
the fore: e.g. why is HTTP the connection protocol, and not RMI, IIOP,
or something else likely more suitable, in the first place?

If the client is browser-based then a non-declarative login form is
likely used and a '401' response, or '401' /in/ the response, is at the
very least superfluous to the application's logic and flow.  Of course,
the browser's built-in form can still be relied upon but, in either
case, a return to the "drawing board" is a serious consideration with an
especial eye this time focused toward the declarative model.

In the second scenario above, the same questions and issues are equally
raised.

>   + If a servlet does not consume a request body, the container must
>     read it if persistent sessions are to work.
>

I disagree.  If the servlet does not consume a request body, then what
the application likely desires to be persisted is that which has already
been consumed.  The container has no place in this determination.

If, however, you mean in case of server failure between the container's
receipt of the request and its passing to the servlet, then I think what
you really mean here is "persistent requests" and this is something
which definitely falls outside the scope of the servlet spec.  It /may/,
of course, be provided by the spec. implementor as a value-add, but this
again is outside the scope of the container's specified contract.

In a clustered -- aka "distributed" -- environment the spec. requires
only that the existing session object and it's contents are migrated,
such as in a fail-over situation.  It makes no concessions to migrating
requests.  A contextual exception is load-balancing but that occurs
before any request processing begins.  A high availability configuration
likewise cares little for the request as it generally occurs after
object invocation (e.g. servlet).  The only configuration which does
concern itself with the request in a fail-over is a fault tolerant one
but /true/ fault tolerance is a complex and difficult, and expensive,
enough solution to implement that even high-end providers like IBM an
BEA do not offer it "out of the box".

> So the HTTP spec is implemented as a cooperation between container and
> servlets.
>
> In the case of the Expect header:
>
>  + the container can send 417 if it does not know about the expected value.
>  + the container can send a 100 if it needs to read the body for form authentication.
>
> But after that, the container cannot decide if it is "efficient or appropriate" for
> the body to be sent.   But the servlet can signal to the container that it intends to
> handle the body - by reading from the input stream!
>
> So my proposal is to use that signal to trigger the sending of a 100-continues.
>
If I understand correctly, the sequence looks as follows, in the case of
FORM authentication /processing/:

* the client sends a request with an Expect:100 header

* the container passes the request to the servlet and waits for the
servlet to obtain the input stream

* upon the servlet obtaining the input stream, the container responds
with a 100-continue to the client and waits for the follow-on request;
_keeping the servlet in a state of limbo_

* /if/ the follow-on request is received the container authenticates and
/if/ the user is authenticated the container gives the servlet the input
stream...

I see many problems with this approach, e.g.:

1. it meshes the FORM post with the original client POST (or PUT) that
is being constrained and, thus, involves the servlet /before/
authentication processing... this is a violation of the container's
contract as well as the servlet's non-platform/server specificity

2. FORM auth handling is done without involvement of '401' response
codes, whether during first or second scenario stage, which: a)
separates the solution from Tony's original issue; and b) still doesn't
answer when, or if, a 100 response is sent with regard to BASIC auth.

3. by "input stream" does the solution also take into account parameter
reads (getParameter...)?

4. if the servlet/request URI is a controller, then the request might be
forwarded before any access to it even occurs; for that matter, other
resources may be included, or a response started in general, all
/before/ the container sends a response for the Expect header back to
the client; so, how is the "output stream" handled in relation to the
proposed solution?

5. if authentication fails, how is the servlet thread/instance handled?
  how long is it left in its limbo state waiting for the request stream?
  what about if authentication is not even a consideration but the
client still sends an Expect header and crashes thereafter; at what
point does the container decide that the client is not /going to/ send
the request body?

There are others but that should be a sufficient start for further
consideration.

> cheers

Regards,

Anthony

vschade.vcf (226 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Yet More Expect-Continue

Greg Wilkins-5
Anthony Cook wrote:

> In the second scenario above, the same questions and issues are equally
> raised.
>
>>   + If a servlet does not consume a request body, the container must
>>     read it if persistent sessions are to work.
>>
>
> I disagree.  If the servlet does not consume a request body, then what
> the application likely desires to be persisted is that which has already
> been consumed.  The container has no place in this determination.

I meant persistent connections - ie not closing the connection.
The container must consume the content of the request so it can access
the next request header.



>> In the case of the Expect header:
>>
>>  + the container can send 417 if it does not know about the expected
>> value.
>>  + the container can send a 100 if it needs to read the body for form
>> authentication.
>>
>> But after that, the container cannot decide if it is "efficient or
>> appropriate" for
>> the body to be sent.   But the servlet can signal to the container
>> that it intends to
>> handle the body - by reading from the input stream!
>>
>> So my proposal is to use that signal to trigger the sending of a
>> 100-continues.
>>
>
> If I understand correctly, the sequence looks as follows, in the case of
> FORM authentication /processing/:
>
> * the client sends a request with an Expect:100 header

yep

> * the container passes the request to the servlet and waits for the
> servlet to obtain the input stream

check

> * upon the servlet obtaining the input stream, the container responds
> with a 100-continue to the client and waits for the follow-on request;
> _keeping the servlet in a state of limbo_

Not a special limbo state.  The servlet just blocks on reading input
as it would normally.  The 100-continue has been sent, so the client must
respond with the content.


> * /if/ the follow-on request is received the container authenticates and
> /if/ the user is authenticated the container gives the servlet the input
> stream...

Not really - form authentication is a bad example for this, as the request body
must be consumed to perform the authentication.

But there is not *if* about the follow-on.  Once the 100 continue is sent, the
client is duty bound to respond.


> I see many problems with this approach, e.g.:
>
> 1. it meshes the FORM post with the original client POST (or PUT) that
> is being constrained and, thus, involves the servlet /before/
> authentication processing... this is a violation of the container's
> contract as well as the servlet's non-platform/server specificity

FORM post is not really changed at all - as the body is consumed to do the
auth, so the 100-continue will always be sent (unless a data constraint is not met).


> 2. FORM auth handling is done without involvement of '401' response
> codes, whether during first or second scenario stage, which: a)
> separates the solution from Tony's original issue; and b) still doesn't
> answer when, or if, a 100 response is sent with regard to BASIC auth.

As I said, FORM auth is a bad example for this.

But for BASIC auth, the 100 response is sent ONLY once basic auth has been passed
and the target servlet is consuming the input body.  If the basic auth is not passed,
then nobody reads the content and 100 is not sent and a 401 is sent instead.

 
> 3. by "input stream" does the solution also take into account parameter
> reads (getParameter...)?

Yes.  getParameter is really just a conveniance method to read input of a certain format.

> 4. if the servlet/request URI is a controller, then the request might be
> forwarded before any access to it even occurs; for that matter, other
> resources may be included, or a response started in general, all
> /before/ the container sends a response for the Expect header back to
> the client; so, how is the "output stream" handled in relation to the
> proposed solution?

If output is sent before the input is read, then that is the response that
is sent and a 100 is never sent and thus the request content is never sent.

I think I need to update my patch to make sure there are no nasty error
states in this strange ordering.

> 5. if authentication fails, how is the servlet thread/instance handled?
>  how long is it left in its limbo state waiting for the request stream?
>  what about if authentication is not even a consideration but the client
> still sends an Expect header and crashes thereafter; at what point does
> the container decide that the client is not /going to/ send the request
> body?

It is not a limbo state.  It is just blocking on input as it would anyway.
It is no different to any other request where the client might not send
the request body.  Timeouts to the rescue.


> There are others but that should be a sufficient start for further
> consideration.

I still think it is the right way to proceed. So bring on your other concerns :-)

cheers


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Yet More Expect-Continue

Anthony Cook
Greg Wilkins wrote:
> I meant persistent connections - ie not closing the connection.
> The container must consume the content of the request so it can access
> the next request header.
>

Which, in turn, further invalidates the whole notion of the client using
an Expect header to test for a security constraint before transmitting
content.  As already pointed out, this is an inappropriate use for the
header and falls outside its specified scope and purpose.

>>* upon the servlet obtaining the input stream, the container responds
>>with a 100-continue to the client and waits for the follow-on request;
>>_keeping the servlet in a state of limbo_
>
> Not a special limbo state.  The servlet just blocks on reading input
 >

By "limbo" I meant "blocked/waiting" while the container completes the
rest of the negotiation.

> as it would normally.  The 100-continue has been sent, so the client must
> respond with the content.
>

This assumes the client is subsequently able to respond, at all, during
the same communication cycle.  If it can not then the servlet thread and
any resources it has already acquired are left "littered about" until
cleaned up.  This has the potential to seriously impact scalability and
performance of not only the servlet but of any resources it uses.

>>* /if/ the follow-on request is received the container authenticates and
>>/if/ the user is authenticated the container gives the servlet the input
>>stream...
>
> Not really - form authentication is a bad example for this, as the request body
> must be consumed to perform the authentication.
>

Agreed. :)

> But there is not *if* about the follow-on.  Once the 100 continue is sent, the
> client is duty bound to respond.
>

As above.

>>I see many problems with this approach, e.g.:
>>
>>1. it meshes the FORM post with the original client POST (or PUT) that
>>is being constrained and, thus, involves the servlet /before/
>>authentication processing... this is a violation of the container's
>>contract as well as the servlet's non-platform/server specificity
>
> FORM post is not really changed at all - as the body is consumed to do the
> auth, so the 100-continue will always be sent (unless a data constraint is not met).
>
By "meshing" I meant the authentication operation and the application
operation become too closely interrelated.

In this scenario -- whether good example or bad, since it must still be
supported -- at which time is the '100' returned: before the client is
presented with the FORM, along with the FORM, or after receiving the
FORM parameters (authentication is performed)?

If the former, then the client's original purpose, which is to receive
either a '100' or (presumably, in the original case) a '401', is
defeated or, at least, is superfluous and again returns the issue to a
question of application design.  A design which must now take into
consideration this specific behaviour of the container.  Additionally,
it violates the container's contract which stipulates that when the user
is unauthenticated, 'Expect' header or not, the login form is
immediately returned to the client.  Doing so with the '100' is also
unsatisfactory as that now incurs a violation of both HTTP /and/
container specifications.

If, however, the '100' is returned following the authentication process
then this, too, is a violation of the container's contract which
stipulates that a /'200'/ response is to be returned regardless of the
authentication outcome.  Nevermind that this, too, still defeats the
client's original purpose.

> As I said, FORM auth is a bad example for this.
>
> But for BASIC auth, the 100 response is sent ONLY once basic auth has been passed
> and the target servlet is consuming the input body.  If the basic auth is not passed,
> then nobody reads the content and 100 is not sent and a 401 is sent instead.
>

OK, I can seen how this would seem to be a suitable solution in /this
case/.  However, the container still has, potentially, three other
authentication models to try and support with this work-around,
/including/ FORM based.

>>3. by "input stream" does the solution also take into account parameter
>>reads (getParameter...)?
>
> Yes.  getParameter is really just a conveniance method to read input of a certain format.
>

Understood, and expected. ;)

>>4. if the servlet/request URI is a controller, then the request might be
>>forwarded before any access to it even occurs; for that matter, other
>>resources may be included, or a response started in general, all
>>/before/ the container sends a response for the Expect header back to
>>the client; so, how is the "output stream" handled in relation to the
>>proposed solution?
>
> If output is sent before the input is read, then that is the response that
> is sent and a 100 is never sent and thus the request content is never sent.
>
Which, in turn, not only breaks the client but is also a violation of
HTTP contract for the 'Expect' header.  However, my point was not about
the response actually /being sent/ but about being "started", or
prepared, /before/ any reads are attempted on the input!  A valid output
stream /must/ be given to the servlet regardless of the final outcome of
any read attempts on the input stream.

So, for example, does the container give the servlet a "buffered" or
other interim cacheing/persisting "output stream" and then tie this to
final/real output stream?  If so, then this is potentially more
"garbage" that the container must manage and which may further impact
overall scalability and performance.

This also assumes that the same r/r cycle is used to control application
flow, and not, instead, a Location header.  Yes, that too means the
original request content is never sent and that, too, also may break the
application.  In the end, a servlet writer /must/ be able to count on
certain specified behaviours and be able to rely on their attendant
assumptions: e.g. "if the servlet is invoked, then its constraints must
have /already/ been met".

> It is not a limbo state.  It is just blocking on input as it would anyway.
> It is no different to any other request where the client might not send
> the request body.  Timeouts to the rescue.
>

Also expected. :)  However, in this case there is more than one
"request" that must be considered in the overall cycle.  Furthermore,
which "timeout" exactly has control?  Timeout on the first ('Expect')
request, or on the second (post-continue) request?  If the first, is
execution timing "tracked" and the first's timeout adjusted accordingly?
  If the second, then how can this take into account "timeouts" sitting
behind the servlet, such as on connection pools?  Which, of course, goes
back to the issue of how a servlet's resources are potentially affected
and for which the servlet writer must either consider the matter in the
application's design (meaning this "feature" can not be assumed to be
transparent to the servlet writer), or accept that the application may
be "broken" by the container's behaviour.  Neither case seems
particularly palatable to me. :-/

Of course, the "answer" can therefore be "well then don't use an
'Expect' header if that's a consideration", but then that just returns
us to the issue of what an 'Expect' header is for and whether the
container is actually "compliant".

> I still think it is the right way to proceed. So bring on your other concerns :-)
>

OK, for starters let's take a look at how [HTTP] sessions are now
potentially impacted.  Sessions are for maintaining client state.  A
common case is the creation of a /new/ session to indicate the starting
state within the application.

Going back to the issue of request failures and "timeouts"... the first
time the servlet is invoked it may look for and, upon not finding it,
create the session before proceeding with input negotiation.  A
controller servlet may on subsequent requests forward, or even
"redirect", them upon finding the session.

If the subsequent (post-continue) request, which of course contains the
content to be processed, never arrives then what does the container
clean up upon "timeout"?  Is the session instance included in the
effort?  Can the container reasonably make that determination without
violating contract?  I don't think so, certainly not without tracking
session creation events along with their lifetimes, and also since
session timeouts generally long exceed request timeouts.

Yet, if the session instance is not cleaned up then the application is
broken by the client's next "initiation" effort!  Just more specific
server behaviour that the servlet writer must consider when writing the
application?  Suddenly, I see cross-server deployability being lost and
replaced by the "vendor lock" servlets are meant to avoid.  This is
especially true where knowledge of the client implementation is not
available.

Ultimately, the issue still centers around responsibilities dictated by
the container/servlet contract.  Answering my own previous question,
"yes", 'servlet writers' /are/ HTTP "implementors", and much more so
than the container is.  It is /servlets/ which provide the server with
its extended functionality, not the container which ultimately is little
more than a lifecyle manager and provider of services needed for the
servlet to do its job.

As such services provider, some parts of HTTP the container must handle
directly, e.g. authentication.  This is explicitly specified, along with
a limited subset of other HTTP attributes.  Everything not explicitly
specified for the container to handle is implicitly directed to the
servlet to handle.  When the container is embedded within the Web or
application server, the spec. implicitly declares that the request is to
be passed to servlet "unhandled", with the notable exception of those
things explicitly specified for the container to do so.  This is the
meaning of the container "supporting" HTTP.

Among those headers which the container must pass "unhandled" is, in
fact, the 'Expect' header.  Again, per the HTTP specification this
header is for testing /extension capabilities/ of the "server" (aka
"specified server behaviours").  In this case, those capabilities are
within the realm of the servlet, as a /server extension component/, not
the container, and there are times when this header has a place in the
application's logic.  For corroboration of this, one needs only to look
to the API docs for HttpServletResponse, namely the Field Summary. ;)

With this in mind, I must digress a bit on my original
response/comments.  Jetty, in fact, did not handle the response in the
original issue appropriately, by sending back both a '100' and a '401'
response.  /Jetty/ should not have responded to the 'Expect' header at
all but, instead, handled just the '401' response and subsequently pass
(assuming authentication succeeded), the original request with 'Expect'
header on to the servlet.  However, I'm sure Tony's client would still
have been broken since, probably, the servlet would not have been
"expecting" :) it and consequently responded "inappropriately" (from the
client's point of view).  This, in turn, brings us back to two things:

1. a proper interpretation of the 'Expect' header as a "test" for
required "server", or in this case /servlet/ (more correctly, still,
"application"), functioning and not just a "header check"; and

2. a return to (for Tony) application design.

> cheers
>

Regards,

Anthony

vschade.vcf (226 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Expect-Continue Patch

Tony Seebregts
In reply to this post by Greg Wilkins-5
Hi Greg,

Been testing the patch you put in for Expect-Continue and so far it seems to
work on the things I've tried ... except a POST request with chunked transfer
encoding.

I get an HTTP 500 response and an IllegalStateException for non-premptive
authentication, pre-emptive authentication with invalid credentials, and also a
redirect (which all work normally except when using chunked transfer encoding).

A sample trace of the request-response headers is attached below.

regards

Tony

httpclient >> POST /agent/save HTTP/1.1
httpclient >> Host: 127.0.0.1:8080
httpclient >> Expect: 100-continue
httpclient >> Transfer-Encoding: chunked

WARN  [SocketListener0-1] org.mortbay.http.BasicAuthenticator - AUTH FAILURE:
user XXXX

java.lang.IllegalStateException: Chunking
  at
org.mortbay.http.HttpInputStream.setContentLength(HttpInputStream.java:259)
  at org.mortbay.http.HttpConnection.commit(HttpConnection.java:627)
  at org.mortbay.http.HttpConnection.outputNotify(HttpConnection.java:559)


httpclient << HTTP/1.1 500 Chunking
httpclient << Date: Mon, 27 Jun 2005 10:34:21 GMT
httpclient << Server: Jetty/5.1.x (Windows XP/5.1 x86 java/1.5.0_03
httpclient << WWW-Authenticate: basic realm="agent"
httpclient << Content-Type: text/html
httpclient << Content-Length: 1269
httpclient << Connection: close



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Patch To Expect-Continue Patch

Tony Seebregts
Hi Greg,

Quick one ... tried a simple patch to not throw an IllegalStateException if
using Expect-Continue and it worked fine. Don't know enough to know if there
are be other implications though.

Regards

Tony

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of
[hidden email]
Sent: 27 June 2005 13:33
To: [hidden email]
Subject: [jetty-discuss] Expect-Continue Patch

Hi Greg,

Been testing the patch you put in for Expect-Continue and so far it seems to
work on the things I've tried ... except a POST request with chunked
transfer
encoding.

I get an HTTP 500 response and an IllegalStateException for non-premptive
authentication, pre-emptive authentication with invalid credentials, and
also a
redirect (which all work normally except when using chunked transfer
encoding).

A sample trace of the request-response headers is attached below.

regards

Tony



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Yet More Expect-Continue

Greg Wilkins-5
In reply to this post by Anthony Cook
Anthony,

OK I'm lost now.... too many backs and forths....

let's turn this around.....   when do you think a servlet container should NOT
respond to a expected continues with a 100 response?

The current stable version of Jetty will always send a 100 response if it sees
a Expect and there is no input available to be read.   Do you think this is the
correct solution?     I don't think so because it is simply testing if the
client has sent a body yet....  a poor test and it puts the smarts in the client
when it is meant to be a server side mechanism.

So should the container always send a 100 response?

If not, what logic do you think should be applied?

cheers



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Yet More Expect-Continue

Anthony Cook-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Greg,

Greg Wilkins wrote:
| let's turn this around.....   when do you think a servlet container
should NOT
| respond to a expected continues with a 100 response?
|

It is my contention that the container should not respond to the
'Expect' header at all and, rather, pass it through to the servlet.  I
have just concluded a number of tests to validate this and to determine
how other implementations handle the matter.  My results are both
confirming and somewhat disappointing.  In the end, it appears that none
of the servers tested, nor clients, implement this header as specified.

For this, I wrote a simple test application comprised of the following:

- -- Clients:
* ExpectHeaderClient: a client which uses Jakarta's Commons HttpClient
package (tested against both versions 2.0.2 and 3.0.3rc2).

* ExpectHeaderTest: a client which uses my own HTTP protocol handler
(basically, a wrapper around HttpURLConnection with some similar
functionality to HttpClient).

- -- Servlets:
* HeaderDumpServlet: a simple "dump" servlet that returns the request
headers in text/plain.

* ExpectHeaderServlet: a servlet which takes an action based on the
'expect' header, or not, received in the request, in accordance with the
forms specified for it in HTTP spec.

- -- Scripts:
* HeaderDumpScript.php: a PHP version of the HeaderDumpServlet
* ExpectHeaderScript.php: a PHP version of the ExpectHeaderServlet

Both clients operate identically in that they both fire off a series of
three (or four) requests to the specified URL and logs the resulting
response.  The 'ExpectHeader...' responders also operate identically in
that they both respond to the given 'expect' header in the same
pre-defined (or "specified" :) ) way and log their corresponding actions.

The series of 'expect' headers follow the three basic forms provided in
the HTTP spec. and include:

* 'Expect: 100-continue':
        Expect-continue negotiation; sent by the client as specified, e.g.
"expect: 100-continue".

* 'Expect: _token_':
        A required server operation ("behaviour"), where _token_ indicates the
required operation; used by the client to force back a 417 response,
e.g. "expect: failme".

* 'Expect: _token_=_result_':
        As with just _token_, but which must result as indicated; used by the
client to test container interception -- the resource simply echos it
back for the response, e.g.:

        - client => server (header): "expect: java.vm.version=1.4.2"
        - server => client (resp. body): "Expect (header): java.vm.version=1.4.2".

Note that the HTTP spec. also provides for parameters to be given in the
request for passing to the 'token' operation (e.g. "expect: doThis;
args=val1,val2").  This form was not included in the testing since it is
just an extended form of the last two.

The servers tested include Tomcat 5.5.9 ("reference implementation" ;) )
& SJSAS 8.1, Jetty 5.1.4 (unpatched), Resin 3.0.13, and Apache 1.3.33 +
PHP 4.3.10.

Tomcat, SJSAS (naturally :) ), and Resin all performed identically by
passing all three 'expect' headers to the servlet and returning the
servlet's given response or status code.  This is bared out first by the
'dump' servlet which shows the 'expect' header from each request.

In each case, I know the /server/ is not responding to the header since
my clients log each cycle and those reflect the servlet's actions in
each step.  For example:

[ExpectHeaderServlet]
...
else if( expect.toLowerCase().equals( "100-continue" ) ) {
        logMsg.append( ": 100-continue received; " );
        if( continued == null ) {
                continued = new Boolean( "true" );
                session.setAttribute( "continued", continued );
                if( fakeContinue ) {
                        logMsg.append( "sending accepted (fake continue)...\n" );
                        response.setStatus( response.SC_ACCEPTED );
                }
                else {
                        logMsg.append( "sending continue...\n" );
                        response.setStatus( response.SC_CONTINUE );
                }
        }
        else {
                // Is this an error??
                logMsg.append( "processing body...\n" );
                session.removeAttribute( "continued" );
                String data = getRequestParameters( request );
                logMsg.apend( data );
                resp.append( data );
        }
}
...

Above is the servlet's 'expect-continue' negotiation handling.  When the
'expect: 100-continue' header is received the servlet responds by simply
setting the response status code, logs its action, and returns.  In all
three of the above servers, the 100 status code is shown in both the
servlet's log /and/ the server's access log, so it's fairly certain that
the servlet is seeing the header and the container is sending back the
servlet's given status code.  To be sure of this, I added an init-param
to the servlet to send back a 202 ("accepted"), in place of the 100 and
this too is seen in both logs.  If the server were responding to the
header directly, then the servlet's log would not reflect the client's
actions (vice-verse).

In Jetty and Apache/PHP's cases, /both/ refuse the _token_ version of
the header outright and directly send back a 417 response.  However,
both /do/ pass through the '100-continue' version and return the
resource's response.  Again, there is no indication in either client
that the server is doing something in middle of the negotiation cycle.

As stated above, there are also issues with both clients, but none that
disparage the test results in any way.

In regard to HttpClient, it appears that its handling of '100-continue'
is "broken" (despite handshake is activated with call to
setUseExpectHeader(true), is something else required?).  In case of a
100 response, it complains that it was unexpected and then exceptions
after a timeout (though it reacts normally to the "fake continue").

In regard to the HttpURLConnection client, it appears HttpURLConnection
doesn't understand the 100 response and returns -1 from
getResponseCode().  This proved consistent in the latest releases of the
Blackdown and IBM SDKs (1.4.2), as well as Sun's (1.4.2 and 1.5.0_04;
Linux x86 all cases).

After all is done, 'expect-continue' negotiation is not without its
problems regardless of the implementation used and, thus, which has no
real place in the Servlet spec's, erm, expectations. :)  So, while
Jetty's handling of the header is currently "out of spec", as compared
to the "reference implementation", it's ultimately irrelevant since the
header's use should be avoided by anyone whose client application needs
consistent cross-platform functioning and deployability.

| The current stable version of Jetty will always send a 100 response if
it sees
| a Expect and there is no input available to be read.   Do you think
this is the
| correct solution?     I don't think so because it is simply testing if the
|

As stated, I do not believe it is the /technically/ correct solution for
Jetty to do anything.  In any case, I'm not sure Jetty is doing as you
say in all cases, maybe just when an authentication issue is involved
(something which I did not test)?

| client has sent a body yet....  a poor test and it puts the smarts in
the client
| when it is meant to be a server side mechanism.
|

Agreed, when it is the /container/ that is doing the checking, though it
should be the servlet that is doing so.  There is more in the headers
that the "server" can potentially test for than just a body being
present (which negates a 100 response, anyway, if it is), or the
presence of user credentials.  So, how can the container reasonably know
exactly what to check for (unless it is made an application
configuration option)?

Clearly, Tomcat/Sun feels that the header, regardless of form, may
warrant the servlet's attention (and I believe this to be consistent
with the header's specified purpose).  Part of an implementation's
compliance testing is comparing its operation to that of the
"reference", so shouldn't Jetty "follow suit", if even for no other
reason than to be fully "compatible"?

In any case, it's ultimately "irrelevant" since other more direct and
meaningful methods are available for client and servlet to communicate
"requirements" between them, than doing so through request/response
headers, and which are uniformly implemented across "compliant" platforms.

As well, there are other ways for a client to test for security
constraints on the resource before sending data, and these require no
more communication cycles between the ends than an 'expect-continue'
negotiation does.  Yes, these may require [a small amount of] additional
client-side programming, but does that mean the client is consequently
made "smarter"?  I don't think so, not when the client may potentially
need to test the results of the negotiation cycle in the first place.

| cheers
|

Regards,

Anthony
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)

iD8DBQFCxa7l8KND+nha8AoRAl5MAJ9sPW/WjGn4vKdRhE2QW55mi5BFgQCgjV6P
MUqoYQgeIHOYusofqy/fwQA=
=2ak1
-----END PGP SIGNATURE-----


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Yet More Expect-Continue

Greg Wilkins-5
Anthony,

thanks for the detailed analysis and I tend to agree with the
advice to just avoid the mechanism.

However the problem with the "Jetty should do nothing" conclusion is
that is will punish a client that does correctly implement the spec.

Ie if a client send the expectation and does not send the body
until it has a received the 100 response - then 99% of web applications
will fail if run on a jetty that does nothing with this expectation.

So I feel that Jetty should do something and I'm now leaning back towards
the just send a 100 continues whenever you see the expectation.  Specially
as there appears to be issues with the jetty patch which I have not
had time to look at yet.

But I will wait for the discussion on the JSR EG before changing the
code more.  

cheers



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click



-------------------------------------------------------
This SF.Net email is sponsored by the 'Do More With Dual!' webinar happening
July 14 at 8am PDT/11am EDT. We invite you to explore the latest in dual
core and dual graphics technology at this free one hour event hosted by HP,
AMD, and NVIDIA.  To register visit http://www.hp.com/go/dualwebinar
_______________________________________________
jetty-discuss mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/jetty-discuss