Date
1 - 20 of 23
HTTP request status text is changed
|
Stanley Shen <meteorping@...>
Hello all
I am having an web application running on CF/Diego with version 233, and I notice strange thing about my APP. I have some code to check attachments will be uploaded to system. If the attachment doesn't pass the checking, we will use code like response.sendError(413, "The file cannot be uploaded due to file extension jar ") Where response is HttpServletResponse. In front end, we have file upload widget to try to upload file to system. Before, we can always get statusText which is what we set in response, like "The file cannot be uploaded due to file extension jar" But right now, I always get statusText "Request Entity Too Large", which is the standard status text of error code 413. I tried in non CF environment, and it works as expected. It looks like in somewhere CF changed the statusText based on the statusCode, but I didn't get clue yet. Any information about this? Thanks in advance. |
|
|
|
Daniel Mikusa
Are you sure the request is actually making it to your app? Requests that
toggle quoted message
Show quoted text
come into CF go through a load balancer and the gorouters before they hit your app. ie. browser -> your LB -> gorouters -> app. If the upload is too large for the LB or gorouter it would probably generate a 413 with the standard response. Take a look at the access logs for your app and for the gorouter (i.e. RTR tag in `cf logs`). If you don't see either, it's your LB. If you just see the RTR then it's the gorouter. If you see access log from your app then it is hitting your app and it would seem that something else is causing this. Dan On Mon, Apr 25, 2016 at 9:25 AM, Stanley Shen <meteorping(a)gmail.com> wrote:
Hello all |
|
|
|
Stanley Shen <meteorping@...>
yes, the request goes to my APP.
I can reproduce it with a very small zip file as ZIP format is forbidden in my system. The default value configured in CF is "client_max_body_size: 15M" I am using HA_Proxy, no LB is introduced in my system. in gorouter I can see the log of my request, and its status is 413 indeed. I am not sure where the statusText is changed. |
|
|
|
Sunil Babu <cloudgrp.assist@...>
Can u check on the haproxy config on the web component
toggle quoted message
Show quoted text
On Monday, April 25, 2016, Stanley Shen <meteorping(a)gmail.com> wrote:
yes, the request goes to my APP. --
Thanks & Regards Sunil Babu K C +91-81970-35608 |
|
|
|
Stanley Shen <meteorping@...>
Actually one CF instance is bosh lite one deployed on AWS, I didn't change things related to haproxy.
I also can reproduce it on this bosh lite instance. - default_networks: - name: cf1 static_ips: null instances: 1 name: ha_proxy_z1 networks: - name: cf1 static_ips: - 10.244.0.34 properties: ha_proxy: ssl_pem: |+ -----BEGIN CERTIFICATE----- MIICsjCCAZoCCQC+xvE/1ZQgFzANBgkqhkiG9w0BAQUFADAaMRgwFgYDVQQDFA8q LmJvc2gtbGl0ZS5jb20wIBcNMTUxMDA4MjIwNDQ3WhgPMjI4OTA3MjIyMjA0NDda MBoxGDAWBgNVBAMUDyouYm9zaC1saXRlLmNvbTCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBAK09Q520xrKx75uew3mAS+y4uyRPZPEjt/pYdBl40PXIwaqO X7LGoc9lNZS/eAPX6xeVFmZbLZReQ5+Fm0moeLzsh58W9jjkWWk7oGISmxfoQz9B X9Eh0NHCrtKXMrCPlr+2RI/qLinJDqn87UEZqwX+84JU8hBZ8RD8D7YnfuDteySV SYOEUjkiN/pIWmbJQY1sjEyk1zH1Hiy8kmnait2sX8Td2S/aV6EJBgODOstzEtnf HFDIfoTJxbSK/0TbF6qBaSl0CLaOop9FX2ULEZUgAuIW4dG2k/xnpMLdz7A0ZsSU Haw9okZ5wNuYk1RSqhnqw+9KUWgXwV6RlMvtXMkCAwEAATANBgkqhkiG9w0BAQUF AAOCAQEAShOqAFLIc93yIjhcnN7L4ZXFo+CvOgklJqFeBbwRshsEptbaddDJYmRr ZXzOE7MiTOBM8YzKqtHvl/ZguXmIAXSZlnq6kuJHdPtcZOqu1x2GAvWWOzn9Xl4m T3RmwF3NgiX0jgNMkkm8i8jfT7uN9BnHxMv65b9yKeM0sRFN5XigA43DDQnfF3j4 FQ9jwpmS7zOx2wn6FayOgoE4rgJfV/9637ZprQOMfUbZPKgQQplDn6bvK13rj9g9 zCC9W0fy29l7VDuAOOSI5xzsoYyH6DfX7oySxn291hidSCb/buadNG4dgI4keMGw u5K8QQYmlSY91IJtuRRITYXGmIiPpg== -----END CERTIFICATE----- -----BEGIN RSA PRIVATE KEY----- MIIEogIBAAKCAQEArT1DnbTGsrHvm57DeYBL7Li7JE9k8SO3+lh0GXjQ9cjBqo5f ssahz2U1lL94A9frF5UWZlstlF5Dn4WbSah4vOyHnxb2OORZaTugYhKbF+hDP0Ff 0SHQ0cKu0pcysI+Wv7ZEj+ouKckOqfztQRmrBf7zglTyEFnxEPwPtid+4O17JJVJ g4RSOSI3+khaZslBjWyMTKTXMfUeLLySadqK3axfxN3ZL9pXoQkGA4M6y3MS2d8c UMh+hMnFtIr/RNsXqoFpKXQIto6in0VfZQsRlSAC4hbh0baT/Gekwt3PsDRmxJQd rD2iRnnA25iTVFKqGerD70pRaBfBXpGUy+1cyQIDAQABAoIBACXzdt2UnbbF3jzU QfRbE8bvDSg+MFnXPlWcjQqLehNuAGcxu2s5snbxsBQ/Abat1XWcFoUj0k9feyb2 KPew7YpNssQ6ToRWGfRAuLjjZJCPNDQmSSxSYSGiqZO+xb8CJb8n2ctBPQ2wWwMI Qp1xVxMAMC5MF59XZMUYwwRfkJ8LawB90+S9BjHcU3GqoPECLFkgEeIj3mrnmpAD vhIeYvQj2W5JCpxLUA+7lnyoqnx8OTOXvBPAsKwO1Hx88yCitnxXro7i0ZAw4ErH zrnMgWkFDvRiS3ta/QS2RcBBiZHKX/gRRT/AvqJ+Erveu0BcZ9AVy1UpPB0w9rBK PTxS2BECgYEA3MLd6Og+xQpw4UNhy9EjeDE/b/rZK4w/vfD3WE5J3Nm4HGdSA6Q4 YmQYVg+VuCLR+HHsk58LxEf+cU0MNgDJR1/rFZRmociF+G0i7/7DuhFm891wWWGW Iz7XeGWHi+LIeYWkteuflrkmvy/7xqArgcNqnirGhba6706MZz0G0YUCgYEAyOR5 aF7qRpLXHgMOPOzJKC4ceWA5rY8rcdJZFI7aNq5MJF9o+fNNt8YRJ1hQTzs5K/R+ HwBJel8J6CoPQo9WUXnj0md4M67sCZSBqWANMO/J0f4VkbLS/lwch+ZPS8jt3Z4z umYW4QnloIKXxORfySo7r9DzZSgmxuDE8PVWn3UCgYAFTwpXF36q7l1YjW5EoHrh 4Q1NfBLM4UqHHsxT604LaZDr3fAy9jgE5bNQHn/TNcMm3lZ6FlEKH1EXGGs6wToV 5VCZ7D+rlE7kcntsmgvK5bA8HQ8elyItJs23r3la+9EmWvhjB4+G6FzuLBE57ZAe RrzBoPW1MXe9WX423VjUoQKBgGea5T49jSc+fbDdtI8ZMxkExuyWAskOyEIYUJa4 obOHqn8rsZEOuKspfBlFg42JJpATtKO6WyrALvTMFDiogcTdTvBpKmXFNbgvHbvD bKorUHN7TZZpmkVSLeisj4KvKnWcLGNaWTxQBVwFXc5OVVQC8utWoOAvl+gDba4z aSwtAoGANdquHRNbigPj2y0cRoexYJwKgpfGEK4HXitsKZUUg09gVfagM1HynVFz RA0LVac0oJZFdMYZyU/PXCySS237xUD2/0oySYJIK9E0C4ZxKD+DoAk5Z097z0LM 7rxStMCBWB2x4ommvEnpdgntEKkl4buIDatvmbdmdwkY3+X65Ks= -----END RSA PRIVATE KEY----- metron_agent: zone: z1 router: servers: z1: - 10.244.0.22 z2: [] resource_pool: router_z1 templates: - name: haproxy release: cf - name: metron_agent release: cf - name: consul_agent release: cf update: {} |
|
|
|
Stanley Shen <meteorping@...>
An simple servlet can reproduce my problem on standard bosh lite CF.
1. push to the app the bosh lite cf push test -p ./test.war 2. access the servlet deployed on CF wget http://test.[my-domain]/test --2016-04-26 09:18:49-- http://test.[my-domain]/test Resolving test.[my-domain]... 54.10.100.100 Connecting to test.[my-domain]|54.10.100.100|:80... connected. HTTP request sent, awaiting response... 414 Request URI Too Long 2016-04-26 09:18:51 ERROR 414: Request URI Too Long. 3. check the servlet on non CF wget http://localhost:8080/test/test --2016-04-26 09:20:55-- http://localhost:8080/test/test Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:8080... connected. HTTP request sent, awaiting response... 414 my customized 414 error message 2016-04-26 09:20:55 ERROR 414: my customized 414 error message. "my customized 414 error message" is my customized error message for 414 status in my case, but it's always "Request URI Too Long" on CF instance now. And the servlet is very simple: {code} protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { // TODO Auto-generated method stub response.sendError(414,"my customized 414 error message"); } {code} On Mon, Apr 25, 2016 at 10:54 PM, Stanley Shen <meteorping(a)gmail.com> wrote: Actually one CF instance is bosh lite one deployed on AWS, I didn't changeOn 25 April 2016 at 22:54, Stanley Shen <meteorping(a)gmail.com> wrote: Actually one CF instance is bosh lite one deployed on AWS, I didn't change |
|
|
|
Stanley Shen <meteorping@...>
any clue for this issue?
|
|
|
|
Alvaro Perez-Shirley
Hi Stanley,
Can you provide us with a little bit more information so we can diagnose the issue? Firstly, a SANITIZED version of your deployment manifest. Did you generate the manifest using `./scripts/generate-bosh-lite-dev-manifest` or in some other way? What version of diego release is deployed alongside cf-release? Thanks, Alvaro |
|
|
|
Stanley Shen <meteorping@...>
Versions are:
BOSH LITE stemcell 3147 CF v233 Diego v0.1460.0 Garden LINUX 0.334.0 etcd 38 And yes, I can reproduce it on a manifest generated using './scripts/generate-bosh-lite-dev-manifest' and with a simple java servlet APP you can find in previous email. |
|
|
|
Alvaro Perez-Shirley
Hi Stanley,
We attempted to reproduce this behavior on bosh-lite and were unable to, which leads me to believe the issue might be in some middleware. Are you using the provided java buildpack or a custom buildpack? Are you running tomcat, and if so how is it configured? You could try updating to the latest version of cf-release to see if that makes any difference, as that is the one we tested with. Hope this helps, Alvaro |
|
|
|
Stanley Shen <meteorping@...>
hello, I tried with latest cf/diego release.
I still can reproduce the issue. I am using all things provided by CF without any customization. I just push my servlet APP and access it from command line.(You can find it in a previous reply in this thread) wget http://test.[APP domain]/test --2016-05-02 03:28:15-- http://test.[APP domain]/test Resolving test.[APP domain] (test.[APP domain])... 152.190.109.xx Connecting to test.[APP domain] (test.[APP domain])|152.190.109.xx|:80... connected. HTTP request sent, awaiting response... 414 Request URI Too Long 2016-05-02 03:28:15 ERROR 414: Request URI Too Long. Here the error message is "Request URI Too Long", which is the standard one, but my customized message "my customized 414 error message". +---------------------------------------------+---------------+---------+--------------------------------------+ | Name | OS | Version | CID | +---------------------------------------------+---------------+---------+--------------------------------------+ | bosh-warden-boshlite-ubuntu-trusty-go_agent | ubuntu-trusty | 3147* | 0636a704-8ac9-44a8-7f21-793f17310298 | +---------------------------------------------+---------------+---------+--------------------------------------+ +-------------------+-----------------+-------------+ | Name | Versions | Commit Hash | +-------------------+-----------------+-------------+ | cf | 236+dev.1* | 9a550e9d+ | | diego | 0.1468.0+dev.1* | a54eeaf4+ | | docker | 26* | 7fabb47c+ | | etcd | 45* | 96dab618+ | | garden-linux | 0.337.0* | a7d9ddac | +-------------------+-----------------+-------------+ buildpack position enabled locked filename staticfile_buildpack 1 true false staticfile_buildpack-cached-v1.3.6.zip java_buildpack 2 true false java-buildpack-v3.7.zip ruby_buildpack 3 true false ruby_buildpack-cached-v1.6.16.zip nodejs_buildpack 4 true false nodejs_buildpack-cached-v1.5.12.zip go_buildpack 5 true false go_buildpack-cached-v1.7.5.zip python_buildpack 6 true false python_buildpack-cached-v1.5.5.zip php_buildpack 7 true false php_buildpack-cached-v4.3.10.zip binary_buildpack 8 true false binary_buildpack-cached-v1.0.1.zip admin_console_buildpack 9 true false admin_console_buildpack-cached-v1.6.12.zip |
|
|
|
CF Runtime
Hi Stanley,
We can reproduce your issue when we push your .war file to a CF running on bosh-lite. We actually also see this same behavior when we run the .war file on a local Tomcat with default configuration . So, we believe the issue is due to tomcat configuration - your local tomcat probably has different configuration than the one provided by the buildpack, and what we see on an out-of-the-box tomcat installation. You can see the java buildpack's tomcat configuration here: https://github.com/cloudfoundry/java-buildpack/blob/master/resources/tomcat/conf/server.xml The behavior you're requesting is unusual, so the java buildpack does not support it (it is nothing to do with dea/diego) so you have a few options. If you really need this behavior, you have a few options: - fork the buildpack and change server.xml to your needs - use spring boot which allows you to fully configure the servlet container Thanks, Rob Dimsdale & David Sabeti CF Release Integration Pivotal |
|
|
|
Stanley Shen <meteorping@...>
Thanks for investigation.
Actually my APP is a jetty based application, the APP releases included all required jetty jar and configuration files, so it should not related to this tomcat setting? As tomcat can reproduce my issue easily so I took tomcat for example. |
|
|
|
Stanley Shen <meteorping@...>
So to be precise, my APP is a jetty based application, it includes jetty
toggle quoted message
Show quoted text
jars and related jetty configuration files. When this application is deployed on ubuntu (Non-CF) environment, we can get expect status message we customized. But when deploying to CF, the customized message is always changed to the standard status message. Since all the jetty jars and configuration files are included in our APP already, so there should no difference in jetty part. That's why we suspect the problem is in CF related things, but really got no clue where is it. We do have some code related to this behavior, and since it's standard servlet behavior, so I think we should supports it? http://www.java2s.com/Code/JavaAPI/javax.servlet.http/HttpServletResponsesendErrorintarg0Stringarg1.htm Attached is one simple jetty based application, when I run it locally I can get my customized message. wget http://localhost:8080/hello --2016-05-03 18:15:22-- http://localhost:8080/hello Resolving localhost... 127.0.0.1, ::1 Connecting to localhost|127.0.0.1|:8080... connected. HTTP request sent, awaiting response... 403 my customized 413 error message 2016-05-03 18:15:22 ERROR 403: my customized 413 error message. When pushing to CF, I still got the standard message. HTTP request sent, awaiting response... 403 Forbidden 2016-05-03 18:18:46 ERROR 403: Forbidden. Thanks, Stanley On 3 May 2016 at 10:49, Stanley Shen <meteorping(a)gmail.com> wrote:
Thanks for investigation. |
|
|
|
CF Runtime
Ok thanks for clarifying. It sounds like your java app is a WAR, not a JAR, correct? Or, perhaps it's a jar, but it's a non-executable jar - it cannot be run with `java -jar my-web-app.jar`
If you create a jar that is executable as above, Cloud Foundry will just run it - so you'll get all of your custom jetty configuration. If instead, you require some other command to run this, CF will not be able to infer this, and your servlets will be run inside of a tomcat instance created by the buildpack. If you cannot make your app an executable jar, you will have to rely on the buildpack behavior, which is non-configurable, unless you fork it. Can you post a sanitized version of your actual app (or DM it to someone from the #release-integration channel in cloudfoundry.slack.com). Without your custom configuration we cannot really proceed. Thanks, Rob Dimsdale & Alvaro Perez-Shirley CF Release Integration Pivotal |
|
|
|
Stanley Shen <meteorping@...>
Can you have a try with the attached jettty.zip.txt in previous reply to see if you reproduce it?
It's used to simulate my APP, you can just download it and change to jetty.zip and unzip it. There is a manifest file along with a war file and jetty runner. You can push it to a bosh lite VM and you should see the issue. In this sample, actually there is nothing special configured for jetty. we just run it like "java -jar jetty-runner.jar --port $PORT testId.war" in the container. My APP is quite large and has many dependencies, which make it not easy to post somewhere. |
|
|
|
Ben Hale <bhale@...>
The Java Buildpack creates a very complex command to start applications (since it needs to manage the JVM’s memory space within the container’s memory space). Therefore, it detects standard artifact types listed in the documentation[1] and builds the command based on that. For self-executable JARs, detection is based on the Java standard `Main-Class` key in the `META-INF/MANFIEST.MF`[2]. You’ll need to remember that when you push the WAR or JAR to Cloud Foundry, it is expanded and presented to the buildpack in an exploded form. We read files off the filesystem and execute the application in that layout. So, for example, there is no `testId.war` on the filesystem in the container, only the contents of the WAR that was pushed.
toggle quoted message
Show quoted text
-Ben Hale Cloud Foundry Java Experience [1]: https://github.com/cloudfoundry/java-buildpack#additional-documentation [2]: https://github.com/cloudfoundry/java-buildpack/blob/master/docs/container-java_main.md On May 3, 2016, at 11:25, Stanley Shen <meteorping(a)gmail.com> wrote: |
|
|
|
Stanley Shen <meteorping@...>
more information on this:
I am experiencing to deploy APPs using docker image, which means we actually didn't use java buildpack in this case. But I still reproduced this issue, so it seems it's not related to java buildpack, but still in one component of CF/diego. |
|
|
|
CF Runtime
In order to debug this, could you provide a sample Dockerfile for an app that demonstrates this behavior, and can you push that sample app to dockerhub? Also, if you're overriding anything like the start command or the ports, can you let us know that too?
Thanks, Rob Dimsdale & Alvaro Perez-Shirley CF Release Integration Pivotal |
|
|
|
Stanley Shen <meteorping@...>
I created the docker image, and you can find it "meteorping2/hello".
You can push the image as an APP like "cf push stanley -o meteorping2/hello" When the APP is ready, you can access the servlet like: wget http://stanley.test.io/hello And I got result ======================================= :~ stanleyshen$ wget http://stanley.test.io/hello --2016-05-11 15:23:16-- http://stanley.test.io/hello Resolving stanley.test.io... 11.22.33.44 Connecting to stanley.test.io|11.22.33.44|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2016-05-11 15:23:17 ERROR 403: Forbidden. ======================================= But if you access the servlet deployed in local env via "java -jar jetty-runner.jar testId.war" you will get below result: ======================================= --2016-05-11 07:04:31-- http://127.0.0.1:8080/hello Connecting to 127.0.0.1:8080... connected. HTTP request sent, awaiting response... 403 my customized 413 error message 2016-05-11 07:04:31 ERROR 403: my customized 413 error message. ======================================= And here is the Dockerfile definition: ======================================= FROM java:8 COPY . /usr/src/myapp WORKDIR /usr/src/myapp CMD java -jar jetty-runner.jar testId.war ======================================= Thanks for investigation, let me know if you still cannot reproduce the issue. Regards, Stanley |
|
|