Application only starts when a bogus service is attached


ramonskie
 

we have some application that since our upgrade from 207 to 214 would not run

If I push an app (java web/ Spring) on cf without any services, CF loop on the push and sometime at the end (10 or 15minutes) the app running or not.

I have this trace in the events :

2015-08-26T13:19:28.00+0200 app.crash pocretform index: 0, reason: CRASHED, exit_description: failed to accept connections within health check timeout, exit_status: -1
2015-08-26T13:18:10.00+0200 app.crash pocretform index: 0, reason: CRASHED, exit_description: failed to accept connections within health check timeout, exit_status: -1
2015-08-26T13:16:48.00+0200 app.crash pocretform index: 0, reason: CRASHED, exit_description: failed to accept connections within health check timeout, exit_status: -1

and in the stdout :

2015-08-26T13:16:48.00+0200 [DEA/8] ERR Instance (index 0) failed to start accepting connections

If I bind a fake service (like mongo), the app is pushed without any problem

and in the log (stdout) I have :

2015-08-26T13:29:50.87+0200 [App/0] OUT [CONTAINER] org.apache.catalina.startup.HostConfig INFO Deployment of web application directory /home/vcap/app/.java-buildpack/tomcat/webapps/ROOT has finished in 6,502 ms
2015-08-26T13:29:50.87+0200 [App/0] OUT [CONTAINER] org.apache.coyote.http11.Http11NioProtocol INFO Starting ProtocolHandler ["http-nio-61230"]
2015-08-26T13:29:50.88+0200 [App/0] OUT [CONTAINER] org.apache.tomcat.util.net.NioSelectorPool INFO Using a shared selector for servlet write/read
2015-08-26T13:29:50.88+0200 [App/0] OUT [CONTAINER] org.apache.catalina.startup.Catalina INFO Server startup in 6540 ms

This trace does not appear, if I unbind the service.

I have also tried to push the Tomcat demo app (https://tomcat.apache.org/tomcat-6.0-doc/appdev/sample/), same behaviour. So it's not our app problem

any suggestions?


Daniel Mikusa
 

On Fri, Sep 4, 2015 at 4:32 AM, Ramon Makkelie <ramon.makkelie(a)klm.com>
wrote:

we have some application that since our upgrade from 207 to 214 would not
run
What version of the Java build pack do you have installed? `cf
buildpacks`? What does your `cf push` command & manifest.yml look like?



If I push an app (java web/ Spring) on cf without any services, CF loop on
the push and sometime at the end (10 or 15minutes) the app running or not.

I have this trace in the events :

2015-08-26T13:19:28.00+0200 app.crash pocretform
index: 0, reason: CRASHED, exit_description: failed to accept
connections within health check timeout, exit_status: -1
2015-08-26T13:18:10.00+0200 app.crash pocretform
index: 0, reason: CRASHED, exit_description: failed to accept
connections within health check timeout, exit_status: -1
2015-08-26T13:16:48.00+0200 app.crash pocretform
index: 0, reason: CRASHED, exit_description: failed to accept
connections within health check timeout, exit_status: -1

and in the stdout :

2015-08-26T13:16:48.00+0200 [DEA/8] ERR Instance (index 0) failed to
start accepting connections

If I bind a fake service (like mongo), the app is pushed without any
problem
What do you mean by "fake"? Is it an actual service attached to a DB? Is
it a user provided service with no info?



and in the log (stdout) I have :

2015-08-26T13:29:50.87+0200 [App/0] OUT [CONTAINER]
org.apache.catalina.startup.HostConfig INFO Deployment of
web application directory
/home/vcap/app/.java-buildpack/tomcat/webapps/ROOT has finished in 6,502 ms
2015-08-26T13:29:50.87+0200 [App/0] OUT [CONTAINER]
org.apache.coyote.http11.Http11NioProtocol INFO Starting
ProtocolHandler ["http-nio-61230"]
2015-08-26T13:29:50.88+0200 [App/0] OUT [CONTAINER]
org.apache.tomcat.util.net.NioSelectorPool INFO Using a shared
selector for servlet write/read
2015-08-26T13:29:50.88+0200 [App/0] OUT [CONTAINER]
org.apache.catalina.startup.Catalina INFO Server startup
in 6540 ms

This trace does not appear, if I unbind the service.
Some suggestions for troubleshooting...

1.) Run `cf logs <app>` in a second terminal. Then run `cf push`. After
the build pack completes, you should see a line that says CF is starting
the app. If you see no logging between that and the notice that the app
exited, you might try this. Sometimes when an app exits too fast, logging
is lost.

http://support.run.pivotal.io/entries/82506749-Help-My-app-is-crashing-and-I-don-t-see-any-thing-in-the-logs

2.) Because it's a Java app, you might also try this.

http://support.run.pivotal.io/entries/59869725-Java-Web-Applications-Slow-Startup-or-Failing

(sorry for the PWS links, those should be relevant for anyone running CF
though)



I have also tried to push the Tomcat demo app (
https://tomcat.apache.org/tomcat-6.0-doc/appdev/sample/), same behaviour.
So it's not our app problem
You might try the samples for Tomcat 8, since that's what the Java build
pack has been deploying for a while.

Dan


Fabien LEBRERE
 

Hi,


What version of the Java build pack do you have installed? `cf
buildpacks`? What does your `cf push` command & manifest.yml look like?
The version is : v3.1 | https://github.com/cloudfoundry/java-buildpack.git#7a538fb
There is not manifest.yml

The push cmd was : cf push sampletest -p sample.war

And the logs :
Starting app sampletest in org dinb / space development as admin...
-----> Downloaded app package (8.0K)
-----> Java Buildpack Version: v3.1 | https://github.com/cloudfoundry/java-buildpack.git#7a538fb
-----> Downloading Open Jdk JRE 1.8.0_60 from https://download.run.pivotal.io/openjdk/trusty/x86_64/openjdk-1.8.0_60.tar.gz (7.4s)
Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.0s)
-----> Downloading Open JDK Like Memory Calculator 1.1.1_RELEASE from https://download.run.pivotal.io/memory-calculator/trusty/x86_64/memory-calculator-1.1.1_RELEASE (0.9s)
Memory Settings: -Xmx768M -Xms768M -XX:MaxMetaspaceSize=104857K -XX:MetaspaceSize=104857K -Xss1M
-----> Downloading Tomcat Instance 8.0.24 from https://download.run.pivotal.io/tomcat/tomcat-8.0.24.tar.gz (4.1s)
Expanding Tomcat to .java-buildpack/tomcat (0.1s)
-----> Downloading Tomcat Lifecycle Support 2.4.0_RELEASE from https://download.run.pivotal.io/tomcat-lifecycle-support/tomcat-lifecycle-support-2.4.0_RELEASE.jar (0.2s)
-----> Downloading Tomcat Logging Support 2.4.0_RELEASE from https://download.run.pivotal.io/tomcat-logging-support/tomcat-logging-support-2.4.0_RELEASE.jar (0.2s)
-----> Downloading Tomcat Access Logging Support 2.4.0_RELEASE from https://download.run.pivotal.io/tomcat-access-logging-support/tomcat-access-logging-support-2.4.0_RELEASE.jar (0.2s)

-----> Uploading droplet (51M)

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 down
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 down
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 down
0 of 1 instances running, 1 failing
FAILED
Start unsuccessful



What do you mean by "fake"? Is it an actual service attached to a DB? Is
it a user provided service with no info?
It's a real service service, but not necessary for the app, like mongoDB (small) service
The binding it's just done for the fun, It's sure it's a strange workaround ;)



Some suggestions for troubleshooting...

1.) Run `cf logs <app>` in a second terminal. Then run `cf push`. After
the build pack completes, you should see a line that says CF is starting
the app. If you see no logging between that and the notice that the app
exited, you might try this. Sometimes when an app exits too fast, logging
is lost.

http://support.run.pivotal.io/entries/82506749-Help-My-app-is-crashing-an...
I have the trace of the app ;)
If I remove the service, the last log line after a push is :
o.s.web.context.ContextLoader - Root WebApplicationContext: initialization completed in 9202 ms
If I add the service, the last log line after a push is :
[CONTAINER] org.apache.catalina.startup.Catalina INFO Server startup in 10731 ms



2.) Because it's a Java app, you might also try this.

http://support.run.pivotal.io/entries/59869725-Java-Web-Applications-Slow...

(sorry for the PWS links, those should be relevant for anyone running CF
though)
Solve the problem !!!!



You might try the samples for Tomcat 8, since that's what the Java build
pack has been deploying for a while.

Dan


Amit Gupta
 

Hi all,

Ramon, Fabien, are you both working on the same problem or is Fabien's response to Daniel separate from Ramon's original question?

I couldn't understand Fabien's response. Did Daniel's help provide a satisfactory solution to the problem, or are you still using a workaround (whether it's binding a bogus service, or some other workaround)?

Thanks,
Amit, OSS Release Integration PM


Fabien LEBRERE
 

Hi Amit,

Sorry, you right.
In fact, I'm an user of the CF' Ramon installation ;)

Daniel has solve my problems with the last link (JAVA_OPTS "-Djava.security.egd=file:///dev/urandom").

Cheers
Fabien


Amit Kumar Gupta
 

Okay, great!

Thanks,
Amit

On Wed, Sep 9, 2015 at 4:35 AM, Fabien LEBRERE <yop190(a)gmail.com> wrote:

Hi Amit,

Sorry, you right.
In fact, I'm an user of the CF' Ramon installation ;)

Daniel has solve my problems with the last link (JAVA_OPTS
"-Djava.security.egd=file:///dev/urandom").

Cheers
Fabien