Date
1 - 8 of 8
Cannot start app in new vsphere cf deployment
|
Ramesh Sambandan
awesome!
I changed the property and redeployed. It works! Thanks you very much. -Ramesh |
|
|
|
CF Runtime
Hi Ramesh,
toggle quoted message
Show quoted text
Looking at the original logs you provided, we see the following error when the CC tries to talk to HM9000: "SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed" We're assuming that your certificate used by ha_proxy is a self signed cert? Setting the "ssl.skip_cert_verify" to true in your manifest should make the CC ignore the fact that it is self signed. Joseph & Dan OSS Release Integration team On Mon, Sep 14, 2015 at 9:03 AM, Ramesh Sambandan <rsamban(a)gmail.com> wrote:
Amit, |
|
|
|
Ramesh Sambandan
Amit,
Below are my cf trace output. guests-MacBook-Pro:apps rsamban$ cf logs gs --recent REQUEST: [2015-09-14T09:39:28-06:00] GET /v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps?q=name%3Ags&inline-relations-depth=1 HTTP/1.1 Host: api.192.168.1.103.xip.io Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.3-c0c9a03 / darwin RESPONSE: [2015-09-14T09:39:28-06:00] HTTP/1.1 200 OK Content-Length: 5311 Content-Type: application/json;charset=utf-8 Date: Mon, 14 Sep 2015 15:39:28 GMT Server: nginx X-Cf-Requestid: 3d46e09a-5ef4-4f0d-5793-8d145a1b91a2 X-Content-Type-Options: nosniff X-Vcap-Request-Id: eab0cd51-7f04-4745-6ebf-0461df755620::4395680b-f74f-4325-a952-6d9fb974e3ce { "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "48d31982-7e60-4de2-9d33-fdbf291ec779", "url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779", "created_at": "2015-09-14T13:15:50Z", "updated_at": "2015-09-14T14:02:52Z" }, "entity": { "name": "gs", "production": false, "space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826", "stack_guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8", "buildpack": null, "detected_buildpack": "java-buildpack=v3.1.1-https://github.com/cloudfoundry/java-buildpack#7a538fb java-main open-jdk-like-jre=1.8.0_60 open-jdk-like-memory-calculator=1.1.1_RELEASE spring-auto-reconfiguration=1.10.0_RELEASE", "environment_json": { }, "memory": 1024, "instances": 1, "disk_quota": 1024, "state": "STARTED", "version": "40963450-8b7d-4478-bb84-682dee103a08", "command": null, "console": false, "debug": null, "staging_task_id": "0bd355ee625b4f1e814b7e1bc4cd0157", "package_state": "STAGED", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": "StagingTimeExpired", "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": "2015-09-14T13:16:04Z", "detected_start_command": "CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE -memorySizes=metaspace:64m.. -memoryWeights=heap:75,metaspace:10,stack:5,native:10 -totMemory=$MEMORY_LIMIT) && SERVER_PORT=$PORT $PWD/.java-buildpack/open_jdk_jre/bin/java -cp $PWD/.:$PWD/.java-buildpack/spring_auto_reconfiguration/spring_auto_reconfiguration-1.10.0_RELEASE.jar -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh $CALCULATED_MEMORY org.springframework.boot.loader.JarLauncher", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826", "space": { "metadata": { "guid": "6ecadc21-314d-4f93-8afb-b646cd384826", "url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826", "created_at": "2015-09-14T13:14:52Z", "updated_at": null }, "entity": { "name": "dev", "organization_guid": "6ee4b17e-ca40-47f2-b949-c7692324112a", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/6ee4b17e-ca40-47f2-b949-c7692324112a", "developers_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/developers", "managers_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/managers", "auditors_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/auditors", "apps_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps", "routes_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/routes", "domains_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/domains", "service_instances_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/service_instances", "app_events_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/app_events", "events_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/events", "security_groups_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/security_groups" } }, "stack_url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8", "stack": { "metadata": { "guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8", "url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8", "created_at": "2015-09-14T05:55:54Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/events", "service_bindings_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/service_bindings", "service_bindings": [ ], "routes_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/routes", "routes": [ { "metadata": { "guid": "4b3968ca-8821-496b-b4e9-47b54303f92f", "url": "/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f", "created_at": "2015-09-14T13:15:50Z", "updated_at": null }, "entity": { "host": "gs", "path": "", "domain_guid": "a6cc2662-2771-4737-89de-992a2d606997", "space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826", "domain_url": "/v2/domains/a6cc2662-2771-4737-89de-992a2d606997", "space_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826", "apps_url": "/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f/apps" } } ] } } ] } Connected, dumping recent logs for app gs in org yesVin / space dev as admin... REQUEST: [2015-09-14T09:39:29-06:00] POST /oauth/token HTTP/1.1 Host: login.192.168.1.103.xip.io Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/x-www-form-urlencoded User-Agent: go-cli 6.12.3-c0c9a03 / darwin grant_type=refresh_token&refresh_token=eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJiOWQzMTE5NC1iM2ZhLTQ4MmQtYmE0Ni1hZjRjMjRhYWUwMWYiLCJzdWIiOiIxY2UwZWQ4Yi1hMTdiLTQyZGEtYTE2Ni04MmZlYWQ1OTQyMGMiLCJzY29wZSI6WyJvcGVuaWQiLCJzY2ltLnJlYWQiLCJjbG91ZF9jb250cm9sbGVyLmFkbWluIiwiY2xvdWRfY29udHJvbGxlci5yZWFkIiwicGFzc3dvcmQud3JpdGUiLCJjbG91ZF9jb250cm9sbGVyLndyaXRlIiwiZG9wcGxlci5maXJlaG9zZSIsInNjaW0ud3JpdGUiXSwiaWF0IjoxNDQyMjM2NDU2LCJleHAiOjE0NDQ4Mjg0NTYsImNpZCI6ImNmIiwiY2xpZW50X2lkIjoiY2YiLCJpc3MiOiJodHRwczovL3VhYS4xOTIuMTY4LjEuMTAzLnhpcC5pby9vYXV0aC90b2tlbiIsInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfbmFtZSI6ImFkbWluIiwidXNlcl9pZCI6IjFjZTBlZDhiLWExN2ItNDJkYS1hMTY2LTgyZmVhZDU5NDIwYyIsInJldl9zaWciOiI1NGEzZjhkYyIsImF1ZCI6WyJjZiIsIm9wZW5pZCIsInNjaW0iLCJjbG91ZF9jb250cm9sbGVyIiwicGFzc3dvcmQiLCJkb3BwbGVyIl19.GLBJ3Px81ccR76QppqDF0ilyvxeXQO21j-XwOUVbQTllg3h8nLgPHShMDOuy7eOecBhJUSLbesahJME196LHYZYV1iOad9WhNASO11gdfqJ_0rTEGwZQEzwY2q9ggaoAv_YUjAmZOYCM8052K6LdKtROOqFd67CdTdrC0L8K1eo&scope= RESPONSE: [2015-09-14T09:39:31-06:00] HTTP/1.1 200 OK Transfer-Encoding: chunked Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store, max-age=0, must-revalidate Cache-Control: no-store Content-Type: application/json;charset=UTF-8 Date: Mon, 14 Sep 2015 15:39:31 GMT Expires: 0 Pragma: no-cache Pragma: no-cache Server: Apache-Coyote/1.1 Strict-Transport-Security: max-age=31536000 ; includeSubDomains X-Cf-Requestid: 8c1688dd-2a9e-4b44-4075-a84e88f5d6cd X-Content-Type-Options: nosniff X-Frame-Options: DENY X-Xss-Protection: 1; mode=block 872 {"access_token":"[PRIVATE DATA HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":599,"scope":"cloud_controller.read password.write cloud_controller.write openid doppler.firehose scim.write scim.read cloud_controller.admin","jti":"33c5626e-eb4c-459a-9a2c-f38295a80a22"} 0 FAILED Unauthorized error: You are not authorized. Error: Invalid authorization FAILED Unauthorized error: You are not authorized. Error: Invalid authorization Let me know thanks -Ramesh |
|
|
|
Amit Kumar Gupta
Hi Ramesh,
You mention Diego, but I doubt your app is actually trying to run on diego. You are not defaulting to the Diego backend, and I don't see anything about how you're pushing this specific app that would make it run on Diego. My guess is there is something misconfigured about your loggregator credentials. It appears to say you're not authorized to see logs, and "cf app" may be failing because it needs to talk to the loggregator system to determine health metrics for the app. Can you share the output of "CF_TRACE=true cf logs gs" Best, Amit On Mon, Sep 14, 2015 at 10:54 AM, Ramesh Sambandan <rsamban(a)gmail.com> wrote: I deployed diego release and still have issues. |
|
|
|
Ramesh Sambandan
I deployed diego release and still have issues.
1. Uploaded the app with no-start flag. 2. cf apps works 3. Started the app, got the failed status (app it self is working. I can get int o the app from browser) 4. cf apps and cf logs are failing (irrespective of how many times I try and how long I wait) Following are my command logs and the logs from all the bosh vms is in https://gist.github.com/6c81bed0d8eb888da970 1. diego_cf-deployment.yml - diego cf deployment 2. diego_diego-deployment.yml - diego diego deployment 3. jobsLogDiego.tgz - logs from all bosh vms *******************Uploading app******** guests-MacBook-Pro:apps rsamban$ cf push gs -p gs-serving-web-content-0.1.0.jar --no-start Creating app gs in org yesVin / space dev as admin... OK Creating route gs.192.168.1.103.xip.io... OK Binding gs.192.168.1.103.xip.io to gs... OK Uploading gs... Uploading app files from: gs-serving-web-content-0.1.0.jar Uploading 12.9M, 112 files Done uploading OK *******************cf apps SUCCESS******** guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... OK name requested state instances memory disk urls gs stopped 0/1 1G 1G gs.192.168.1.103.xip.io ******************Staring apps FAILED******** guests-MacBook-Pro:apps rsamban$ cf start gs Warning: error tailing logs Unauthorized error: You are not authorized. Error: Invalid authorization Starting app gs in org yesVin / space dev as admin... FAILED Start app timeout TIP: use 'cf logs gs --recent' for more information *********************cf logs FAILED*********** guests-MacBook-Pro:apps rsamban$ cf logs gs --recent Connected, dumping recent logs for app gs in org yesVin / space dev as admin... FAILED Unauthorized error: You are not authorized. Error: Invalid authorization *********************cf apps fails*********** guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... FAILED Server error, status code: 500, error code: 10001, message: An unknown error occurred. guests-MacBook-Pro:apps rsamban$ The logs for all the bosh vms are in jobsLogDiego.tgz Following is the bosh vms output ramesh(a)ubuntu:~/cloudFoundry/bosh-workplace/logs/6c81bed0d8eb888da970$ bosh vms Acting as user 'admin' on 'bosh2' Deployment `yesVinCloudFoundry' Director task 421 Task 421 done +------------------------------------+---------+---------------+---------------+ | Job/index | State | Resource Pool | IPs | +------------------------------------+---------+---------------+---------------+ | api_worker_z1/0 | running | small_z1 | 192.168.1.122 | | api_worker_z2/0 | running | small_z2 | 192.168.1.222 | | api_z1/0 | running | large_z1 | 192.168.1.120 | | api_z2/0 | running | large_z2 | 192.168.1.220 | | clock_global/0 | running | medium_z1 | 192.168.1.121 | | consul_z1/0 | running | medium_z1 | 192.168.1.117 | | doppler_z1/0 | running | medium_z1 | 192.168.1.107 | | doppler_z2/0 | running | medium_z2 | 192.168.1.207 | | etcd_z1/0 | running | medium_z1 | 192.168.1.114 | | etcd_z1/1 | running | medium_z1 | 192.168.1.115 | | etcd_z2/0 | running | medium_z2 | 192.168.1.213 | | ha_proxy_z1/0 | running | router_z1 | 192.168.1.103 | | hm9000_z1/0 | running | medium_z1 | 192.168.1.123 | | hm9000_z2/0 | running | medium_z2 | 192.168.1.223 | | loggregator_trafficcontroller_z1/0 | running | small_z1 | 192.168.1.108 | | loggregator_trafficcontroller_z2/0 | running | small_z2 | 192.168.1.208 | | nats_z1/0 | running | medium_z1 | 192.168.1.101 | | nats_z2/0 | running | medium_z2 | 192.168.1.201 | | nfs_z1/0 | running | medium_z1 | 192.168.1.102 | | postgres_z1/0 | running | medium_z1 | 192.168.1.112 | | router_z1/0 | running | router_z1 | 192.168.1.105 | | router_z2/0 | running | router_z2 | 192.168.1.205 | | runner_z1/0 | running | runner_z1 | 192.168.1.124 | | runner_z2/0 | running | runner_z2 | 192.168.1.224 | | stats_z1/0 | running | small_z1 | 192.168.1.110 | | uaa_z1/0 | running | medium_z1 | 192.168.1.111 | | uaa_z2/0 | running | medium_z2 | 192.168.1.211 | +------------------------------------+---------+---------------+---------------+ VMs total: 27 Deployment `yesVinCloudFoundry-diego' Director task 422 Task 422 done +--------------------+---------+------------------+---------------+ | Job/index | State | Resource Pool | IPs | +--------------------+---------+------------------+---------------+ | access_z1/0 | running | access_z1 | 192.168.1.152 | | brain_z1/0 | running | brain_z1 | 192.168.1.27 | | cc_bridge_z1/0 | running | cc_bridge_z1 | 192.168.1.29 | | cell_z1/0 | running | cell_z1 | 192.168.1.28 | | database_z1/0 | running | database_z1 | 192.168.1.26 | | route_emitter_z1/0 | running | route_emitter_z1 | 192.168.1.30 | +--------------------+---------+------------------+---------------+ VMs total: 6 I appears that once I start the app, things go awry for some reason. Even though staring app reports failure, the app is actually started I would really appreciate any help thanks -Ramesh |
|
|
|
Ramesh Sambandan
I am sure I created public gist. I think if we remove the .git (https://gist.github.com/6c81bed0d8eb888da970) from the end it works. of git clone https://gist.github.com/6c81bed0d8eb888da970.git also works,
Let me know thanks -Ramesh |
|
|
|
CF Runtime
Hey Ramesh,
It seems that gist you posted is either private or has since been deleted. We're trying to take a closer look at your logs and see what we can find. Thanks, Zak + Dan, CF OSS Integration team On Thu, Sep 10, 2015 at 12:34 PM, Ramesh Sambandan <rsamban(a)gmail.com> wrote: I just deployed cloud foundry in vsphere. I can cf login,create org, space |
|
|
|
Ramesh Sambandan
I just deployed cloud foundry in vsphere. I can cf login,create org, space etc. I can push app with —no -start flag. However, I cannot start the app. Once I start the app, “cf apps” does not work.(cf login still works). But after some time, the “cf apps” lists the apps until I again try to start the app.
The same app works fine when I deploy to bosh lite. guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... OK name requested state instances memory disk urls first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... OK name requested state instances memory disk urls first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io guests-MacBook-Pro:apps rsamban$ cf start first.jar App first.jar is already started guests-MacBook-Pro:apps rsamban$ cf restage first.jar Restaging app first.jar in org yesVin / space dev as admin... Warning: error tailing logs Unauthorized error: You are not authorized. Error: Invalid authorization FAILED Server error, status code: 400, error code: 170001, message: Staging error: no available stagers guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... FAILED Server error, status code: 500, error code: 10001, message: An unknown error occurred. guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... FAILED Server error, status code: 500, error code: 10001, message: An unknown error occurred. guests-MacBook-Pro:apps rsamban$ cf apps Getting apps in org yesVin / space dev as admin... OK name requested state instances memory disk urls first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io I searched the log files of all the vms for errors (Search for "no available stagers”, "Invalid authorization" and "An unknown error occurred" strings in all the bosh jobs). I can see “no available stagers” message in /var/vcap/sys/logs/cloud_controller_ng/cloud_controller_ng.log in api_z1 and other two strings are not found in any of the logs. How would I find the problem. Not sure what I am missing. Below are my logs and cf manifest. https://gist.github.com/6c81bed0d8eb888da970.git Can somebody help me, thanks Ramesh Apologize if it is too much clutter, I wish the mailing list supports some kind of formatting (like markdown) |
|
|