Cannot start app in new vsphere cf deployment


Ramesh Sambandan
 

I just deployed cloud foundry in vsphere. I can cf login,create org, space etc. I can push app with —no -start flag. However, I cannot start the app. Once I start the app, “cf apps” does not work.(cf login still works). But after some time, the “cf apps” lists the apps until I again try to start the app.

The same app works fine when I deploy to bosh lite.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io
guests-MacBook-Pro:apps rsamban$ cf start first.jar
App first.jar is already started
guests-MacBook-Pro:apps rsamban$ cf restage first.jar
Restaging app first.jar in org yesVin / space dev as admin...
Warning: error tailing logs
Unauthorized error: You are not authorized. Error: Invalid authorization
FAILED
Server error, status code: 400, error code: 170001, message: Staging error: no available stagers

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G firstjar.192.168.1.103.xip.io

I searched the log files of all the vms for errors (Search for "no available stagers”, "Invalid authorization" and "An unknown error occurred" strings in all the bosh jobs).
I can see “no available stagers” message in /var/vcap/sys/logs/cloud_controller_ng/cloud_controller_ng.log in api_z1 and other two strings are not found in any of the logs.

How would I find the problem. Not sure what I am missing.

Below are my logs and cf manifest.
https://gist.github.com/6c81bed0d8eb888da970.git

Can somebody help me,

thanks
Ramesh

Apologize if it is too much clutter, I wish the mailing list supports some kind of formatting (like markdown)


CF Runtime
 

Hey Ramesh,

It seems that gist you posted is either private or has since been deleted.
We're trying to take a closer look at your logs and see what we can find.

Thanks,
Zak + Dan, CF OSS Integration team

On Thu, Sep 10, 2015 at 12:34 PM, Ramesh Sambandan <rsamban(a)gmail.com>
wrote:

I just deployed cloud foundry in vsphere. I can cf login,create org, space
etc. I can push app with —no -start flag. However, I cannot start the app.
Once I start the app, “cf apps” does not work.(cf login still works). But
after some time, the “cf apps” lists the apps until I again try to start
the app.

The same app works fine when I deploy to bosh lite.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G
firstjar.192.168.1.103.xip.io
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G
firstjar.192.168.1.103.xip.io
guests-MacBook-Pro:apps rsamban$ cf start first.jar
App first.jar is already started
guests-MacBook-Pro:apps rsamban$ cf restage first.jar
Restaging app first.jar in org yesVin / space dev as admin...
Warning: error tailing logs
Unauthorized error: You are not authorized. Error: Invalid authorization
FAILED
Server error, status code: 400, error code: 170001, message: Staging
error: no available stagers

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.

guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
first.jar started 0/1 1G 1G
firstjar.192.168.1.103.xip.io

I searched the log files of all the vms for errors (Search for "no
available stagers”, "Invalid authorization" and "An unknown error occurred"
strings in all the bosh jobs).
I can see “no available stagers” message in
/var/vcap/sys/logs/cloud_controller_ng/cloud_controller_ng.log in api_z1
and other two strings are not found in any of the logs.

How would I find the problem. Not sure what I am missing.

Below are my logs and cf manifest.
https://gist.github.com/6c81bed0d8eb888da970.git

Can somebody help me,

thanks
Ramesh

Apologize if it is too much clutter, I wish the mailing list supports some
kind of formatting (like markdown)


Ramesh Sambandan
 

I am sure I created public gist. I think if we remove the .git (https://gist.github.com/6c81bed0d8eb888da970) from the end it works. of git clone https://gist.github.com/6c81bed0d8eb888da970.git also works,

Let me know

thanks
-Ramesh


Ramesh Sambandan
 

I deployed diego release and still have issues.

1. Uploaded the app with no-start flag.
2. cf apps works
3. Started the app, got the failed status (app it self is working. I can get int o the app from browser)
4. cf apps and cf logs are failing (irrespective of how many times I try and how long I wait)


Following are my command logs and the logs from all the bosh vms is in https://gist.github.com/6c81bed0d8eb888da970

1. diego_cf-deployment.yml - diego cf deployment
2. diego_diego-deployment.yml - diego diego deployment
3. jobsLogDiego.tgz - logs from all bosh vms

*******************Uploading app********
guests-MacBook-Pro:apps rsamban$ cf push gs -p gs-serving-web-content-0.1.0.jar --no-start
Creating app gs in org yesVin / space dev as admin...
OK

Creating route gs.192.168.1.103.xip.io...
OK

Binding gs.192.168.1.103.xip.io to gs...
OK

Uploading gs...
Uploading app files from: gs-serving-web-content-0.1.0.jar
Uploading 12.9M, 112 files
Done uploading
OK
*******************cf apps SUCCESS********
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
gs stopped 0/1 1G 1G gs.192.168.1.103.xip.io
******************Staring apps FAILED********
guests-MacBook-Pro:apps rsamban$ cf start gs
Warning: error tailing logs
Unauthorized error: You are not authorized. Error: Invalid authorization
Starting app gs in org yesVin / space dev as admin...

FAILED
Start app timeout

TIP: use 'cf logs gs --recent' for more information
*********************cf logs FAILED***********
guests-MacBook-Pro:apps rsamban$ cf logs gs --recent
Connected, dumping recent logs for app gs in org yesVin / space dev as admin...

FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization
*********************cf apps fails***********
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.
guests-MacBook-Pro:apps rsamban$

The logs for all the bosh vms are in jobsLogDiego.tgz
Following is the bosh vms output
ramesh(a)ubuntu:~/cloudFoundry/bosh-workplace/logs/6c81bed0d8eb888da970$ bosh vms
Acting as user 'admin' on 'bosh2'
Deployment `yesVinCloudFoundry'

Director task 421

Task 421 done

+------------------------------------+---------+---------------+---------------+
| Job/index | State | Resource Pool | IPs |
+------------------------------------+---------+---------------+---------------+
| api_worker_z1/0 | running | small_z1 | 192.168.1.122 |
| api_worker_z2/0 | running | small_z2 | 192.168.1.222 |
| api_z1/0 | running | large_z1 | 192.168.1.120 |
| api_z2/0 | running | large_z2 | 192.168.1.220 |
| clock_global/0 | running | medium_z1 | 192.168.1.121 |
| consul_z1/0 | running | medium_z1 | 192.168.1.117 |
| doppler_z1/0 | running | medium_z1 | 192.168.1.107 |
| doppler_z2/0 | running | medium_z2 | 192.168.1.207 |
| etcd_z1/0 | running | medium_z1 | 192.168.1.114 |
| etcd_z1/1 | running | medium_z1 | 192.168.1.115 |
| etcd_z2/0 | running | medium_z2 | 192.168.1.213 |
| ha_proxy_z1/0 | running | router_z1 | 192.168.1.103 |
| hm9000_z1/0 | running | medium_z1 | 192.168.1.123 |
| hm9000_z2/0 | running | medium_z2 | 192.168.1.223 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 | 192.168.1.108 |
| loggregator_trafficcontroller_z2/0 | running | small_z2 | 192.168.1.208 |
| nats_z1/0 | running | medium_z1 | 192.168.1.101 |
| nats_z2/0 | running | medium_z2 | 192.168.1.201 |
| nfs_z1/0 | running | medium_z1 | 192.168.1.102 |
| postgres_z1/0 | running | medium_z1 | 192.168.1.112 |
| router_z1/0 | running | router_z1 | 192.168.1.105 |
| router_z2/0 | running | router_z2 | 192.168.1.205 |
| runner_z1/0 | running | runner_z1 | 192.168.1.124 |
| runner_z2/0 | running | runner_z2 | 192.168.1.224 |
| stats_z1/0 | running | small_z1 | 192.168.1.110 |
| uaa_z1/0 | running | medium_z1 | 192.168.1.111 |
| uaa_z2/0 | running | medium_z2 | 192.168.1.211 |
+------------------------------------+---------+---------------+---------------+

VMs total: 27
Deployment `yesVinCloudFoundry-diego'

Director task 422

Task 422 done

+--------------------+---------+------------------+---------------+
| Job/index | State | Resource Pool | IPs |
+--------------------+---------+------------------+---------------+
| access_z1/0 | running | access_z1 | 192.168.1.152 |
| brain_z1/0 | running | brain_z1 | 192.168.1.27 |
| cc_bridge_z1/0 | running | cc_bridge_z1 | 192.168.1.29 |
| cell_z1/0 | running | cell_z1 | 192.168.1.28 |
| database_z1/0 | running | database_z1 | 192.168.1.26 |
| route_emitter_z1/0 | running | route_emitter_z1 | 192.168.1.30 |
+--------------------+---------+------------------+---------------+

VMs total: 6

I appears that once I start the app, things go awry for some reason. Even though staring app reports failure, the app is actually started

I would really appreciate any help

thanks
-Ramesh


Amit Kumar Gupta
 

Hi Ramesh,

You mention Diego, but I doubt your app is actually trying to run on
diego. You are not defaulting to the Diego backend, and I don't see
anything about how you're pushing this specific app that would make it run
on Diego.

My guess is there is something misconfigured about your loggregator
credentials. It appears to say you're not authorized to see logs, and "cf
app" may be failing because it needs to talk to the loggregator system to
determine health metrics for the app.

Can you share the output of "CF_TRACE=true cf logs gs"

Best,
Amit

On Mon, Sep 14, 2015 at 10:54 AM, Ramesh Sambandan <rsamban(a)gmail.com>
wrote:

I deployed diego release and still have issues.

1. Uploaded the app with no-start flag.
2. cf apps works
3. Started the app, got the failed status (app it self is working. I can
get int o the app from browser)
4. cf apps and cf logs are failing (irrespective of how many times I try
and how long I wait)


Following are my command logs and the logs from all the bosh vms is in
https://gist.github.com/6c81bed0d8eb888da970

1. diego_cf-deployment.yml - diego cf deployment
2. diego_diego-deployment.yml - diego diego deployment
3. jobsLogDiego.tgz - logs from all bosh vms

*******************Uploading app********
guests-MacBook-Pro:apps rsamban$ cf push gs -p
gs-serving-web-content-0.1.0.jar --no-start
Creating app gs in org yesVin / space dev as admin...
OK

Creating route gs.192.168.1.103.xip.io...
OK

Binding gs.192.168.1.103.xip.io to gs...
OK

Uploading gs...
Uploading app files from: gs-serving-web-content-0.1.0.jar
Uploading 12.9M, 112 files
Done uploading
OK
*******************cf apps SUCCESS********
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
OK

name requested state instances memory disk urls
gs stopped 0/1 1G 1G
gs.192.168.1.103.xip.io
******************Staring apps FAILED********
guests-MacBook-Pro:apps rsamban$ cf start gs
Warning: error tailing logs
Unauthorized error: You are not authorized. Error: Invalid authorization
Starting app gs in org yesVin / space dev as admin...

FAILED
Start app timeout

TIP: use 'cf logs gs --recent' for more information
*********************cf logs FAILED***********
guests-MacBook-Pro:apps rsamban$ cf logs gs --recent
Connected, dumping recent logs for app gs in org yesVin / space dev as
admin...

FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization
*********************cf apps fails***********
guests-MacBook-Pro:apps rsamban$ cf apps
Getting apps in org yesVin / space dev as admin...
FAILED
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.
guests-MacBook-Pro:apps rsamban$

The logs for all the bosh vms are in jobsLogDiego.tgz
Following is the bosh vms output
ramesh(a)ubuntu:~/cloudFoundry/bosh-workplace/logs/6c81bed0d8eb888da970$
bosh vms
Acting as user 'admin' on 'bosh2'
Deployment `yesVinCloudFoundry'

Director task 421

Task 421 done


+------------------------------------+---------+---------------+---------------+
| Job/index | State | Resource Pool | IPs
|

+------------------------------------+---------+---------------+---------------+
| api_worker_z1/0 | running | small_z1 |
192.168.1.122 |
| api_worker_z2/0 | running | small_z2 |
192.168.1.222 |
| api_z1/0 | running | large_z1 |
192.168.1.120 |
| api_z2/0 | running | large_z2 |
192.168.1.220 |
| clock_global/0 | running | medium_z1 |
192.168.1.121 |
| consul_z1/0 | running | medium_z1 |
192.168.1.117 |
| doppler_z1/0 | running | medium_z1 |
192.168.1.107 |
| doppler_z2/0 | running | medium_z2 |
192.168.1.207 |
| etcd_z1/0 | running | medium_z1 |
192.168.1.114 |
| etcd_z1/1 | running | medium_z1 |
192.168.1.115 |
| etcd_z2/0 | running | medium_z2 |
192.168.1.213 |
| ha_proxy_z1/0 | running | router_z1 |
192.168.1.103 |
| hm9000_z1/0 | running | medium_z1 |
192.168.1.123 |
| hm9000_z2/0 | running | medium_z2 |
192.168.1.223 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 |
192.168.1.108 |
| loggregator_trafficcontroller_z2/0 | running | small_z2 |
192.168.1.208 |
| nats_z1/0 | running | medium_z1 |
192.168.1.101 |
| nats_z2/0 | running | medium_z2 |
192.168.1.201 |
| nfs_z1/0 | running | medium_z1 |
192.168.1.102 |
| postgres_z1/0 | running | medium_z1 |
192.168.1.112 |
| router_z1/0 | running | router_z1 |
192.168.1.105 |
| router_z2/0 | running | router_z2 |
192.168.1.205 |
| runner_z1/0 | running | runner_z1 |
192.168.1.124 |
| runner_z2/0 | running | runner_z2 |
192.168.1.224 |
| stats_z1/0 | running | small_z1 |
192.168.1.110 |
| uaa_z1/0 | running | medium_z1 |
192.168.1.111 |
| uaa_z2/0 | running | medium_z2 |
192.168.1.211 |

+------------------------------------+---------+---------------+---------------+

VMs total: 27
Deployment `yesVinCloudFoundry-diego'

Director task 422

Task 422 done

+--------------------+---------+------------------+---------------+
| Job/index | State | Resource Pool | IPs |
+--------------------+---------+------------------+---------------+
| access_z1/0 | running | access_z1 | 192.168.1.152 |
| brain_z1/0 | running | brain_z1 | 192.168.1.27 |
| cc_bridge_z1/0 | running | cc_bridge_z1 | 192.168.1.29 |
| cell_z1/0 | running | cell_z1 | 192.168.1.28 |
| database_z1/0 | running | database_z1 | 192.168.1.26 |
| route_emitter_z1/0 | running | route_emitter_z1 | 192.168.1.30 |
+--------------------+---------+------------------+---------------+

VMs total: 6

I appears that once I start the app, things go awry for some reason. Even
though staring app reports failure, the app is actually started

I would really appreciate any help

thanks
-Ramesh


Ramesh Sambandan
 

Amit,

Below are my cf trace output.
guests-MacBook-Pro:apps rsamban$ cf logs gs --recent

REQUEST: [2015-09-14T09:39:28-06:00]
GET /v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps?q=name%3Ags&inline-relations-depth=1 HTTP/1.1
Host: api.192.168.1.103.xip.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / darwin



RESPONSE: [2015-09-14T09:39:28-06:00]
HTTP/1.1 200 OK
Content-Length: 5311
Content-Type: application/json;charset=utf-8
Date: Mon, 14 Sep 2015 15:39:28 GMT
Server: nginx
X-Cf-Requestid: 3d46e09a-5ef4-4f0d-5793-8d145a1b91a2
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: eab0cd51-7f04-4745-6ebf-0461df755620::4395680b-f74f-4325-a952-6d9fb974e3ce

{
"total_results": 1,
"total_pages": 1,
"prev_url": null,
"next_url": null,
"resources": [
{
"metadata": {
"guid": "48d31982-7e60-4de2-9d33-fdbf291ec779",
"url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779",
"created_at": "2015-09-14T13:15:50Z",
"updated_at": "2015-09-14T14:02:52Z"
},
"entity": {
"name": "gs",
"production": false,
"space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"stack_guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8",
"buildpack": null,
"detected_buildpack": "java-buildpack=v3.1.1-https://github.com/cloudfoundry/java-buildpack#7a538fb java-main open-jdk-like-jre=1.8.0_60 open-jdk-like-memory-calculator=1.1.1_RELEASE spring-auto-reconfiguration=1.10.0_RELEASE",
"environment_json": {

},
"memory": 1024,
"instances": 1,
"disk_quota": 1024,
"state": "STARTED",
"version": "40963450-8b7d-4478-bb84-682dee103a08",
"command": null,
"console": false,
"debug": null,
"staging_task_id": "0bd355ee625b4f1e814b7e1bc4cd0157",
"package_state": "STAGED",
"health_check_type": "port",
"health_check_timeout": null,
"staging_failed_reason": "StagingTimeExpired",
"staging_failed_description": null,
"diego": false,
"docker_image": null,
"package_updated_at": "2015-09-14T13:16:04Z",
"detected_start_command": "CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE -memorySizes=metaspace:64m.. -memoryWeights=heap:75,metaspace:10,stack:5,native:10 -totMemory=$MEMORY_LIMIT) && SERVER_PORT=$PORT $PWD/.java-buildpack/open_jdk_jre/bin/java -cp $PWD/.:$PWD/.java-buildpack/spring_auto_reconfiguration/spring_auto_reconfiguration-1.10.0_RELEASE.jar -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh $CALCULATED_MEMORY org.springframework.boot.loader.JarLauncher",
"enable_ssh": true,
"docker_credentials_json": {
"redacted_message": "[PRIVATE DATA HIDDEN]"
},
"space_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"space": {
"metadata": {
"guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"created_at": "2015-09-14T13:14:52Z",
"updated_at": null
},
"entity": {
"name": "dev",
"organization_guid": "6ee4b17e-ca40-47f2-b949-c7692324112a",
"space_quota_definition_guid": null,
"allow_ssh": true,
"organization_url": "/v2/organizations/6ee4b17e-ca40-47f2-b949-c7692324112a",
"developers_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/developers",
"managers_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/managers",
"auditors_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/auditors",
"apps_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps",
"routes_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/routes",
"domains_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/domains",
"service_instances_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/service_instances",
"app_events_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/app_events",
"events_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/events",
"security_groups_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/security_groups"
}
},
"stack_url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8",
"stack": {
"metadata": {
"guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8",
"url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8",
"created_at": "2015-09-14T05:55:54Z",
"updated_at": null
},
"entity": {
"name": "cflinuxfs2",
"description": "Cloud Foundry Linux-based filesystem"
}
},
"events_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/events",
"service_bindings_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/service_bindings",
"service_bindings": [

],
"routes_url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/routes",
"routes": [
{
"metadata": {
"guid": "4b3968ca-8821-496b-b4e9-47b54303f92f",
"url": "/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f",
"created_at": "2015-09-14T13:15:50Z",
"updated_at": null
},
"entity": {
"host": "gs",
"path": "",
"domain_guid": "a6cc2662-2771-4737-89de-992a2d606997",
"space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"domain_url": "/v2/domains/a6cc2662-2771-4737-89de-992a2d606997",
"space_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"apps_url": "/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f/apps"
}
}
]
}
}
]
}
Connected, dumping recent logs for app gs in org yesVin / space dev as admin...


REQUEST: [2015-09-14T09:39:29-06:00]
POST /oauth/token HTTP/1.1
Host: login.192.168.1.103.xip.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.12.3-c0c9a03 / darwin

grant_type=refresh_token&refresh_token=eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJiOWQzMTE5NC1iM2ZhLTQ4MmQtYmE0Ni1hZjRjMjRhYWUwMWYiLCJzdWIiOiIxY2UwZWQ4Yi1hMTdiLTQyZGEtYTE2Ni04MmZlYWQ1OTQyMGMiLCJzY29wZSI6WyJvcGVuaWQiLCJzY2ltLnJlYWQiLCJjbG91ZF9jb250cm9sbGVyLmFkbWluIiwiY2xvdWRfY29udHJvbGxlci5yZWFkIiwicGFzc3dvcmQud3JpdGUiLCJjbG91ZF9jb250cm9sbGVyLndyaXRlIiwiZG9wcGxlci5maXJlaG9zZSIsInNjaW0ud3JpdGUiXSwiaWF0IjoxNDQyMjM2NDU2LCJleHAiOjE0NDQ4Mjg0NTYsImNpZCI6ImNmIiwiY2xpZW50X2lkIjoiY2YiLCJpc3MiOiJodHRwczovL3VhYS4xOTIuMTY4LjEuMTAzLnhpcC5pby9vYXV0aC90b2tlbiIsInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfbmFtZSI6ImFkbWluIiwidXNlcl9pZCI6IjFjZTBlZDhiLWExN2ItNDJkYS1hMTY2LTgyZmVhZDU5NDIwYyIsInJldl9zaWciOiI1NGEzZjhkYyIsImF1ZCI6WyJjZiIsIm9wZW5pZCIsInNjaW0iLCJjbG91ZF9jb250cm9sbGVyIiwicGFzc3dvcmQiLCJkb3BwbGVyIl19.GLBJ3Px81ccR76QppqDF0ilyvxeXQO21j-XwOUVbQTllg3h8nLgPHShMDOuy7eOecBhJUSLbesahJME196LHYZYV1iOad9WhNASO11gdfqJ_0rTEGwZQEzwY2q9ggaoAv_YUjAmZOYCM8052K6LdKtROOqFd67CdTdrC0L8K1eo&scope=

RESPONSE: [2015-09-14T09:39:31-06:00]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-store
Content-Type: application/json;charset=UTF-8
Date: Mon, 14 Sep 2015 15:39:31 GMT
Expires: 0
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Cf-Requestid: 8c1688dd-2a9e-4b44-4075-a84e88f5d6cd
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

872
{"access_token":"[PRIVATE DATA HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":599,"scope":"cloud_controller.read password.write cloud_controller.write openid doppler.firehose scim.write scim.read cloud_controller.admin","jti":"33c5626e-eb4c-459a-9a2c-f38295a80a22"}
0


FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization
FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization

Let me know

thanks
-Ramesh


CF Runtime
 

Hi Ramesh,

Looking at the original logs you provided, we see the following error when
the CC tries to talk to HM9000:

"SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B:
certificate verify failed"

We're assuming that your certificate used by ha_proxy is a self signed
cert? Setting the "ssl.skip_cert_verify" to true in your manifest should
make the CC ignore the fact that it is self signed.

Joseph & Dan
OSS Release Integration team

On Mon, Sep 14, 2015 at 9:03 AM, Ramesh Sambandan <rsamban(a)gmail.com> wrote:

Amit,

Below are my cf trace output.
guests-MacBook-Pro:apps rsamban$ cf logs gs --recent

REQUEST: [2015-09-14T09:39:28-06:00]
GET
/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps?q=name%3Ags&inline-relations-depth=1
HTTP/1.1
Host: api.192.168.1.103.xip.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.12.3-c0c9a03 / darwin



RESPONSE: [2015-09-14T09:39:28-06:00]
HTTP/1.1 200 OK
Content-Length: 5311
Content-Type: application/json;charset=utf-8
Date: Mon, 14 Sep 2015 15:39:28 GMT
Server: nginx
X-Cf-Requestid: 3d46e09a-5ef4-4f0d-5793-8d145a1b91a2
X-Content-Type-Options: nosniff
X-Vcap-Request-Id:
eab0cd51-7f04-4745-6ebf-0461df755620::4395680b-f74f-4325-a952-6d9fb974e3ce

{
"total_results": 1,
"total_pages": 1,
"prev_url": null,
"next_url": null,
"resources": [
{
"metadata": {
"guid": "48d31982-7e60-4de2-9d33-fdbf291ec779",
"url": "/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779",
"created_at": "2015-09-14T13:15:50Z",
"updated_at": "2015-09-14T14:02:52Z"
},
"entity": {
"name": "gs",
"production": false,
"space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"stack_guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8",
"buildpack": null,
"detected_buildpack": "java-buildpack=v3.1.1-
https://github.com/cloudfoundry/java-buildpack#7a538fb java-main
open-jdk-like-jre=1.8.0_60 open-jdk-like-memory-calculator=1.1.1_RELEASE
spring-auto-reconfiguration=1.10.0_RELEASE",
"environment_json": {

},
"memory": 1024,
"instances": 1,
"disk_quota": 1024,
"state": "STARTED",
"version": "40963450-8b7d-4478-bb84-682dee103a08",
"command": null,
"console": false,
"debug": null,
"staging_task_id": "0bd355ee625b4f1e814b7e1bc4cd0157",
"package_state": "STAGED",
"health_check_type": "port",
"health_check_timeout": null,
"staging_failed_reason": "StagingTimeExpired",
"staging_failed_description": null,
"diego": false,
"docker_image": null,
"package_updated_at": "2015-09-14T13:16:04Z",
"detected_start_command":
"CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && SERVER_PORT=$PORT
$PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/.:$PWD/.java-buildpack/spring_auto_reconfiguration/spring_auto_reconfiguration-1.10.0_RELEASE.jar
-Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY org.springframework.boot.loader.JarLauncher",
"enable_ssh": true,
"docker_credentials_json": {
"redacted_message": "[PRIVATE DATA HIDDEN]"
},
"space_url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"space": {
"metadata": {
"guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"url": "/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"created_at": "2015-09-14T13:14:52Z",
"updated_at": null
},
"entity": {
"name": "dev",
"organization_guid": "6ee4b17e-ca40-47f2-b949-c7692324112a",
"space_quota_definition_guid": null,
"allow_ssh": true,
"organization_url":
"/v2/organizations/6ee4b17e-ca40-47f2-b949-c7692324112a",
"developers_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/developers",
"managers_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/managers",
"auditors_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/auditors",
"apps_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/apps",
"routes_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/routes",
"domains_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/domains",
"service_instances_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/service_instances",
"app_events_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/app_events",
"events_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/events",
"security_groups_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826/security_groups"
}
},
"stack_url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8",
"stack": {
"metadata": {
"guid": "4d62cffe-549a-4952-8fe7-977cc2d562d8",
"url": "/v2/stacks/4d62cffe-549a-4952-8fe7-977cc2d562d8",
"created_at": "2015-09-14T05:55:54Z",
"updated_at": null
},
"entity": {
"name": "cflinuxfs2",
"description": "Cloud Foundry Linux-based filesystem"
}
},
"events_url":
"/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/events",
"service_bindings_url":
"/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/service_bindings",
"service_bindings": [

],
"routes_url":
"/v2/apps/48d31982-7e60-4de2-9d33-fdbf291ec779/routes",
"routes": [
{
"metadata": {
"guid": "4b3968ca-8821-496b-b4e9-47b54303f92f",
"url": "/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f",
"created_at": "2015-09-14T13:15:50Z",
"updated_at": null
},
"entity": {
"host": "gs",
"path": "",
"domain_guid": "a6cc2662-2771-4737-89de-992a2d606997",
"space_guid": "6ecadc21-314d-4f93-8afb-b646cd384826",
"domain_url":
"/v2/domains/a6cc2662-2771-4737-89de-992a2d606997",
"space_url":
"/v2/spaces/6ecadc21-314d-4f93-8afb-b646cd384826",
"apps_url":
"/v2/routes/4b3968ca-8821-496b-b4e9-47b54303f92f/apps"
}
}
]
}
}
]
}
Connected, dumping recent logs for app gs in org yesVin / space dev as
admin...


REQUEST: [2015-09-14T09:39:29-06:00]
POST /oauth/token HTTP/1.1
Host: login.192.168.1.103.xip.io
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.12.3-c0c9a03 / darwin


grant_type=refresh_token&refresh_token=eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJiOWQzMTE5NC1iM2ZhLTQ4MmQtYmE0Ni1hZjRjMjRhYWUwMWYiLCJzdWIiOiIxY2UwZWQ4Yi1hMTdiLTQyZGEtYTE2Ni04MmZlYWQ1OTQyMGMiLCJzY29wZSI6WyJvcGVuaWQiLCJzY2ltLnJlYWQiLCJjbG91ZF9jb250cm9sbGVyLmFkbWluIiwiY2xvdWRfY29udHJvbGxlci5yZWFkIiwicGFzc3dvcmQud3JpdGUiLCJjbG91ZF9jb250cm9sbGVyLndyaXRlIiwiZG9wcGxlci5maXJlaG9zZSIsInNjaW0ud3JpdGUiXSwiaWF0IjoxNDQyMjM2NDU2LCJleHAiOjE0NDQ4Mjg0NTYsImNpZCI6ImNmIiwiY2xpZW50X2lkIjoiY2YiLCJpc3MiOiJodHRwczovL3VhYS4xOTIuMTY4LjEuMTAzLnhpcC5pby9vYXV0aC90b2tlbiIsInppZCI6InVhYSIsImdyYW50X3R5cGUiOiJwYXNzd29yZCIsInVzZXJfbmFtZSI6ImFkbWluIiwidXNlcl9pZCI6IjFjZTBlZDhiLWExN2ItNDJkYS1hMTY2LTgyZmVhZDU5NDIwYyIsInJldl9zaWciOiI1NGEzZjhkYyIsImF1ZCI6WyJjZiIsIm9wZW5pZCIsInNjaW0iLCJjbG91ZF9jb250cm9sbGVyIiwicGFzc3dvcmQiLCJkb3BwbGVyIl19.GLBJ3Px81ccR76QppqDF0ilyvxeXQO21j-XwOUVbQTllg3h8nLgPHShMDOuy7eOecBhJUSLbesahJME196LHYZYV1iOad9WhNASO11gdfqJ_0rTEGwZQEzwY2q9ggaoAv_YUjAmZOYCM8052K6LdKtROOqFd67CdTdrC0L8K1eo&scope=

RESPONSE: [2015-09-14T09:39:31-06:00]
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Access-Control-Allow-Origin: *
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-store
Content-Type: application/json;charset=UTF-8
Date: Mon, 14 Sep 2015 15:39:31 GMT
Expires: 0
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Cf-Requestid: 8c1688dd-2a9e-4b44-4075-a84e88f5d6cd
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

872
{"access_token":"[PRIVATE DATA
HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA
HIDDEN]","expires_in":599,"scope":"cloud_controller.read password.write
cloud_controller.write openid doppler.firehose scim.write scim.read
cloud_controller.admin","jti":"33c5626e-eb4c-459a-9a2c-f38295a80a22"}
0


FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization
FAILED
Unauthorized error: You are not authorized. Error: Invalid authorization

Let me know

thanks
-Ramesh


Ramesh Sambandan
 

awesome!
I changed the property and redeployed.
It works!

Thanks you very much.

-Ramesh