Date   

Re: Diego IAAS settings for vsphere

Daya Shetty <daya.shetty@...>
 

Hi, Eric,

FYI.. staging a buildpack-based app against the Diego backend also gave me the same Failure (InsufficientResources) but I was able to deploy and start the same buildpack based app using the Warden container.

I will check the logs you mentioned above to see if I can debug this problem. Surprising thing is that I was able to download the same apps successfully on my bosh-lite instance using the Diego backend.

Thanks
Daya


Re: Diego IAAS settings for vsphere

Daya Shetty <daya.shetty@...>
 

Hi, Eric,

FYI.. staging a buildpack-based app against the Diego backend also gave me the same Failure (InsufficientResources) but I was able to deploy and start the same buildpack based app using the Warden container.

I will check the logs you mentioned above to see if I can debug this problem. Surprising thing is that I was able to download the same apps successfully on my bosh-lite instance using the Diego backend.

Thanks
Daya


Re: etcd fails to start when trying to deploy diego with

Amit Kumar Gupta
 

That looks off. This is the code in the etcd startup script that would be
running in that version of cf-release:

https://github.com/cloudfoundry-incubator/etcd-release/blob/f3e1eca6fb4b0ee9464a07d48e232836580ba0d2/jobs/etcd/templates/etcd_bosh_utils.sh.erb#L48-L54

It should not be trying to sync with
https://etcd-z2-0.etcd.service.cf.internal:4001 on job etcd_z2/0, which is
the etcd cluster used by loggregator and routing-api, not diego. And in
that cluster, etcd does not run in SSL mode, and should not be trying to
sync with something.etcd.service.cf.internal, it should be trying to sync
with STATIC_IP_OF_ETCD_Z2/0:4001. Have you set "require_ssl: true" for
this etcd cluster in your manifest?

On Wed, Jan 27, 2016 at 8:38 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

What bosh instances --ps give to you ?

On Wed, Jan 27, 2016 at 8:03 AM, Martin Jackson <
martin(a)uncommonsense-uk.com> wrote:

Hi there I'm trying to deploy a CF release with Diego but I'm getting:

Started updating job etcd_z2 > etcd_z2/0 (canary). Failed: `etcd_z2/0' is
not running after update (00:10:46)
Error 400007: `etcd_z2/0' is not running after update

When I check the `etcd_ctl.err.log` on the failing node I can see:
```Error: cannot sync with the cluster using endpoints
https://etcd-z2-0.etcd.service.cf.internal:4001

# dig etcd-z2-0.etcd.service.cf.internal

; <<>> DiG 9.9.5-3ubuntu0.5-Ubuntu <<>> etcd-z2-0.etcd.service.cf.internal
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 60655
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;etcd-z2-0.etcd.service.cf.internal. IN A

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jan 26 09:14:40 UTC 2016
;; MSG SIZE rcvd: 52

So I can not resolv anything under the services domain.

deployment details:

+-----------+----------------------+----------------------------------------------+--------------+
| Name | Release(s) | Stemcell(s)
| Cloud Config |

+-----------+----------------------+----------------------------------------------+--------------+
| pulverize | cf/225 |
bosh-aws-xen-hvm-ubuntu-trusty-go_agent/3104 | none |
| | diego/0.1441.0 |
| |
| | etcd/18 |
| |
| | garden-linux/0.327.0 |
| |
| | nginx/2 |
| |

+-----------+----------------------+----------------------------------------------+--------------+

Regards

Martin Jackson


Re: Error 500 when testing new v229 CF deployment

Dieu Cao <dcao@...>
 

Hi James,

That sounds like you might be using a self signed cert and CC is failing
trying to do cert verification.
You can try specifying in your manifest properties.ssl.skip_cert_verify:
true
https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L44
We have a bug created recently to try to return a more helpful error
instead of a 500 for this case.
https://www.pivotaltracker.com/story/show/112371013

Hope that helps.

-Dieu
CF CAPI PM

On Wed, Jan 27, 2016 at 3:59 AM, James Leavers <james(a)cloudhelix.io> wrote:

Hi,

I have just set up a fresh v229 install to do some testing with - the bosh
deployment appeared to complete successfully and all the VMs are running,
however anything after initial authentication fails with an error 500:

iMac:~ jim$ cf login
API endpoint: https://api.app.x.y

Email> admin

Password>
Authenticating...
OK

FAILED
Error finding available orgs
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.


API endpoint: https://api.app.x.y (API version: 2.47.0)
User: admin
No org or space targeted, use 'cf target -o ORG -s SPACE'

iMac:~ jim$ CF_TRACE=true cf orgs
Getting orgs as admin...


REQUEST: [2016-01-27T11:48:25Z]
GET /v2/organizations HTTP/1.1
Host: api.app.x.y
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.15.0+fa1bfe2 / darwin



RESPONSE: [2016-01-27T11:48:26Z]
HTTP/1.1 500 Internal Server Error
Content-Length: 99
Content-Type: application/json;charset=utf-8
Date: Wed, 27 Jan 2016 11:48:26 GMT
Server: nginx
X-Cf-Requestid: fbf3ba39-65fd-4e21-67a5-f3ba7f9c0330
X-Content-Type-Options: nosniff
X-Vcap-Request-Id:
471ad2c7-0e09-46a9-6a31-6fe5024a1084::f9173c33-ee11-4513-a124-6ab04a37484d

{
"error_code": "UnknownError",
"description": "An unknown error occurred.",
"code": 10001
}

FAILED
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.
FAILED
Server error, status code: 500, error code: 10001, message: An unknown
error occurred.

I had a look in cloud_controller_ng.log on api_z1/0 & api_z2/0 and can see
the requests failing as follows:

{"timestamp":1453833820.1395981,"message":"Started GET
\"/v2/organizations\" for 86.152.144.203 with vcap-request-id:
d44f2a12-873f-437f-7d2c-f261f503640f::1d2ea0bc-1a78-49ea-85ff-bfb9c11d2f2b
at 2016-01-26 18:43:40
UTC","log_level":"info","source":"cc.api","data":{},"thread_id":47241509083720,"fiber_id":47241507190360,"process_id":5222,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb","lineno":12,"method":"call"}
{"timestamp":1453833820.1408544,"message":"Request failed: 500:
{\"code\"=>10001, \"description\"=>\"Neither PUB key nor PRIV key: header
too long\", \"error_code\"=>\"CF-RSAError\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in
`initialize'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in
`new'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in
`normalize_options'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:158:in
`initialize'\",
\"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:62:in
`new'\", \"/var/vcap/data/packag
es/cloud
_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:62:in
`decode_token_with_key'\",
\"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:53:in
`decode_token_with_asymmetric_key'\",
\"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:29:in
`decode_token'\",
\"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/cloud_controller/security/security_context_configurer.rb:22:in
`decode_token'\",
\"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controll
er_ng/li
b/cloud_controller/security/security_context_configurer.rb:10:in
`configure'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/controllers/base/front_controller.rb:26:in
`block in <class:FrontController>'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in
`block in compile!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in
`[]'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in
`block in process_route'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in
`catch'\", \"/var/vcap/packag
es/cloud
_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in
`process_route'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in
`block in filter!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in
`each'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in
`filter!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1083:in
`block in dispatch!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in
`block in invoke'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/si
natra/ba
se.rb:1066:in `catch'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in
`invoke'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1081:in
`dispatch!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in
`block in call!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in
`block in invoke'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in
`catch'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in
`invoke'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/r
uby/2.2.
0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in `call!'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:894:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/base.r
b:49:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/nulllogger.rb:9:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/head.rb:13:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:181:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:2021:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/urlmap.rb:66:in
`block in call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/r
ack/urlm
ap.rb:50:in `each'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb:21:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/vcap_request_id.rb:14:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:47:in
`call_app'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:12:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_metrics.rb:12:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/builder.rb:153:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:86:in
`block in pre_process'\", \"/var/vcap/packages
/cloud_c
ontroller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in
`catch'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in
`pre_process'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.8/lib/eventmachine.rb:1062:in
`call'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.8/lib/eventmachine.rb:1062:in
`block in
spawn_threadpool'\"]}","log_level":"error","source":"cc.api","data":{"request_guid":"d44f2a12-873f-437f-7d2c-f261f503640f::1d2ea0bc-1a78-49ea-85ff-bfb9c11d2f2b"},"thread_id":47241509083720,"fiber_id":47241507190360,"process_id":5222,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/lib/sinatra/vcap.rb","lineno":53,"method":"block
in registered"}

Any thoughts on what might be causing this? I guess it could be
SSL-related or UAA related from the above, however I haven't found anything
in any other logs to explain it yet (of course, I could be looking in the
wrong place!)

Thanks
James


AUFS bug in Linux kernel

Will Pragnell <wpragnell@...>
 

A bug with AUFS [1] was introduced in version 3.19.0-40 of the linux
kernel. This bug can cause containers to end up with unkillable zombie
processes with high CPU usage. This can happen any time a container is
supposed to be destroyed.

This affects both Garden-Linux and Warden (and Docker). If you see
significant slowdown or increased CPU usage on DEAs or Diego cells, it
might well be this. It will probably build slowly up over time, so you may
not notice anything for a while depending on the rate of app instance churn
on your deployment.

The bad version of the kernel is present in stemcell 3160 and later. I
can't recommend using older stemcells because the bad kernel versions also
include fixes for several high severity security vulnerabilities (at least
[2-5], there may be others I've missed). Were it not for these, rolling
back to stemcell 3157 would be the fix.

We're waiting for a fix to make its way into the kernel, and the BOSH team
will produce a stemcell with the fix as soon as possible. In the meantime,
I'd suggest simply keeping a closer eye than usual on your DEAs and Diego
cells.

If this issue occurs, the only solution is to recreate that machine. While
we've not had any actual reports of this issue occurring for Cloud Foundry
deployments in the wild yet, we're confident that the issue will be
occurring. The Diego team have seen it in testing, and several teams have
encountered the issue with their Concourse workers, which also use
Garden-Linux.

As always, please get in touch out if you have any questions.

Will - Garden PM

[1]: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1533043
[2]: http://www.ubuntu.com/usn/usn-2857-1/
[3]: http://www.ubuntu.com/usn/usn-2868-1/
[4]: http://www.ubuntu.com/usn/usn-2869-1/
[5]: http://www.ubuntu.com/usn/usn-2871-2/


Re: etcd fails to start when trying to deploy diego with

Gwenn Etourneau
 

What bosh instances --ps give to you ?

On Wed, Jan 27, 2016 at 8:03 AM, Martin Jackson <martin(a)uncommonsense-uk.com
wrote:
Hi there I'm trying to deploy a CF release with Diego but I'm getting:

Started updating job etcd_z2 > etcd_z2/0 (canary). Failed: `etcd_z2/0' is
not running after update (00:10:46)
Error 400007: `etcd_z2/0' is not running after update

When I check the `etcd_ctl.err.log` on the failing node I can see:
```Error: cannot sync with the cluster using endpoints
https://etcd-z2-0.etcd.service.cf.internal:4001

# dig etcd-z2-0.etcd.service.cf.internal

; <<>> DiG 9.9.5-3ubuntu0.5-Ubuntu <<>> etcd-z2-0.etcd.service.cf.internal
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 60655
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;etcd-z2-0.etcd.service.cf.internal. IN A

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jan 26 09:14:40 UTC 2016
;; MSG SIZE rcvd: 52

So I can not resolv anything under the services domain.

deployment details:

+-----------+----------------------+----------------------------------------------+--------------+
| Name | Release(s) | Stemcell(s)
| Cloud Config |

+-----------+----------------------+----------------------------------------------+--------------+
| pulverize | cf/225 |
bosh-aws-xen-hvm-ubuntu-trusty-go_agent/3104 | none |
| | diego/0.1441.0 |
| |
| | etcd/18 |
| |
| | garden-linux/0.327.0 |
| |
| | nginx/2 |
| |

+-----------+----------------------+----------------------------------------------+--------------+

Regards

Martin Jackson


etcd fails to start when trying to deploy diego with

Martin Jackson
 

Hi there I'm trying to deploy a CF release with Diego but I'm getting:

Started updating job etcd_z2 > etcd_z2/0 (canary). Failed: `etcd_z2/0' is not running after update (00:10:46)
Error 400007: `etcd_z2/0' is not running after update

When I check the `etcd_ctl.err.log` on the failing node I can see:
```Error: cannot sync with the cluster using endpoints https://etcd-z2-0.etcd.service.cf.internal:4001

# dig etcd-z2-0.etcd.service.cf.internal

; <<>> DiG 9.9.5-3ubuntu0.5-Ubuntu <<>> etcd-z2-0.etcd.service.cf.internal
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 60655
;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;etcd-z2-0.etcd.service.cf.internal. IN A

;; Query time: 1 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Tue Jan 26 09:14:40 UTC 2016
;; MSG SIZE rcvd: 52

So I can not resolv anything under the services domain.

deployment details:
+-----------+----------------------+----------------------------------------------+--------------+
| Name | Release(s) | Stemcell(s) | Cloud Config |
+-----------+----------------------+----------------------------------------------+--------------+
| pulverize | cf/225 | bosh-aws-xen-hvm-ubuntu-trusty-go_agent/3104 | none |
| | diego/0.1441.0 | | |
| | etcd/18 | | |
| | garden-linux/0.327.0 | | |
| | nginx/2 | | |
+-----------+----------------------+----------------------------------------------+--------------+

Regards

Martin Jackson


Re: push a python django app to cloudfoundry

Lynn Lin
 

Got your point,Dan

Thanks

Sent from my iPhone

Loris:leverage cloud foundry for @Scale DevOps


在 2016年1月27日,下午10:06,Daniel Mikusa <dmikusa(a)pivotal.io<mailto:dmikusa(a)pivotal.io>> 写道:

Correct. There's a couple ways you could do it.
- You can package the installer and run that before your app starts (assuming the installer can be run as a non-root user). You can use a `.profile.d` script to run the installer or just shell out in your app.
- You can pick individual binaries. This is trickier because then you need to pull in all of the dependencies (shared libraries, files, etc) of the binary. I find that the `ldd` is helpful to find the shared libraries required by a binary.

The first option is generally easier, while the second might allow you to package a smaller amount of files with your app.

Dan

Dan

On Wed, Jan 27, 2016 at 8:18 AM, Lin, Lynn <lynn.lin(a)emc.com<mailto:lynn.lin(a)emc.com>> wrote:
Thanks Dan
Before I check if we need root access,we not only package the binary but also its all dependency,correct?

Sent from my iPhone

Loris:leverage cloud foundry for @Scale DevOps


在 2016年1月27日,下午9:12,Daniel Mikusa <dmikusa(a)pivotal.io<mailto:dmikusa(a)pivotal.io>> 写道:

Apps don't have root access, so you can't `sudo` or `su`. Can you run this not as root? If so, you could package the binaries you need with the app.

Dan


On Wed, Jan 27, 2016 at 1:36 AM, Lynn Lin <Lynn.Lin(a)emc.com<mailto:Lynn.Lin(a)emc.com>> wrote:
Hi , we have a python django web app . In this web app, we will call VMware ovf tool (https://my.vmware.com/group/vmware/details?productId=491&downloadGroup=OVFTOOL410) .This tool is needed to be installed (sudo bash VMware-ovftool-4.1.0-2459827<tel:4.1.0-2459827>-lin.x86_64.bundle),How can we support this in CF V2?


Re: Enviroment variables HTTP_PROXY for CF CLI doen't work

JT Archie <jarchie@...>
 

Yu,

The environment variable HTTP_PROXY needs to be attached to the app, not
just the box.

On your app set the environment variable with `cf set-env <app_name>
HTTP_PROXY <proxy_server>`.

Then try to `cf push`.

Kind Regards,

JT

On Wed, Jan 27, 2016 at 4:40 AM, Yu, Yongcong <yuyc(a)cn.fujitsu.com> wrote:

Hello CF-dev community members!

I deployed CF on a PC with cf_nise_installer.
When pushing a app with a online buildpack as following, it's failed.
########################################################
cf push -b https://github.com/cloudfoundry/java-buildpack
Using manifest file /home/yuyc/svn/SampleAPP/Java/manifest.yml

Creating app business in org DevBox / space sp1 as admin...
OK

Creating route business.10.167.227.48.xip.io...
OK

Binding business.10.167.227.48.xip.io to business...
OK

Uploading business...
Done uploading
OK

Starting app business in org DevBox / space sp1 as admin...
-----> Downloaded app package (36K)
Cloning into '/tmp/buildpacks/java-buildpack'...
fatal: unable to access 'https://github.com/cloudfoundry/java-buildpack/':
Failed to connect to 10.167.227.38 port 1080: Connection refused
Cloning into '/tmp/buildpacks/java-buildpack'...
fatal: unable to access 'https://github.com/cloudfoundry/java-buildpack/':
Failed to connect to 10.167.227.38 port 1080: Connection refused
/var/vcap/packages/dea_next/buildpacks/lib/git.rb:23:in `clone': Git clone
failed: git clone --recursive
https://github.com/cloudfoundry/java-buildpack
/tmp/buildpacks/java-buildpack (RuntimeError)
from
/var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:154:in
`clone_buildpack'
from
/var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:120:in `build_pack'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:83:in
`block in compile_with_timeout'
from /usr/lib/ruby/1.9.1/timeout.rb:69:in `timeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:82:in
`compile_with_timeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:58:in
`block in stage_application'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in
`chdir'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in
`stage_application'
from /var/vcap/packages/dea_next/buildpacks/bin/run:10:in `<main>'


FAILED
StagingError
########################################################

Git clone has also been successfully worked on the host of CF where setup
the same environment variables HTTP_PROXY.
So, it looks like that HTTP_PROXY doesn't work when pushing app.

The problem happens at the time of starting app, so it seems that the
warden container can't access to the internet via HTTP_PROXY.
I also login the warden container and get the route of it, but I can't
ping the gateway on it.
The gateway of the warden container is assigned by the warden server, but
I found the gateway doesn't have a IP of IPV4
Cloud you tell me how to fix this problem?

Best wishes for you.
------------------------------------------------------------
Yu Yongcong
Development Dept. I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No.6 Wenzhu Road, Software Avenue, Nanjing, 210012, China
TEL: +86+25-86630566-8626
COINS: 7998-8626
FAX: +86+25-83317685
http://www.fujitsu.com/cn/fnst/ MAIL: yuyc(a)cn.fujitsu.com
------------------------------------------------------------





Re: push a python django app to cloudfoundry

Daniel Mikusa
 

Correct. There's a couple ways you could do it.
- You can package the installer and run that before your app starts
(assuming the installer can be run as a non-root user). You can use a
`.profile.d` script to run the installer or just shell out in your app.
- You can pick individual binaries. This is trickier because then you
need to pull in all of the dependencies (shared libraries, files, etc) of
the binary. I find that the `ldd` is helpful to find the shared libraries
required by a binary.

The first option is generally easier, while the second might allow you to
package a smaller amount of files with your app.

Dan

Dan

On Wed, Jan 27, 2016 at 8:18 AM, Lin, Lynn <lynn.lin(a)emc.com> wrote:

Thanks Dan
Before I check if we need root access,we not only package the binary but
also its all dependency,correct?

Sent from my iPhone

Loris:leverage cloud foundry for @Scale DevOps


在 2016年1月27日,下午9:12,Daniel Mikusa <dmikusa(a)pivotal.io> 写道:

Apps don't have root access, so you can't `sudo` or `su`. Can you run
this not as root? If so, you could package the binaries you need with the
app.

Dan


On Wed, Jan 27, 2016 at 1:36 AM, Lynn Lin <Lynn.Lin(a)emc.com> wrote:

Hi , we have a python django web app . In this web app, we will call
VMware ovf tool (
https://my.vmware.com/group/vmware/details?productId=491&downloadGroup=OVFTOOL410)
.This tool is needed to be installed (sudo bash VMware-ovftool-
4.1.0-2459827-lin.x86_64.bundle),How can we support this in CF V2?


Re: push a python django app to cloudfoundry

Lynn Lin
 

Thanks Dan
Before I check if we need root access,we not only package the binary but also its all dependency,correct?

Sent from my iPhone

Loris:leverage cloud foundry for @Scale DevOps


在 2016年1月27日,下午9:12,Daniel Mikusa <dmikusa(a)pivotal.io<mailto:dmikusa(a)pivotal.io>> 写道:

Apps don't have root access, so you can't `sudo` or `su`. Can you run this not as root? If so, you could package the binaries you need with the app.

Dan

On Wed, Jan 27, 2016 at 1:36 AM, Lynn Lin <Lynn.Lin(a)emc.com<mailto:Lynn.Lin(a)emc.com>> wrote:
Hi , we have a python django web app . In this web app, we will call VMware ovf tool (https://my.vmware.com/group/vmware/details?productId=491&downloadGroup=OVFTOOL410) .This tool is needed to be installed (sudo bash VMware-ovftool-4.1.0-2459827<tel:4.1.0-2459827>-lin.x86_64.bundle),How can we support this in CF V2?


Re: push a python django app to cloudfoundry

Daniel Mikusa
 

Apps don't have root access, so you can't `sudo` or `su`. Can you run this
not as root? If so, you could package the binaries you need with the app.

Dan

On Wed, Jan 27, 2016 at 1:36 AM, Lynn Lin <Lynn.Lin(a)emc.com> wrote:

Hi , we have a python django web app . In this web app, we will call
VMware ovf tool (
https://my.vmware.com/group/vmware/details?productId=491&downloadGroup=OVFTOOL410)
.This tool is needed to be installed (sudo bash VMware-ovftool-
4.1.0-2459827-lin.x86_64.bundle),How can we support this in CF V2?


Error 500 when testing new v229 CF deployment

James Leavers
 

Hi,

I have just set up a fresh v229 install to do some testing with - the bosh deployment appeared to complete successfully and all the VMs are running, however anything after initial authentication fails with an error 500:

iMac:~ jim$ cf login
API endpoint: https://api.app.x.y

Email> admin

Password>
Authenticating...
OK

FAILED
Error finding available orgs
Server error, status code: 500, error code: 10001, message: An unknown error occurred.


API endpoint: https://api.app.x.y (API version: 2.47.0)
User: admin
No org or space targeted, use 'cf target -o ORG -s SPACE'

iMac:~ jim$ CF_TRACE=true cf orgs
Getting orgs as admin...


REQUEST: [2016-01-27T11:48:25Z]
GET /v2/organizations HTTP/1.1
Host: api.app.x.y
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/json
User-Agent: go-cli 6.15.0+fa1bfe2 / darwin



RESPONSE: [2016-01-27T11:48:26Z]
HTTP/1.1 500 Internal Server Error
Content-Length: 99
Content-Type: application/json;charset=utf-8
Date: Wed, 27 Jan 2016 11:48:26 GMT
Server: nginx
X-Cf-Requestid: fbf3ba39-65fd-4e21-67a5-f3ba7f9c0330
X-Content-Type-Options: nosniff
X-Vcap-Request-Id: 471ad2c7-0e09-46a9-6a31-6fe5024a1084::f9173c33-ee11-4513-a124-6ab04a37484d

{
"error_code": "UnknownError",
"description": "An unknown error occurred.",
"code": 10001
}

FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.
FAILED
Server error, status code: 500, error code: 10001, message: An unknown error occurred.

I had a look in cloud_controller_ng.log on api_z1/0 & api_z2/0 and can see the requests failing as follows:

{"timestamp":1453833820.1395981,"message":"Started GET \"/v2/organizations\" for 86.152.144.203 with vcap-request-id: d44f2a12-873f-437f-7d2c-f261f503640f::1d2ea0bc-1a78-49ea-85ff-bfb9c11d2f2b at 2016-01-26 18:43:40 UTC","log_level":"info","source":"cc.api","data":{},"thread_id":47241509083720,"fiber_id":47241507190360,"process_id":5222,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb","lineno":12,"method":"call"}
{"timestamp":1453833820.1408544,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"Neither PUB key nor PRIV key: header too long\", \"error_code\"=>\"CF-RSAError\", \"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in `initialize'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in `new'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:51:in `normalize_options'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/cache/cf-uaa-lib-b1e11235dc6c/lib/uaa/token_coder.rb:158:in `initialize'\", \"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:62:in `new'\", \"/var/vcap/data/packages/cloud
_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:62:in `decode_token_with_key'\", \"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:53:in `decode_token_with_asymmetric_key'\", \"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/vcap/uaa_token_decoder.rb:29:in `decode_token'\", \"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/lib/cloud_controller/security/security_context_configurer.rb:22:in `decode_token'\", \"/var/vcap/data/packages/cloud_controller_ng/e2284b8d57577947694deb55d61cbb3a2c87ac06.1-7f1e9cd9c57e152bf8c7a71d84131f5a43476974/cloud_controller_ng/li
b/cloud_controller/security/security_context_configurer.rb:10:in `configure'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/controllers/base/front_controller.rb:26:in `block in <class:FrontController>'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1610:in `block in compile!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in `[]'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1014:in `block in process_route'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in `catch'\", \"/var/vcap/packages/cloud
_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1012:in `process_route'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in `block in filter!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in `each'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:965:in `filter!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1083:in `block in dispatch!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `block in invoke'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/ba
se.rb:1066:in `catch'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `invoke'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1081:in `dispatch!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in `block in call!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `block in invoke'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `catch'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:1066:in `invoke'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.
0/gems/sinatra-1.4.6/lib/sinatra/base.rb:906:in `call!'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:894:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in
`call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/nulllogger.rb:9:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/head.rb:13:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:181:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/sinatra-1.4.6/lib/sinatra/base.rb:2021:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/urlmap.rb:66:in `block in call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/urlm
ap.rb:50:in `each'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/urlmap.rb:50:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb:21:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/vcap_request_id.rb:14:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:47:in `call_app'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:12:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_metrics.rb:12:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/builder.rb:153:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:86:in `block in pre_process'\", \"/var/vcap/packages/cloud_c
ontroller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `catch'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `pre_process'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.8/lib/eventmachine.rb:1062:in `call'\", \"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.8/lib/eventmachine.rb:1062:in `block in spawn_threadpool'\"]}","log_level":"error","source":"cc.api","data":{"request_guid":"d44f2a12-873f-437f-7d2c-f261f503640f::1d2ea0bc-1a78-49ea-85ff-bfb9c11d2f2b"},"thread_id":47241509083720,"fiber_id":47241507190360,"process_id":5222,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/lib/sinatra/vcap.rb","lineno":53,"method":"block in registered"}

Any thoughts on what might be causing this? I guess it could be SSL-related or UAA related from the above, however I haven't found anything in any other logs to explain it yet (of course, I could be looking in the wrong place!)

Thanks
James


Re: Need help for diego deployment

Kinjal Doshi
 

Hi Eric,

Thanks a lot for the detailed response to my query.

I used the minimal-aws.yml configuration (
https://github.com/cloudfoundry/cf-release/tree/v226/example_manifests) to
create my deployment manifest which does not have the consul VMs set up. I
am guessing that the first step would be to change this.

In this case should I use the script generators to generate the CF
deployment manifest and re-deploy cloud foundry, or are there any other
techniques/shorter path for doing this?

Thanks in advance,
Kinjal

On Mon, Jan 25, 2016 at 6:57 AM, Eric Malm <emalm(a)pivotal.io> wrote:

Hi, Kinjal,

The stub I included in-line in my previous email may not have come through
so well for all mail clients, so I've also included it in a public gist at
https://gist.github.com/ematpl/149ac1bac691caae0722.

Thanks,
Eric

On Fri, Jan 22, 2016 at 6:32 PM, Eric Malm <emalm(a)pivotal.io> wrote:

Hi, Kinjal,

Thanks for asking: this is an area in which the Diego team is looking
forward to improving documentation and tooling in the near term. For the
time being, here are some more manual instructions:

Assuming you have AWS infrastructure already provisioned for your CF
deployment (VPC, subnets, NAT box, ELBs, etc.), you should need only to add
one or more additional subnets for the VMs in the Diego deployment, and
optionally an ELB for the SSH proxy routing tier (you can also use the
HAproxy in the CF deployment to do the same load-balancing, but you'll need
to give it an Elastic IP). If you're brave, and can coordinate the reserved
sections in the CF and Diego deployment manifests' networking configs
correctly, you could even share the same subnet(s) between the two
deployments.

Once you have those subnets provisioned, you'll need to translate their
properties into the iaas-settings.yml stub that you supply to the
generate-deployment-manifest script in diego-release. Since you're
deploying CF v226, we recommend you use Diego final version v0.1442.0 and
the associated manifest-generation script in that version of the release.
The other stubs should be independent of that iaas-settings one, and should
be pretty much the same as the ones for the BOSH-Lite deployment. You'll
likely want to provide different secrets and credentials in the
property-overrides stub, though, and perhaps different instance counts
depending on the availability needs of your deployment. I've included at
the end of this email a representative iaas-settings.yml file from one of
the Diego team's environments, with any specific identifiers for AWS
entities replaced by PLACEHOLDER values.

As a side note, if you don't already have the consul VMs deployed in your
CF deployment, you'll need to enable them so that the Diego components can
use it to communicate. We recommend you operate an odd number of consul
VMs: 1 if don't need high availability, and 3 or 5 if you do (as in a
production environment). You can enable them by changing the instance count
on the consul_z1 and consul_z2 jobs in the CF manifest.

After you've customized those stubs and adjusted your CF manifest if
necessary, you can generate the Diego manifest by running something like
the following from your diego-release directory:

$ ./scripts/generate-deployment-manifest \
PATH/TO/MY/CUSTOMIZED-PROPERTY-OVERRIDES.YML \
PATH/TO/MY/CUSTOMIZED-INSTANCE-COUNT-OVERRIDES.YML \
manifest-generation/bosh-lite-stubs/persistent-disk-overrides.yml \
PATH/TO/MY/CUSTOMIZED-IAAS-SETTINGS.YML \
manifest-generation/bosh-lite-stubs/additional-jobs.yml \
manifest-generation/bosh-lite-stubs/release-versions.yml \
PATH/TO/MY/MANIFEST/DIRECTORY \
> PATH/TO/MY/MANIFEST/DIRECTORY/diego.yml

'PATH/TO/MY/MANIFEST/DIRECTORY' should contain your CF manifest in a
file named 'cf.yml'. Also, please note that if you move to CF v227 or
later, which recommend Diego v0.1445.0 or later, the manifest-generation
script has changed to take its stub arguments via flags, instead of as
these positional arguments, and some of the stubs have changed slightly.

We also realize this is currently an obscure and potentially error-prone
process, and the Diego team does have a couple stories queued up to do soon
to provide more information about how to set up Diego on AWS:

- We plan in https://www.pivotaltracker.com/story/show/100909610 to
parametrize, document, and publish the tools and additional templates we
use to provision the AWS environments we use for CI and for our developers'
experiments and investigations, all the way from an empty account to a VPC
with BOSH, CF, and Diego.
- We plan in https://www.pivotaltracker.com/story/show/100909610 to
provide more manual instructions to set up a Diego environment compatible
with the 'minimal-aws' CF deployment manifest and infrastructure settings,
including provisioning any additional infrastructure such as subnets and
translating their information into the stubs for the diego-release
manifest-generation script.

We'll also be eager to adopt and to integrate with the tooling the CF
Infrastructure and CF Release Integration teams will produce at some point
to automate environment bootstrapping and CF manifest generation as much as
possible.

Please let me and the rest of the team know here if you need further
assistance or clarification.

Thanks again,
Eric, CF Runtime Diego PM

*****

Example iaas-settings.yml file, with PLACEHOLDER entries for your
environment's info:

iaas_settings:
compilation_cloud_properties:
availability_zone: us-east-1a
instance_type: c3.large
resource_pool_cloud_properties:
- cloud_properties:
availability_zone: us-east-1a
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z1
- cloud_properties:
availability_zone: us-east-1b
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z2
- cloud_properties:
availability_zone: us-east-1c
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: brain_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: brain_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: brain_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: cc_bridge_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: cc_bridge_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: cc_bridge_z3
- cloud_properties:
availability_zone: us-east-1a
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z1
- cloud_properties:
availability_zone: us-east-1b
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z2
- cloud_properties:
availability_zone: us-east-1c
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.large
name: colocated_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.large
name: colocated_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.large
name: colocated_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.large
name: database_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.large
name: database_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.large
name: database_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: route_emitter_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: route_emitter_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: route_emitter_z3
stemcell:
name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent
version: latest
subnet_configs:
- name: diego1
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-A
dns:
- 10.10.0.2
gateway: 10.10.5.1
range: 10.10.5.0/24
reserved:
- 10.10.5.2 - 10.10.5.9
static:
- 10.10.5.10 - 10.10.5.63
- name: diego2
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-B
dns:
- 10.10.0.2
gateway: 10.10.6.1
range: 10.10.6.0/24
reserved:
- 10.10.6.2 - 10.10.6.9
static:
- 10.10.6.10 - 10.10.6.63
- name: diego3
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-C
dns:
- 10.10.0.2
gateway: 10.10.7.1
range: 10.10.7.0/24
reserved:
- 10.10.7.2 - 10.10.7.9
static:
- 10.10.7.10 - 10.10.7.63


On Fri, Jan 22, 2016 at 4:28 AM, Kinjal Doshi <kindoshi(a)gmail.com> wrote:

Hi,

After deploying CF version 226 on AWS using microbosh, I am trying to
understand how to deploy Diego now to work with this version of CF but have
not been able to figure out much yet. I was able to find steps for
deploying Diego on BOSH-Lite at
https://github.com/cloudfoundry-incubator/diego-release#deploying-diego-to-bosh-lite
but not for BOSH.

Would appreciate some pointers in this direction .

Thanks in advance,
Kinjal


Re: [abacus] MongoDB client

Hristo Iliev
 

Hi,

I merged the new client as "mongoclient" module.

For now the only way to try it out is to rename it to dbclient and replace the existing CouchDB client with it.

Regards,
Hristo Iliev


Re: Cloud Foundry being used for an EU social learning games platform

Juan Antonio Breña Moral <bren at juanantonio.info...>
 

Good morning Danny,

"This is interesting and I'd like to obtain a better understanding of the use cases this library supports that drove you to publish it"

Can you describe in detail the question?
I suppose that the use case could be similar than Java does. In this stage the development has the focus on app deployment and a basic interaction with logging component and uaa provided by pivotal/bluemix and local instance from nise installer: https://github.com/yudai/cf_nise_installer but In next months, I would like to add bosh-lite in the tests. Besides, I would like to increate the tests with the deployment with Docker containers. In this area, I have some tests, but I recognice that I have to evolve to bosh-lite:
https://github.com/prosociallearnEU/cf-nodejs-client/blob/master/test/lib/model/cloudcontroller/DockerTests.js

Cheers

Juan Antonio


Enviroment variables HTTP_PROXY for CF CLI doen't work

Yu, Yongcong <yuyc@...>
 

Hello CF-dev community members!

I deployed CF on a PC with cf_nise_installer.
When pushing a app with a online buildpack as following, it's failed.
########################################################
cf push -b https://github.com/cloudfoundry/java-buildpack
Using manifest file /home/yuyc/svn/SampleAPP/Java/manifest.yml

Creating app business in org DevBox / space sp1 as admin...
OK

Creating route business.10.167.227.48.xip.io...
OK

Binding business.10.167.227.48.xip.io to business...
OK

Uploading business...
Done uploading
OK

Starting app business in org DevBox / space sp1 as admin...
-----> Downloaded app package (36K)
Cloning into '/tmp/buildpacks/java-buildpack'...
fatal: unable to access 'https://github.com/cloudfoundry/java-buildpack/': Failed to connect to 10.167.227.38 port 1080: Connection refused
Cloning into '/tmp/buildpacks/java-buildpack'...
fatal: unable to access 'https://github.com/cloudfoundry/java-buildpack/': Failed to connect to 10.167.227.38 port 1080: Connection refused
/var/vcap/packages/dea_next/buildpacks/lib/git.rb:23:in `clone': Git clone failed: git clone --recursive https://github.com/cloudfoundry/java-buildpack /tmp/buildpacks/java-buildpack (RuntimeError)
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:154:in `clone_buildpack'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:120:in `build_pack'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:83:in `block in compile_with_timeout'
from /usr/lib/ruby/1.9.1/timeout.rb:69:in `timeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:82:in `compile_with_timeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:58:in `block in stage_application'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in `chdir'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:53:in `stage_application'
from /var/vcap/packages/dea_next/buildpacks/bin/run:10:in `<main>'


FAILED
StagingError
########################################################

Git clone has also been successfully worked on the host of CF where setup the same environment variables HTTP_PROXY.
So, it looks like that HTTP_PROXY doesn't work when pushing app.

The problem happens at the time of starting app, so it seems that the warden container can't access to the internet via HTTP_PROXY.
I also login the warden container and get the route of it, but I can't ping the gateway on it.
The gateway of the warden container is assigned by the warden server, but I found the gateway doesn't have a IP of IPV4
Cloud you tell me how to fix this problem?

Best wishes for you.
------------------------------------------------------------
Yu Yongcong
Development Dept. I
Nanjing Fujitsu Nanda Software Tech. Co., Ltd.(FNST)
No.6 Wenzhu Road, Software Avenue, Nanjing, 210012, China
TEL: +86+25-86630566-8626
COINS: 7998-8626
FAX: +86+25-83317685
http://www.fujitsu.com/cn/fnst/ MAIL: yuyc(a)cn.fujitsu.com
------------------------------------------------------------


Re: Diego IAAS settings for vsphere

Eric Malm <emalm@...>
 

Hi, Daya,

I'd start by looking at the auctioneer logs at
`/var/vcap/sys/log/auctioneer/auctioneer.stdout.log` on the brain VMs and
the BBS logs at `/var/vcap/sys/log/bbs/bbs.stdout.log` on the database VMs.
You could also get the state response from each cell by curling `
http://CELL_IP:1800/state` <http://CELL_IP:1800/state> to see the resources
it advertises to the auctioneer. I think CC by default configures staging
tasks to use 6G of disk, so it's possible that none of the cells in the
deployment have the remaining capacity to accept the work, especially if
they're already running other workloads.

Are you able to stage or run a buildpack-based app against the Diego
backend?

Thanks,
Eric

On Tue, Jan 26, 2016 at 4:00 PM, Daya Shetty <daya.shetty(a)bnymellon.com>
wrote:

Hi Eric,

Thanks for your helpI I could conjure up an iass-settings for vsphere and
was able to deploy diego successfully but was having the same access_z1 VM
failing problem but was able to bring up diego after incorporating your
comments on this thread .


https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/JDD3SEP7CV6ONGED3DFAFYN2I54JQSIT/

Trying to push a docker app and I’m getting

cf push my-tcp-receiver -o cloudfoundry/tcp-sample-receiver
Creating app my-tcp-receiver in org BNYMellon / space testspace as admin...
OK

Creating route my-tcp-receiver.poc-apps.bnymellon.net...
OK

Binding my-tcp-receiver.poc-apps.bnymellon.net to my-tcp-receiver...
OK

Starting app my-tcp-receiver in org BNYMellon / space testspace as admin...
FAILED
InsufficientResources

I checked the stager.stdout.log on the cc-bridge and it shows the
following:

"timestamp":"1453833835.995078564","source":"stager","message":"stager.starting","log_level":1,"data":{}}
{"timestamp":"1453833835.995591879","source":"stager","message":"stager.Listening
for staging requests!","log_level":1,"data":{}}

{"timestamp":"1453850050.902931690","source":"stager","message":"stager.docker.build-recipe.staging-request","log_level":1,"data":{"app-id":"6992baee-ef29-4805-92ce-acac5268b78e","session":"2.1","staging-guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6"}}

{"timestamp":"1453850050.903894424","source":"stager","message":"stager.staging-handler.staging-request.desiring-task","log_level":1,"data":{"callback_url":"
http://stager.service.cf.internal:8888/v1/staging/6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6/completed
","session":"3.1","staging-guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6","task_guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6"}}

{"timestamp":"1453850051.046576500","source":"stager","message":"stager.completion-handler.task-complete-callback-received.posting-staging-complete","log_level":1,"data":{"guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6","payload":"eyJlcnJvciI6eyJpZCI6Ikluc3VmZmljaWVudFJlc291cmNlcyIsIm1lc3NhZ2UiOiJpbnN1ZmZpY2llbnQgcmVzb3VyY2VzIn19","session":"4.1"}}
{"timestamp":"1453850051.047005653","source":"stager","message":"stager.completion-handler.task-complete-callback-received.cc-client.delivering-staging-response","log_level":1,"data":{"guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6","payload":"{\"error\":{\"id\":\"InsufficientResources\",\"message\":\"insufficient
resources\"}}","session":"4.1.1"}}

{"timestamp":"1453850051.143590689","source":"stager","message":"stager.completion-handler.task-complete-callback-received.cc-client.delivered-staging-response","log_level":1,"data":{"guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6","session":"4.1.1"}}

{"timestamp":"1453850051.143915415","source":"stager","message":"stager.completion-handler.task-complete-callback-received.posted-staging-complete","log_level":1,"data":{"guid":"6992baee-ef29-4805-92ce-acac5268b78e-6e72ac5779384f68a04a245ade7184b6","session":"4.1”}}

I have the following releases deployed:

root(a)rsomtapae181 cf-release]# bosh releases
Acting as user 'admin' on 'bosh2'

+----------------------------+------------+-------------+
| Name | Versions | Commit Hash |
+----------------------------+------------+-------------+
| cf | 222+dev.1* | e4eb9f4b+ |
| diego | 0.1437.0* | 7a972628 |
| etcd | 16* | 793d1c6b |
| garden-linux | 0.308.0* | b5eced17 |
+----------------------------+------------+-------------+
(*) Currently deployed
(+) Uncommitted changes

Any ideas on where I could look to debug this issue?

Thanks
Daya


Re: Cloud Foundry being used for an EU social learning games platform

Amit Kumar Gupta
 

On Wed, Jan 27, 2016 at 12:40 AM, Danny Rosen <drosen(a)pivotal.io> wrote:

@Amit: Could you embellish on the use cases you've encountered that have
attempted to drive interactions with CF via a python library?

@Juan: This is interesting and I'd like to obtain a better understanding
of the use cases this library supports that drove you to publish it.

Thanks in advance for the insight,
Danny


On Wed, Jan 27, 2016 at 3:12 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Juan, good stuff! I've seen some people struggling recently to drive
interactions with CF via a Python library, it would be cool if there were
well-maintained client libraries in a bunch of different languages. It's
great you've written one for Node.

I don't know how the cloudfoundry-community GH org is administered, I'd
ask Dr Nic (cc'd).

Cheers,
Amit

On Tue, Jan 26, 2016 at 8:12 AM, Juan Antonio Breña Moral <
bren(a)juanantonio.info> wrote:

Good afternoon community,

after some months of, we have developed a stable client for cloud
foundry:
https://github.com/prosociallearnEU/cf-nodejs-client
https://travis-ci.org/prosociallearnEU/cf-nodejs-client
Docs: https://prosociallearneu.github.io/cf-nodejs-client/docs/v0.13.0/

I would like to explore the idea to share the development with the
community and how to integrate the development on the repo:
https://github.com/cloudfoundry-community/

Some useful developments are:
https://github.com/prosociallearnEU/cf-nodejs-dashboard
https://github.com/jthomas/cfbot

Cheers


Re: Cloud Foundry being used for an EU social learning games platform

Danny Rosen
 

@Amit: Could you embellish on the use cases you've encountered that have
attempted to drive interactions with CF via a python library?

@Juan: This is interesting and I'd like to obtain a better understanding of
the use cases this library supports that drove you to publish it.

Thanks in advance for the insight,
Danny

On Wed, Jan 27, 2016 at 3:12 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Juan, good stuff! I've seen some people struggling recently to drive
interactions with CF via a Python library, it would be cool if there were
well-maintained client libraries in a bunch of different languages. It's
great you've written one for Node.

I don't know how the cloudfoundry-community GH org is administered, I'd
ask Dr Nic (cc'd).

Cheers,
Amit

On Tue, Jan 26, 2016 at 8:12 AM, Juan Antonio Breña Moral <
bren(a)juanantonio.info> wrote:

Good afternoon community,

after some months of, we have developed a stable client for cloud foundry:
https://github.com/prosociallearnEU/cf-nodejs-client
https://travis-ci.org/prosociallearnEU/cf-nodejs-client
Docs: https://prosociallearneu.github.io/cf-nodejs-client/docs/v0.13.0/

I would like to explore the idea to share the development with the
community and how to integrate the development on the repo:
https://github.com/cloudfoundry-community/

Some useful developments are:
https://github.com/prosociallearnEU/cf-nodejs-dashboard
https://github.com/jthomas/cfbot

Cheers

5881 - 5900 of 9425