Date   

Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Kayode,

I am using the default bosh-lite-v231.yml file and the instances for nfs server is set to 0:

vagrant(a)agent-id-bosh-0:~$ egrep -i "name:.*nfs|instances" bosh-lite-v231.yml.1603041454
etc...
- instances: 0
- instances: 0
- instances: 0
name: nfs_z1
- name: debian_nfs_server
- instances: 1
- instances: 1
- instances: 1
etc...

So it is not running.

Thanks,


Update Parallelization in Cloud Foundry

Omar Elazhary <omazhary@...>
 

Hello everyone,

I know it is possible to update and redeploy components in parallel in cloud foundry by setting the "serial" property in the deployment manifest to "false". However, is such a thing recommended? Are there particular job dependencies that I need to pay attention to?

Regards,
Omar


Re: New CF Service Broker "chaos-galago" - a chaos-monkey for your Cloud Foundry

Sam Bryant
 

For anyone interested we have also now added a smoke tests project for chaos-galago that can be used to monitor the service-broker. This can be found: https://github.com/FidelityInternational/chaos-galago-smoke-tests

Details are also on the README for chaos-galago.

Regards,
Sam


Reg the minimal-openstack yml files

Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM@Cisco) <ngnanase at cisco.com...>
 

Hi

We are trying to upgrade our deployment with the latest cloud-foundry, from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. Can we have a similar one available for openstack environment, with which we can deploy the basic cloud foundry and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided by the errors given by the generate_deployment_manifest script. Kindly let us know if this is the correct way to generate the manifest.


Regards
Nithiyasri


Re: `api_z1/0' is not running after update to CF v231

Paul Bakare
 

Wayne, is the nfs_server-partition running?

On Mon, Mar 7, 2016 at 1:43 AM, Wayne Ha <wayne.h.ha(a)gmail.com> wrote:

I checked the blobstore is running:

root(a)e83575d2-dfbf-4f7c-97ee-5112560fa137:/var/vcap/sys/log# monit summary
The Monit daemon 5.2.4 uptime: 4h 14m
Process 'consul_agent' running
Process 'metron_agent' running
Process 'blobstore_nginx' running
Process 'route_registrar' running
System 'system_e83575d2-dfbf-4f7c-97ee-5112560fa137' running

But there are thousands of errors saying DopplerForwarder: can't forward
message, loggregator client pool is empty:

root(a)e83575d2-dfbf-4f7c-97ee-5112560fa137:/var/vcap/sys/log# find . -name
"*.log" | xargs grep -i error | cut -c 73-500 | sort -u
,"process_id":246,"source":"metron","log_level":
"error","message":"DopplerForwarder: can't forward message","data":{
"error":"loggregator client pool is empty"},

"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/writers/dopplerforwarder/doppler_forwarder.go",
"line":104,

"method":"metron/writers/dopplerforwarder.(*DopplerForwarder).networkWrite"}

Not sure what is wrong.


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

I checked the blobstore is running:

root(a)e83575d2-dfbf-4f7c-97ee-5112560fa137:/var/vcap/sys/log# monit summary
The Monit daemon 5.2.4 uptime: 4h 14m
Process 'consul_agent' running
Process 'metron_agent' running
Process 'blobstore_nginx' running
Process 'route_registrar' running
System 'system_e83575d2-dfbf-4f7c-97ee-5112560fa137' running

But there are thousands of errors saying DopplerForwarder: can't forward message, loggregator client pool is empty:

root(a)e83575d2-dfbf-4f7c-97ee-5112560fa137:/var/vcap/sys/log# find . -name "*.log" | xargs grep -i error | cut -c 73-500 | sort -u
,"process_id":246,"source":"metron","log_level":
"error","message":"DopplerForwarder: can't forward message","data":{
"error":"loggregator client pool is empty"},
"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/writers/dopplerforwarder/doppler_forwarder.go",
"line":104,
"method":"metron/writers/dopplerforwarder.(*DopplerForwarder).networkWrite"}

Not sure what is wrong.


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Amit,

Thanks for letting me know I might have looked at the wrong log files. I
saw the following in cloud_controller log files:

root(a)7a1f2221-c31a-494b-b16c-d4a97c16c9ab:/var/vcap/sys/log# tail
./cloud_controller_ng_ctl.log
[2016-03-06 22:40:28+0000] ------------ STARTING cloud_controller_ng_ctl at
Sun Mar 6 22:40:28 UTC 2016 --------------
[2016-03-06 22:40:28+0000] Checking for blobstore availability
[2016-03-06 22:41:03+0000] Blobstore is not available

root(a)7a1f2221-c31a-494b-b16c-d4a97c16c9ab:/var/vcap/sys/log# tail
./cloud_controller_worker_ctl.log
[2016-03-06 22:41:13+0000] Killing
/var/vcap/sys/run/cloud_controller_ng/cloud_controller_worker_2.pid: 12145
[2016-03-06 22:41:13+0000] .Stopped
[2016-03-06 22:41:36+0000] Blobstore is not available
[2016-03-06 22:41:48+0000] ------------ STARTING
cloud_controller_worker_ctl at Sun Mar 6 22:41:48 UTC 2016 --------------
[2016-03-06 22:41:48+0000] Checking for blobstore availability
[2016-03-06 22:41:48+0000] Removing stale pidfile...

So maybe the cause is Blobstore is not available?

Thanks,

On Sun, Mar 6, 2016 at 1:15 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

The log lines saying "/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock
is not found" is probably just a symptom of the problem, not the root
cause. You're probably seeing those in the nginx logs? Cloud Controller
is failing to start, hence it is not establishing a connection on the
socket. You need to dig deeper into failures in logs in
/var/vcap/sys/log/cloud_controller_ng.

On Sun, Mar 6, 2016 at 10:00 AM, sridhar vennela <
sridhar.vennela(a)gmail.com> wrote:

Hi Wayne,

Looks like it, It is trying to connect to loggregator and failing I guess.


https://github.com/cloudfoundry/cloud_controller_ng/blob/master/app/controllers/runtime/syslog_drain_urls_controller.rb

Thank you,
Sridhar


Re: app auto-scaling in OSS CF contribution

Padmashree B
 

Hi,

Is the solution same as the one offered in IBM Bluemix?
Where can I find more information on IBM's solution [open-Autoscaler], current/planned features, their roadmap, timeline etc. ?

Kind Regards,
Padma


Re: `api_z1/0' is not running after update to CF v231

Amit Kumar Gupta
 

The log lines saying
"/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock
is not found" is probably just a symptom of the problem, not the root
cause. You're probably seeing those in the nginx logs? Cloud Controller
is failing to start, hence it is not establishing a connection on the
socket. You need to dig deeper into failures in logs in
/var/vcap/sys/log/cloud_controller_ng.

On Sun, Mar 6, 2016 at 10:00 AM, sridhar vennela <sridhar.vennela(a)gmail.com>
wrote:

Hi Wayne,

Looks like it, It is trying to connect to loggregator and failing I guess.


https://github.com/cloudfoundry/cloud_controller_ng/blob/master/app/controllers/runtime/syslog_drain_urls_controller.rb

Thank you,
Sridhar


Re: `api_z1/0' is not running after update to CF v231

sridhar vennela
 

Hi Wayne,

Looks like it, It is trying to connect to loggregator and failing I guess.

https://github.com/cloudfoundry/cloud_controller_ng/blob/master/app/controllers/runtime/syslog_drain_urls_controller.rb

Thank you,
Sridhar


Re: monit definitions

Benjamin Gandon
 

Hi,
I'm no expert but “monit” is a component of BOSH, not Cloud Foundry.
Your question would get answered if asked on the “bosh-dev” mailing-list.
Cheers

Le 18 févr. 2016 à 11:19, Nitta, Minoru <minoru.nitta(a)jp.fujitsu.com> a écrit :

Hi guys,

I know monit executes connection testing to some processes in CloudFoundry VM
by issuing HTTP requests.
e.g. (in UAA case)
if failed port <%= p('uaa.port') %> protocol http
request "/healthz"
with timeout 60 seconds for 10 cycles
then restart

I am wondering how timeout and cycles configuration values are decided. I mean, are there
any policies or guidelines to set these values? It seems that different values are set to
each process so I guessed there might be any policies in CloudFoundry.

Regards,
Minoru Nitta


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Since it is complaining /var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock is not found, I thought I would just touch that file. Now I get:

2016/03/06 17:14:11 [error] 18497#0: *5 connect() to unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock failed (111: Connection refused) while connecting to upstream, client: <bosh director>,
server: _, request: "GET /v2/syslog_drain_urls?batch_size=1000 HTTP/1.1", upstream: "http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock:/v2/syslog_drain_urls?batch_size=1000", host: "api.bosh-lite.com"

Maybe there is network configuration problem in my environment?


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Sridhar,

Thanks for your response. I have tried your suggestion and it doesn't
help. But I might have misled you with the consul error. That error only
got logged once at the beginning. So like you said, maybe VM was not able
to join consul server before it came up. But after that, the following
error keeps logging every minute or so:

2016/03/06 17:04:41 [crit] 11480#0: *4 connect() to
unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock failed (2:
No such file or directory) while connecting to upstream,
server: _, request: "GET /v2/syslog_drain_urls?batch_size=1000 HTTP/1.1",
upstream: "http://unix:/var/vcap/sys/run/cloud_controller_ng/cloud_controller.sock:/v2/syslog_drain_urls?batch_size=1000",
host: "api.bosh-lite.com"

So maybe the above is the cause of the problem?

Thanks,

On Sun, Mar 6, 2016 at 12:51 AM, sridhar vennela <sridhar.vennela(a)gmail.com>
wrote:

Hi Wayne,

Somehow VM is not able to join consul server. You can try below steps.

ps -ef | grep consul

kill consul-serverpid

monit restart <consul-job>

Thank you,
Sridhar


Re: `api_z1/0' is not running after update to CF v231

sridhar vennela
 

Hi Wayne,

Somehow VM is not able to join consul server. You can try below steps.

ps -ef | grep consul

kill consul-serverpid

monit restart <consul-job>

Thank you,
Sridhar


Re: `api_z1/0' is not running after update to CF v231

Wayne Ha <wayne.h.ha@...>
 

Sridhar,

Thanks for your response. I found the VM is listening to port 8500:

root(a)c6822dcb-fb02-4858-ae5d-3ab45d593896:/var/vcap/sys/log# netstat -anp |
grep LISTEN
tcp 0 0 127.0.0.1:8400 0.0.0.0:*
LISTEN 18162/consul
tcp 0 0 127.0.0.1:8500 0.0.0.0:*
LISTEN 18162/consul
tcp 0 0 127.0.0.1:53 0.0.0.0:*
LISTEN 18162/consul
tcp 0 0 127.0.0.1:2822 0.0.0.0:*
LISTEN 72/monit
tcp 0 0 0.0.0.0:22 0.0.0.0:*
LISTEN 31/sshd
tcp 0 0 10.244.0.138:8301 0.0.0.0:*
LISTEN 18162/consul

If I run "monit stop all" then it only listens to the following:

root(a)c6822dcb-fb02-4858-ae5d-3ab45d593896:/var/vcap/sys/log# netstat -anp |
grep LISTEN
tcp 0 0 127.0.0.1:2822 0.0.0.0:*
LISTEN 72/monit
tcp 0 0 0.0.0.0:22 0.0.0.0:*
LISTEN 31/sshd

Note that 10.244.0.138 is the IP of this VM.

Thanks,

On Sat, Mar 5, 2016 at 12:58 AM, sridhar vennela <sridhar.vennela(a)gmail.com>
wrote:

Hi Wayne,

Can you please verify port 8500 listening? Maybe output of netstat -anp
will help.

{"timestamp":"1457136496.397377968","source":"confab","message":"confab.agent-client.verify-joined.members.request.failed","log_level":2,"data":{"error":"Get
http://127.0.0.1:8500/v1/agent/members: dial tcp 127.0.0.1:8500:
getsockopt: connection refused","wan":false}}

Thank you,
Sridhar


Re: User defined variable "key" validation doesn't happen at cf set-env phase

Nicholas Calugar
 

Hi Ponraj,

I don't think the CC can make any determination regarding the validity of
environment variables as the CC doesn't (and shouldn't) know how each
buildpack will use these environment variables.

Thanks,

Nick

On Thu, Mar 3, 2016 at 9:22 AM Ponraj E <ponraj.e(a)gmail.com> wrote:

Hi CF Colleagues,

I see various PaaS providers provide the UI for entering the User provided
variables' key and value, but they dont seem to validate the "key" at the
save ["set-env"] phase, the validation happens only at
the restage phase. This is also because CF does the same. Is there any
reason that CC doesnt validate keys of user defined environment varibles at
the cf set-env phase ?

Examples:
1.cf set-env spring-music !@#$$%% "foobar" succeeds, but the restage
fails throwing /bin/bash: line 6: export: `!@#49%%=foobar': not a valid
identifier
2.cf set-env spring-music "!@#$$%%" "foobar" succeeds, the restage also
succeeds but the same as above is thrown as message here.

P.S: The above variables are used for only testing purposes and these
would give errros at the runtime usage in the application is implicit here.


Regards,
Ponraj


Re: How to get the IP addresses of the instances of the same application

Casey West
 

Indeed, I've had great success using the `jq` command-line tool to parse
JSON along with `cf app app-name --guid` and `cf curl`. Like so:

$ cf curl /v2/apps/$(cf app app-name --guid)/stats | jq '.[].stats.host'
"10.10.112.75"
"10.10.116.51"
"10.10.113.72"
"10.10.116.53"
"10.10.113.69"

More on jq if needed: https://github.com/stedolan/jq

— Casey

On Sat, Mar 5, 2016 at 8:15 PM DHR <lists(a)dhrapson.com> wrote:

The router_status endpoint of the router node gives a nice point-in-time
snapshot of the IPs / ports in is for each app URL, in json, andis
continually kept up to date by the CF platform.

On 4 Mar 2016, at 22:31, Jason Sherron <jsherron(a)pivotal.io> wrote:

I'm also very interested to hear about what you might need to do with
direct container-to-container communication. We've started an effort to
enable it at the network layer, and as Dan gets at, service discovery is a
thorny area, and we'd like to hear your feedback.

On Fri, Mar 4, 2016 at 9:07 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Thu, Mar 3, 2016 at 10:32 PM, Malmee Weerasinghe <malmee(a)wso2.com>
wrote:

Hi,
When I push an application to cloud foundry, "cf scale myApp -i 5" will
increase the number of instances of my application to 5. I want to get the
ip addresses of these instances. Is there any way to get the ip addresses
and ports of the 5 instances of my application? Is there any API?
Look at the environment variables in the container, things starting with
VCAP_ or CF_ provide you with details about the container, app and bound
services. It sounds like you're looking for CF_INSTANCE_ADDR in particular.

That said, what are you planning to do with this information? You don't
really want to try and talk directly to specific app instances. Those
instance can come and go and the IP's can change. If you're talking
directly then you have the tedious task of tracking all that information.
If instead you talk through the URL mapped to your app then you never have
to worry about things changing.

It's probably also worth mentioning that some CF installations will
prohibit direct container to container communication.

Dan


I am running cloud foundry with diego in bosh-lite. In documents I have
seen that garden API gives container information, but how to call the
garden API? I highly appreciate your responses regarding this issue.

Thank you.


Re: Org and Space Quota management

Padmashree B
 

Hi,

Any inputs on the current behavior of the APIs would be of great help, thanks !

Regards,
Padma

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.


Re: CF env reveals passwords and secrets ...

Marco Voelz
 

Dear Padma, Mathias, all,

BOSH has just included the functionality to replace all properties with the string "<redacted>" already *on the server side*.
Maybe that is an option the Cloud Controller could follow here?

Note that we're currently discussing what default behavior should be and if there should be an option to turn redacting on or off, respectively. [1]

Warm regards
Marco

[1] https://github.com/cloudfoundry/bosh/issues/1158

On 05/03/16 10:51, "Padmashree B" <padmashree.b(a)sap.com> wrote:

It will be interesting if there are any solutions to handle this from CF.
As an alternative solution, every client tools should build some logic to hide the secure variables. However, it will not be straight forward for the client tools to identify those variables to be handled in a secure way since at the end it is just a key-value pair.
This is also applicable for user-defined variables which is usually defined by application developers and there is no way to mark certain variables as secure.

Thanks,
Padma


Re: CF env reveals passwords and secrets ...

Padmashree B
 

It will be interesting if there are any solutions to handle this from CF.
As an alternative solution, every client tools should build some logic to hide the secure variables. However, it will not be straight forward for the client tools to identify those variables to be handled in a secure way since at the end it is just a key-value pair.
This is also applicable for user-defined variables which is usually defined by application developers and there is no way to mark certain variables as secure.

Thanks,
Padma

5361 - 5380 of 9425