1. cf-dev@lists.cloudfoundry.org
  2. Messages
   Date   
Re: [ PHP ] Best config practices

Nicholas Calugar
 

You may also want to look into [1] .user.ini files.

+Steven

Would you consider compiling PHP with this option:

--with-config-file-scan-dir /home/vcap/app/php/etc/php.d

And then providing some mechanism for the buildpack to copy .ini files from
a known location into that directory?



[1] http://php.net/manual/en/configuration.file.per-user.php
--
Nicholas Calugar

On January 20, 2017 at 5:36:45 AM, Daniel Mikusa (dmikusa(a)pivotal.io) wrote:

You can include a `.profile.d/` script or a `.profile` script. It depends
on exactly what you're trying to do as to which one would be better to use,
but I believe most of the time you'd want the `.profile` script. You can
see more about it in the docs here.

https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html#profile

You can essentially do whatever you want in this script. Plus you'll have
access to everything that the PHP build pack has configured for you and
you'll have access to all the environment variables. This means you can
run PHP scripts or examine VCAP_SERVICES. The only catch is that these
scripts runs prior to your application starting so they must run and your
application must start within the defined timeout (default 60s, max 180s on
most platforms).

If you want to integrate with the build pack and do something during
staging or if you need more time (cause staging typically runs for up to
900s), you can add a build pack extension. These are written in Python and
run as a part of the build pack. Instructions on doing that can be found
here.

https://github.com/cloudfoundry/php-buildpack#extensions

Hope that helps!

Dan

On Thu, Jan 19, 2017 at 4:01 PM, Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

I have a php monolith app that requires tons of .ini files config previous
to the deployment I want to know if any of you have a "best practice" to
generate all this ini files depending from the environmental variables.

?
attachment.html

CF Java Client

Chenulu P
 

I am trying build the Java Client from https://github.com/cloudfoundry/cf-java-client.git. However none of the jar files generated contains the classes referred to in the sample code https://docs.cloudfoundry.org/buildpacks/java/java-client.html#sample-code

I would really appreciate if anyone can provide me how to build the client jar so that I can reference the client lib methods.

TIA.
Re: [ PHP ] Best config practices

Daniel Mikusa
 

You can include a `.profile.d/` script or a `.profile` script. It depends
on exactly what you're trying to do as to which one would be better to use,
but I believe most of the time you'd want the `.profile` script. You can
see more about it in the docs here.

https://docs.cloudfoundry.org/devguide/deploy-apps/deploy-app.html#profile

You can essentially do whatever you want in this script. Plus you'll have
access to everything that the PHP build pack has configured for you and
you'll have access to all the environment variables. This means you can
run PHP scripts or examine VCAP_SERVICES. The only catch is that these
scripts runs prior to your application starting so they must run and your
application must start within the defined timeout (default 60s, max 180s on
most platforms).

If you want to integrate with the build pack and do something during
staging or if you need more time (cause staging typically runs for up to
900s), you can add a build pack extension. These are written in Python and
run as a part of the build pack. Instructions on doing that can be found
here.

https://github.com/cloudfoundry/php-buildpack#extensions

Hope that helps!

Dan

On Thu, Jan 19, 2017 at 4:01 PM, Leandro David Cacciagioni <
leandro.21.2008(a)gmail.com> wrote:

I have a php monolith app that requires tons of .ini files config previous
to the deployment I want to know if any of you have a "best practice" to
generate all this ini files depending from the environmental variables.

?
attachment.html

Re: Proposed changes to Gorouter log message format

Gwenn Etourneau
 

Thanks Shannon ! that's a pretty good news hope we can increase perf more.
:)

On Fri, Jan 20, 2017 at 6:20 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Thanks Shannon.

FWIW we strip the first timestamp from log messages sent to Splunk because
Splunk doesn't detect the message as JSON unless the entire event is
properly formatted json. Either way, I think always second position should
be fine.

Mike

On Thu, Jan 19, 2017 at 12:44 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Mike, thank you for your response. The Routing team tells me we can count
on the library-provided timestamp appearing in the second position. To be
clear, a full log message has two timestamps, one provided by the logging
library (epoch), and one that our Gorouter code prepends every message with
(human readable).

[2017-01-13 22:05:58+0000] {"log_level":1,"timestamp":148
4345158.9532504,"message":"starting","source":"vcap.gorouter","data":{}}

If you're referring to the human readable one, that will continue to
prepend the JSON object.

Etourneau, we've seen 10-20% improvement in throughput by swapping lager
for zap in this use case. There are still valid reasons for other CF
components to continue using lager, including features it supports that zap
does not. However, we don't have requirements for those features in
Gorouter, and performance is particularly important in this component. We
will be publishing a blog post as a sequel to one we published recently, in
which we'll share the tooling we've developed for exploring performance
improvements, what we've tried that worked and didn't, and our results.


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jan 18, 2017 at 9:50 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

4. Key reordering. My expectation is that no one should be using key
order for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Our Splunk configuration requires the timestamp to be early in the log
message. Should be fine as the second item. But, might cause issues any
further into the message than that.

Mike

?
attachment.html

Re: Proposed changes to Gorouter log message format

Mike Youngstrom <youngm@...>
 

Thanks Shannon.

FWIW we strip the first timestamp from log messages sent to Splunk because
Splunk doesn't detect the message as JSON unless the entire event is
properly formatted json. Either way, I think always second position should
be fine.

Mike

On Thu, Jan 19, 2017 at 12:44 PM, Shannon Coen <scoen(a)pivotal.io> wrote:

Mike, thank you for your response. The Routing team tells me we can count
on the library-provided timestamp appearing in the second position. To be
clear, a full log message has two timestamps, one provided by the logging
library (epoch), and one that our Gorouter code prepends every message with
(human readable).

[2017-01-13 22:05:58+0000] {"log_level":1,"timestamp":
1484345158.9532504,"message":"starting","source":"vcap.
gorouter","data":{}}

If you're referring to the human readable one, that will continue to
prepend the JSON object.

Etourneau, we've seen 10-20% improvement in throughput by swapping lager
for zap in this use case. There are still valid reasons for other CF
components to continue using lager, including features it supports that zap
does not. However, we don't have requirements for those features in
Gorouter, and performance is particularly important in this component. We
will be publishing a blog post as a sequel to one we published recently, in
which we'll share the tooling we've developed for exploring performance
improvements, what we've tried that worked and didn't, and our results.


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jan 18, 2017 at 9:50 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

4. Key reordering. My expectation is that no one should be using key
order for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Our Splunk configuration requires the timestamp to be early in the log
message. Should be fine as the second item. But, might cause issues any
further into the message than that.

Mike

?
attachment.html

[ PHP ] Best config practices

Leandro David Cacciagioni
 

I have a php monolith app that requires tons of .ini files config previous
to the deployment I want to know if any of you have a "best practice" to
generate all this ini files depending from the environmental variables.

?
attachment.html

Re: Proposed changes to Gorouter log message format

Shannon Coen
 

Mike, thank you for your response. The Routing team tells me we can count
on the library-provided timestamp appearing in the second position. To be
clear, a full log message has two timestamps, one provided by the logging
library (epoch), and one that our Gorouter code prepends every message with
(human readable).

[2017-01-13 22:05:58+0000]
{"log_level":1,"timestamp":1484345158.9532504,"message":"starting","source":"vcap.gorouter","data":{}}

If you're referring to the human readable one, that will continue to
prepend the JSON object.

Etourneau, we've seen 10-20% improvement in throughput by swapping lager
for zap in this use case. There are still valid reasons for other CF
components to continue using lager, including features it supports that zap
does not. However, we don't have requirements for those features in
Gorouter, and performance is particularly important in this component. We
will be publishing a blog post as a sequel to one we published recently, in
which we'll share the tooling we've developed for exploring performance
improvements, what we've tried that worked and didn't, and our results.


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Wed, Jan 18, 2017 at 9:50 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

4. Key reordering. My expectation is that no one should be using key order
for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Our Splunk configuration requires the timestamp to be early in the log
message. Should be fine as the second item. But, might cause issues any
further into the message than that.

Mike

?
attachment.html

Re: smallest number of VMs for CF deployment

Daniel Jones
 

Hi,

You may find this great talk from Altoros helpful:
https://www.youtube.com/watch?v=RJx9tLoa2ug

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
@DanielJonesEB <https://twitter.com/DanielJonesEB>
*EngineerBetter* Ltd <http://www.engineerbetter.com> - UK Cloud Foundry
Specialists

On 19 January 2017 at 15:41, srinivas A <shreeadada(a)gmail.com> wrote:

Hi Nic,

I am trying to deploy CF with less Vms , can you please help me provide
any link which can guide me or any sample yaml file . I have checked the
above link but looks like thats been removed as it was old.
THanks

?
attachment.html

Re: smallest number of VMs for CF deployment

srinivas A
 

Hi Nic,

I am trying to deploy CF with less Vms , can you please help me provide any link which can guide me or any sample yaml file . I have checked the above link but looks like thats been removed as it was old.
THanks
Best Way to Customize Name Servers

Suren R
 

I have some recommendation from Infrastructure to customize the
/etc/resolv.conf file in all CF VMs and the app containers. It is not just
name server and search domain entries. There are other entries as well.
What would be the best way to achieve this?

?
attachment.html

Re: Proposed changes to Gorouter log message format

Mike Youngstrom <youngm@...>
 


4. Key reordering. My expectation is that no one should be using key order
for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Our Splunk configuration requires the timestamp to be early in the log
message. Should be fine as the second item. But, might cause issues any
further into the message than that.

Mike

?
attachment.html

Re: Proposed changes to Gorouter log message format

Etourneau Gwenn
 

Shannon,

Can you share about the improvements you are talking about ?

Thanks

2017-01-19 9:46 GMT+09:00 Shannon Coen <scoen(a)pivotal.io>:

The Routing team is moving from cloudfoundry/lager to uber-go/zap logging
library for the Gorouter log, as we have observed improvements to
throughput performance using zap. This change has no impact on the
access.log, only to gorouter.log.

The Gorouter log message will change in the following ways. Please let us
know if any of these are a concern. Thank you!

1. "session" removed from "data" key. Does anyone use this?

lager
"data":{"host":"10.0.32.21:4222","session":"1"}

zap
"data":{"host":"10.0.32.21:4222"}


2. Values for "source" key more granular. Seems helpful for filtering
messages for like operations

lager
"source":"vcap.gorouter"

zap
"source":"vcap.gorouter"
"source":"vcap.gorouter.nats"
"source":"vcap.gorouter.subscriber"
"source":"vcap.gorouter.router"
"source":"vcap.gorouter.registry"


3. Source no longer prepends value of "message" key, as it is redundant
to the "source" key.

lager
"message":"vcap.gorouter.starting"

zap
"message":"starting"

4. Key reordering. My expectation is that no one should be using key order
for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Full example log messages: https://gist.github.com/shalako/
79c8d7e8bf6d26b9934bb2679914c856

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

?
attachment.html

Proposed changes to Gorouter log message format

Shannon Coen
 

The Routing team is moving from cloudfoundry/lager to uber-go/zap logging
library for the Gorouter log, as we have observed improvements to
throughput performance using zap. This change has no impact on the
access.log, only to gorouter.log.

The Gorouter log message will change in the following ways. Please let us
know if any of these are a concern. Thank you!

1. "session" removed from "data" key. Does anyone use this?

lager
"data":{"host":"10.0.32.21:4222","session":"1"}

zap
"data":{"host":"10.0.32.21:4222"}


2. Values for "source" key more granular. Seems helpful for filtering
messages for like operations

lager
"source":"vcap.gorouter"

zap
"source":"vcap.gorouter"
"source":"vcap.gorouter.nats"
"source":"vcap.gorouter.subscriber"
"source":"vcap.gorouter.router"
"source":"vcap.gorouter.registry"


3. Source no longer prepends value of "message" key, as it is redundant to
the "source" key.

lager
"message":"vcap.gorouter.starting"

zap
"message":"starting"

4. Key reordering. My expectation is that no one should be using key order
for log analysis.

lager
{"timestamp","source","message","log_level","data"}

zap
{"log_level","timestamp","message","source","data"}


Full example log messages:
https://gist.github.com/shalako/79c8d7e8bf6d26b9934bb2679914c856

Thank you,

Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

?
attachment.html

Re: 2017 CF Summit Silicon Valley Contributor Code

Chip Childers <cchilders@...>
 

Hi all!

On today's CAB call, Dr. Max asked if this thread could be bumped so
everyone is reminded to see it.

Contributors: Don't forget to register!

-chip

On Wed, Nov 30, 2016 at 12:22 PM Chip Childers <cchilders(a)cloudfoundry.org>
wrote:

Hi all!

Some of you may have noticed that the registration is now open for the
upcoming CF Summit in Silicon Valley, and we are offering free passes for
contributors to the project again.

This code can be used by anyone that is a contributor to a Cloud Foundry
or BOSH project. We consider contributions to be project leads, dedicated
committers or even if you have sent in a pull request to one of the
projects.

Use of the code is on the honor system... ;-)


https://www.cloudfoundry.org/summit2017/?utm_source=flash&utm_campaign=summit_2017_sv&utm_medium=eml&utm_term=cloud%20foundry%20summit


Code: CFSV17CONT
Feel free to reach out to me or to events(a)cloudfoundry.org if you have
any questions.

See you there!
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815 <(267)%20250-0815>
--
Chip Childers
CTO, Cloud Foundry Foundation
1.267.250.0815

?
attachment.html

Update to Garden RunC vulnerability notice

Molly Crowther
 

Hello - we originally thought we were not vulnerable to a vulnerability in
RunC but turns out it is complicated. Please see updated notice below.

https://www.cloudfoundry.org/cve-2016-9962/

Please let me know if you have any questions.

Thanks,
Molly Crowther
CFF Security Team

RunC Exec Vulnerability

Severity

Medium/Low
Vendor

Open Containers Initiative
Description

RunC allowed additional container processes via runc exec to be ptraced by
the pid 1 of the container. This allows the main processes of the
container, if running as root, to gain access to file-descriptors of these
new processes during the initialization and can lead to container escapes
or modification of runC state before the process is fully placed inside the
container.
Affected Cloud Foundry Products and Versions

- Garden-runC versions prior to 1.1.1

The Cloud Foundry team has determined that the project is not exposed to
this particular vulnerability and therefore does not require any upgrades.
As Cloud Foundry never runs user processes as pid 1 and runs all buildpack
containers as unprivileged users in a user namespace, and as Cloud Foundry
uses apparmor to prevent ptrace, the specific exploit in the CVE is not
possible.

However, the CVE patch from runC also worked around an Ubuntu kernel bug
that resulted in file descriptors which were inherited by a new process
being available for a very short time when they should have been
automatically closed. This could result in a container being able to access
files on the host, although not with elevated permissions.
Mitigation

OSS users are encouraged to follow one of the mitigations below:

- Upgrade garden-runC to version 1.1.1 or later.

Credit

Credit for this discovery goes to Aleksa Sarai from SUSE and Tõnis Tiigi
from Docker.
References

- Original CVE from Docker: http://seclists.org/oss-sec/2017/q1/54
- Ubuntu tracking for kernel bug:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1654602

?
attachment.html

Proposal: BOSH-aware DNS server

Amit Kumar Gupta
 

Hi all,

In service of the initiative to remove the hard dependency on Consul for Locks
& Service Discovery in CF Runtime
<https://docs.google.com/document/d/16O5Rk5tOmHc2Qeya7soVHoASAgrH9jYZ0V3fS58dY14/edit>,
we are proposing BOSH-native DNS in the form of a BOSH-aware DNS server.
Please see the proposal document
<https://docs.google.com/document/d/1GsS1S7XFJoeLR1kYY46u4pcX08lRDvXUrjkssuyKI4s/edit#>
for more details and discussion.

Regards,
Amit

?
attachment.html

Re: Locks & Service Discovery in CF Runtime

Mike Youngstrom <youngm@...>
 

I like the direction. This change will be good.

Mike

On Tue, Jan 17, 2017 at 11:23 AM, Matthew Kocher <mkocher(a)pivotal.io> wrote:

Hi All - We're proposing prioritizing work among all core Cloud Foundry
components to move necessary locks from a shared distributed data store
into components’ own data stores and allow for generic DNS based service
discovery. This is in response to operational difficulties with Consul as
well as seeing multiple places where app availability has depended on
Consul.

Please see the Locks & Service Discovery in CF Runtime
<https://docs.google.com/document/d/16O5Rk5tOmHc2Qeya7soVHoASAgrH9jYZ0V3fS58dY14/edit> proposal
for more details and discussion.

Matthew Kocher
Sr. Director of Engineering, Cloud Foundry @ Pivotal

?
attachment.html

Locks & Service Discovery in CF Runtime

Matthew Kocher <mkocher@...>
 

Hi All - We're proposing prioritizing work among all core Cloud Foundry
components to move necessary locks from a shared distributed data store
into components’ own data stores and allow for generic DNS based service
discovery. This is in response to operational difficulties with Consul as
well as seeing multiple places where app availability has depended on
Consul.

Please see the Locks & Service Discovery in CF Runtime
<https://docs.google.com/document/d/16O5Rk5tOmHc2Qeya7soVHoASAgrH9jYZ0V3fS58dY14/edit>
proposal
for more details and discussion.

Matthew Kocher
Sr. Director of Engineering, Cloud Foundry @ Pivotal

?
attachment.html

Re: API to get instance details at the space level

Ronak Banka
 

Hi,

For detailed info, you have to iterate over app endpoint for each app in
space.

Thanks
Ronak

On Tue, Jan 17, 2017 at 19:21 Patil, Sukhil <sukhil.patil(a)sap.com> wrote:

Thanks for your quick response paolo but the apps
call(/v2/spaces/$space-id/apps) will not give any information about
instances like this


https://apidocs.cloudfoundry.org/242/apps/get_the_instance_information_for_a_started_app.html
.



-----Original Message-----

From: Stivanin, Paolo [mailto:paolo.stivanin(a)sap.com]

Sent: 17 January, 2017 3:07 PM

To: Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>

Subject: [cf-dev] Re: API to get instance details at the space level



Hi,

You can query /v2/spaces/$space-id/apps.



BR,



Paolo





On 17/01/2017, 10:28, "Sukhil Patil" <sukhil.patil(a)sap.com> wrote:



Hi,



We are interested in getting app instance details for all apps within
a space. space summary
'/v2/spaces/50ae42f6-346d-4eca-9e97-f8c9e04d5fbe/summary' call gives only
info about 'running_instances' and 'instances', but not about instance
status such as RUNNING, STARTING and so on.

So is there any way where we can achieve this with a single API call ??





?
attachment.html

Re: API to get instance details at the space level

Patil, Sukhil
 

Thanks for your quick response paolo but the apps call(/v2/spaces/$space-id/apps) will not give any information about instances like this
https://apidocs.cloudfoundry.org/242/apps/get_the_instance_information_for_a_started_app.html.

-----Original Message-----
From: Stivanin, Paolo [mailto:paolo.stivanin(a)sap.com]
Sent: 17 January, 2017 3:07 PM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>
Subject: [cf-dev] Re: API to get instance details at the space level

Hi,
You can query /v2/spaces/$space-id/apps.

BR,

Paolo


On 17/01/2017, 10:28, "Sukhil Patil" <sukhil.patil(a)sap.com> wrote:

Hi,

We are interested in getting app instance details for all apps within a space. space summary '/v2/spaces/50ae42f6-346d-4eca-9e97-f8c9e04d5fbe/summary' call gives only info about 'running_instances' and 'instances', but not about instance status such as RUNNING, STARTING and so on.
So is there any way where we can achieve this with a single API call ??
3121 - 3140 of 9429