1. cf-dev@lists.cloudfoundry.org
   Date   
Re: Proposed BOSH logging interface

Carlo Alberto Ferraris
 

Jesse,
sorry for the late reply. One thing we normally add to our internal bosh releases - something that is especially useful for the control/drain scripts, but that we do almost for every single process, is the ability to add a timestamp to each line and the PID of the process generating the logs. It would be very neat if the logging interface had an (opt-in?) provision to handle this so that release authors don't have to do it manually.
Database support for UAA

Enrique Cano
 

I understand that UAA can run against MySQL and Postgres as the backend database, and Oracle was considered in the past but was discarded due to the enterprise licensing model (https://github.com/cloudfoundry/uaa/issues/291).
My question is whether there is a way to develop support for Oracle (or any other database) without having to integrate the code within the UAA repo. In other words, is there a way to externalise the components to support a different database and then make it work alongside UAA's war file?

Thanks

Enrique
Re: Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

Aaron Huber
 

After further testing with both the 1709 and 1803 versions of Windows Server 2016 it does appear that the WinNAT component being used by the Host Network Service in Windows does not play well with IPSec.  We've confirmed with both the open source Cloud Foundry and commercial PCF deployments that traffic to the containers stops working once IPSec is enabled.  We've also tested using out-of-the-box installations of both 1709 and 1803 with Docker for Windows and can replicate the same results - traffic to a container over the NAT connection to an exposed port stops working as soon as IPSec is enabled.

If anyone has successfully been able to get this working, please reply with any details.  In the mean time we'll be trying to reach out to Microsoft to confirm if this is a bug that can be fixed or if this is working as intended.

Aaron
(action requested) Changing CF App Display

Abby Chau
 

Hello cf-dev,

In order to provide a better user experience, the Cloud Foundry CLI team is considering changing the   cf app app-name  key-value and table display information. Towards this effort, we are asking the community to complete a short survey. 

Actions: Please fill out this short survey [1] to help us understand how you use the cf app app-name command display, and your thoughts on changing the display to default to the V3-app version of the command. The survey will close by end of day, June 15th. 

Thanks in advance for your time and participation.

Best,

Abby Chau
CF Product Manager - CLI
Re: CF CLI v6.37.0 Released Today: Variable Substitution

Abby Chau
 

Hi Ronak,

We don't currently have that prioritised currently for the CF CLI, but please create a github issue for visibility and future consideration.

Thanks,

Abby


On Thu, May 31, 2018 at 11:44 PM, Ronak Banka <ronakbanka.cse@...> wrote:
Hi Abby,

Going forward is there something on roadmap to use ops files for cf CLI too, like we do for Bosh CLI or yaml patch?

Use case can be CI/CD pipelines where part of manifest can be injected for different environment.

Thanks
Ronak

On Thu, 31 May 2018 at 7:37 AM, Abby Chau <achau@...> wrote:
Hi Benjamin,

Thanks for reaching out; we will update the documentation for clarification. The partial value use case, for example, is a manifest.yml with the following:

- route: ((host)).example.com

and a variable.yml file which invokes the value by passing in the hostname:

host: test

When you push with the manifest and variable file, the route test.example.com will be created. Hope that makes sense. 

Best,

Abby



On Wed, May 30, 2018 at 7:30 AM, Benjamin Gandon <benjamin@...> wrote:
Hi Abby,

In the documentation for variable substitution, when it’s said:

Template variables can also be used as partial values, such as in:

It’s quite unclear what is meant by “partial values”, and the example doesn’t bring an illustration to this. Indeed, correct me if I’m wrong but this second example looks the same as the first one, because using --vars-file <(cat secrets.yml) should be quite the same as writing --vars-file secrets.yml.

Maybe “partial values” was a way to say that multiple --vars-file arguments can be used?

Benjamin


Le 30 mai 2018 à 03:17, Abby Chau <achau@...> a écrit :

Hi all,

The CF CLI team released cf CLI v6.37.0; please see release notes for full details. 

Highlights include:
  • Variable substitution - (replaces inheritance) which allows app developers to create app manifests with values shared across all applicable environments in combination with references to environment-specific differences defined in separate files
  • Now the CLI reads username and password from environment variables
  • If you are using a API version older than 2.69.0, a warning displays asking you to update your API version - as the CLI no longer supports versions older than 2.69.0
Bug Fixes:
  • Fixes a bug whereby X-Cf-Warnings were not being unescaped when displayed to user
  • When using CF_TRACE=1, passwords are now sanitized

Let us know if you have any comments or feedback, thanks!

Best,

Abby Chau
Product Manager, CF CLI
Re: Deprecation Notice: bosh-hm-forwarder is moving

Mike Youngstrom
 

Thanks Joseph,

Perhaps just sending everything would be fine as long as the deployment were added as a tag to the metric or something.  I'll keep thinking about it, look at the code now that it is available, and create an issue in the release repository if I want to discuss it further before putting together any kind of PR or anything.

Thanks,
Mike

On Mon, Jun 4, 2018 at 1:42 PM, Joseph Rodriguez <jrodriguez@...> wrote:
Bosh system metrics can not filter based on deployment (or any other criteria).

We haven't heard about this use case yet but it does sound interesting. If filtering were an option, would you have one deployment of CF with no filter enabled to receive metrics from deployments for non-cf releases (e.g. postgres, redis, et cetera)?

In terms of security, there are no deployment-level permissions on the forwarder client: any consumer that successfully connects to the bosh-system-metrics-server has access to the entire stream. Filtering could occur on either end (director or forwarder), but either way, all metrics will be accessible to any deployment.

It would be relatively straight-forward to fork and add a deployment filter option to the code (1).

1. This is where the bosh-system-metrics-forwarder receives metrics from the bosh-system-metrics-server: https://github.com/cloudfoundry/bosh-system-metrics-forwarder-release/blob/master/src/github.com/pivotal-cf/bosh-system-metrics-forwarder/pkg/ingress/ingress.go#L182


Re: Deprecation Notice: bosh-hm-forwarder is moving

Joseph Rodriguez <jrodriguez@...>
 

Bosh system metrics can not filter based on deployment (or any other criteria).

We haven't heard about this use case yet but it does sound interesting. If filtering were an option, would you have one deployment of CF with no filter enabled to receive metrics from deployments for non-cf releases (e.g. postgres, redis, et cetera)?

In terms of security, there are no deployment-level permissions on the forwarder client: any consumer that successfully connects to the bosh-system-metrics-server has access to the entire stream. Filtering could occur on either end (director or forwarder), but either way, all metrics will be accessible to any deployment.

It would be relatively straight-forward to fork and add a deployment filter option to the code (1).

1. This is where the bosh-system-metrics-forwarder receives metrics from the bosh-system-metrics-server: https://github.com/cloudfoundry/bosh-system-metrics-forwarder-release/blob/master/src/github.com/pivotal-cf/bosh-system-metrics-forwarder/pkg/ingress/ingress.go#L182
NOTICE: [php-buildpack] Default PHP version will change from 5.6.x to 7.2.x after 2018-07-02

Scott Sisil
 

The default version of PHP will be updated to 7.2.x in the first release after 2018-07-02.


Per the PHP supported versions website, all support for the 5.6.x series of releases will end on 2018-12-31, roughly four years after the initial release[1]. We are giving users a 30 day notice before we change the default version of PHP to 7.2.x. We will continue to support 5.6.x until the end of the year, but if you are still using PHP 5.6.x, we strongly encourage you to upgrade to a more recent, fully supported version of PHP.


[1] http://php.net/supported-versions.php




Scott


CF Buildpacks PM
Re: Oratos Incubation Proposal #loggregator

Krannich, Bernd <bernd.krannich@...>
 

Hi Adam,

 

Congrats, excited to see this as an incubation proposal, following the Eirini example directly in the Runtime PMC. It is great to see these pieces of technology moving into the core of CF right away.

 

I left a few smaller cosmetic feedbacks inside the docs you shared.

 

Regards,

Bernd

 

 

Bernd Krannich

SAP Cloud Platform

SAP SE

Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany

 

E bernd.krannich@...

 

Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum

 

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

 

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
Re: Oratos Incubation Proposal #loggregator

Dr Nic Williams
 

I hope all K8s operators and users are excited for CF logging system!

Question: why did Naomi have to wait 6 months for Cody (to “bosh deploy” CFCR?) :)

From: cf-dev@... <cf-dev@...> on behalf of Adam Hevenor <ahevenor@...>
Sent: Saturday, June 2, 2018 12:06:00 AM
To: cf-dev@...
Subject: Re: [cf-dev] Oratos Incubation Proposal #loggregator
 
Re: Oratos Incubation Proposal #loggregator

Adam Hevenor
 

Oratos Incubation Proposal #loggregator

Adam Hevenor
 

As many of you know, the Loggregator team has been experimenting with containerizing the Loggregator and Log cache components for use in a K8s context. I posted about these efforts previously[1], and we would like to officially nominate these efforts for incubation within the Runtime PMC (we have also considered extensions - but this would mean twice as many meetings for me ;-) ). We feel that assigning a specific team to this effort is worthwhile to formalize the efforts around product research and design and also want better communicate our intentions to the CF community to gather feedback. Additionally we have identified a first up feature proposal[2] to focus on. 

Please review this proposal and FAQ[3] and provide your feedback and comments here or in the document. 

1. https://lists.cloudfoundry.org/g/cf-dev/topic/oratos_containerized/18106811?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,40,18106811
2. https://docs.google.com/document/d/1PjbaImDrSs3qj1oqu46lSChGgJ6ka_N5AuQv0HVkBbI/edit#
3. https://docs.google.com/document/d/1oX4f3VLbTB9AXHOGUA8auGdwK19OVv1dRcKQYc8QsWk/edit#
Re: Deprecation Notice: bosh-hm-forwarder is moving

Adam Hevenor
 

Mike - 

I am not sure, but probably. I'll ping some folks that would know and have them chime in. 

Adam 
Re: CF CLI v6.37.0 Released Today: Variable Substitution

Ronak Banka
 

Hi Abby,

Going forward is there something on roadmap to use ops files for cf CLI too, like we do for Bosh CLI or yaml patch?

Use case can be CI/CD pipelines where part of manifest can be injected for different environment.

Thanks
Ronak


On Thu, 31 May 2018 at 7:37 AM, Abby Chau <achau@...> wrote:
Hi Benjamin,

Thanks for reaching out; we will update the documentation for clarification. The partial value use case, for example, is a manifest.yml with the following:

- route: ((host)).example.com

and a variable.yml file which invokes the value by passing in the hostname:

host: test

When you push with the manifest and variable file, the route test.example.com will be created. Hope that makes sense. 

Best,

Abby



On Wed, May 30, 2018 at 7:30 AM, Benjamin Gandon <benjamin@...> wrote:
Hi Abby,

In the documentation for variable substitution, when it’s said:

Template variables can also be used as partial values, such as in:

It’s quite unclear what is meant by “partial values”, and the example doesn’t bring an illustration to this. Indeed, correct me if I’m wrong but this second example looks the same as the first one, because using --vars-file <(cat secrets.yml) should be quite the same as writing --vars-file secrets.yml.

Maybe “partial values” was a way to say that multiple --vars-file arguments can be used?

Benjamin


Le 30 mai 2018 à 03:17, Abby Chau <achau@...> a écrit :

Hi all,

The CF CLI team released cf CLI v6.37.0; please see release notes for full details. 

Highlights include:
  • Variable substitution - (replaces inheritance) which allows app developers to create app manifests with values shared across all applicable environments in combination with references to environment-specific differences defined in separate files
  • Now the CLI reads username and password from environment variables
  • If you are using a API version older than 2.69.0, a warning displays asking you to update your API version - as the CLI no longer supports versions older than 2.69.0
Bug Fixes:
  • Fixes a bug whereby X-Cf-Warnings were not being unescaped when displayed to user
  • When using CF_TRACE=1, passwords are now sanitized

Let us know if you have any comments or feedback, thanks!

Best,

Abby Chau
Product Manager, CF CLI
Re: CF CLI v6.37.0 Released Today: Variable Substitution

Abby Chau
 

Hey Carlo, thanks for reaching out.

Arrays and hashes are not supported for the Variable Substitution feature. Let us know if you have any additional questions. Thanks.

Best,

Abby


On Wed, May 30, 2018 at 7:48 PM, Carlo Alberto Ferraris <carloalberto.ferraris@...> wrote:
Abby,
Is inserting arrays/hashes also supported? If that's the case it may be a good idea to show it in the examples.

Carlo

On Thu, May 31, 2018, 09:37 Abby Chau <achau@...> wrote:
Hi Benjamin,

Thanks for reaching out; we will update the documentation for clarification. The partial value use case, for example, is a manifest.yml with the following:

- route: ((host)).example.com

and a variable.yml file which invokes the value by passing in the hostname:

host: test

When you push with the manifest and variable file, the route test.example.com will be created. Hope that makes sense. 

Best,

Abby



On Wed, May 30, 2018 at 7:30 AM, Benjamin Gandon <benjamin@...> wrote:
Hi Abby,

In the documentation for variable substitution, when it’s said:

Template variables can also be used as partial values, such as in:

It’s quite unclear what is meant by “partial values”, and the example doesn’t bring an illustration to this. Indeed, correct me if I’m wrong but this second example looks the same as the first one, because using --vars-file <(cat secrets.yml) should be quite the same as writing --vars-file secrets.yml.

Maybe “partial values” was a way to say that multiple --vars-file arguments can be used?

Benjamin


Le 30 mai 2018 à 03:17, Abby Chau <achau@...> a écrit :

Hi all,

The CF CLI team released cf CLI v6.37.0; please see release notes for full details. 

Highlights include:
  • Variable substitution - (replaces inheritance) which allows app developers to create app manifests with values shared across all applicable environments in combination with references to environment-specific differences defined in separate files
  • Now the CLI reads username and password from environment variables
  • If you are using a API version older than 2.69.0, a warning displays asking you to update your API version - as the CLI no longer supports versions older than 2.69.0
Bug Fixes:
  • Fixes a bug whereby X-Cf-Warnings were not being unescaped when displayed to user
  • When using CF_TRACE=1, passwords are now sanitized

Let us know if you have any comments or feedback, thanks!

Best,

Abby Chau
Product Manager, CF CLI
Re: Deprecation Notice: bosh-hm-forwarder is moving

Mike Youngstrom
 

Hi Adam,

Is it possible to tell bosh-system-metrics to only forward metrics from certain deployments to loggregator?  We deploy multiple CFs on a single bosh and would like to only forward metrics related to a given CF onto that CF's loggregator stream.

Thanks,
Mike

On Thu, May 31, 2018 at 7:50 AM, Adam Hevenor <ahevenor@...> wrote:
Hi Benjamin - 

The bosh-hm-fowrarder is still available outside of the Loggregator repo, just not supported by the Loggregator team any longer. The supported replacement called bosh-system-metrics forwarder has gone through OSS approval and should be available very soon as well. 

Adam


Re: Deprecation Notice: bosh-hm-forwarder is moving

Adam Hevenor
 

Hi Benjamin - 

The bosh-hm-fowrarder is still available outside of the Loggregator repo, just not supported by the Loggregator team any longer. The supported replacement called bosh-system-metrics forwarder has gone through OSS approval and should be available very soon as well. 

Adam
Re: CF CLI v6.37.0 Released Today: Variable Substitution

Carlo Alberto Ferraris
 

Abby,
Is inserting arrays/hashes also supported? If that's the case it may be a good idea to show it in the examples.

Carlo

On Thu, May 31, 2018, 09:37 Abby Chau <achau@...> wrote:
Hi Benjamin,

Thanks for reaching out; we will update the documentation for clarification. The partial value use case, for example, is a manifest.yml with the following:

- route: ((host)).example.com

and a variable.yml file which invokes the value by passing in the hostname:

host: test

When you push with the manifest and variable file, the route test.example.com will be created. Hope that makes sense. 

Best,

Abby



On Wed, May 30, 2018 at 7:30 AM, Benjamin Gandon <benjamin@...> wrote:
Hi Abby,

In the documentation for variable substitution, when it’s said:

Template variables can also be used as partial values, such as in:

It’s quite unclear what is meant by “partial values”, and the example doesn’t bring an illustration to this. Indeed, correct me if I’m wrong but this second example looks the same as the first one, because using --vars-file <(cat secrets.yml) should be quite the same as writing --vars-file secrets.yml.

Maybe “partial values” was a way to say that multiple --vars-file arguments can be used?

Benjamin


Le 30 mai 2018 à 03:17, Abby Chau <achau@...> a écrit :

Hi all,

The CF CLI team released cf CLI v6.37.0; please see release notes for full details. 

Highlights include:
  • Variable substitution - (replaces inheritance) which allows app developers to create app manifests with values shared across all applicable environments in combination with references to environment-specific differences defined in separate files
  • Now the CLI reads username and password from environment variables
  • If you are using a API version older than 2.69.0, a warning displays asking you to update your API version - as the CLI no longer supports versions older than 2.69.0
Bug Fixes:
  • Fixes a bug whereby X-Cf-Warnings were not being unescaped when displayed to user
  • When using CF_TRACE=1, passwords are now sanitized

Let us know if you have any comments or feedback, thanks!

Best,

Abby Chau
Product Manager, CF CLI
Re: CF CLI v6.37.0 Released Today: Variable Substitution

Abby Chau
 

Hi Benjamin,

Thanks for reaching out; we will update the documentation for clarification. The partial value use case, for example, is a manifest.yml with the following:

- route: ((host)).example.com

and a variable.yml file which invokes the value by passing in the hostname:

host: test

When you push with the manifest and variable file, the route test.example.com will be created. Hope that makes sense. 

Best,

Abby



On Wed, May 30, 2018 at 7:30 AM, Benjamin Gandon <benjamin@...> wrote:
Hi Abby,

In the documentation for variable substitution, when it’s said:

Template variables can also be used as partial values, such as in:

It’s quite unclear what is meant by “partial values”, and the example doesn’t bring an illustration to this. Indeed, correct me if I’m wrong but this second example looks the same as the first one, because using --vars-file <(cat secrets.yml) should be quite the same as writing --vars-file secrets.yml.

Maybe “partial values” was a way to say that multiple --vars-file arguments can be used?

Benjamin


Le 30 mai 2018 à 03:17, Abby Chau <achau@...> a écrit :

Hi all,

The CF CLI team released cf CLI v6.37.0; please see release notes for full details. 

Highlights include:
  • Variable substitution - (replaces inheritance) which allows app developers to create app manifests with values shared across all applicable environments in combination with references to environment-specific differences defined in separate files
  • Now the CLI reads username and password from environment variables
  • If you are using a API version older than 2.69.0, a warning displays asking you to update your API version - as the CLI no longer supports versions older than 2.69.0
Bug Fixes:
  • Fixes a bug whereby X-Cf-Warnings were not being unescaped when displayed to user
  • When using CF_TRACE=1, passwords are now sanitized

Let us know if you have any comments or feedback, thanks!

Best,

Abby Chau
Product Manager, CF CLI
rep fd keep increasing until 'too many open files' and cell in bad status

Qiu Jie QJ Li <liqiujie@...>
 

Hi, CF developers
We met a problem that rep fd keep increasing until 'too many open files'.

Our cloudfoundry env was built on kubenetes cluster, it had 3 VMs under it.  1 for diego-cell (4core * 16G) and 2 for others.   When we did stress test, we used 10+ threads to push/start/stop/../delete apps continuously with 10s thinktime between each step. It began with 0 errors, but always ended with cell in bad status hours later.    App stage failed with 'can't communicate with compatible cells' and 'too many open files' in rep.stdout.log . We began to monitor the # of files under /proc/<rep-pid>/fd due to the 'too many open files' hint and noticed that the # of files was steady at first, then from a point, it kept increasing, even after the push app test was completely stopped, the increasing file number seems like the cause of 'too many open files' and most likely would cause the node(VM) unreachable in the end.

Why would this fd keep increasing? Was there some leak or something couldn't be released?  

I had opened an issue in rep repository https://github.com/cloudfoundry/rep/issues/21with more details. Please let us know what extra detailed info you need to know.

Thanks a lot.

Regards,
Qiu Jie (Sophy) Li