Date   

Re: Buildpack Cache

Ben Hale <bhale@...>
 

Different buildpacks use this cache for different things. Ben Hale (CCd) may have more details about how the Java buildpack handles caching. In general, the purpose of the cache is to speed up build time and/or reduce data transfer during staging.
The Java Buildpack treats the Buildpack cache as an opaque directory. This means that there are larger system policies about whether it's cached per-cell, per-foundation, etc. but from the perspective of the Java Buildpack, there is only a directory that may or may not contain cached artifacts.

All artifacts that the Java Buildpack might download are cached upon retrieval. So if you are using an online buildpack and download a JVM, we'll only download once per buildpack-cache (again, the system-wide policies are what really govern how often this happens) and store it for future staging. In addition to the artifact itself, we also store the `Last-Modified` timestamp and the `ETag` if either of these is returned when the artifact is downloaded. For all subsequent uses of an artifact, the buildpack first attempts a download with `If-Modified-Since` and `If-None-Match`. If all goes well (i.e. the artifact hasn't changed on the server), a `304 Not Modified` response will be returned and we know that our cached copies are safe for use and they'll be served up to the application without and bytes downloaded.

Cache hits are called out as part of staging output:

```
-----> Java Buildpack v4.12 (offline) | https://github.com/cloudfoundry/java-buildpack.git#5dca820
-----> Downloading Jvmkill Agent 1.12.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/jvmkill-1.12.0_RELEASE.so (found in cache)
-----> Downloading Open Jdk JRE 1.8.0_172 from https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_172.tar.gz (found in cache)
Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.1s)
JVM DNS caching disabled in lieu of BOSH DNS caching
-----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-3.13.0_RELEASE.tar.gz (found in cache)
Loaded Classes: 16818, Threads: 250
-----> Downloading Client Certificate Mapper 1.6.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.6.0_RELEASE.jar (found in cache)
-----> Downloading Container Security Provider 1.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.13.0_RELEASE.jar (found in cache)
-----> Downloading Spring Auto Reconfiguration 2.4.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.4.0_RELEASE.jar (found in cache)
```

versus:

```
-----> Java Buildpack 222ce62 | https://github.com/cloudfoundry/java-buildpack.git#222ce62
-----> Downloading Jvmkill Agent 1.14.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/jvmkill-1.14.0_RELEASE.so (0.0s)
-----> Downloading Open Jdk JRE 1.8.0_172 from https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_172.tar.gz (0.9s)
Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.4s)
JVM DNS caching disabled in lieu of BOSH DNS caching
-----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-3.13.0_RELEASE.tar.gz (0.0s)
Loaded Classes: 16818, Threads: 250
-----> Downloading Client Certificate Mapper 1.6.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.6.0_RELEASE.jar (0.0s)
-----> Downloading Container Security Provider 1.14.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.14.0_RELEASE.jar (0.0s)
-----> Downloading Spring Auto Reconfiguration 2.4.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.4.0_RELEASE.jar (0.0s)
```


-Ben Hale
Cloud Foundry Java Experience


Re: Buildpack Cache

Stephen Levine
 

The system buildpacks are cached on the cell and bind mounted read-only into each staging container. Buildpack URLs (to a git repo or a zip file) are downloaded every time staging occurs.


On Mon, Jun 18, 2018 at 12:50 PM Stephan Merker <stephan.merker@...> wrote:

Thanks for the explanation.

 

I have one more question: Is there any caching for the buildpack itself?

The CC coding looks like the buildpack is downloaded for each staging operation (either as zip file or via git clone).

Is there a difference between system buildpacks and custom buildpacks when it comes to download and caching?

 

Best regards,

Stephan

 

From: cf-dev@... [mailto:cf-dev@...] On Behalf Of Stephen Levine
Sent: Donnerstag, 14. Juni 2018 22:04
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev@...>; Ben Hale <bhale@...>
Subject: Re: [cf-dev] Buildpack Cache

 

Hi Stephan,

 

The buildpack cache (aka "app cache," or "build artifacts cache") is a per-app cache that is stored as a tarball in the blob store and recovered when an existing app is restaged (regardless of app code or buildpack changes).

 

Different buildpacks use this cache for different things. Ben Hale (CCd) may have more details about how the Java buildpack handles caching. In general, the purpose of the cache is to speed up build time and/or reduce data transfer during staging.

 

 

-Stephen

 

 

On Thu, Jun 14, 2018 at 10:31 AM Stephan Merker <stephan.merker@...> wrote:

Hi,

 

is there documentation available that describes how the buildpack cache works?

 

I found the following info:

https://docs.cloudfoundry.org/concepts/how-applications-are-staged.html

“The Task downloads buildpacks and the app’s buildpack cache, if present.”

 

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/extending-caches.md

 

I wonder how often the buildpacks and artifacts like JVM, Tomcat etc. are downloaded and how they are cached? Is this cache per app, per diego cell, CF global?

I’m talking about the default online buildpacks.

 

Thanks,

Stephan

 


Re: Buildpack Cache

Stephan Merker
 

Thanks for the explanation.

 

I have one more question: Is there any caching for the buildpack itself?

The CC coding looks like the buildpack is downloaded for each staging operation (either as zip file or via git clone).

Is there a difference between system buildpacks and custom buildpacks when it comes to download and caching?

 

Best regards,

Stephan

 

From: cf-dev@... [mailto:cf-dev@...] On Behalf Of Stephen Levine
Sent: Donnerstag, 14. Juni 2018 22:04
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev@...>; Ben Hale <bhale@...>
Subject: Re: [cf-dev] Buildpack Cache

 

Hi Stephan,

 

The buildpack cache (aka "app cache," or "build artifacts cache") is a per-app cache that is stored as a tarball in the blob store and recovered when an existing app is restaged (regardless of app code or buildpack changes).

 

Different buildpacks use this cache for different things. Ben Hale (CCd) may have more details about how the Java buildpack handles caching. In general, the purpose of the cache is to speed up build time and/or reduce data transfer during staging.

 

 

-Stephen

 

 

On Thu, Jun 14, 2018 at 10:31 AM Stephan Merker <stephan.merker@...> wrote:

Hi,

 

is there documentation available that describes how the buildpack cache works?

 

I found the following info:

https://docs.cloudfoundry.org/concepts/how-applications-are-staged.html

“The Task downloads buildpacks and the app’s buildpack cache, if present.”

 

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/extending-caches.md

 

I wonder how often the buildpacks and artifacts like JVM, Tomcat etc. are downloaded and how they are cached? Is this cache per app, per diego cell, CF global?

I’m talking about the default online buildpacks.

 

Thanks,

Stephan

 


Re: Feedback on service instance sharing (experimental)

Lukas Lehmann
 

Great news, thans a lot for the update Matt!

Am Mo., 18. Juni 2018 um 12:22 Uhr schrieb Matt McNeeney <mmcneeney@...>:

Hey Lukas,

Thanks for checking in! We have had further feedback from users who have been trying out this feature and so far of the feedback has been very positive. Given that, we are likely to look into what we need to do to make this feature GA in the next few weeks. You can track this story to keep updated!

Matt


On Mon, Jun 18, 2018 at 9:43 AM Lukas Lehmann <lukas92lehmann@...> wrote:
Hi Matt,

Just wanted to check back if there is something we can help to make this feature GA?
Did you get any further feedback within the last month?

Best regards,
Lukas

Am Mi., 16. Mai 2018 um 13:15 Uhr schrieb Matt McNeeney <mmcneeney@...>:
Hey Lukas,

Very pleased to hear the feature is working as you expected! As things stand, the feature should be released as GA within the next couple of months, and so far we haven't received any feedback that has made us rethink about how the feature works today.

Thanks,
Matt



On Wed, May 16, 2018 at 11:38 AM Lukas Lehmann <lukas92lehmann@...> wrote:
Hi Matt,

We have tested this feature in Swisscom and it works great and as expected. 
Is there a timeline on your side to release this future finally?
Is there something we can further help you with testing or so?

Best regards,
Lukas

2018-03-27 16:05 GMT+02:00 Matt McNeeney <mmcneeney@...>:
I'm pleased to announce that the ability to share service instances across orgs and spaces has been released!

In order to use this feature, please make sure you are running:
To share a service instance from one space into another, you just need to run cf share-service SERVICE-INSTANCE -s SPACE

Note that this needs to be enabled by both admins (via a feature-flag) and by the author of the service you want to share.

For more information, please check out the following docs:

I'd encourage you all to try this out with a supportive service broker (like this example broker). This feature is currently experimental (beta), so feedback at this time would be extremely valuable.

Best wishes,
The CF Services API team




--
Freundliche Grüsse
 
Lukas Lehmann
 



--
Freundliche Grüsse
 
Lukas Lehmann
 



--
Freundliche Grüsse
 
Lukas Lehmann
 


Re: Feedback on service instance sharing (experimental)

Matt McNeeney
 

Hey Lukas,

Thanks for checking in! We have had further feedback from users who have been trying out this feature and so far of the feedback has been very positive. Given that, we are likely to look into what we need to do to make this feature GA in the next few weeks. You can track this story to keep updated!

Matt


On Mon, Jun 18, 2018 at 9:43 AM Lukas Lehmann <lukas92lehmann@...> wrote:
Hi Matt,

Just wanted to check back if there is something we can help to make this feature GA?
Did you get any further feedback within the last month?

Best regards,
Lukas

Am Mi., 16. Mai 2018 um 13:15 Uhr schrieb Matt McNeeney <mmcneeney@...>:
Hey Lukas,

Very pleased to hear the feature is working as you expected! As things stand, the feature should be released as GA within the next couple of months, and so far we haven't received any feedback that has made us rethink about how the feature works today.

Thanks,
Matt



On Wed, May 16, 2018 at 11:38 AM Lukas Lehmann <lukas92lehmann@...> wrote:
Hi Matt,

We have tested this feature in Swisscom and it works great and as expected. 
Is there a timeline on your side to release this future finally?
Is there something we can further help you with testing or so?

Best regards,
Lukas

2018-03-27 16:05 GMT+02:00 Matt McNeeney <mmcneeney@...>:
I'm pleased to announce that the ability to share service instances across orgs and spaces has been released!

In order to use this feature, please make sure you are running:
To share a service instance from one space into another, you just need to run cf share-service SERVICE-INSTANCE -s SPACE

Note that this needs to be enabled by both admins (via a feature-flag) and by the author of the service you want to share.

For more information, please check out the following docs:

I'd encourage you all to try this out with a supportive service broker (like this example broker). This feature is currently experimental (beta), so feedback at this time would be extremely valuable.

Best wishes,
The CF Services API team




--
Freundliche Grüsse
 
Lukas Lehmann
 



--
Freundliche Grüsse
 
Lukas Lehmann
 


Re: Feedback on service instance sharing (experimental)

Lukas Lehmann
 

Hi Matt,

Just wanted to check back if there is something we can help to make this feature GA?
Did you get any further feedback within the last month?

Best regards,
Lukas

Am Mi., 16. Mai 2018 um 13:15 Uhr schrieb Matt McNeeney <mmcneeney@...>:

Hey Lukas,

Very pleased to hear the feature is working as you expected! As things stand, the feature should be released as GA within the next couple of months, and so far we haven't received any feedback that has made us rethink about how the feature works today.

Thanks,
Matt



On Wed, May 16, 2018 at 11:38 AM Lukas Lehmann <lukas92lehmann@...> wrote:
Hi Matt,

We have tested this feature in Swisscom and it works great and as expected. 
Is there a timeline on your side to release this future finally?
Is there something we can further help you with testing or so?

Best regards,
Lukas

2018-03-27 16:05 GMT+02:00 Matt McNeeney <mmcneeney@...>:
I'm pleased to announce that the ability to share service instances across orgs and spaces has been released!

In order to use this feature, please make sure you are running:
To share a service instance from one space into another, you just need to run cf share-service SERVICE-INSTANCE -s SPACE

Note that this needs to be enabled by both admins (via a feature-flag) and by the author of the service you want to share.

For more information, please check out the following docs:

I'd encourage you all to try this out with a supportive service broker (like this example broker). This feature is currently experimental (beta), so feedback at this time would be extremely valuable.

Best wishes,
The CF Services API team




--
Freundliche Grüsse
 
Lukas Lehmann
 



--
Freundliche Grüsse
 
Lukas Lehmann
 


Re: (action requested) Changing CF App Display

Abby Chau
 

Hi all,

The CF CLI app display feedback survey will close today; if you haven't had a chance to take the short survey, we would appreciate your feedback. 

Thanks!

Best,

Abby


On Mon, Jun 11, 2018 at 9:49 AM, Abby Chau <achau@...> wrote:
Hello cf-dev,

Thanks to those who have taken the short survey; we very much appreciate your feedback. If you haven't had a chance to take the survey, please take a look and give us your feedback on the cf app display.

Best,

Abby


On Tue, Jun 5, 2018 at 6:49 PM, Abby Chau <achau@...> wrote:
Hello cf-dev,

In order to provide a better user experience, the Cloud Foundry CLI team is considering changing the   cf app app-name  key-value and table display information. Towards this effort, we are asking the community to complete a short survey. 

Actions: Please fill out this short survey [1] to help us understand how you use the cf app app-name command display, and your thoughts on changing the display to default to the V3-app version of the command. The survey will close by end of day, June 15th. 

Thanks in advance for your time and participation.

Best,

Abby Chau
CF Product Manager - CLI




Cloud Foundry Java Client 2.x EOL

Paul Harris
 

Hi All,

We are planning to end support for version 2.x of the Cloud Foundry Java Client on December 31st, 2018. Updates will continue until then, but we recommend all users upgrade [0] to 3.x early. The 3.x line currently contains all the same functionality as the 2.x line, but benefits from more up-to-date dependencies.

Cheers, Paul


Calling Cloud Foundry Meetup Organizers

Swarna Podila
 

If you're a Cloud Foundry meetup organizer (or a frequent speaker at Cloud Foundry meetups), can you please connect with me directly via email or slack? 

Thank you,
-- ​Swarna Podila
​Senior
 Director
​, Community​
 | Cloud Foundry Foundation
​spodila
@cloudfoundry.org
@
​skpodila​


Re: Buildpack Cache

Stephen Levine
 

Hi Stephan,

The buildpack cache (aka "app cache," or "build artifacts cache") is a per-app cache that is stored as a tarball in the blob store and recovered when an existing app is restaged (regardless of app code or buildpack changes).

Different buildpacks use this cache for different things. Ben Hale (CCd) may have more details about how the Java buildpack handles caching. In general, the purpose of the cache is to speed up build time and/or reduce data transfer during staging.


-Stephen


On Thu, Jun 14, 2018 at 10:31 AM Stephan Merker <stephan.merker@...> wrote:

Hi,

 

is there documentation available that describes how the buildpack cache works?

 

I found the following info:

https://docs.cloudfoundry.org/concepts/how-applications-are-staged.html

“The Task downloads buildpacks and the app’s buildpack cache, if present.”

 

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/extending-caches.md

 

I wonder how often the buildpacks and artifacts like JVM, Tomcat etc. are downloaded and how they are cached? Is this cache per app, per diego cell, CF global?

I’m talking about the default online buildpacks.

 

Thanks,

Stephan

 


CF Routing release 0.179.0

Shubha Anjur Tupil
 

We cut routing release 0.179.0 yesterday with a bug fix for an issue where backends were not being pruned on a TLS handshake failure potentially leading to stale routes in the routing tables, now backends are pruned when the TLS handshake fails, for details

Please let us know if you have thoughts or questions. 

CF-Routing Team


Re: Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

A William Martin
 

Thanks, Aaron.

A couple of notes: The Garden Windows team has started working on contributing to the Envoy Windows support. We're betting on this as the most likely path forward for data-in-motion security, along with Istio support.

We're still planning how long the team (along with BOSH Windows) can maintain 2012 R2 support (as supporting a new Windows OS every 6 months is important but tedious). Our current thinking is to maintain it for about 12 months from now to give us time to achieve parity on the 2016 stack.

William


On Thu, Jun 14, 2018 at 12:04 PM Aaron Huber <aaron.m.huber@...> wrote:
Just to close on this Microsoft has confirmed that this is expected for now, using IPSec along with WinNAT is not supported in 1709, 1803, or the upcoming Windows Server 2019.  They are considering it for inclusion in a future release but there is no timeline.  For now there is no way to encrypt the traffic between the gorouter and the containers on Windows which will prevent us (and others I'm sure) from moving off of Windows Server 2012 R2 for legacy .NET apps.  Hopefully Envoy will be working on Windows soon (https://github.com/envoyproxy/envoy/issues/129) so we can remove the IPSec dependency.

Aaron


Re: Is anyone successfully using IPSec along with Windows Server 2016 (1709)?

Aaron Huber
 

Just to close on this Microsoft has confirmed that this is expected for now, using IPSec along with WinNAT is not supported in 1709, 1803, or the upcoming Windows Server 2019.  They are considering it for inclusion in a future release but there is no timeline.  For now there is no way to encrypt the traffic between the gorouter and the containers on Windows which will prevent us (and others I'm sure) from moving off of Windows Server 2012 R2 for legacy .NET apps.  Hopefully Envoy will be working on Windows soon (https://github.com/envoyproxy/envoy/issues/129) so we can remove the IPSec dependency.

Aaron


Buildpack Cache

Stephan Merker
 

Hi,

 

is there documentation available that describes how the buildpack cache works?

 

I found the following info:

https://docs.cloudfoundry.org/concepts/how-applications-are-staged.html

“The Task downloads buildpacks and the app’s buildpack cache, if present.”

 

https://github.com/cloudfoundry/java-buildpack/blob/master/docs/extending-caches.md

 

I wonder how often the buildpacks and artifacts like JVM, Tomcat etc. are downloaded and how they are cached? Is this cache per app, per diego cell, CF global?

I’m talking about the default online buildpacks.

 

Thanks,

Stephan

 


CAB call for June is Wednesday 06/20 @ 8a PST or 11a EST

Michael Maximilien
 

FYI...


Please remember to join Wednesday the Zoom call [0] June 20th at 8a Pacific for QAs, highlights, and two presentations:


1. CF Stratos UI update and demo of latest version by Neil MacDougall of SuSe [1] 


2. Project Blockheads: Ethereum Smart Contract Open Service Broker for CF and Kube by Nima Kaviani of IBM (winner of the CF Summit -Boston Hackathon)  [2] 


Zoom soon. Best,




Feedback request - Polyglot service discovery for container networking

Preethi Varambally
 

Hi,


The CF Networking team is working towards making Polyglot service discovery GA [1]. As part of this, operators can configure their own internal domains and we envision the following 2 scenarios.


Fresh install

- Operators use the CC APIs to configure internal domains

- Operators also have to configure a BOSH property "internal_domains" on the BOSH DNS adaptor job with the same internal domains


Upgrade from experimental service discovery

- apps.internal is already seeded into the cloud controller. No action is needed.

- Operators have to configure the BOSH property "internal_domains" on the BOSH DNS adaptor job to [apps.internal] to avoid downtime for apps using service discovery


If you are using the experimental feature, we would like to hear from you about any questions or concerns you may have on the upgrade process above.



Thanks,

CF Networking team




Feedback request: Proposed changes to capi-release nfs_server.share_path

Tim Downey
 

Hello all,

Recently on the CAPI team we've been updating our BOSH jobs to run within BPM. BPM is opinionated with regards to which directories can be mounted within its containerized jobs. This means that in order to properly support NFS, we will need to update the default nfs_server.share_path to be located within the "/var/vcap/data" directory on Cloud Controller BOSH instances.

The nfs_server.share_path value is currently configurable across several jobs within capi-release, but we are considering removing this configurability in order to ensure that the NFS volume is mounted in the proper location.

We are wondering what the impact of this change would be and are interested in hearing from anyone who is currently setting nfs_server.share_path to a location other than the default path of "/var/vcap/nfs".

In summary: 
  • at a minimum we will need to change the default value of nfs_server.share_path to support BPM in capi-release
  • we are also proposing that we make it no longer configurable to ensure that jobs running within BPM have access to the NFS mount
Thanks!
Tim Downey and Elena Sharma
CAPI team members


#advocacy on CF Slack #advocacy

Swarna Podila
 

Let us use #advocacy channel to share information on upcoming industry events, "call for paper" deadlines, and other speaking opportunities that might be of interest for y'all.


Re: (action requested) Changing CF App Display

Abby Chau
 

Hello cf-dev,

Thanks to those who have taken the short survey; we very much appreciate your feedback. If you haven't had a chance to take the survey, please take a look and give us your feedback on the cf app display.

Best,

Abby


On Tue, Jun 5, 2018 at 6:49 PM, Abby Chau <achau@...> wrote:
Hello cf-dev,

In order to provide a better user experience, the Cloud Foundry CLI team is considering changing the   cf app app-name  key-value and table display information. Towards this effort, we are asking the community to complete a short survey. 

Actions: Please fill out this short survey [1] to help us understand how you use the cf app app-name command display, and your thoughts on changing the display to default to the V3-app version of the command. The survey will close by end of day, June 15th. 

Thanks in advance for your time and participation.

Best,

Abby Chau
CF Product Manager - CLI



Re: Proposal: Metadata for API resources

Stephan Merker
 

+1

 

We had to introduce a number of workarounds / extra book keeping to keep track for e.g. ‘system applications’, orgs belonging to the installation and not to customers, orgs requiring exceptions from quota handling etc.

Annotations are the natural place to store such information.

 

Best regards,

Stephan

 

From: cf-dev@... [mailto:cf-dev@...] On Behalf Of Zach Robinson
Sent: Donnerstag, 7. Juni 2018 19:46
To: cf-dev@...
Subject: [cf-dev] Proposal: Metadata for API resources

 

Hey all,

The CAPI team would like to share a proposal to apply metadata to API resources.  Please have a look and add comments for any questions or concerns.

https://docs.google.com/document/d/1Ebko_wNu4wWLnAdHg6wmydEMTx_IdwYnb4Ys0mxVmM4/edit?usp=sharing

Thanks!
-Zach, CAPI Project Lead

1321 - 1340 of 9387