1. Messages
  2. Buildpack Cache

Re: Buildpack Cache

Stephen Levine
 

The buildpack cache in CF is always per-app and foundation-wide. It can only be erased by deleting the app or dropping all cache artifacts on the platform.

The system buildpacks are cached on the cell, and if you use an offline buildpack with dependencies included, the buildpack will generally not need to download those dependencies. Therefore, they won't get cached in the app's buildpack cache (but other things might still be).


On Tue, Jun 19, 2018 at 11:04 AM Stephan Merker <stephan.merker@...> wrote:
> This means that there are larger system policies about whether it's cached per-cell, per-foundation, etc.

Do you mean with "larger system policies" the usage of online vs. offline buildpacks? Or are there configuration parameters in e.g. the CC  or Diego to configure the caching policy?
In other words: what is the global buildpack artifact caching policy for a standard https://github.com/cloudfoundry/cf-deployment/releases/tag/v1.40.0

Best regards,
Stephan

-----Original Message-----
From: cf-dev@... [mailto:cf-dev@...] On Behalf Of Ben Hale
Sent: Montag, 18. Juni 2018 20:23
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev@...>
Subject: Re: [cf-dev] Buildpack Cache

> Different buildpacks use this cache for different things. Ben Hale (CCd) may have more details about how the Java buildpack handles caching. In general, the purpose of the cache is to speed up build time and/or reduce data transfer during staging.

The Java Buildpack treats the Buildpack cache as an opaque directory.  This means that there are larger system policies about whether it's cached per-cell, per-foundation, etc. but from the perspective of the Java Buildpack, there is only a directory that may or may not contain cached artifacts.

All artifacts that the Java Buildpack might download are cached upon retrieval.  So if you are using an online buildpack and download a JVM, we'll only download once per buildpack-cache (again, the system-wide policies are what really govern how often this happens) and store it for future staging.  In addition to the artifact itself, we also store the `Last-Modified` timestamp and the `ETag` if either of these is returned when the artifact is downloaded.  For all subsequent uses of an artifact, the buildpack first attempts a download with `If-Modified-Since` and `If-None-Match`.  If all goes well (i.e. the artifact hasn't changed on the server), a `304 Not Modified` response will be returned and we know that our cached copies are safe for use and they'll be served up to the application without and bytes downloaded.

Cache hits are called out as part of staging output:

```
-----> Java Buildpack v4.12 (offline) | https://github.com/cloudfoundry/java-buildpack.git#5dca820
-----> Downloading Jvmkill Agent 1.12.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/jvmkill-1.12.0_RELEASE.so (found in cache)
-----> Downloading Open Jdk JRE 1.8.0_172 from https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_172.tar.gz (found in cache)
       Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.1s)
       JVM DNS caching disabled in lieu of BOSH DNS caching
-----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-3.13.0_RELEASE.tar.gz (found in cache)
       Loaded Classes: 16818, Threads: 250
-----> Downloading Client Certificate Mapper 1.6.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.6.0_RELEASE.jar (found in cache)
-----> Downloading Container Security Provider 1.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.13.0_RELEASE.jar (found in cache)
-----> Downloading Spring Auto Reconfiguration 2.4.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.4.0_RELEASE.jar (found in cache)
```

versus:

```
-----> Java Buildpack 222ce62 | https://github.com/cloudfoundry/java-buildpack.git#222ce62
-----> Downloading Jvmkill Agent 1.14.0_RELEASE from https://java-buildpack.cloudfoundry.org/jvmkill/trusty/x86_64/jvmkill-1.14.0_RELEASE.so (0.0s)
-----> Downloading Open Jdk JRE 1.8.0_172 from https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_172.tar.gz (0.9s)
       Expanding Open Jdk JRE to .java-buildpack/open_jdk_jre (1.4s)
       JVM DNS caching disabled in lieu of BOSH DNS caching
-----> Downloading Open JDK Like Memory Calculator 3.13.0_RELEASE from https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-3.13.0_RELEASE.tar.gz (0.0s)
       Loaded Classes: 16818, Threads: 250
-----> Downloading Client Certificate Mapper 1.6.0_RELEASE from https://java-buildpack.cloudfoundry.org/client-certificate-mapper/client-certificate-mapper-1.6.0_RELEASE.jar (0.0s)
-----> Downloading Container Security Provider 1.14.0_RELEASE from https://java-buildpack.cloudfoundry.org/container-security-provider/container-security-provider-1.14.0_RELEASE.jar (0.0s)
-----> Downloading Spring Auto Reconfiguration 2.4.0_RELEASE from https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-2.4.0_RELEASE.jar (0.0s)
```


-Ben Hale
Cloud Foundry Java Experience
Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.