Including Jim Campbell to make sure this reaches him.



On 20/05/16 13:41, "Chawki, Amin" <amin.chawki(a)sap.com> wrote:



Hi,



by upgrading to CF v234 (including pre-release v232) we lost all our
monitoring capabilities regarding DEA and HM9000 (we were still using
Collector). By migrating to Firehose only a fraction of the metrics was
available. Very important metrics for our productive systems like
‘available_memory_ratio’ were just added in CF v235. In the meantime, we
were pretty much “flying blind”.



We replaced not existing metrics like ‘DEA…mem_used_bytes’ and
‘HM9000…healthy’, which were available via Collector, with metrics from
Bosh. Is this the way to go or are there any plans to add them again?



Best Regards,

Amin

Is it correct to say that in the context of CF, `rootfs` is a Linux-only
abstraction, and is not necessarily relevant to Garden (whereas it may be
relevant to one or more of the Garden backends)?

On 19 May 2016, at 18:36, Lingesh Mouleeshwaran <lingeshmouleeshwaran(a)gmail.com> wrote:

Hi Cloudfoundry ,

Is there any simple way, we can update or install new libraries in existing cf stacks (cflinuxf2) without creating new custom stacks ??

Example : Our modified java buildpack not running on existing stacks because of few of the ruby gems are missing in the file system. so we would require to install few ruby gem and use the modified java buildpack.

Share your thoughts and suggestions.

Regards
Lingesh M

sorry JK - it's still early am (ignore what I just said) - the stack is
the specific rootfs - the container image is a combination of a droplet
plus a stack. The container image will then run in your garden process.
For a windows stack you would need a windows vm/cell to run it.

On Thu, May 19, 2016 at 8:43 AM Duncan Winn <dwinn(a)pivotal.io> wrote:

Hey JK, a stack is a combination of the rootfs and droplet. This is
conceptually a "container image" that will then run in your garden
container. For a windows stack you would need a windows vm/cell to run it.

On Thu, May 19, 2016 at 8:26 AM J K <falconwing(a)gmail.com> wrote:

Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?

--
Duncan Winn
Cloud Foundry PCF Services

--
Duncan Winn
Cloud Foundry PCF Services

Hey JK, a stack is a combination of the rootfs and droplet. This is
conceptually a "container image" that will then run in your garden
container. For a windows stack you would need a windows vm/cell to run it.

On Thu, May 19, 2016 at 8:26 AM J K <falconwing(a)gmail.com> wrote:

Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?

--
Duncan Winn
Cloud Foundry PCF Services

Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?

In the future we may consider implementing a json feed. If we did go
forward with this work, would you be interested in creating example CI / CD
implementations as references?

On Tue, May 17, 2016 at 10:30 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Hi,
Any json feed /api ?
Can be nice and more easy to integrate with any CI/CD tool.

Thanks

On Wed, May 18, 2016 at 11:15 AM, taichi nakashima <nsd22843(a)gmail.com>
wrote:

Great,

I hope cloudfoundry/cli will provide the same thing.

cf.
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/K3BEBY4A2WSUKS7YS5IF2UDQHHSU35A7/

Taichi Nakashima

2016年5月18日(水) 6:20 David Jahn <djahn(a)pivotal.io>:

Dear Cloud Foundry Users,

To help operators and users of Cloud Foundry establish a "chain of
custody" for buildpacks, we have launched the following checksum site:

https://buildpackverify.cloudfoundry.org

This site provides a checksum for all cached buildpack release zip
files (except for the java-buildpack). Whenever the buildpacks team
generates a new buildpack release, we will immediately compute the SHA256
checksum of that file and upload it to this website.

The site is hosted on a different repository from the main buildpack
github repositories. It allows operators to validate that the zip file we
produced is the same artifact that has been downloaded and installed.

Additionally, if an operator wishes to further investigate the
components of a buildpack, the "manifest.yml" in each buildpack root
directory (for
example,
https://github.com/cloudfoundry/go-buildpack/blob/master/manifest.yml)
provides a catalog of every third party component in the buildpack, a URL
of that component's location, and an MD5 checksum of that component.

We hope that this will assist people in auditing the source of their
buildpack code!

Cheers,
Buildpacks Team

hi,

I'm learning about Cloud Foundry's ideas of how to organize applications
and how it works under the covers. At the moment I'm having a little
trouble understanding the relationship between a few ideas in CF: "rootfs",
"Garden", and "stem cells".

Can someone explain:

(1) their understanding of what these ideas are; and
(2) how they are related?

Thank you!

Hi Nick,

Thanks for the update!
Are there any plans of bringing it to V2 APIs as well?

Kind Regards,
Padma

Hi,

To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage

metrics (the rest basically)

2. Store the time-based metrics in a separate DB(s)

Your proposal looks good to me. Piotr, Kevin and Raj and I had several
design discussions on this topic in the last few days and we've come up
with a few more ideas on top of what you're describing here:

- The distinction between time-based and discrete resource usage metering
could also be understood as usage metering of a stateful vs stateless
resource or metric. In the stateful case, we simply store the state of the
resource instance in a separate DB like you're proposing (e.g. we store the
fact that an app or a container is currently running, or has stopped), and
update that state in place when it changes. Then to compute and report
usage later on we just need to the current resource instance state from
that DB.

- We could continue to store the metrics in the current historical
databases as well (on top of that new DB) to preserve the resource instance
history as many users typically want to know their past usage.

- Some of us were not sure if the time-based / discrete distinction should
be at the resource type level or at the metric level... IMO your proposal
to do that at the metric level is cleaner so I'm happy with it :)

- The dataflow module will probably need a few minor code changes to
detect the case where some of the output docs need to go to a separate DB
(IIRC you or someone else also mentioned that on one of our scrums or on
slack...)

- Like you said, we may still need to maintain 2 DBs to purge old entries.
If that's easier, we could also adjust the usage accumulator service and
the dataflow module a bit to delete entries for inactive resource instances
right away (e.g. when an app or container stops.)

Thoughts?

P.S. I'll add these comments to issue #88 as well to make it be easier to
follow up there.

- Jean-Sebastien

On Thu, May 12, 2016 at 5:54 AM, Hristo Iliev <hsiliev(a)gmail.com> wrote:

Hi,

We're trying to fix Abacus issue 88: Missing aggregated usage for the
running application [1].

Background
=========

See the jsdelfino comment in the GitHub issue [2]. TL;DR: Resource
providers have to send a 'ping' doc per month for time-based metrics.

Proposed solution
==============

We decided to implement a solution in Abacus that frees the usage
providers from sending the 'ping' submission.

To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage
metrics (the rest basically)
2. Store the time-based metrics in a separate DB(s)

We already drafted a proposal for adding measurement type in the usage
plans with PR #320 [3].

We're about to spike on storing the time-based metrics in their own
Database, but we wanted to get the community opinion on the topic.

Motivation
========

The discrete usage submitted to Abacus is:
* stored in partitioned databases, due to their size/number
* like an event log, storing the history of the usage/resources

In contrast the current time-based metrics are:
* limited number (usually around 2 million on a loaded CF system)
* storing just the app resources usage state (GB/h consumed so far, GB/h
consuming currently)

Therefore it looks like a good idea to separate the two usage metrics
types and store the time-based metrics in a separate database. This will
allow us not only to solve the issue, but also to store and query the data
more effectively.

We may still need to maintain 2 databases and swap new/old (irrelevant)
metrics to reduce the DB size on the month boundaries.


Regards,
Hristo & Adriana

[1] https://github.com/cloudfoundry-incubator/cf-abacus/issues/88
[2]
https://github.com/cloudfoundry-incubator/cf-abacus/issues/88#issuecomment-148498164
[3] https://github.com/cloudfoundry-incubator/cf-abacus/pull/320

but we all are saying same thing .. believe me after running multiple

bosh

services managing marketplace in a large org is kind of a messy jobs ..

I cannot agree more!