Application Metric Forwarding and Developer Segmented Firehose


Adam Hevenor
 

Hi Everyone -

The Loggregator team has been discussing two related features that surface more capabilities to App Developers. There are two separate concepts being proposed here. The first is the ability for developers to forward metrics into the firehose [1]. This expands abilities for developers to inspect custom aspects of their application and build automation on top of existing standards and tooling. The second concept is for developers to have access to a firehose segmented to their org/space[2]. This builds further on existing tooling, but exposes the tooling to development teams at a scope level they have access to on the platform. There is an intentional ordering to these separate but related proposals.

I appreciate your feedback and comments.

Thanks
Adam

[1] - https://docs.google.com/document/d/1hjMO3plNBDwtqCgVIYsayhCcl0k-H0Hzvaaam4SSkto/edit
[2] - https://docs.google.com/document/d/1z5aVaUn0J3sG3q5tGB1viDxAF70HZ1BYnF9vUe-c0j4/edit


Maxwell Eshleman
 

These sound like great ideas! What is the difference between the app metric
forwarding mentioned here and what Scott's team is doing?

-Max

On Wed, Jul 19, 2017 at 2:35 PM Adam Hevenor <ahevenor(a)pivotal.io> wrote:

Hi Everyone -

The Loggregator team has been discussing two related features that surface
more capabilities to App Developers. There are two separate concepts being
proposed here. The first is the ability for developers to forward metrics
into the firehose [1]. This expands abilities for developers to inspect
custom aspects of their application and build automation on top of existing
standards and tooling. The second concept is for developers to have access
to a firehose segmented to their org/space[2]. This builds further on
existing tooling, but exposes the tooling to development teams at a scope
level they have access to on the platform. There is an intentional ordering
to these separate but related proposals.

I appreciate your feedback and comments.

Thanks
Adam

[1] -
https://docs.google.com/document/d/1hjMO3plNBDwtqCgVIYsayhCcl0k-H0Hzvaaam4SSkto/edit
[2] -
https://docs.google.com/document/d/1z5aVaUn0J3sG3q5tGB1viDxAF70HZ1BYnF9vUe-c0j4/edit


Adam Hevenor
 

Max - The intent is the same and the UX is nearly identical (Scott's uses a service binding but that's the only real difference). This would be part of the OSS distribution is the key difference.


Bret Mogilefsky
 

How can one find out the state of a proposal? I'd like to know more about what happened with this one, but the proposal listing doesn't give me any hints.


Adam Hevenor
 

Bret - 

Thanks for inquiring. I try to keep the following github project board up to date with status of proposals and work in progress. Regarding these particular proposals we are making progress on these problems but have taken a slightly different course for egress thus far. Specifically we have been leveraging the work we did for cf-syslog-drain-release (formerly known as scalable syslog). A revised proposal for this approach is available here.  

If you are specifically looking to allow app developers to send and receive custom app metrics we are actually very close to having a full open source path available for that (again through syslog-drains) and the help of the metrics-forwarder service that Pivotal is open sourcing. I'll be sure to post to the list again when all these components are available and we'll provide some experimental ops files to make it simple to deploy.

Adam