cf-dev@lists.cloudfoundry.org | Proposal to end-of-life the varz nozzle

Home Messages Hashtags Subgroups

Date Date 1 - 5 of 5

Proposal to end-of-life the varz nozzle

Jim CF Campbell #3843 The initial implementation of CF metrics used the Collector to poll /varz endpoints of various components and then emit them to consumers such as Datadog, Graphite and others. With the implementation of Loggregator and the firehose, the default metric path became nozzles attached to the firehose. The varz nozzle was created to consume metrics from the firehose and then present them via polled /varz endpoints to the Collector to allow backward compatibility. This is a long-deprecated technique and the Loggregator team is proposing to end support and maintenance of the varz nozzle. This will be followed soon by an effort to end-of-life the Collector itself. We would like community input on this. -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 attachment.html attachment.html
More All Messages By This Member
Marco Voelz #3853 Dear Jim, I understand that you want to get rid of a deprecated approach. However, I'm not sure if there is a parity for the new approach yet and if we've seen a grace period allowing people to switch. Can you give us an up-to-date overview about which metrics are available over the firehose and which only over /varz? For example, for quite a long time the metrics of DEAs were not available in the firehose. I know that this has recently changed [1], but there hasn't been a cf-release include these changes. As this is only one example I remember from the top of my head I'd like to * be sure that we actually have everything available in the firehose that we're getting from /varz today * after that is the case wait some time so people can switch and use the new approach Thanks and warm regards Marco [1] https://github.com/cloudfoundry/dea_ng/commit/a1aa10a887c5facb10270f82bba6214529fcfa3d#diff-9edf5787563a04c03ce32d85b1f6d7ab toggle quoted message Show quoted text On 19/02/16 01:21, "Jim CF Campbell" <jcampbell(a)pivotal.io<mailto:jcampbell(a)pivotal.io>> wrote: The initial implementation of CF metrics used the Collector to poll /varz endpoints of various components and then emit them to consumers such as Datadog, Graphite and others. With the implementation of Loggregator and the firehose, the default metric path became nozzles attached to the firehose. The varz nozzle was created to consume metrics from the firehose and then present them via polled /varz endpoints to the Collector to allow backward compatibility. This is a long-deprecated technique and the Loggregator team is proposing to end support and maintenance of the varz nozzle. This will be followed soon by an effort to end-of-life the Collector itself. We would like community input on this. -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 attachment.html attachment.html
More All Messages By This Member
Jim CF Campbell #3870 Hi Marco, the varz nozzle is only for metrics that are already in the firehose. It connects to the firehose and offers a /varz endpoint to the metrics that come through the firehose. It is designed to support systems that only want their metrics via varz. There are metrics from DEA and HM9000 that are currently not in the firehose and only available via /varz. Does that make sense? toggle quoted message Show quoted text On Fri, Feb 19, 2016 at 11:47 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote: Dear Jim, I understand that you want to get rid of a deprecated approach. However, I'm not sure if there is a parity for the new approach yet and if we've seen a grace period allowing people to switch. Can you give us an up-to-date overview about which metrics are available over the firehose and which only over /varz? For example, for quite a long time the metrics of DEAs were not available in the firehose. I know that this has recently changed [1], but there hasn't been a cf-release include these changes. As this is only one example I remember from the top of my head I'd like to - be sure that we actually have everything available in the firehose that we're getting from /varz today - after that is the case wait some time so people can switch and use the new approach Thanks and warm regards Marco [1] https://github.com/cloudfoundry/dea_ng/commit/a1aa10a887c5facb10270f82bba6214529fcfa3d#diff-9edf5787563a04c03ce32d85b1f6d7ab On 19/02/16 01:21, "Jim CF Campbell" <jcampbell(a)pivotal.io> wrote: The initial implementation of CF metrics used the Collector to poll /varz endpoints of various components and then emit them to consumers such as Datadog, Graphite and others. With the implementation of Loggregator and the firehose, the default metric path became nozzles attached to the firehose. The varz nozzle was created to consume metrics from the firehose and then present them via polled /varz endpoints to the Collector to allow backward compatibility. This is a long-deprecated technique and the Loggregator team is proposing to end support and maintenance of the varz nozzle. This will be followed soon by an effort to end-of-life the Collector itself. We would like community input on this. -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 attachment.html attachment.html
More All Messages By This Member
Marco Voelz #3889 Dear Jim, thanks for your answer and the detailed explanation of the /varz nozzle. I was more worried about the latter part of your first email, in which you say you want to EOL the collector "soon". For that to happen, I'd like to make sure that we don't lose any metrics we currently rely on. In terms of removing the /varz nozzle: I'd probably leave that in for as long as the collector is there. Forcing people to get metrics from two different sources to get the full picture is a possible workaround, but not the nicest solution here. Warm regards Marco toggle quoted message Show quoted text On 19/02/16 23:50, "Jim CF Campbell" <jcampbell(a)pivotal.io<mailto:jcampbell(a)pivotal.io>> wrote: Hi Marco, the varz nozzle is only for metrics that are already in the firehose. It connects to the firehose and offers a /varz endpoint to the metrics that come through the firehose. It is designed to support systems that only want their metrics via varz. There are metrics from DEA and HM9000 that are currently not in the firehose and only available via /varz. Does that make sense? On Fri, Feb 19, 2016 at 11:47 AM, Voelz, Marco <marco.voelz(a)sap.com<mailto:marco.voelz(a)sap.com>> wrote: Dear Jim, I understand that you want to get rid of a deprecated approach. However, I'm not sure if there is a parity for the new approach yet and if we've seen a grace period allowing people to switch. Can you give us an up-to-date overview about which metrics are available over the firehose and which only over /varz? For example, for quite a long time the metrics of DEAs were not available in the firehose. I know that this has recently changed [1], but there hasn't been a cf-release include these changes. As this is only one example I remember from the top of my head I'd like to * be sure that we actually have everything available in the firehose that we're getting from /varz today * after that is the case wait some time so people can switch and use the new approach Thanks and warm regards Marco [1] https://github.com/cloudfoundry/dea_ng/commit/a1aa10a887c5facb10270f82bba6214529fcfa3d#diff-9edf5787563a04c03ce32d85b1f6d7ab On 19/02/16 01:21, "Jim CF Campbell" <jcampbell(a)pivotal.io<mailto:jcampbell(a)pivotal.io>> wrote: The initial implementation of CF metrics used the Collector to poll /varz endpoints of various components and then emit them to consumers such as Datadog, Graphite and others. With the implementation of Loggregator and the firehose, the default metric path became nozzles attached to the firehose. The varz nozzle was created to consume metrics from the firehose and then present them via polled /varz endpoints to the Collector to allow backward compatibility. This is a long-deprecated technique and the Loggregator team is proposing to end support and maintenance of the varz nozzle. This will be followed soon by an effort to end-of-life the Collector itself. We would like community input on this. -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963<tel:303.618.0963> -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 attachment.html attachment.html
More All Messages By This Member
Jim CF Campbell #3960 Hi Marco, All metrics are now available from the firehose. The only metrics continuing to flow through the Collector are from DEA, which are also flowing through the firehose. The collector currently supports emitting metrics to Datadog - Graphite - OpenTSDB - AWS Cloud Watch All are supported via nozzles except Cloud Watch. So except for Cloud Watch, everyone can continue working in their own way with the Nozzle instead of the Collector. Cloud Watch people can continue to use Collector, we just won't maintain it. It has not had a commit since Sept 2015. We'd be happy to advise someone who wanted to build a Cloud Watch nozzle. Jim toggle quoted message Show quoted text On Mon, Feb 22, 2016 at 3:01 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote: Dear Jim, thanks for your answer and the detailed explanation of the /varz nozzle. I was more worried about the latter part of your first email, in which you say you want to EOL the collector "soon". For that to happen, I'd like to make sure that we don't lose any metrics we currently rely on. In terms of removing the /varz nozzle: I'd probably leave that in for as long as the collector is there. Forcing people to get metrics from two different sources to get the full picture is a possible workaround, but not the nicest solution here. Warm regards Marco On 19/02/16 23:50, "Jim CF Campbell" <jcampbell(a)pivotal.io> wrote: Hi Marco, the varz nozzle is only for metrics that are already in the firehose. It connects to the firehose and offers a /varz endpoint to the metrics that come through the firehose. It is designed to support systems that only want their metrics via varz. There are metrics from DEA and HM9000 that are currently not in the firehose and only available via /varz. Does that make sense? On Fri, Feb 19, 2016 at 11:47 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote: Dear Jim, I understand that you want to get rid of a deprecated approach. However, I'm not sure if there is a parity for the new approach yet and if we've seen a grace period allowing people to switch. Can you give us an up-to-date overview about which metrics are available over the firehose and which only over /varz? For example, for quite a long time the metrics of DEAs were not available in the firehose. I know that this has recently changed [1], but there hasn't been a cf-release include these changes. As this is only one example I remember from the top of my head I'd like to - be sure that we actually have everything available in the firehose that we're getting from /varz today - after that is the case wait some time so people can switch and use the new approach Thanks and warm regards Marco [1] https://github.com/cloudfoundry/dea_ng/commit/a1aa10a887c5facb10270f82bba6214529fcfa3d#diff-9edf5787563a04c03ce32d85b1f6d7ab On 19/02/16 01:21, "Jim CF Campbell" <jcampbell(a)pivotal.io> wrote: The initial implementation of CF metrics used the Collector to poll /varz endpoints of various components and then emit them to consumers such as Datadog, Graphite and others. With the implementation of Loggregator and the firehose, the default metric path became nozzles attached to the firehose. The varz nozzle was created to consume metrics from the firehose and then present them via polled /varz endpoints to the Collector to allow backward compatibility. This is a long-deprecated technique and the Loggregator team is proposing to end support and maintenance of the varz nozzle. This will be followed soon by an effort to end-of-life the Collector itself. We would like community input on this. -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 -- Jim Campbell \| Product Manager \| Cloud Foundry \| Pivotal.io \| 303.618.0963 attachment.html attachment.html
More All Messages By This Member

1 - 5 of 5

1

Previous Topic Next Topic

Home Hashtags Subgroups Terms