|
|
|
|
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning, so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Here are some example consumers and the documentation of the Log Cache:
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol. This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a CF deployment.
Please let us know if you have any comments or concerns.
Johannes
|
|
|
Johannes, just echoing what Neil is saying below:
We also feel that three months is probably too short of a runway for this.
Let's discuss the details, but 3 months is likely not good enough for us,
too. From:
"Neil MacDougall"
<neil.macdougall@...>To:
cf-dev@...Date:
12/02/2019 18:05Subject:
Re: [cf-dev]
[Proposal] Deprecation of the firehose endpointSent by:
cf-dev@...
Johannes, Thank your for the notification. We use the firehose in Stratos to stream logs for applications
and the firehose itself for users/admins to view in the UI. We also use the CF Firehose exporter ( https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease)
to retrieve metrics from the firehose and store those in Prometheus for
display within Stratos. Do you know what the plan is for these to be updated? It is a large amount of work for us to make the changes
for the removal of the firehose and I think 3 months is too aggressive
a time frame for us to be able to complete those - this is not something
we’ve factored into our planning, so I think we’d need a 12 month window
on the firehose removal to be safe. We have users using Stratos on a variety of Cloud Foundry
distributions, which we won’t be able to guarantee will be updated in
your proposed three month window, so we will most likely have to make updates
to allow Stratos to work with systems that use the current firehose and
those that use the newer APIs. I’m happy to discuss this further with you directly if
that would help. Regards, Neil
toggle quoted message
Show quoted text
|
|
|
Richard Towers <richard.towers@...>
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
Thanks, Richard
On Thu, 14 Feb 2019 at 15:22, Simon D Moser < smoser@...> wrote: Johannes,
just echoing what Neil is saying below:
We also feel that three months is probably too short of a runway for this.
Let's discuss the details, but 3 months is likely not good enough for us,
too.
From:
"Neil MacDougall"
<neil.macdougall@...>
To:
cf-dev@...
Date:
12/02/2019 18:05
Subject:
Re: [cf-dev]
[Proposal] Deprecation of the firehose endpoint
Sent by:
cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications
and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease)
to retrieve metrics from the firehose and store those in Prometheus for
display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes
for the removal of the firehose and I think 3 months is too aggressive
a time frame for us to be able to complete those - this is not something
we’ve factored into our planning, so I think we’d need a 12 month window
on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry
distributions, which we won’t be able to guarantee will be updated in
your proposed three month window, so we will most likely have to make updates
to allow Stratos to work with systems that use the current firehose and
those that use the newer APIs.
I’m happy to discuss this further with you directly if
that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...>
wrote:
Hi there,
Following the last thread about the deprecation
of the /containermetrics endpoint, the Loggregator team would like to continue
on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations,
so we will provide reference implementations and documents for migrating
away from the firehose to the newer loggregator endpoints (namely, the
RLP Gateway and LogCache). Depending on the feedback we receive to this
email, we would like to proceed with the removal of the firehose endpoint
in three months - meaning that the loggregator release being cut in three
month’s time won't have support for that endpoint anymore.
You can find some example consumers and the
documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation
of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller
is based on our plan to get rid of the trafficcontroller - and then everything
in loggregator that still references the V1 api and protocol. This will
help us to significantly reduce code complexity, but it will also allow
us to operate much more efficiently. This should result in reduced overhead
for the logging pipeline and ultimately lower infrastructure costs for
the logging components in a CF deployment.
Please let us know if you have any comments
or concerns.
Johannes
PM of #loggregator
|
|
|
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
toggle quoted message
Show quoted text
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
|
Mike Youngstrom <youngm@...>
We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
|
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
|
Mike Youngstrom <youngm@...>
Thanks Johannes,
Though I'm not on the cf-java-client team, you reaching out to them to see if you can help speed anything along couldn't hurt. :)
Thanks, Mike
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
|
Migrating the logsearch piece would definitely help cloud.gov (which otherwise would similarly would be hard-pressed to keep up if it was removed in 3 months).
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
|
Stanislav German-Evtushenko
Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for:
- storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle
- getting all platform metrics for monitoring purposes using kafka-firehose-nozzle
- getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards,
Stanislav
|
|
|
Hi everybody,
thanks for the continuous stream of feedback. We are definitely going to support the firehose-exporter and firehose-to-syslog projects off the firehose to the RLP. The stories in our backlog are coming up soon. Stanislav, do you think that looking at what we did with the firehose-to-syslog project would help you to migrate the kafka-firehose-nozzle?
Also, based on the feedback I have received so far I am open to increasing the deprecation window to 6 months. Would that be acceptable?
Thanks, Johannes
Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for:
- storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle
- getting all platform metrics for monitoring purposes using kafka-firehose-nozzle
- getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards,
Stanislav
|
|
|
Johannes,
A 6 month deprecation window works for Stratos - that should be fine for us to update.
If you can work with the firehose-exporter project to get that updated and off the firehose within the next 2 or 3 months, that would be super helpful and give us time to integrate and test, assuming its a drop-in replacement for the current firehose based version.
Many thanks,
Hi everybody,
thanks for the continuous stream of feedback. We are definitely going to support the firehose-exporter and firehose-to-syslog projects off the firehose to the RLP. The stories in our backlog are coming up soon. Stanislav, do you think that looking at what we did with the firehose-to-syslog project would help you to migrate the kafka-firehose-nozzle?
Also, based on the feedback I have received so far I am open to increasing the deprecation window to 6 months. Would that be acceptable?
Thanks, Johannes Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for:
- storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle
- getting all platform metrics for monitoring purposes using kafka-firehose-nozzle
- getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards,
Stanislav
|
|
|
Hi Neil,
Thanks for the feedback. We actually created PRs for both, firehose-exporter[0] and to firehose-to-syslog[1]. We will continue working with the maintainers to get these pulled in.
Best, Johannes
Johannes,
A 6 month deprecation window works for Stratos - that should be fine for us to update.
If you can work with the firehose-exporter project to get that updated and off the firehose within the next 2 or 3 months, that would be super helpful and give us time to integrate and test, assuming its a drop-in replacement for the current firehose based version.
Many thanks,
Neil
Hi everybody,
thanks for the continuous stream of feedback. We are definitely going to support the firehose-exporter and firehose-to-syslog projects off the firehose to the RLP. The stories in our backlog are coming up soon. Stanislav, do you think that looking at what we did with the firehose-to-syslog project would help you to migrate the kafka-firehose-nozzle?
Also, based on the feedback I have received so far I am open to increasing the deprecation window to 6 months. Would that be acceptable?
Thanks, Johannes Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for:
- storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle
- getting all platform metrics for monitoring purposes using kafka-firehose-nozzle
- getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards,
Stanislav
|
|
|
Hello everybody,
after a lot of deliberation and many good conversations, we came to the conclusion that we will go ahead and remove the firehose endpoint in six months. So, in the first release published after September 14th, there will be no more firehose endpoint. Until then, we will continue to help the community to migrate commonly used firehose integrations over to using the new endpoints located on the Reverse Log Proxy.
Just to reiterate the benefit of this effort, this is an important step to lower overhead and complexity for cloud foundry operator. Also, it will bring more adoption to the new endpoints which should be easier to consume and put less stress on the logging system.
Thanks again for everybody who reached out to us either on this list or on Slack or a separate email thread. Your input was very valuable.
Please let me know if you have any questions or concerns, Johannes
Hi Neil,
Thanks for the feedback. We actually created PRs for both, firehose-exporter[0] and to firehose-to-syslog[1]. We will continue working with the maintainers to get these pulled in.
Best, Johannes
Johannes,
A 6 month deprecation window works for Stratos - that should be fine for us to update.
If you can work with the firehose-exporter project to get that updated and off the firehose within the next 2 or 3 months, that would be super helpful and give us time to integrate and test, assuming its a drop-in replacement for the current firehose based version.
Many thanks,
Neil
Hi everybody,
thanks for the continuous stream of feedback. We are definitely going to support the firehose-exporter and firehose-to-syslog projects off the firehose to the RLP. The stories in our backlog are coming up soon. Stanislav, do you think that looking at what we did with the firehose-to-syslog project would help you to migrate the kafka-firehose-nozzle?
Also, based on the feedback I have received so far I am open to increasing the deprecation window to 6 months. Would that be acceptable?
Thanks, Johannes Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for:
- storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle
- getting all platform metrics for monitoring purposes using kafka-firehose-nozzle
- getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards,
Stanislav
|
|
|
Hello, all,
We wanted to follow up on this deprecation timeline.
We still plan to deprecate the v1 firehose, and our target is now the end of the year (the end of December 2019).
Sincerely,
The Loggregator Team
|
|
|
Steffen Uhlig
Hi Jesse and Loggregator team, we are using the gRPC endpoint[1] on the Dopplers to export logs and metrics into an internal system. Can you confirm that this endpoint is not affected by this change? [1] https://github.com/cloudfoundry/loggregator-release/blob/e923f2fb3875d99ed3f8b5c8777bb9ffac0dd52b/jobs/doppler/spec#L21
Best regards / Mit freundlichen Grüßen
Steffen
--
Steffen Uhlig
Senior Software Engineer, IBM Cloud Development
Tel.: +49 7031 16-2199, e-mail: Steffen.Uhlig@...
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Matthias Hartmann
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
|
|
|
Hi Steffen, confirmed. Dopplers are part of the v2 firehose, so the gRPC endpoint will still be available when v1 is deprecated.
Regards, Mitch
Hi Jesse and Loggregator team,
we are using the gRPC endpoint[1] on the Dopplers to export logs and metrics into an internal system. Can you confirm that this endpoint is not affected by this change?
[1] https://github.com/cloudfoundry/loggregator-release/blob/e923f2fb3875d99ed3f8b5c8777bb9ffac0dd52b/jobs/doppler/spec#L21
Best regards / Mit freundlichen Grüßen
Steffen
--
Steffen Uhlig
Senior Software Engineer, IBM Cloud Development
IBM Deutschland Research & Development GmbH
Vorsitzender des Aufsichtsrats: Matthias Hartmann
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
-- Mitchel Seaman
Associate Director, Product Management @mitchelseaman
|
|
|
