#Spring #java buildpack #Spring Autoreconfiguration
#spring
#java
I am trying to deploy the legacy spring (version 1.1) application in cloud foundry. I want to use spring auto-reconfiguration feature for the app but the feature is not enabled for my app.But if I try changing higher versions of spring, I could see this feature is enabled in the java build pack. Can't we use spring auto-reconfiguration feature for spring older versions like 1.1 and 2.0.?Thanks in Advance
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Hi everybody,
thanks for the continuous stream of feedback. We are definitely going to support the firehose-exporter and firehose-to-syslog projects off the firehose to the RLP. The stories in our backlog are coming up soon. Stanislav, do you think that looking at what we did with the firehose-to-syslog project would help you to migrate the kafka-firehose-nozzle?
Also, based on the feedback I have received so far I am open to increasing the deprecation window to 6 months. Would that be acceptable?
Thanks, Johannes
toggle quoted messageShow quoted text
Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for: - storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle - getting all platform metrics for monitoring purposes using kafka-firehose-nozzle - getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards, Stanislav
|
|
Re: [Proposal] Deprecation /containermetrics endpoint on the loggregator_trafficcontroller
Hi there,
we haven't heard from anyone who is currently consuming the /containermetrics endpoint. We went ahead an removed it. It will be gone as of loggregator release 105.
Please let us know if you have any questions. You can reach us in the Cloud Foundry slack in the #loggregator channel.
Best, Johannes
toggle quoted messageShow quoted text
Hi there,
Therefore, we would like to remove this endpoint to keep our API and code base lean and clean. If there are consumers of this endpoint besides the Cloud Controller, please let us know.
Thanks, Johannes
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Stanislav German-Evtushenko
Hi Johannes,
We are heavy users of loggregator and we have the same concerns as others in the thread.
We use loggregator for: - storing all application logs for 6 month (regulations requirement) using kafka-firehose-nozzle - getting all platform metrics for monitoring purposes using kafka-firehose-nozzle - getting all applications metrics to provide monitoring dashboard using kafka-firehose-nozzle and cf-metrics-refinery (see https://cloudfoundry.slack.com/messages/C6WFZ5K0F/p1531268612000101)
Another concern is autoscaler (https://github.com/cloudfoundry-incubator/app-autoscaler-release). It is relying on loggregator and I haven't seen any updates on moving out of it yet.
Best regards, Stanislav
|
|
Re: Retrieving events for a custom UAA Installation
#cf
|
|
Feedback needed on UAA Microsoft SQL server Usage
TL;DR
If you are using Microsoft SQL server (MS-SQL) as a database backend for UAA, please respond to this survey with your usage details. It will take less than a minute.
Hi UAA Open Source Contributors,
We are in the process of adding some improvements around UAA’s TLS connectivity to databases. More details can be found here. In this regard, we would like to understand the usage patterns around UAA database types.
At this time, we are not aware of any MS-SQL usage with UAA. We are aware of deployments mainly using either MySQL or PostGres.
Action Needed:
If you are using MS-SQL as a UAA backend, we would like to understand more about your current usage and deployment. Please respond with the following details in this survey.
The information above will help us in making an informed decision around either continuing to support MS-SQL or deprecating its support.
Thank you for continued support on making this product better!
Thanks, UAA Team 02/22/2019
|
|
Retrieving events for a custom UAA Installation
#cf
We have deployed our own installation of UAA in cloud foundry which is used as an OAuth 2 Server for applications. As a part of the security feature, we want to review the events on a daily basis. However, there does not seem to be any apis to retrieve the events for a date range. What is the best way to retrieve these events ? The events and logs are also streamed to an Elastic Search instance with a Kibana front end. I can search in Kibana with a keyword "Audit" for the UAA application and the records will show up. The problem with this is that the search will show up entries with Audit anywhere in the message and that may not be an UAA Audit Event. Also, is there an easy way to pull this using an API ?
Please advise !
|
|
CF CLI v6.43.0 Released Today
Thomas Viehman <tviehman@...>
Hey everyone, The CF CLI team released cf CLI v6.43.0 today; please see release notes for full details. Highlights IncludeMulti-Service Registration epicCF now allows multiple service brokers to offer services with the same name and or to have the same catalogs. (However, brokers themselves must still be given a unique name.) Note: Multi-service registration is only supported on CC API version 2.125.0 or greater. Users can specify which broker to use with a new -b flag, which is available on the following commands: cf create-service cf enable-service-access cf disable-service-access cf purge-service-offering cf service - now display broker namescf marketplace now display broker names
Important Note: If you have two service instances with the same name, the commands above will now require the -b flag to disambiguate which service instance and broker you want to operate on. For more information: For more information regarding this feature and other services-related work in this release, reach out to cf-services-api@... or #sapi on Cloud Foundry Slack. Enhancementscf curl supports a new --fail flag (primarily for scripting purposes) which returns exit code 22 for server errors story- Improves
cf delete-orphaned-routes such that it uses a different endpoint, reducing the chance of a race condition when two users are simultaneously deleting orphaned routes and associating routes with applications story - we've improved the speed of cf services - it now hits a single endpoint instead of making individual API calls
Minimum Version CleanupOur minimum version policy changed in January 2019 to support CC API 2.100/3.35. This release removes code which support CC API below those versions. story Bug FixesSecurity- Fixes issue with running
cf login in verbose mode whereby passwords which contains regex were not completely redacted - Fixes issue whilst running commands in verbose mode refresh tokens were not completely redacted
32-bit systems- Fixes a bug for users on 32-bit systems where the CLI would fail to unmarshall responses from the server because the response contained integer values that, when unmarshalled, would overflow 32-bit integers. For example, now users on 32-bit systems, can now set their memory usage larger than 2GB for
cf push . See the list of commands below which we've applied the fix for. story story
CC Version | Resource | Field | Affected Commands | |
---|
v2 | App | HealthCheckTimeout | push | | v2 | App Instance Status | Disk | create-app-manifest | | v2 | App Instance Status | DiskQuota | create-app-manifest | | v2 | App Instance Status | Memory | create-app-manifest | | v2 | App Instance Status | Memory | create-app-manifest | | v3 | Process | Health Check Invocation Timeout | v3-set-health-check , v3-get-health-check | | v3 | Process | Index | | | v3 | Process Instance | Uptime | push , restage , restart , start , app | | v3 | Task | Sequence ID | tasks , run-task | | v3 | Job | Code | v3-delete , v3-apply-manifest | |
Other Bug Fixes- Updates help text for
cf curl story - Now refresh tokens work properly whilst using
cf curl with V3 CC API endpoints story - Fixes performance degradation for
cf services story cf delete-service requires that you are targeting a space storycf enable-service access for a service in an org will succeed if you have already enabled access for that service in that org story
Plugin Additions/UpdatesContributors: Thomas Viehman, Jennifer Spinney, Will Murphy, Magesh Kumar Murali, Alex Shan, Brendan Smith, Abby Chau, Aarti Kriplani, Alex Blease, Georgi Lozev, Henry Stanley, Laurel Gray, Niki Maslarski, Oleksii Fedorov, William Martin, Adam Eijdenberg (for the plugin page anchor links pull request), David Grizzanti (for the delete orphaned routes pr) Thank you to all our Community contributors, we appreciate the pull requests! Note: The minimum version of the CC API this CF CLI release is compatible with is CC API v2.100.0 (3.35). See our minimum supported version policy for more information.
Thanks, CF CLI Team
|
|
FINAL REMINDER: CF CAB call for February is Wednesday February 20th @ 8a PST
fyi...
Tomorrow 8a Pacific. Zoom soon. Best,
------ dr.max ibm ☁ silicon valley, ca maximilien.org
toggle quoted messageShow quoted text
----- Original message ----- From: Michael Maximilien/Almaden/IBM To: cf-dev@... Cc: Subject: CF CAB call for February is Wednesday February 20th @ 8a PST Date: Thu, Feb 7, 2019 12:55 AM
Hi, all,
First thing is that the CAB call survey will close this Monday 2/11. Link again here, two minutes of time max:
Second, reminder that the CAB call for February is Wednesday February 20th @ 8a PST (in two weeks).
We will have our regular PMCs highlights and two talks:
1. Summary of CAB 2019 survey results by me 2. Silk for CFCR prototype project by Konstantin Kiess from Stark and Wayne [1]
All other info in agenda here [0].
Zoom soon. Best,
dr.max ibm ☁ silicon valley, ca
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Migrating the logsearch piece would definitely help cloud.gov (which otherwise would similarly would be hard-pressed to keep up if it was removed in 3 months).
toggle quoted messageShow quoted text
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
We just cut routing-release 0.186.0 with a few bug fixes and a implementation for the single B3 header for distributed tracing. Details here.
CF Routing Team
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Thanks Johannes,
Though I'm not on the cf-java-client team, you reaching out to them to see if you can help speed anything along couldn't hurt. :)
Thanks, Mike
toggle quoted messageShow quoted text
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Thank you all for the feedback. This is very helpful. I understand your concern about the short timeframe. I am very open to extending the timeframe to allow everybody to move of the firehose in a non-stressful way, although I hope that we don't have to extend it to 12 months.
What help would most to migrate away from the firehose faster? So far in this thread, I saw a few open source firehose integrations mentioned:
1. firehose-exporter (for prometheus) 2. firehose-to-syslog (for logsearch) 3. cf-java-client
If we (the loggregator-team) helped migrating these integrations to the log-stream endpoint (from the RLP), would that help? If not, are there other things we could do to improve the migration process?
Thanks again for the feedback. We really appreciate it and are listening.
Johannes
toggle quoted messageShow quoted text
On Thu, Feb 14, 2019 at 12:19 PM Mike Youngstrom < youngm@...> wrote: We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
Re: [Proposal] Deprecation of the firehose endpoint
We have developed some custom monitoring using the firehose and Splunk based on the cf-java-client firehose support. The jmxconsumer has some known memory leaks and would prefer to wait for official cf-java-client support ( https://github.com/cloudfoundry/cf-java-client/issues/904). The sooner cf-java-client has 2.x support the sooner we'll be off. As it stands 3 months is a little quick for us as well for that reason.
Thanks, Mike
toggle quoted messageShow quoted text
On Thu, Feb 14, 2019 at 11:58 AM Jon Price < jon.price@...> wrote:
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Same concerns for us, we too run the open source CF deployment along with the prometheus-boshrelease and logsearch-for-cloudfoundry release.
Jon Price
Intel Corp.
toggle quoted messageShow quoted text
From: cf-dev@... <cf-dev@...>
On Behalf Of via Lists.Cloudfoundry.Org
Sent: Thursday, February 14, 2019 9:10 AM
To: cf-dev@...
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
On Thu, 14 Feb 2019 at 15:22, Simon D Moser <smoser@...> wrote:
Johannes,
just echoing what Neil is saying below: We also feel that three months is probably too short of a runway for this. Let's discuss the details, but 3 months is likely not good enough for us, too.
From: "Neil MacDougall" <neil.macdougall@...>
To: cf-dev@...
Date: 12/02/2019 18:05
Subject: Re: [cf-dev] [Proposal] Deprecation of the firehose endpoint
Sent by: cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease) to retrieve metrics from
the firehose and store those in Prometheus for display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes for the removal of the firehose and I think 3 months is too aggressive a time frame for us to be able to complete those - this is not something we’ve factored into our planning,
so I think we’d need a 12 month window on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry distributions, which we won’t be able to guarantee will be updated in your proposed three month window, so we will most likely have to make updates to allow Stratos to
work with systems that use the current firehose and those that use the newer APIs.
I’m happy to discuss this further with you directly if that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...> wrote:
Hi there,
Following the last thread about the deprecation of the /containermetrics endpoint, the Loggregator team would like to continue on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations, so we will provide reference implementations and documents for migrating away from the firehose to the newer loggregator endpoints (namely, the RLP Gateway and LogCache). Depending on
the feedback we receive to this email, we would like to proceed with the removal of the firehose endpoint in three months - meaning that the loggregator release being cut in three month’s time won't have support for that endpoint anymore.
You can find some example consumers and the documentation of the RLP here:
Go: https://github.com/cloudfoundry/log-stream-cli
Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go
Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer
Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation of the Log Cache:
Go: https://github.com/cloudfoundry/log-cache-cli
Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client
Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller is based on our plan to get rid of the trafficcontroller - and then everything in loggregator that still references the V1 api and protocol.
This will help us to significantly reduce code complexity, but it will also allow us to operate much more efficiently. This should result in reduced overhead for the logging pipeline and ultimately lower infrastructure costs for the logging components in a
CF deployment.
Please let us know if you have any comments or concerns.
Johannes
PM of #loggregator
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Richard Towers <richard.towers@...>
Hi Johannes,
I work on GOV.UK PaaS - we run an open source CF deployment. Our deployment also uses the firehose exporter that Neil mentions (via prometheus-boshrelease).
We probably don't have enough people to help much with migrating these things, but they form a core part of our platform monitoring, so it would be a problem for us if they stopped working.
I agree with Neil and Simon that three months may not be enough time for the community to adopt the new things.
Thanks, Richard
toggle quoted messageShow quoted text
On Thu, 14 Feb 2019 at 15:22, Simon D Moser < smoser@...> wrote: Johannes,
just echoing what Neil is saying below:
We also feel that three months is probably too short of a runway for this.
Let's discuss the details, but 3 months is likely not good enough for us,
too.
From:
"Neil MacDougall"
<neil.macdougall@...> To:
cf-dev@... Date:
12/02/2019 18:05 Subject:
Re: [cf-dev]
[Proposal] Deprecation of the firehose endpoint Sent by:
cf-dev@...
Johannes,
Thank your for the notification.
We use the firehose in Stratos to stream logs for applications
and the firehose itself for users/admins to view in the UI.
We also use the CF Firehose exporter (https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease)
to retrieve metrics from the firehose and store those in Prometheus for
display within Stratos. Do you know what the plan is for these to be updated?
It is a large amount of work for us to make the changes
for the removal of the firehose and I think 3 months is too aggressive
a time frame for us to be able to complete those - this is not something
we’ve factored into our planning, so I think we’d need a 12 month window
on the firehose removal to be safe.
We have users using Stratos on a variety of Cloud Foundry
distributions, which we won’t be able to guarantee will be updated in
your proposed three month window, so we will most likely have to make updates
to allow Stratos to work with systems that use the current firehose and
those that use the newer APIs.
I’m happy to discuss this further with you directly if
that would help.
Regards,
Neil
On 11 Feb 2019, at 23:54, Johannes Tuchscherer <jtuchscherer@...>
wrote:
Hi there,
Following the last thread about the deprecation
of the /containermetrics endpoint, the Loggregator team would like to continue
on its path of deprecations. Next on the chopping block is the /firehose
endpoint. We understand that this endpoint is used by a few integrations,
so we will provide reference implementations and documents for migrating
away from the firehose to the newer loggregator endpoints (namely, the
RLP Gateway and LogCache). Depending on the feedback we receive to this
email, we would like to proceed with the removal of the firehose endpoint
in three months - meaning that the loggregator release being cut in three
month’s time won't have support for that endpoint anymore.
You can find some example consumers and the
documentation of the RLP here: Go: https://github.com/cloudfoundry/log-stream-cli Go Client: https://github.com/cloudfoundry/go-loggregator/blob/master/rlp_gateway_client.go Java: https://github.com/cloudfoundry-community/jmx-consumer-release/tree/develop/src/jmxconsumer Docs: https://github.com/cloudfoundry/loggregator/blob/master/docs/rlp_gateway.md
Here are some example consumers and the documentation
of the Log Cache: Go: https://github.com/cloudfoundry/log-cache-cli Go Client: https://github.com/cloudfoundry/log-cache/tree/master/pkg/client Docs: https://github.com/cloudfoundry/log-cache/blob/master/README.md
Our desire to remove endpoints on the trafficcontroller
is based on our plan to get rid of the trafficcontroller - and then everything
in loggregator that still references the V1 api and protocol. This will
help us to significantly reduce code complexity, but it will also allow
us to operate much more efficiently. This should result in reduced overhead
for the logging pipeline and ultimately lower infrastructure costs for
the logging components in a CF deployment.
Please let us know if you have any comments
or concerns.
Johannes PM of #loggregator
|
|
Re: [Proposal] Deprecation of the firehose endpoint
Johannes, just echoing what Neil is saying below:
We also feel that three months is probably too short of a runway for this.
Let's discuss the details, but 3 months is likely not good enough for us,
too. From:
"Neil MacDougall"
<neil.macdougall@...>To:
cf-dev@...Date:
12/02/2019 18:05Subject:
Re: [cf-dev]
[Proposal] Deprecation of the firehose endpointSent by:
cf-dev@...
Johannes, Thank your for the notification. We use the firehose in Stratos to stream logs for applications
and the firehose itself for users/admins to view in the UI. We also use the CF Firehose exporter ( https://github.com/bosh-prometheus/firehose_exporter)
as does https://github.com/bosh-prometheus/prometheus-boshrelease)
to retrieve metrics from the firehose and store those in Prometheus for
display within Stratos. Do you know what the plan is for these to be updated? It is a large amount of work for us to make the changes
for the removal of the firehose and I think 3 months is too aggressive
a time frame for us to be able to complete those - this is not something
we’ve factored into our planning, so I think we’d need a 12 month window
on the firehose removal to be safe. We have users using Stratos on a variety of Cloud Foundry
distributions, which we won’t be able to guarantee will be updated in
your proposed three month window, so we will most likely have to make updates
to allow Stratos to work with systems that use the current firehose and
those that use the newer APIs. I’m happy to discuss this further with you directly if
that would help. Regards, Neil
toggle quoted messageShow quoted text
|
|
Announcement: CC API v2 Deprecation Plan
Greetings cf-dev!
The V3 Acceleration Team has been hard at work expanding the CC v3 API to cover resources that were previously only available on the v2 API. We are now ready to announce our CC V2 API Deprecation Plan.
As per the plan, we will announce the start of the v2 deprecation window at a future date. We recognize that transitioning from the v2 API will be an involved process for some clients, so we recommend starting now for resources that are available on the v3 API.
Please feel free to comment on the document, respond to this email, or message us in the #v3-acceleration-team channel on the Cloud Foundry Slack if you have any questions.
In the meantime, we are still soliciting feedback on the remaining v3 API proposals and the v3 API in general. We would like to give a shout out to the Stratos team for their in-depth notes about their experiences using v3.
Thanks! V3 Acceleration Team
|
|
Re: CF Application Runtime PMC: CF Infrastructure Project Lead Call for Nominations
Thanks Evan and your team over the years!
toggle quoted messageShow quoted text
From: cf-dev@... on behalf of Steve Taylor <staylor@...>
Sent: Thursday, February 14, 2019 6:38 am
To: cf-dev@...
Subject: Re: [cf-dev] CF Application Runtime PMC: CF Infrastructure Project Lead Call for Nominations
Thank you, Evan, for all of your work on Infrastructure!
--Steve
On Wed, Feb 13, 2019 at 12:07 PM Eric Malm < emalm@...> wrote:
Hi, everyone,
Pivotal is nominating Preethi Varambally for the CF Infrastructure Project Lead in the Application Runtime PMC.
Preethi has been working as a product manager on the CF Infrastructure team for the past two months, and prior to that had been the Project Lead for the Container Networking team since July 2018.
Preethi previously worked as a technical product owner and business analyst at a GIS company on a customer-facing web application. Here, along with managing the product, she also worked on designing application layer data flow architecture for efficiently
getting data from various source systems. Prior to this, she worked as an engineer, building data tracking and reporting tools using MVC framework and gradually moved to a product owner role managing a team of onshore and offshore engineers.
Preethi holds a Masters degree in Computer Science from University of Texas, Dallas.
Please send any other nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.
Thanks,
Eric Malm
On Wed, Feb 13, 2019 at 12:04 PM Eric Malm < emalm@...> wrote:
Hi, everyone,
Evan Farrar, the Project Lead for the CF Infrastructure team within the Application Runtime PMC, is stepping down from the project. We thank him for his years of service as the CF Infrastructure Project Lead.
The CF Infrastructure team, based in Santa Monica, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM
PST on Friday, February 22, 2018.
Thanks,
Eric Malm, CF Application Runtime PMC Lead
|
|
Re: CF Application Runtime PMC: CF Infrastructure Project Lead Call for Nominations
Thank you, Evan, for all of your work on Infrastructure!
--Steve
toggle quoted messageShow quoted text
On Wed, Feb 13, 2019 at 12:07 PM Eric Malm < emalm@...> wrote: Hi, everyone,
Pivotal is nominating Preethi Varambally for the CF Infrastructure Project Lead in the Application Runtime PMC.
Preethi has been working as a product manager on the CF Infrastructure team for the past two months, and prior to that had been the Project Lead for the Container Networking team since July 2018.
Preethi previously worked as a technical product owner and business analyst at a GIS company on a customer-facing web application. Here, along with managing the product, she also worked on designing application layer data flow architecture for efficiently getting data from various source systems. Prior to this, she worked as an engineer, building data tracking and reporting tools using MVC framework and gradually moved to a product owner role managing a team of onshore and offshore engineers.
Preethi holds a Masters degree in Computer Science from University of Texas, Dallas.
Please send any other nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.
Thanks, Eric Malm
On Wed, Feb 13, 2019 at 12:04 PM Eric Malm < emalm@...> wrote: Hi, everyone,
Evan Farrar, the Project Lead for the CF Infrastructure team within the Application Runtime PMC, is stepping down from the project. We thank him for his years of service as the CF Infrastructure Project Lead.
The CF Infrastructure team, based in Santa Monica, now has an opening for its project lead. Project leads must be nominated by a Cloud Foundry Foundation member. Please send nominations directly to me or in reply to this message no later than 11:59 PM PST on Friday, February 22, 2018.
Thanks, Eric Malm, CF Application Runtime PMC Lead
|
|