1. cf-dev@lists.cloudfoundry.org
  2. Messages
   Date   
Updates from the world of .NET! 🎉

Ashley Hathaway
 

Why hello there!


In an effort to solicit more feedback and understand actual user problems to help define our roadmap & backlog I wanted to reach out and say howdy (howdy!) and share a bit about the exciting times in Windows CF land.


The Windows runtime team here at Pivotal in NYC have been super productive streamlining and improving a lot of our processes as well as adding new features to Cloud Foundry’s Windows runtime. Here are the highlights:


  • 😎The Windows2016 stack/runtime has been hanging out in experimental for a while. It’s soon (like within this month) to be supported in OSS. Kind of sort of a huge deal. This means Windows containers for realsies.

  • 🔐Another pretty epic update is cf ssh capabilities. With the cf CLI you can now securely log into Windows remote host VMs running CF app instances.

  • 🐈One of the newer improvements has been around acceptance suites. The standard version is CATS (CF Acceptance Test Suite) but we’ve always had a second version for Windows (aka WATS). But no more! We aim to have one test suite to rule them all. The team has worked to identify any overlapping processes and are just one PR away from integrating the suites! Yay simplicity! Feel free to check out the epic here.

  • 🔈We’re working on Volume Services with the Diego Persistence team and hope to ship sometime soon.

  • 🚙We’ve started work to bring Envoy and route integrity to Windows.

  • 🎁The ContainerD runtime initiative with Windows will begin in April w/ the Garden team. Hooray simplicity.

  • 🎊Finally, the Cloud Foundry Summit! Next month! Are you going? Give us a shout! And be sure to check out our sessions on Remote Debugging with two of our engineers and a short demo with our two of our PM’s.


And if you haven’t already head over to the CF Docs to get started w/ the Windows CLI.


So dear developer, what are we missing!? Are docs the best they can be? What can we improve? Feel free to reach out with any feedback or questions or ping us on the #garden-windows CF Slack!


Til next time,

Ashley


@Ash_Hathaway
Re: Rotating cf-deployment certificates

Iryna Shustava
 

Hey Mike,

Check out this doc regarding CA rotation with CredHub: https://github.com/pivotal-cf/credhub-release/blob/master/docs/ca-rotation.md.

Cheers,
Iryna


On Tue, Mar 6, 2018 at 2:44 PM, Aaron Huber <aaron.m.huber@...> wrote:
This one-liner will grab all the certs out of the vars files used by the bosh-cli and print out the expiration dates which is useful for a quick check:

openssl crl2pkcs7 -nocrl -certfile <(sed -n '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' *vars.yml | sed -e 's/^[ \t]*//') | openssl pkcs7 -print_certs -text -noout | sed -e 's/^[ \t]*//' | grep -E "Issuer:|Subject:|Not\ After\ :" | awk '{ if ((NR % 3) == 1) printf("\n*******\n\n"); print; }'

Aaron
Proposal for modest readability improvements to Diego component logs

Eric Malm <emalm@...>
 

Hi, all,

The Diego team is planning to make some modest improvements to the readability of the Diego component logs. Primarily, we'd like to make each log-line's timestamp ISO 8601/RFC 3339 compliant (that is, of the form "2018-03-06T12:34:56.789012345Z") and its log level a human-readable string. Details are in the proposal document at https://docs.google.com/document/d/1D3GK2IUGQz_3fCuuLPNWz7Yb8eiKKZ9k-78LwNHWtuU/edit, on which we would certainly appreciate your feedback.

Thanks,
Eric Malm, CF Diego PM
Re: Rotating cf-deployment certificates

Aaron Huber
 

This one-liner will grab all the certs out of the vars files used by the bosh-cli and print out the expiration dates which is useful for a quick check:

openssl crl2pkcs7 -nocrl -certfile <(sed -n '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' *vars.yml | sed -e 's/^[ \t]*//') | openssl pkcs7 -print_certs -text -noout | sed -e 's/^[ \t]*//' | grep -E "Issuer:|Subject:|Not\ After\ :" | awk '{ if ((NR % 3) == 1) printf("\n*******\n\n"); print; }'

Aaron
Re: Rotating cf-deployment certificates

Mike Youngstrom
 

So, reading through the document you provided Iryna and re-reading David's rotation steps everything now makes sense when using bosh-cli generated certs.

Are there steps to do the same with credhub managed certificates?

Thanks,
Mike

On Tue, Mar 6, 2018 at 1:22 PM, Mike Youngstrom <youngm@...> wrote:
Thanks for the clarification Iryna!  I'll do some more studying and respond if I have further questions.

Mike

On Tue, Mar 6, 2018 at 12:07 PM, Iryna Shustava <ishustava@...> wrote:
Hey Mike,

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

We expect no app routability or log availability downtime during the 3 step CA/cert rotation. That is because during Step 1 we make all components trust both CAs - the old one and the new one. During step 2, when we roll out new leaf certificates, all components should trust both CAs, so the certificate switch will happen without downtime. You will, however, see some cf push downtime, as David mentioned.

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

The default expiration is the same for CredHub and BOSH CLI. BOSH CLI does not allow you to change certificate expiration period, but CredHub does. You can do so by adding the duration property measured in days to your certificate or CA variable in the manifest.

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

If you're interested, this doc describes reasons behind cf push downtime during CA rotation.

Thanks!
Iryna, CF Release Integration Team


On Tue, Mar 6, 2018 at 9:21 AM, Mike Youngstrom <youngm@...> wrote:
Thanks for the heads up David.  I have questions about the rotation process.

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

Thanks,
Mike

On Fri, Mar 2, 2018 at 11:32 AM, David Sabeti <dsabeti@...> wrote:
Hey cf-dev,

The Release Integration team has had a few reports from other CF engineering teams that their long-running environments have had their internal TLS certificates expire. Since certificates generated by the BOSH CLI get a one-year expiration date, and it's been about a year since early adopters started using cf-deployment, we suspect that some older environments in the CF community are fast approaching this issue as well. We hope to provide enough of a warning that folks in the community can address this.

Check your certificate expiration dates
This is pretty simple to do. You can copy a certificate -- service_cf_internal_ca is a good one to try -- and paste it into the form on this site: https://www.sslshopper.com/certificate-decoder.html. You'll find the expiration date in the "Valid To" section. If your certificates going to expire soon, continue to the process below.

How to rotate certificates
This is not an easy process, but it's doable. I'll warn you right now that, during the transition, your CF will experience `cf push` downtime, but apps should remain available. Also, if you're deploying with the windows-cell.yml or secure-service-credentials.yml ops-files, the process will be a bit more complicated, so please reach out to the RelInt team for help.
  1. Deploy with concatenated CA certificates
    1. Generate new certs by running
    2. bosh int cf-deployment.yml [-o ... ] --vars-store new-vars.yml -v system_domain=$SYSTEM_DOMAIN
    3. For each new CA cert, concatenate the new CA certificate to both the `ca` and `certificate` field.
    4. Deploy
  2. Deploy with new leaf certificates
    1. For each leaf certificate in your vars-store, replace with the corresponding certificate from new-vars.yml. These leaf certificates are signed by the new CA's.
    2. Deploy. When the api instances roll, users will no longer be able to push apps, until you remove the old CA certificates.
  3. Deploy without the old CA certificates.
    1. For each CA certificate in your vars-store, remove the first certificate in the `ca` and `certificate` fields. The result should be that only the new CA certificates created in step 1.1 should be included in your vars-store.
The RelInt team has also worked through a process for rotating certificates that have already expired. If you have any questions or concerns, jump into the #release-integration channel in the Cloud Foundry slack and feel free to get a hold of the team there.

Thanks!
CF Release Integration
Re: Rotating cf-deployment certificates

Mike Youngstrom
 

Thanks for the clarification Iryna!  I'll do some more studying and respond if I have further questions.

Mike

On Tue, Mar 6, 2018 at 12:07 PM, Iryna Shustava <ishustava@...> wrote:
Hey Mike,

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

We expect no app routability or log availability downtime during the 3 step CA/cert rotation. That is because during Step 1 we make all components trust both CAs - the old one and the new one. During step 2, when we roll out new leaf certificates, all components should trust both CAs, so the certificate switch will happen without downtime. You will, however, see some cf push downtime, as David mentioned.

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

The default expiration is the same for CredHub and BOSH CLI. BOSH CLI does not allow you to change certificate expiration period, but CredHub does. You can do so by adding the duration property measured in days to your certificate or CA variable in the manifest.

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

If you're interested, this doc describes reasons behind cf push downtime during CA rotation.

Thanks!
Iryna, CF Release Integration Team


On Tue, Mar 6, 2018 at 9:21 AM, Mike Youngstrom <youngm@...> wrote:
Thanks for the heads up David.  I have questions about the rotation process.

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

Thanks,
Mike

On Fri, Mar 2, 2018 at 11:32 AM, David Sabeti <dsabeti@...> wrote:
Hey cf-dev,

The Release Integration team has had a few reports from other CF engineering teams that their long-running environments have had their internal TLS certificates expire. Since certificates generated by the BOSH CLI get a one-year expiration date, and it's been about a year since early adopters started using cf-deployment, we suspect that some older environments in the CF community are fast approaching this issue as well. We hope to provide enough of a warning that folks in the community can address this.

Check your certificate expiration dates
This is pretty simple to do. You can copy a certificate -- service_cf_internal_ca is a good one to try -- and paste it into the form on this site: https://www.sslshopper.com/certificate-decoder.html. You'll find the expiration date in the "Valid To" section. If your certificates going to expire soon, continue to the process below.

How to rotate certificates
This is not an easy process, but it's doable. I'll warn you right now that, during the transition, your CF will experience `cf push` downtime, but apps should remain available. Also, if you're deploying with the windows-cell.yml or secure-service-credentials.yml ops-files, the process will be a bit more complicated, so please reach out to the RelInt team for help.
  1. Deploy with concatenated CA certificates
    1. Generate new certs by running
    2. bosh int cf-deployment.yml [-o ... ] --vars-store new-vars.yml -v system_domain=$SYSTEM_DOMAIN
    3. For each new CA cert, concatenate the new CA certificate to both the `ca` and `certificate` field.
    4. Deploy
  2. Deploy with new leaf certificates
    1. For each leaf certificate in your vars-store, replace with the corresponding certificate from new-vars.yml. These leaf certificates are signed by the new CA's.
    2. Deploy. When the api instances roll, users will no longer be able to push apps, until you remove the old CA certificates.
  3. Deploy without the old CA certificates.
    1. For each CA certificate in your vars-store, remove the first certificate in the `ca` and `certificate` fields. The result should be that only the new CA certificates created in step 1.1 should be included in your vars-store.
The RelInt team has also worked through a process for rotating certificates that have already expired. If you have any questions or concerns, jump into the #release-integration channel in the Cloud Foundry slack and feel free to get a hold of the team there.

Thanks!
CF Release Integration
Re: Rotating cf-deployment certificates

Iryna Shustava
 

Hey Mike,

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

We expect no app routability or log availability downtime during the 3 step CA/cert rotation. That is because during Step 1 we make all components trust both CAs - the old one and the new one. During step 2, when we roll out new leaf certificates, all components should trust both CAs, so the certificate switch will happen without downtime. You will, however, see some cf push downtime, as David mentioned.

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

The default expiration is the same for CredHub and BOSH CLI. BOSH CLI does not allow you to change certificate expiration period, but CredHub does. You can do so by adding the duration property measured in days to your certificate or CA variable in the manifest.

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

If you're interested, this doc describes reasons behind cf push downtime during CA rotation.

Thanks!
Iryna, CF Release Integration Team


On Tue, Mar 6, 2018 at 9:21 AM, Mike Youngstrom <youngm@...> wrote:
Thanks for the heads up David.  I have questions about the rotation process.

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

Thanks,
Mike

On Fri, Mar 2, 2018 at 11:32 AM, David Sabeti <dsabeti@...> wrote:
Hey cf-dev,

The Release Integration team has had a few reports from other CF engineering teams that their long-running environments have had their internal TLS certificates expire. Since certificates generated by the BOSH CLI get a one-year expiration date, and it's been about a year since early adopters started using cf-deployment, we suspect that some older environments in the CF community are fast approaching this issue as well. We hope to provide enough of a warning that folks in the community can address this.

Check your certificate expiration dates
This is pretty simple to do. You can copy a certificate -- service_cf_internal_ca is a good one to try -- and paste it into the form on this site: https://www.sslshopper.com/certificate-decoder.html. You'll find the expiration date in the "Valid To" section. If your certificates going to expire soon, continue to the process below.

How to rotate certificates
This is not an easy process, but it's doable. I'll warn you right now that, during the transition, your CF will experience `cf push` downtime, but apps should remain available. Also, if you're deploying with the windows-cell.yml or secure-service-credentials.yml ops-files, the process will be a bit more complicated, so please reach out to the RelInt team for help.
  1. Deploy with concatenated CA certificates
    1. Generate new certs by running
    2. bosh int cf-deployment.yml [-o ... ] --vars-store new-vars.yml -v system_domain=$SYSTEM_DOMAIN
    3. For each new CA cert, concatenate the new CA certificate to both the `ca` and `certificate` field.
    4. Deploy
  2. Deploy with new leaf certificates
    1. For each leaf certificate in your vars-store, replace with the corresponding certificate from new-vars.yml. These leaf certificates are signed by the new CA's.
    2. Deploy. When the api instances roll, users will no longer be able to push apps, until you remove the old CA certificates.
  3. Deploy without the old CA certificates.
    1. For each CA certificate in your vars-store, remove the first certificate in the `ca` and `certificate` fields. The result should be that only the new CA certificates created in step 1.1 should be included in your vars-store.
The RelInt team has also worked through a process for rotating certificates that have already expired. If you have any questions or concerns, jump into the #release-integration channel in the Cloud Foundry slack and feel free to get a hold of the team there.

Thanks!
CF Release Integration
Re: Rotating cf-deployment certificates

Mike Youngstrom
 

Thanks for the heads up David.  I have questions about the rotation process.

Although all applications may remain up while re-deploying I imagine things like loggregator will stop working mid deploy when doppler and metron certs no longer match.  Perhaps reps will be unable to properly drain when their certs don't match?  Does that sound correct?

Is the expiration default the same for certificates created by credhub?  Are you aware of any way to increase the default expiration date for credhub or bosh-cli?

Long term are core teams working towards zero downtime cert rotation capabilities?  Or do you foresee the need to rotate with some service impact an issue long term?

Thanks,
Mike

On Fri, Mar 2, 2018 at 11:32 AM, David Sabeti <dsabeti@...> wrote:
Hey cf-dev,

The Release Integration team has had a few reports from other CF engineering teams that their long-running environments have had their internal TLS certificates expire. Since certificates generated by the BOSH CLI get a one-year expiration date, and it's been about a year since early adopters started using cf-deployment, we suspect that some older environments in the CF community are fast approaching this issue as well. We hope to provide enough of a warning that folks in the community can address this.

Check your certificate expiration dates
This is pretty simple to do. You can copy a certificate -- service_cf_internal_ca is a good one to try -- and paste it into the form on this site: https://www.sslshopper.com/certificate-decoder.html. You'll find the expiration date in the "Valid To" section. If your certificates going to expire soon, continue to the process below.

How to rotate certificates
This is not an easy process, but it's doable. I'll warn you right now that, during the transition, your CF will experience `cf push` downtime, but apps should remain available. Also, if you're deploying with the windows-cell.yml or secure-service-credentials.yml ops-files, the process will be a bit more complicated, so please reach out to the RelInt team for help.
  1. Deploy with concatenated CA certificates
    1. Generate new certs by running
    2. bosh int cf-deployment.yml [-o ... ] --vars-store new-vars.yml -v system_domain=$SYSTEM_DOMAIN
    3. For each new CA cert, concatenate the new CA certificate to both the `ca` and `certificate` field.
    4. Deploy
  2. Deploy with new leaf certificates
    1. For each leaf certificate in your vars-store, replace with the corresponding certificate from new-vars.yml. These leaf certificates are signed by the new CA's.
    2. Deploy. When the api instances roll, users will no longer be able to push apps, until you remove the old CA certificates.
  3. Deploy without the old CA certificates.
    1. For each CA certificate in your vars-store, remove the first certificate in the `ca` and `certificate` fields. The result should be that only the new CA certificates created in step 1.1 should be included in your vars-store.
The RelInt team has also worked through a process for rotating certificates that have already expired. If you have any questions or concerns, jump into the #release-integration channel in the Cloud Foundry slack and feel free to get a hold of the team there.

Thanks!
CF Release Integration
Re: Rotating cf-deployment certificates

Benjamin Gandon
 

Hello Carlo,

I'm definitely interested by your step that checks if any of the certs are close to the expiration date!
If you can share this on a Github somewhere it would be perfect!

Cdt,
/Benjamin GANDON (depuis mon iPhone)

Le 6 mars 2018 à 03:05, Carlo Alberto Ferraris <carlo.ferraris@...> a écrit :

Just a couple of random notes about this:
- since we have a lot of certificates in our deployment manifest (not just the CF/diego ones) we actually have a step in our deployment process that automatically checks if any of them is close to the expiration date (or invalid for other reasons) if anybody is interested we can publish it out somewhere
- would be nice to have the cert generation scripts prompt for the desired validity of the certificates (to avoid surprises)

Re: Cache synchronisation

Rozenszajn, Sergio
 

I refer to the first proposal since the second option (re-start the instance) would clear all applications from the cache and not only the one that was deleted from persistency. It will lead to additional calls to persistency to re-build the cache.

 

RE central cache: as you correctly said à it is centralized, it means that:

  1. for each request to the reverse proxy we need an additional hoop to the central cache.
  2. We used NGINX for its reverse proxy and caching capabilities in a distributed architecture, we will lose this
  3. We need an additional runtime dependency to the central cache = potential failure point

 

Imagine this flow:

  1. API returns that 5 instances exist
  2. While calling in a loop from instance 1 to instance 5, 2 new instances are created by CF

Question: will CF create the new 2 instances as index 6,7 (so they won’t be called) or they can be introduced in the middle (e.g.: moving the previous 4,5 to 6,7)?

 

From: <cf-dev@...> on behalf of Carlo Alberto Ferraris <carlo.ferraris@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Tuesday, 6 March 2018 at 3:52
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] Cache synchronisation

 

> Was the current maximum number of instances really implemented?.
No, it wasn't implemented yet. But it's just an improvement to avoid calling the API.

> I see your point however it sounds like something that should be provided centrally.
The normal way to do this is to have a central cache (e.g. a redis/cassandra/memcache) and have your reverse proxy use it. Then when you want to flush it you call any instance and the instance clears the backing data store. This is arguably "safest" approach, but it has a big downside: it is centralized.

> Do you consider this as production API?
I don't know to which of my two proposals you are referring. I think both are fine, since they are both supported. We routinely use X-CF-INSTANCE-INDEX for monitoring/debugging purposes (it's provided for this very reason, and if you squint hard enough "clearing a cache" can actually be considered a debugging operation, since cache invalidation should ideally be handled automatically by the reverse proxy+backend application). And we routinely use the CF API (the one that would be exclusively used for my second proposal for restarting all instances) for controlling every single application.

> I had the impression that it is only used for development purposes.
I have the feeling you're mixing two concepts here. "Used by developers" (i.e. the users of the platform) is not the same as "used for development" (i.e. not for production). By this logic the CF API should only be used for development, not production... and that's clearly not the case. The CF API can be used for production, but only by developers.
Re: Rotating cf-deployment certificates

Carlo Alberto Ferraris
 

Just a couple of random notes about this:
- since we have a lot of certificates in our deployment manifest (not just the CF/diego ones) we actually have a step in our deployment process that automatically checks if any of them is close to the expiration date (or invalid for other reasons) if anybody is interested we can publish it out somewhere
- would be nice to have the cert generation scripts prompt for the desired validity of the certificates (to avoid surprises)
Re: Cache synchronisation

Carlo Alberto Ferraris
 

> Was the current maximum number of instances really implemented?.
No, it wasn't implemented yet. But it's just an improvement to avoid calling the API.

> I see your point however it sounds like something that should be provided centrally.
The normal way to do this is to have a central cache (e.g. a redis/cassandra/memcache) and have your reverse proxy use it. Then when you want to flush it you call any instance and the instance clears the backing data store. This is arguably "safest" approach, but it has a big downside: it is centralized.

> Do you consider this as production API?
I don't know to which of my two proposals you are referring. I think both are fine, since they are both supported. We routinely use X-CF-INSTANCE-INDEX for monitoring/debugging purposes (it's provided for this very reason, and if you squint hard enough "clearing a cache" can actually be considered a debugging operation, since cache invalidation should ideally be handled automatically by the reverse proxy+backend application). And we routinely use the CF API (the one that would be exclusively used for my second proposal for restarting all instances) for controlling every single application.

> I had the impression that it is only used for development purposes.
I have the feeling you're mixing two concepts here. "Used by developers" (i.e. the users of the platform) is not the same as "used for development" (i.e. not for production). By this logic the CF API should only be used for development, not production... and that's clearly not the case. The CF API can be used for production, but only by developers.
Re: Cache synchronisation

Rozenszajn, Sergio
 

Hi Carlo,

 

Was the current maximum number of instances really implemented?.

 

I see your point however it sounds like something that should be provided centrally. Do you consider this as production API?. I had the impression that it is only used for development purposes.

 

From: <cf-dev@...> on behalf of Carlo Alberto Ferraris <carlo.ferraris@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Monday, 5 March 2018 at 3:24
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] Cache synchronisation

 

[Edited Message Follows]

Sergio,
there are multiple ways to do this:
- call the API to get the number of instances and then send the request to each instance, making sure that each instance confirms that the cache was cleared
- call the API to get the number of instances and then restart each instance in turn (this has the nice side effect that you don't need to implement the cache clearing API)

Even assuming that you add/remove instances after calling the API and before you call/restart each instance, this won't cause problems:
- if an instance is added (or an existing instance restarts for other reasons), its cache is guaranteed to be empty
- if an instance is removed, it's not running anymore - so the cache does not even exist

The only downside is having to call the API. I proposed https://github.com/cloudfoundry/gorouter/issues/197 a while ago to avoid the API call in these scenarios.

Re: Cache synchronisation

Carlo Alberto Ferraris
 
Edited

Sergio,
there are multiple ways to do this:
- call the API to get the number of instances and then send the request to each instance, making sure that each instance confirms that the cache was cleared
- call the API to get the number of instances and then restart each instance in turn (this has the nice side effect that you don't need to implement the cache clearing API)

Even assuming that you add/remove instances after calling the API and before you call/restart each instance, this won't cause problems:
- if an instance is added (or an existing instance restarts for other reasons), its cache is guaranteed to be empty
- if an instance is removed, it's not running anymore - so the cache does not even exist

The only downside is having to call the API. I proposed https://github.com/cloudfoundry/gorouter/issues/197 a while ago to avoid the API call in these scenarios.
Re: Proposal for incubation in the Extensions PMC: MS-SQL Service Broker

Michael Maximilien
 

Thanks Zach and team for sharing. Looking forward to the details during next call and super happy to consider a broker for a Windows Service. I believe this would be a first! Hopefully not last.

Cheers all,

Mac


On Sun, Mar 4, 2018 at 4:30 PM Zach Brown <zbrown@...> wrote:
Hi All,

There's a SQL Server service broker currently in the incubator, but it
appears to be abandoned. We'd like to propose replacing it with this
newer broker built by Jared Gordon and Mallika Iyer from Pivotal.

At the same time, a team from Microsoft has agreed to take over
ownership and ongoing maintenance of this new broker.

Feel free to ask any questions you may have in this thread. If
possible we'd like to discuss this proposal at the March PMC meeting.

View the complete proposal here:
https://docs.google.com/document/d/1cUjY2fqdHn8GPjqp4jY-wjGlPIhZMYj4Qv7zeGf_6T8/


--
Zach Brown
zbrown@...



--
dr.max Sent from my iPhone http://maximilien.org
Proposal for incubation in the Extensions PMC: MS-SQL Service Broker

Zach Brown
 

Hi All,

There's a SQL Server service broker currently in the incubator, but it
appears to be abandoned. We'd like to propose replacing it with this
newer broker built by Jared Gordon and Mallika Iyer from Pivotal.

At the same time, a team from Microsoft has agreed to take over
ownership and ongoing maintenance of this new broker.

Feel free to ask any questions you may have in this thread. If
possible we'd like to discuss this proposal at the March PMC meeting.

View the complete proposal here:
https://docs.google.com/document/d/1cUjY2fqdHn8GPjqp4jY-wjGlPIhZMYj4Qv7zeGf_6T8/


--
Zach Brown
zbrown@...
Re: Cache synchronisation

Rozenszajn, Sergio
 

Hi Chris,

 

I know this API. I don’t think it is suitable for productive scenarios since it might break if an instance was added/removed while creating the requests.

I was looking for some central Go Router functionality such as “send to all instances” that makes sure that also newly created instances get called.

 

Thanks, Sergio

 

From: <cf-dev@...> on behalf of Chris Selzo <cselzo@...>
Reply-To: "cf-dev@..." <cf-dev@...>
Date: Saturday, 3 March 2018 at 23:23
To: "cf-dev@..." <cf-dev@...>
Subject: Re: [cf-dev] Cache synchronisation

 

Hi Sergio,

 

 

-Chris
Re: Cache synchronisation

Chris Selzo
 

Hi Sergio,


-Chris
Cache synchronisation

Rozenszajn, Sergio
 

Hi cf-dev,

 

In one of our projects we use an NGINX based application that uses NGINX cache capabilities. We store in cache static file content that belong to a UI application.

From time to time we need to invalidate this cache externally, for example when the application is deleted. This can be supported in NGINX by implementing a “purge” endpoint that deletes the corresponding cache keys. NGINX also supports a “mirroring” mechanism that synchronizes the cache in all nodes. For that the containers internal addresses should be known, which is not the case in CF.

 

Our requirement would be to implement a “purge” rest API that really reaches all application instances. Is there a way to implement such behavior via CF Router?

 

Best Regards

Sergio

 
Rotating cf-deployment certificates

David Sabeti
 

Hey cf-dev,

The Release Integration team has had a few reports from other CF engineering teams that their long-running environments have had their internal TLS certificates expire. Since certificates generated by the BOSH CLI get a one-year expiration date, and it's been about a year since early adopters started using cf-deployment, we suspect that some older environments in the CF community are fast approaching this issue as well. We hope to provide enough of a warning that folks in the community can address this.

Check your certificate expiration dates
This is pretty simple to do. You can copy a certificate -- service_cf_internal_ca is a good one to try -- and paste it into the form on this site: https://www.sslshopper.com/certificate-decoder.html. You'll find the expiration date in the "Valid To" section. If your certificates going to expire soon, continue to the process below.

How to rotate certificates
This is not an easy process, but it's doable. I'll warn you right now that, during the transition, your CF will experience `cf push` downtime, but apps should remain available. Also, if you're deploying with the windows-cell.yml or secure-service-credentials.yml ops-files, the process will be a bit more complicated, so please reach out to the RelInt team for help.
  1. Deploy with concatenated CA certificates
    1. Generate new certs by running
    2. bosh int cf-deployment.yml [-o ... ] --vars-store new-vars.yml -v system_domain=$SYSTEM_DOMAIN
    3. For each new CA cert, concatenate the new CA certificate to both the `ca` and `certificate` field.
    4. Deploy
  2. Deploy with new leaf certificates
    1. For each leaf certificate in your vars-store, replace with the corresponding certificate from new-vars.yml. These leaf certificates are signed by the new CA's.
    2. Deploy. When the api instances roll, users will no longer be able to push apps, until you remove the old CA certificates.
  3. Deploy without the old CA certificates.
    1. For each CA certificate in your vars-store, remove the first certificate in the `ca` and `certificate` fields. The result should be that only the new CA certificates created in step 1.1 should be included in your vars-store.
The RelInt team has also worked through a process for rotating certificates that have already expired. If you have any questions or concerns, jump into the #release-integration channel in the Cloud Foundry slack and feel free to get a hold of the team there.

Thanks!
CF Release Integration
1581 - 1600 of 9398