Date   

Re: Loggregator Architecture Change: Independently Scalable Syslog

Altenhofen, Michael <michael.altenhofen@...>
 

Hi Adam,

as you might have guessed from my questions, we're interested in more resilient setups for syslog drains.
Not sure whether you've had the chance to listen to my colleague Istvan Ballok's talk at the CF Summit (http://sched.co/AJmc) where he reported on our journey regarding application logging.

Our current solution is implemented as a managed service where the service binding will include a syslog_drain_url of one of our so-called SLEEVE forwarder instances. If one of those instances is going down, I guess it's not that easy to have the corresponding doppler instance (in the current setup) re-connect to a different instance. So, currently we think about using a load balancer in front of those instance cluster, but that introduces another component that may fail.

What "hurts" us at the moment, too, is that we have to re-establish existing bindings on instance restarts, e.g. during an update. This is needed to re-cover "meta data" (app name, space name, org name) for logs that we want to attach to logs. As we currently restrict ourselves to using the official CF API for that, this may take fairly long on large landscapes. We know that there are internal APIs, like the bulk APIs, but as they are marked as "internal", there's no guarantee that they will not change and we want to avoid being trapped by API changes that we didn't spot.

Best

Michael


Re: Proposal: Platform support zero downtime deployments

Zach Robinson
 

Hi Stefan,

Thanks for checking in. We've currently identified the Diego features required to implement this. The Diego team has fleshed out BBS API changes. The next step will be for them to incept and begin work, at which point there will be some tracker stories to look at. Eric M, may be able to add some info on when they are planning to begin work. As the Diego team progresses then CAPI will be able to start integrating with their new APIs to bring the zero downtime features to apps. The CAPI epic is here https://www.pivotaltracker.com/epic/show/3658601, but currently unpopulated.

Thanks!
-Zach


Re: Incubation Proposal: BOSH Backup and Restore

Michael Maximilien
 

Hi, Therese,

All addressed. Best,

Max

On Thu, Jul 13, 2017 at 3:44 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Max,
Hope you had a great holiday.
I've replied to your comments, and added clarification in a few sections.
Look forward to your reply.
thanks,
Therese

On Tue, Jul 11, 2017 at 8:04 PM, Michael Maximilien <mmaximilien(a)gmail.com
wrote:
Hi, Therese,

Finally got some time after holiday and Resolved all except for a few
that I could not. See my additional comment.

I'll try to look at any response from you at least daily to help move my
comments along.

Best,

Max

On Thu, Jun 29, 2017 at 8:49 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
That works perfectly, thanks.
I've responded to pending comments, would welcome thoughts on whether
there's further discussion or clarification needed.
best,
Therese

On Wed, Jun 28, 2017 at 6:20 PM, Michael Maximilien <
mmaximilien(a)gmail.com> wrote:

Hi, Therese,

Noted.

Let me suggest the following schedule:

1. Please address all pending comments making sure each commenter
accepts (Resolve in Google Doc) their comments.

2. After that, let's (you and I) make a last call for comments with a
deadline of the next CAB call on 7/19 or so. This gives a final chance for
all to comment.

3. Assuming no unresolved comments or dissent then we can schedule the
vote on next CF-Extensions call which is a little more than one week after
CAB call on 7/31.

Does that work? Best,

Max

On Tue, Jun 27, 2017 at 6:20 AM Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
Thanks for the comments, I've replied and added detail to the
document. I don't think there's anything contentious but happy to discuss
if there's interest.
Can we aim for a vote at the next PMC Extensions call, please?
best,
Therese

On Tue, Jun 27, 2017 at 12:23 AM, Michael Maximilien <
mmaximilien(a)gmail.com> wrote:

Hi, Therese,

Sorry about the long delay. I am finally back in normal work mode
after weeks and weeks of traveling.

Anyhow, as promise at CF-summit, I have reviewed and left various
comments. Some more important than others. Let's have a discussion about
them in the Google Doc commenting system. If you think we should take some
time to chat then let's schedule it or use part of next month's PMC
Extensions call.

Any approach in the open is fine with me. Let me know.

Best,

max

On Mon, Jun 19, 2017 at 7:04 AM, Therese Stowell <tstowell(a)pivotal.io
wrote:
Hello Stefan,

The B+R scripts are part of CF OSS for all the components aside from
MySQL, as to back up MySQL for Pivotal CF (ERT) we use the closed-source
backup-prepare node. We have plans to add a B+R scripts for OSS MySQL soon.

These docs https://docs.pivotal.io/pivotalcf/1-11/customizing/back
up-restore/backup-pcf-bbr.html#backup-prepare-node explain how to
back up and restore PCF 1.11. BBR can be used to back up any BOSH director
or deployment.

best,
Therese

On Sun, Jun 18, 2017 at 8:40 AM, Stefan Mayr <stefan(a)mayr-stefan.de>
wrote:

Hi

Am 13.06.2017 um 19:16 schrieb Therese Stowell:
Hello,



Pivotal would like to propose to the CF Extensions PMC a new
incubation
project , “BOSH Backup and Restore”, focusing on backing up and
restoring BOSH deployments and BOSH directors. This project aims
to
enable consistent, reliable backups of CF.



Project name: BOSH Backup and Restore



Project proposal:

https://docs.google.com/document/d/1xQ--FzwoADAepTPjxjcbu0at
3jtrp0anvLQE89SViuw/edit#heading=h.7nvp15pyiad8



Project overview presentation:

https://drive.google.com/file/d/0BxHFe1wja8KpaEY1TjU3MWRVZ1k
/view?usp=sharing



Video demo:

https://drive.google.com/file/d/0BxHFe1wja8KpZ1ppM0Rxb05qX00
/view?usp=sharing



Proposed Project Lead: Therese Stowell (Pivotal)

Development Operating Model: Pairing Model

Technical Approach: See "Examples" in the proposal

Initial team committed: 4 engineers from Pivotal



Please let us know if you have any questions.



Thanks,

Therese Stowell, Pivotal


I've seen BBR is already mentiond in the recently released PCF 1.11
docs. Are these B+R scripts already part of CF OSS or is this
currently
exclusive to Pivotal customers?

One thing I found in PCFs docs is the MySQL backup prepare node
(https://docs.pivotal.io/pivotalcf/1-11/customizing/backup-r
estore/backup-pcf-bbr.html#backup-prepare-node).
It is not mentioned on http://www.boshbackuprestore.io. What is
this
nodes task in the context of BBR?

Thank you,

Stefan

--
max
http://maximilien.org
http://blog.maximilien.com
--
dr.max Sent from my iPhone http://maximilien.org

--
max
http://maximilien.org
http://blog.maximilien.com

--
max
http://maximilien.org
http://blog.maximilien.com


Re: Loggregator Architecture Change: Independently Scalable Syslog

Adam Hevenor
 

Hi Michael -

Thanks for these questions. I'll answer them here:

Our intent is to keep the reconnection logic the same. That said we recently realized that we do not have an equivalent back off strategy for HTTPS drains. We are working on a feature to match this to the previous functionality.

The release does not currently allow for load balancing of a single drain. That certainly is an interesting idea and this architecture is better positioned to consider this feature. Is this a common use case for you?

Adam


CF space application sharing

John jerrby
 

I know that every application the deploy to CF space are deployed to isolated container .
application in space can share service instance which is not the case for application in different spaces ...
my question is that: I know that the application from different spaces doesn't have any way to impact other application
but in case of two applications are deployed to the same space, there is a way that it have "privileges " to harm (from security perspective...) other application in the space
which is not available to applications that deployed to different spaces ?


Re: Incubation Proposal: BOSH Backup and Restore

Therese Stowell
 

Hi Max,
Hope you had a great holiday.
I've replied to your comments, and added clarification in a few sections.
Look forward to your reply.
thanks,
Therese

On Tue, Jul 11, 2017 at 8:04 PM, Michael Maximilien <mmaximilien(a)gmail.com>
wrote:

Hi, Therese,

Finally got some time after holiday and Resolved all except for a few that
I could not. See my additional comment.

I'll try to look at any response from you at least daily to help move my
comments along.

Best,

Max

On Thu, Jun 29, 2017 at 8:49 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
That works perfectly, thanks.
I've responded to pending comments, would welcome thoughts on whether
there's further discussion or clarification needed.
best,
Therese

On Wed, Jun 28, 2017 at 6:20 PM, Michael Maximilien <
mmaximilien(a)gmail.com> wrote:

Hi, Therese,

Noted.

Let me suggest the following schedule:

1. Please address all pending comments making sure each commenter
accepts (Resolve in Google Doc) their comments.

2. After that, let's (you and I) make a last call for comments with a
deadline of the next CAB call on 7/19 or so. This gives a final chance for
all to comment.

3. Assuming no unresolved comments or dissent then we can schedule the
vote on next CF-Extensions call which is a little more than one week after
CAB call on 7/31.

Does that work? Best,

Max

On Tue, Jun 27, 2017 at 6:20 AM Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
Thanks for the comments, I've replied and added detail to the document.
I don't think there's anything contentious but happy to discuss if there's
interest.
Can we aim for a vote at the next PMC Extensions call, please?
best,
Therese

On Tue, Jun 27, 2017 at 12:23 AM, Michael Maximilien <
mmaximilien(a)gmail.com> wrote:

Hi, Therese,

Sorry about the long delay. I am finally back in normal work mode
after weeks and weeks of traveling.

Anyhow, as promise at CF-summit, I have reviewed and left various
comments. Some more important than others. Let's have a discussion about
them in the Google Doc commenting system. If you think we should take some
time to chat then let's schedule it or use part of next month's PMC
Extensions call.

Any approach in the open is fine with me. Let me know.

Best,

max

On Mon, Jun 19, 2017 at 7:04 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hello Stefan,

The B+R scripts are part of CF OSS for all the components aside from
MySQL, as to back up MySQL for Pivotal CF (ERT) we use the closed-source
backup-prepare node. We have plans to add a B+R scripts for OSS MySQL soon.

These docs https://docs.pivotal.io/pivotalcf/1-11/customizing/back
up-restore/backup-pcf-bbr.html#backup-prepare-node explain how to
back up and restore PCF 1.11. BBR can be used to back up any BOSH director
or deployment.

best,
Therese

On Sun, Jun 18, 2017 at 8:40 AM, Stefan Mayr <stefan(a)mayr-stefan.de>
wrote:

Hi

Am 13.06.2017 um 19:16 schrieb Therese Stowell:
Hello,



Pivotal would like to propose to the CF Extensions PMC a new
incubation
project , “BOSH Backup and Restore”, focusing on backing up and
restoring BOSH deployments and BOSH directors. This project aims to
enable consistent, reliable backups of CF.



Project name: BOSH Backup and Restore



Project proposal:

https://docs.google.com/document/d/1xQ--FzwoADAepTPjxjcbu0at
3jtrp0anvLQE89SViuw/edit#heading=h.7nvp15pyiad8



Project overview presentation:

https://drive.google.com/file/d/0BxHFe1wja8KpaEY1TjU3MWRVZ1k
/view?usp=sharing



Video demo:

https://drive.google.com/file/d/0BxHFe1wja8KpZ1ppM0Rxb05qX00
/view?usp=sharing



Proposed Project Lead: Therese Stowell (Pivotal)

Development Operating Model: Pairing Model

Technical Approach: See "Examples" in the proposal

Initial team committed: 4 engineers from Pivotal



Please let us know if you have any questions.



Thanks,

Therese Stowell, Pivotal


I've seen BBR is already mentiond in the recently released PCF 1.11
docs. Are these B+R scripts already part of CF OSS or is this
currently
exclusive to Pivotal customers?

One thing I found in PCFs docs is the MySQL backup prepare node
(https://docs.pivotal.io/pivotalcf/1-11/customizing/backup-r
estore/backup-pcf-bbr.html#backup-prepare-node).
It is not mentioned on http://www.boshbackuprestore.io. What is this
nodes task in the context of BBR?

Thank you,

Stefan

--
max
http://maximilien.org
http://blog.maximilien.com
--
dr.max Sent from my iPhone http://maximilien.org

--
max
http://maximilien.org
http://blog.maximilien.com


CF Networking logging enhancements

Usha Ramachandran
 

Hello,

I wanted to share a document describing logging feature enhancements that
the CF networking team is working on. Feedback is welcome, please add your
comments/questions to the proposal doc.

*Proposal - Converging on logging for ASGs and container networking*
*Summary:* Operators want a way to log all application traffic with an easy
way to identify the source of traffic captured in the logs.
Link to proposal -
*https://docs.google.com/document/d/1Gh_Z0NNg3VhrMnJpW_I6JpgNyFw-3vq0smnRZClP3i4/edit?usp=sharing
<https://docs.google.com/document/d/1Gh_Z0NNg3VhrMnJpW_I6JpgNyFw-3vq0smnRZClP3i4/edit?usp=sharing>*

Thanks,
Usha Ramachandran
CF Networking PM




--
Usha Ramachandran | Senior Product Manager | Pivotal Cloud Foundry - San
Francisco


CF CAB call for July 2017 is next Wednesday, July 19th @ 8a PDT [15:00 UTC]

Michael Maximilien
 

Hi, all,

Quick reminder of the CAB call for July is next Wednesday, July 19th @ 8a PDT. All info in link [1].

Remember, no more status update but rather discussions, so come ready with your questions. Also we will have David Sabeti of Pivotal giving an update on cf-deployment [2].

Finally, I am looking for one more 15 mins presentation. Please contact me replying here or on the slack.cloudfoundry.org #CAB channel if you have suggestions and to browse previous and future discussions.

Talk to you all next week. I'll send one more reminder on this list next week.

Best,

dr.max

ibm cloud labs
sillicon valley, ca
usa
maximilien.org

Sent from my iPhone

[1] https://docs.google.com/document/d/1SCOlAquyUmNM-AQnekCOXiwhLs6gveTxAcduvDcW_xI/edit#heading=h.o44xhgvum2we

[2] https://github.com/cloudfoundry/cf-deployment


Re: Unable to decode log messages using protocol buffers decode_raw: "Failed to parse input."

Ben Moss <bmoss@...>
 

You might try using noaa to get logs if you can:
https://github.com/cloudfoundry/noaa

Alternatively you might be able to look at the source of that (it’s a
relatively small program) to figure out why you’re having trouble decoding
the protobufs.

Ben


On Tue, Jul 11, 2017 at 3:12 PM, Pawel Kijowski <pawel.kijowski(a)gmail.com>
wrote:

Could somebody help me understand why I am not able to decode log message
using protoc decode_raw?

I've tried to decode the entire output:

```

curl -X GET -H 'Authorization: bearer UAA_TOKEN'
https://doppler.ng.bluemix.net/apps/APP_GUID/recentlogs 2>/dev/null |
protoc --decode_raw

```

as well as a single LogMessage from an Event Envelope:

```

echo 0a08676f726f75746572100530a3dbabda8c8897e81442a1040ae4036d6f
62696c652d63666576656e74732d636f6e73756d65722e6d79626c75656d
69782e6e6574202d205b323031372d30372d31315431393a30383a32392e
3236372b303030305d2022504f5354202f6576656e747320485454502f31
2e312220323030203131353520313620222d2220222d2220223130382e31
36382e3235302e3135323a35353039342220223136392e34372e3230302e
3136373a36323134332220785f666f727761726465645f666f723a223233
2e3234362e3139392e38312220785f666f727761726465645f70726f746f
3a2268747470732220766361705f726571756573745f69643a2234623864
656533622d333234322d346334632d366564622d33343461623932616564
36642220726573706f6e73655f74696d653a302e30303534363035353920
6170705f69643a2232383562323633392d336530632d346265622d393831
622d36356630366139333234653822206170705f696e6465783a22302220
785f676c6f62616c5f7472616e73616374696f6e5f69643a223432313734
36323130392220785f62335f747261636569643a22323833623065626637
306431306235312220785f62335f7370616e69643a223238336230656266
37306431306235312220785f6
2335f706
172656e747370616e69643a222d220a1001188dcbabda8c8897e81422243
2383562323633392d336530632d346265622d393831622d3635663036613
933323465382a035254523201336a0963662d77617264656e7206726f757
465727a2464323134653534332d353662372d343539662d616462612d626
63535383738643062666382010c37352e3132362e33372e3734333838383
8383720776974682073746174757320315c6e2a2063616e63656c6c65645
c6e2a2063616e63656c6c6564222c22726561736f6e223a2243524153484
544227d2c2273706163655f67756964223a2231376238393239622d38666
1382d346637362d393234352d653739323137613236373331222c226f726
7616e697a6174696f6e5f67756964223a2262396439626133382d3330613
42d346536632d616263322d626131333065633661333131222c226163746
f725f69616d5f6964223a22222c2261637465655f69616d5f6964223a222
27d2c22726567696f6e223a2275732d736f757468227d100118d6b7e8e1c
58c96e814222432383562323633392d336530632d346265622d393831622
d3635663036613933323465382a034150503201306a18736361732d797a2
d70726f642d646965676f2d64616c3130720a63656c6c5f64616c31307a2
437393263643735362d373738322d
34386436
2d623435312d34303239653535303366393982010e3136392e34372e3230
302e313637392d336530632d346265622d393831622d3635663036613933
323465382a034150503201306a18736361732d797a2d70726f642d646965
676f2d64616c3130720a63656c6c5f64616c31307a243739326364373536
2d373738322d343864362d623435312d3430323965353530336639398201
0e3136392e34372e3230302e3136372e3230302e3136372e3230302e3136
373982010e3136392e34372e3230302e313637 | xxd -r -p | protoc --decode_raw

```

Both fail with:

```

Failed to parse input.

```

Thanks in advance for your help.


Unable to decode log messages using protocol buffers decode_raw: "Failed to parse input."

Pawel Kijowski
 

Could somebody help me understand why I am not able to decode log message using protoc decode_raw?

I've tried to decode the entire output:

```

curl -X GET -H 'Authorization: bearer UAA_TOKEN' https://doppler.ng.bluemix.net/apps/APP_GUID/recentlogs 2>/dev/null | protoc --decode_raw

```

as well as a single LogMessage from an Event Envelope:

```

echo 0a08676f726f75746572100530a3dbabda8c8897e81442a1040ae4036d6f62696c652d63666576656e74732d636f6e73756d65722e6d79626c75656d69782e6e6574202d205b323031372d30372d31315431393a30383a32392e3236372b303030305d2022504f5354202f6576656e747320485454502f312e312220323030203131353520313620222d2220222d2220223130382e3136382e3235302e3135323a35353039342220223136392e34372e3230302e3136373a36323134332220785f666f727761726465645f666f723a2232332e3234362e3139392e38312220785f666f727761726465645f70726f746f3a2268747470732220766361705f726571756573745f69643a2234623864656533622d333234322d346334632d366564622d3334346162393261656436642220726573706f6e73655f74696d653a302e303035343630353539206170705f69643a2232383562323633392d336530632d346265622d393831622d36356630366139333234653822206170705f696e6465783a22302220785f676c6f62616c5f7472616e73616374696f6e5f69643a22343231373436323130392220785f62335f747261636569643a22323833623065626637306431306235312220785f62335f7370616e69643a22323833623065626637306431306235312220785f62335f706
172656e747370616e69643a222d220a1001188dcbabda8c8897e814222432383562323633392d336530632d346265622d393831622d3635663036613933323465382a035254523201336a0963662d77617264656e7206726f757465727a2464323134653534332d353662372d343539662d616462612d62663535383738643062666382010c37352e3132362e33372e37343338383838383720776974682073746174757320315c6e2a2063616e63656c6c65645c6e2a2063616e63656c6c6564222c22726561736f6e223a2243524153484544227d2c2273706163655f67756964223a2231376238393239622d386661382d346637362d393234352d653739323137613236373331222c226f7267616e697a6174696f6e5f67756964223a2262396439626133382d333061342d346536632d616263322d626131333065633661333131222c226163746f725f69616d5f6964223a22222c2261637465655f69616d5f6964223a22227d2c22726567696f6e223a2275732d736f757468227d100118d6b7e8e1c58c96e814222432383562323633392d336530632d346265622d393831622d3635663036613933323465382a034150503201306a18736361732d797a2d70726f642d646965676f2d64616c3130720a63656c6c5f64616c31307a2437393263643735362d373738322d34386436
2d623435312d34303239653535303366393982010e3136392e34372e3230302e313637392d336530632d346265622d393831622d3635663036613933323465382a034150503201306a18736361732d797a2d70726f642d646965676f2d64616c3130720a63656c6c5f64616c31307a2437393263643735362d373738322d343864362d623435312d34303239653535303366393982010e3136392e34372e3230302e3136372e3230302e3136372e3230302e3136373982010e3136392e34372e3230302e313637 | xxd -r -p | protoc --decode_raw

```

Both fail with:

```

Failed to parse input.

```

Thanks in advance for your help.


Re: Incubation Proposal: BOSH Backup and Restore

Michael Maximilien
 

Hi, Therese,

Finally got some time after holiday and Resolved all except for a few that
I could not. See my additional comment.

I'll try to look at any response from you at least daily to help move my
comments along.

Best,

Max

On Thu, Jun 29, 2017 at 8:49 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
That works perfectly, thanks.
I've responded to pending comments, would welcome thoughts on whether
there's further discussion or clarification needed.
best,
Therese

On Wed, Jun 28, 2017 at 6:20 PM, Michael Maximilien <mmaximilien(a)gmail.com
wrote:
Hi, Therese,

Noted.

Let me suggest the following schedule:

1. Please address all pending comments making sure each commenter accepts
(Resolve in Google Doc) their comments.

2. After that, let's (you and I) make a last call for comments with a
deadline of the next CAB call on 7/19 or so. This gives a final chance for
all to comment.

3. Assuming no unresolved comments or dissent then we can schedule the
vote on next CF-Extensions call which is a little more than one week after
CAB call on 7/31.

Does that work? Best,

Max

On Tue, Jun 27, 2017 at 6:20 AM Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hi Dr Max,
Thanks for the comments, I've replied and added detail to the document.
I don't think there's anything contentious but happy to discuss if there's
interest.
Can we aim for a vote at the next PMC Extensions call, please?
best,
Therese

On Tue, Jun 27, 2017 at 12:23 AM, Michael Maximilien <
mmaximilien(a)gmail.com> wrote:

Hi, Therese,

Sorry about the long delay. I am finally back in normal work mode after
weeks and weeks of traveling.

Anyhow, as promise at CF-summit, I have reviewed and left various
comments. Some more important than others. Let's have a discussion about
them in the Google Doc commenting system. If you think we should take some
time to chat then let's schedule it or use part of next month's PMC
Extensions call.

Any approach in the open is fine with me. Let me know.

Best,

max

On Mon, Jun 19, 2017 at 7:04 AM, Therese Stowell <tstowell(a)pivotal.io>
wrote:

Hello Stefan,

The B+R scripts are part of CF OSS for all the components aside from
MySQL, as to back up MySQL for Pivotal CF (ERT) we use the closed-source
backup-prepare node. We have plans to add a B+R scripts for OSS MySQL soon.

These docs https://docs.pivotal.io/pivotalcf/1-11/customizing/back
up-restore/backup-pcf-bbr.html#backup-prepare-node explain how to
back up and restore PCF 1.11. BBR can be used to back up any BOSH director
or deployment.

best,
Therese

On Sun, Jun 18, 2017 at 8:40 AM, Stefan Mayr <stefan(a)mayr-stefan.de>
wrote:

Hi

Am 13.06.2017 um 19:16 schrieb Therese Stowell:
Hello,



Pivotal would like to propose to the CF Extensions PMC a new
incubation
project , “BOSH Backup and Restore”, focusing on backing up and
restoring BOSH deployments and BOSH directors. This project aims to
enable consistent, reliable backups of CF.



Project name: BOSH Backup and Restore



Project proposal:

https://docs.google.com/document/d/1xQ--FzwoADAepTPjxjcbu0at
3jtrp0anvLQE89SViuw/edit#heading=h.7nvp15pyiad8



Project overview presentation:

https://drive.google.com/file/d/0BxHFe1wja8KpaEY1TjU3MWRVZ1k
/view?usp=sharing



Video demo:

https://drive.google.com/file/d/0BxHFe1wja8KpZ1ppM0Rxb05qX00
/view?usp=sharing



Proposed Project Lead: Therese Stowell (Pivotal)

Development Operating Model: Pairing Model

Technical Approach: See "Examples" in the proposal

Initial team committed: 4 engineers from Pivotal



Please let us know if you have any questions.



Thanks,

Therese Stowell, Pivotal


I've seen BBR is already mentiond in the recently released PCF 1.11
docs. Are these B+R scripts already part of CF OSS or is this
currently
exclusive to Pivotal customers?

One thing I found in PCFs docs is the MySQL backup prepare node
(https://docs.pivotal.io/pivotalcf/1-11/customizing/backup-
restore/backup-pcf-bbr.html#backup-prepare-node).
It is not mentioned on http://www.boshbackuprestore.io. What is this
nodes task in the context of BBR?

Thank you,

Stefan

--
max
http://maximilien.org
http://blog.maximilien.com
--
dr.max Sent from my iPhone http://maximilien.org

--
max
http://maximilien.org
http://blog.maximilien.com


Re: Proposal: Platform support zero downtime deployments

Stefan Mayr
 

Hi

Am 29.03.2017 um 20:41 schrieb Zach Robinson:
Hey CF-Dev

We have been working on a proposal to pull basic zero downtime app deployments into the platform. The proposal is linked below. We would love to hear thoughts on whether this would address most use cases.

https://docs.google.com/document/d/116I_mOWjZcPeIbUvvsh-jAcwpoE_mGPD_SkCel5xXuU/edit?usp=sharing

Thanks!
Zach
CAPI PM

What's the state of this propsal? Are there any public tracker stories
we can follow?

Thanks,

Stefan


CATs timeout fixup

David Sabeti
 

Hi CF Dev,

I wanted to let everyone know about a potentially disruptive change coming
to CATs <https://github.com/cloudfoundry/cf-acceptance-tests>. If you've
ever run that beloved test suite before, you've probably configured a
timeout or two. Well, it turns out that the test suite is pretty
inconsistent about how it measures timeouts, sometimes preferring seconds
and at other times preferring minutes
<https://github.com/cloudfoundry/cf-acceptance-tests/blob/master/helpers/config/config_struct.go#L588-L618>.
That's not great.

We're going to instead do the sane thing and measure everything in same
unit, specifically in seconds. On Wednesday morning at 9:30 AM PST, we're
going to merge this PR to use seconds for all timeouts
<https://github.com/cloudfoundry/cf-acceptance-tests/pull/219>. Hopefully,
that should be enough time for everyone to update their timeout
configuration to use seconds, rather than minutes. *The timeout most likely
to be seriously affected is the `cf_push_timeout`.*

Feel free to reach out if you any questions or concerns.

Thanks!
CF Release Integration


Re: [Proposal] Sharing service instances across orgs and spaces

Matt McNeeney
 

Thanks all for the great feedback. There are a lot of good ideas here that
we will use to help guide the implementation of this.

Gabriel; the relationship service instances have with ASGs is interesting.
I don't think we should automatically modify security groups when service
instances are shared, as there could be security implications in doing so.
But this is definitely something we will look into in detail.

Jouke; permanent migrations of services isn't something we've taken into
account in this sharing proposal, however I can see the use cases. I wonder
if these could be mitigated via another solution to another CF CLI command,
for example, making backing up and restoring data services easier or
available to application developers.

Daniel; I believe that idea has been raised a few times but, as I'm sure
you know, breaking the services layer out of the cloud controller could be
very tricky and time consuming. Zach R would have more thoughts on this I'm
sure.

Dies; we believe enforcing service sharing to be enabled by default is a
more secure approach. I appreciate your point that right now application
developers can effectively workaround this and share services via passing
credentials and using *cf cups*, but we know that some CF users are looking
to hide credentials as best as possible. In a more secure world where
credentials are not visible to application developers and are protected in
something like CredHub, this new workflow would be the only way to share
services, and we'd like to ensure that this new workflow is as secure as
possible.


On Wed, Jul 5, 2017 at 6:37 AM tommyoshields71 <tommyoshields71(a)gmail.com>
wrote:

Tell me is this that expensive coin that I have

On Jul 4, 2017 12:40 AM, "Koper, Dies [via CF Dev]" <[hidden email]
<http:///user/SendEmail.jtp?type=node&node=7101&i=0>> wrote:

Hi Matt,



Can you explain the background of the service sharing enabling feature?

Who would want to disable this and for what reason?



I can imagine there are services that should not be bound to multiple
apps, maybe because of locking issues.

If that’s not a limitation/feature of the service, it may be a constraint
of the (initial) developer who created this service for a single-user
purpose.

For those cases, I feel it would make sense for the “enable” flag to be
an attribute of the service itself, or of the person (role) creating it.



How does the Org Manager come in play?

Wouldn’t I currently be copying the credentials over to another
org/person (maybe to another app developer) to store in a user-provided
service, so the additional enable step just creates an additional step
offering no protection?



Regards,

Dies Koper
Cloud Foundry Product Manager - CLI




*From:* Matthew McNeeney [mailto:[hidden email]
<http:///user/SendEmail.jtp?type=node&node=7097&i=0>]
*Sent:* Thursday, June 29, 2017 11:36 PM
*To:* Discussions about Cloud Foundry projects and the system overall. <[hidden
email] <http:///user/SendEmail.jtp?type=node&node=7097&i=1>>
*Subject:* [cf-dev] [Proposal] Sharing service instances across orgs and
spaces

Many Cloud Foundry users have expressed a desire to share service
instances across orgs and spaces. Whilst this could be considered an
anti-pattern for some data services, there are many use cases for which the
ability to do this is important. Two examples are sharing config servers
and messaging queues.



The workarounds that exist today (e.g. creating user-provided services)
require credentials to be passed around in some out-of-band way and will
prevent the platform from being able to do things like automatic rotation
of credentials in the future.



We'd like to propose a new workflow that looks like this:



$ cf share-service SERVICE_INSTANCE TARGET_ORG TARGET_SPACE



A SpaceDeveloper in the target org/space will only be able to bind/unbind
to/from the shared service instance, and running cf service will show
that the service instance has been shared.



To manage any security concerns around this, a CF admin would have to
enable one-way sharing between two spaces with a command like:



$ cf enable-service-sharing SERVICE SOURCE_ORG SOURCE_SPACE TARGET_ORG
TARGET_SPACE





We'd love to hear feedback from the community on this proposal. If you
have any other use cases that this could help with, please let us know
about those too.



Matt



------------------------------
If you reply to this email, your message will be added to the discussion
below:

http://cf-dev.70369.x6.nabble.com/cf-dev-Proposal-Sharing-service-instances-across-orgs-and-spaces-tp7076p7097.html
To start a new topic under CF Dev, email [hidden email]
<http:///user/SendEmail.jtp?type=node&node=7101&i=1>
To unsubscribe from CF Dev, click here.
NAML
<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
*IMG_20170704_081255.jpg* (1M) Download Attachment
<http://cf-dev.70369.x6.nabble.com/attachment/7101/0/IMG_20170704_081255.jpg>

------------------------------
View this message in context: Re: [cf-dev] Re: [Proposal] Sharing service
instances across orgs and spaces
<http://cf-dev.70369.x6.nabble.com/Re-cf-dev-Re-Proposal-Sharing-service-instances-across-orgs-and-spaces-tp7101.html>
Sent from the CF Dev mailing list archive
<http://cf-dev.70369.x6.nabble.com/> at Nabble.com.


Re: Route service example in .NET?

Zach Brown
 

Hi JK,

Looks like you're breaking some new ground here. :)

I asked around at Pivotal and got this reply from one of our consultants.

If you make progress, please share source (if you can) for the benefit of
the community.

------
yes started to play with that, but used java oauth2 route service in the
end - as it is anyway agnostic of backend service.

while researching found these two approaches pretty good :
- owin middleware from asp.net team https://github.com/aspnet/Proxy and
https://github.com/SharpTools/SharpReverseProxy

- or using delegating MessageHandlers in webapi
https://kasperholdum.dk/2016/03/09/reverse-proxy-in-asp-net-web-api/

On Mon, Jun 12, 2017 at 2:00 PM, J K <falconwing(a)gmail.com> wrote:

I'm looking for a simple .NET route service that does authentication or
authorization of some kind and runs on CF.

Do any reference examples exist? We're evaluating .NET on CF at my company
and this would be helpful for understanding things.
--

*Zach Brown*


Re: Proposal: Plan to Deprecate cf-release

David Sabeti
 

Hi Bernd,

Thanks for the feedback. I responded to your comment in the doc. Let me
know there what you think.

David

On Wed, Jul 5, 2017 at 6:20 AM Krannich, Bernd <bernd.krannich(a)sap.com>
wrote:

Hi David,



From talking to our teams here locally, one point that is important for us
to have the same out of the box IaaS coverage than what we have in
cf-release today (for us, specifically this means OpenStack, AWS, Azure,
Google Cloud).



Not sure if this is already considered in the current planning and it’s
also two days after your deadline, but I have the feeling it’s better to
raise this as an input from our end before a GA announcement is made.



Thanks in advance,

Bernd



*Bernd Krannich*

SAP Cloud Platform

*SAP SE*

Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany



Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum
<http://www.sap.com/company/legal/impressum.epx/>



Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige
vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich
erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine
Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte
benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen
Dank.



This e-mail may contain trade secrets or privileged, undisclosed, or
otherwise confidential information. If you have received this e-mail in
error, you are hereby notified that any review, copying, or distribution of
it is strictly prohibited. Please inform us immediately and destroy the
original transmittal. Thank you for your cooperation.





*From: *David Sabeti <dsabeti(a)pivotal.io>
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall." <cf-dev(a)lists.cloudfoundry.org>
*Date: *Saturday, 24. June 2017 at 00:56
*To: *"Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
*Subject: *[cf-dev] Re: Re: Proposal: Plan to Deprecate cf-release



Hi Benjamin,



The README is still accurate, and we're working away at being able to make
some guarantees about uptime during deploys -- especially the deploy to
transition from cf-release to cf-deployment. This is the tracker milestone
<https://www.pivotaltracker.com/story/show/143199685> to keep an eye on,
which we're currently targeting for end of July. Once we feel comfortable
with the uptime measurement, we'll announce it pretty loudly that
cf-deployment is ready to go and give it a 1.0.0 version.



David



On Fri, Jun 23, 2017 at 12:50 PM Benjamin Gandon <benjamin(a)gandon.org>
wrote:

Super cool to get to know this new “cf-deployment-transition” repo!



But about cf-deployment *in production*, is it still blocked on “Downtime
testing” as the README states? Or has there been progress on that without
it being reflected in that README?



If it is still a blocker, could you give us insights about what is
actually blocking, and any link to a tracker story so that we can watch
progress about that?

Thanks.

/Benjamin (depuis mon iPhone)





Bien cordialement,

/Benjamin Gandon (depuis mon iPhone)

Le 23 juin 2017 à 01:55, David Sabeti <dsabeti(a)pivotal.io> a écrit :

Hi cf-dev,



As cf-deployment <https://github.com/cloudfoundry/cf-deployment> moves
closer to general availability (GA), we want to propose a schedule for
deprecating cf-release <https://github.com/cloudfoundry/cf-release> and
helping operators move from deployments based on cf-release to deployments
based on cf-deployment. Please take a look at the proposal included here:




https://docs.google.com/document/d/1KLl4UIQbl92SvYom4fO-LcEoMK1D45KmjA988MwnOR4/edit?usp=sharing



*We're hoping to finalize a plan by July 3rd, so please leave any feedback
on the document by then.*



Thanks!

David Sabeti

Project Lead, CF Release Integration


Communication to blobstore is failing

nitin padalia
 

Hi,

I am trying to upgrade my cf to v251, when I ran bosh deploy, it gave error during build pack upload to blobstore after compilation:
caused by: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.

Later when I tried to deploy again, it is giving me:
Failed to download S3 object, code 1, output: '', error: '2017/07/05 17:36:10 performing operation get: ServiceUnavailable: The server is temporarily unavailable. Please try again later

I tried aws cli s3 ls and was able to list folders there using the credentials.

What could be causing this?


Re: UAAdb duplicated useres detection and deletion

Filip Hanik
 

the unique key for a user is

username+origin

where origin is the authentication source of the user (the identity
provider)

On Wed, Jul 5, 2017 at 15:34 Gowrisankar M <gowrisankarbeece(a)gmail.com>
wrote:

Hi Colleagues,

I have experienced an issue that User exists twice with the same user name
in the uaa db. Is there any script that does detection of ducplicated users
and auto removal ?

Thanks,
GS


UAAdb duplicated useres detection and deletion

Gowrisankar M
 

Hi Colleagues,

I have experienced an issue that User exists twice with the same user name
in the uaa db. Is there any script that does detection of ducplicated users
and auto removal ?

Thanks,
GS


Re: Proposal: Plan to Deprecate cf-release

Krannich, Bernd <bernd.krannich@...>
 

Hi David,

From talking to our teams here locally, one point that is important for us to have the same out of the box IaaS coverage than what we have in cf-release today (for us, specifically this means OpenStack, AWS, Azure, Google Cloud).

Not sure if this is already considered in the current planning and it’s also two days after your deadline, but I have the feeling it’s better to raise this as an input from our end before a GA announcement is made.

Thanks in advance,
Bernd

Bernd Krannich
SAP Cloud Platform
SAP SE
Dietmar-Hopp-Allee 16, 69190 Walldorf, Germany

Pflichtangaben/Mandatory Disclosure Statement: www.sap.com/impressum<http://www.sap.com/company/legal/impressum.epx/>

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.


From: David Sabeti <dsabeti(a)pivotal.io>
Reply-To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org>
Date: Saturday, 24. June 2017 at 00:56
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org>
Subject: [cf-dev] Re: Re: Proposal: Plan to Deprecate cf-release

Hi Benjamin,

The README is still accurate, and we're working away at being able to make some guarantees about uptime during deploys -- especially the deploy to transition from cf-release to cf-deployment. This is the tracker milestone<https://www.pivotaltracker.com/story/show/143199685> to keep an eye on, which we're currently targeting for end of July. Once we feel comfortable with the uptime measurement, we'll announce it pretty loudly that cf-deployment is ready to go and give it a 1.0.0 version.

David

On Fri, Jun 23, 2017 at 12:50 PM Benjamin Gandon <benjamin(a)gandon.org<mailto:benjamin(a)gandon.org>> wrote:
Super cool to get to know this new “cf-deployment-transition” repo!

But about cf-deployment in production, is it still blocked on “Downtime testing” as the README states? Or has there been progress on that without it being reflected in that README?


If it is still a blocker, could you give us insights about what is actually blocking, and any link to a tracker story so that we can watch progress about that?
Thanks.

/Benjamin (depuis mon iPhone)


Bien cordialement,
/Benjamin Gandon (depuis mon iPhone)
Le 23 juin 2017 à 01:55, David Sabeti <dsabeti(a)pivotal.io<mailto:dsabeti(a)pivotal.io>> a écrit :
Hi cf-dev,

As cf-deployment<https://github.com/cloudfoundry/cf-deployment> moves closer to general availability (GA), we want to propose a schedule for deprecating cf-release<https://github.com/cloudfoundry/cf-release> and helping operators move from deployments based on cf-release to deployments based on cf-deployment. Please take a look at the proposal included here:

https://docs.google.com/document/d/1KLl4UIQbl92SvYom4fO-LcEoMK1D45KmjA988MwnOR4/edit?usp=sharing

We're hoping to finalize a plan by July 3rd, so please leave any feedback on the document by then.

Thanks!
David Sabeti
Project Lead, CF Release Integration

2361 - 2380 of 9389