Date   

Re: Proposal for Service Discovery within Elastic Clusters

Amit Kumar Gupta
 

Hi all,

Sorry for the confusion, I originally added some extraneous docs to the
Required Reading list that people have been trying and failing to access.
Those links have been removed, the most up-to-date information (about the
general Elastic Clusters feature narrative) is in what's now the first
link, titled "Cloud Foundry Elastic Clusters".

Cheers,
Amit

On Wed, Apr 6, 2016 at 6:52 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi all,

If you are familiar with the proposed Elastic Clusters feature narrative
(or even if you're not), I welcome you to take a look at the below proposal
for Consul-based service discovery for components within an Elastic
Clusters configuration.


https://docs.google.com/document/d/1aMpIXsPpB6O_oJsoazOl7kGajc3sRnlrflQFCtysfx8/edit

Feedback is eagerly solicited.

Links to background information on Elastic Clusters can be found within
the above doc.

Thanks,
Amit, CF Infrastructure team PM


Re: How can we customized "404 Not Found"

Mike Youngstrom
 

We customized our 404 mapping wildcard routes to an app. We also display a
503 if the route exists.

Mike

On Apr 7, 2016 9:18 AM, "Stanley Shen" <meteorping(a)gmail.com> wrote:

When we accessing an APP which is not up or not existed, the CF will
display error message like:

404 Not Found: Requested route ('test.domain.name') does not exist.

Can we customize it, and how can we do that?


Thanks in advance.


How can we customized "404 Not Found"

Stanley Shen <meteorping@...>
 

When we accessing an APP which is not up or not existed, the CF will display error message like:

404 Not Found: Requested route ('test.domain.name') does not exist.

Can we customize it, and how can we do that?


Thanks in advance.


Re: OpenStack Leads

David <David@...>
 

Did you get a chance to review my previous email?



Let me know your thoughts!



~David



From: David [mailto:David(a)lead2tech.com]
Sent: Tuesday, April 05, 2016 8:42 AM
To: 'cf-dev(a)lists.cloudfoundry.org'
Subject: OpenStack Leads





Hi,



I hope this email finds you well.



As your organization is a part of " OpenStack Summit 2016 " I'm writing to
check if you would be interested in a leads of OpenStack users for marketing
campaign and lead generation initiative.



You may choose job titles such as: IT Architects, CTOs, IT Directors, Data
Centre Managers, Developers and more.



Each lead will be delivered to you with multi-channel data fields such as
Company name, Web address, Contact Name, Job Title, Phone Number, Direct
Business Emails, Postal address, Industry, employee size, Revenue size, SIC
code, Fax number, Application/Technology usage etc.



Appreciate your time and look forward to hearing from you.



Regards,



David Reid| Lead Generation Specialist
Lead2Tech LLC| Technology Division

If you do not wish to receive any more emails please reply to this email
with Leave out in the subject line.











<http://1816396.sigclick.mailinfo.com/sigclick/09000306/0000054D/0E0B0607/04
450118820393.jpg>


Automated Deployment of window cell for diego

Long Nguyen
 

Anyone know about automated deployment of window cell for diego? I've done it manually a few months back and it was very manual and error prone. Does anyone have anything that automate process?


Re: Internal/external domains

Geoff Franks <geoff@...>
 

We built the cf-haproxy-boshrelease for this (https://github.com/cloudfoundry-community/cf-haproxy-boshrelease). Check out https://blog.starkandwayne.com/2015/06/17/securing-private-domains-on-a-public-cloudfoundry/ for more details. The feature-set has been updated quite a bit since that post. It now supports enough features that should allow it to work behind tcp or http-mode load balancers (you can force x-forwarded-proto if behind a tcp-only loadbalancer that does ssl termination), handle websockets on standard http ports, serve multiple ssl certs, and do generic tcp proxying for things like app-ssh.

My ideal setup would be a set of http-mode load balancers in front of the 'public_haproxy_z*' nodes, which would filter out non-allowed domains, and send the remaining requests to the gorouters. For internal-only requests, I'd use a non-public http-mode load balancer directly in front of the gorouters.


Static IP setup for routers on AWS

Engelke, Johannes <info@...>
 

Hi,
does anybody know, why the routers got static ips in the cf-infrastructure-aws.yml file? https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-aws.yml#L173 <https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-aws.yml#L173>

Bosh is assigning the instances to ELB’s during deploy time, so there should be no need to have static addresses here.

If nobody know’s a good reason should we remove them ;-)

Cheers
Johannes


CF 212 - service in "delete failed" -> how to remove it properly?

Rafal Radecki
 

Hi :)

I have several organizations in which I have services in "delete failed" state. They prevent me from deleting the organization.

What is the best way to remove metadata about such service?

BR,
Rafal.


[ Logging properties ] change buffer size for Containers SDOUT

Fabien Guichard
 

Hello,

while developping an extension ofr a buildpack, related to logging event from applications, we found out that STDOUT logs where being flushed with a fixed size (because we're receving troncated strings in our external aggregating system).

it seems to us that the buffer size is around 4000 caracters, which is not what we would like.

We went through doppler metron and loggregator configurations & codes, without success, to find out anything related to fixed-size logging property.

Do ou know where we should go to configure the size of the buffer being flushed please ?

Many thanks for the time you took to read our very first question on this ML.

Regards,
Fabien.


Re: Internal/external domains

Matthew Sykes <matthew.sykes@...>
 

Container networking is aimed at your requirement. Once it's ready, you
would deploy your apps without external routes and rely on the private
network for communication.

We expect we'll have facilities in place to find apps via a DNS
implementation using app names or services names but we''re beginning to
move in a direction where apps are associated with explicit service names
that are orthogonal to the app name. We may also adopt a VIP pattern for
services to remove the need name resolution entirely.

For status, we have containers talking via n overlay with guardian and
we're in the process of integrating with diego. We hope to demonstrate what
we have at summit. It will be a while before things are "baked" but the
project exists and we're moving forward.

Finally, there are plans to implement several policy controls around
endpoint communication that enable users to express a supported
connectivity graph without relying on apps living in the same space for
communication.

Hope that helps. Let us know if you have more questions.

On Thu, Apr 7, 2016 at 1:28 AM, Krannich, Bernd <bernd.krannich(a)sap.com>
wrote:

Hi all,

Dr. Nic’s blog post [1] addresses a requirement that we keep hearing
frequently: To have certain applications that can only be accessed from
other apps running in the same Cloud Foundry deployment.

I wanted to check with the community here to see if anybody has come
across this requirement as well. Is this something one should actually be
advocating in terms of architecture?

Also, the solution suggested in [1] in our opinion is prone to bypassing
if implemented in the described way (see top-most comment in the same blog
post). Of course, one could work around that by terminating SSL at the two
load balancers, introspecting the header fields there. On the other hand,
this requires a much more complex “internal/external” DNS setup.

A logical next step would be to extend the requirement to only allow calls
from apps being deployed into certain orgs and spaces. This can no longer
be achieved by a setup “external” to CF given that apps from all types of
orgs and spaces are usually deployed on the very same runners with calls
being “indistinguishable” once a call has left the runner.

As alternatives, we came up with the following potential options:

1. Avoiding the notion of “private” services altogether, making sure that
each application is secured by - for example - a JWT token carrying a
shared secret.
2. Using messaging via a backing service instead of HTTP-based
communication.
3. Recently suggested/implemented CF functionality like “container
networking for applications” [2] (what’s the status on this one, actually?)
and “Context Path Routing” [3] (this was discussed on the mailing list some
time back, even though I’m not sure how you would actually use it to solve
the above scenario). Is any of that allowing to implement the above
requirement?

Any thoughts, opinions, suggestions are welcome.

Thanks in advance,
Bernd

[1]
https://blog.starkandwayne.com/2014/10/31/public-and-private-microservices-on-the-same-cloud-foundry
[2]
https://docs.google.com/document/d/1zQJqIEk4ldHH5iE5zat_oKIK8Ejogkgd_lySpg_oV_s/edit?usp=sharing
[3]
http://berlin2015.cfsummit.com/sites/berlin2015.cfsummit.com/files//pages/files/summit-berlin.pdf

Bernd Krannich
Products & Innovation Technology
SAP SE
Dietmar-Hopp-Allee 16
69190 Walldorf Germany
mailto:bernd.krannich(a)sap.com
www.sap.com

Pflichtangaben/Mandatory Disclosure Statements:
http://www.sap.com/company/legal/impressum.epx

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige
vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich
erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine
Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte
benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen
Dank.

This e-mail may contain trade secrets or privileged, undisclosed, or
otherwise confidential information. If you have received this e-mail in
error, you are hereby notified that any review, copying, or distribution of
it is strictly prohibited. Please inform us immediately and destroy the
original transmittal. Thank you for your cooperation.


--
Matthew Sykes
matthew.sykes(a)gmail.com


HAproxy server connection limitation

MaggieMeng
 

Hi,

Is there connection limitation on HAproxy server? We had encountered a connection issue recently. HAproxy server failed to accept new connection request after connection number reached 2000+. Is there a way to change the connection limitation?

The CF version is 197 and below is the content of haproxy server configuration file.

global
log 127.0.0.1 syslog info
daemon
user vcap
group vcap
maxconn 64000
spread-checks 4

defaults
log global
timeout connect 30000ms

frontend http-in
mode http
bind :80
option httplog
option forwardfor
reqadd X-Forwarded-Proto:\ http
default_backend http-routers



frontend https-in
mode http
bind :443 ssl crt /var/vcap/jobs/haproxy/config/cert.pem no-sslv3 ciphers
option httplog
option forwardfor
option http-server-close
reqadd X-Forwarded-Proto:\ https
default_backend http-routers

frontend ssl-in
mode tcp
bind :4443 ssl crt /var/vcap/jobs/haproxy/config/cert.pem no-sslv3 ciphers
default_backend tcp-routers


backend http-routers
mode http
balance roundrobin

Thanks,
Maggie


Re: April CAB call next week on Wednesday April 13th, 2016

Dr Nic Williams <drnicwilliams@...>
 

Hurray I'll be in a useful timezone and can join!
"Living in Australia" and "CAB calls" are mutually exclusive :)
Nic

On Wed, Apr 6, 2016 at 5:36 PM -0700, "Michael Maximilien" <maxim(a)us.ibm.com> wrote:












Hi, all,

Quick reminder of the CAB call next Wednesday, April 13th @ 8a PDT. All info in link:

https://docs.google.com/document/d/1SCOlAquyUmNM-AQnekCOXiwhLs6gveTxAcduvDcW_xI/edit#heading=h.o44xhgvum2we

Remember, no more status update but rather discussions, so come ready with your questions.

Join the slack.cloudfoundry.org and join the #CAB channel for previous and future discussions.

Talk to you all next week. We'll send one more reminder on this list.

Best,

Chip, James, and Max

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


Internal/external domains

Krannich, Bernd <bernd.krannich@...>
 

Hi all,

Dr. Nic’s blog post [1] addresses a requirement that we keep hearing frequently: To have certain applications that can only be accessed from other apps running in the same Cloud Foundry deployment.

I wanted to check with the community here to see if anybody has come across this requirement as well. Is this something one should actually be advocating in terms of architecture?

Also, the solution suggested in [1] in our opinion is prone to bypassing if implemented in the described way (see top-most comment in the same blog post). Of course, one could work around that by terminating SSL at the two load balancers, introspecting the header fields there. On the other hand, this requires a much more complex “internal/external” DNS setup.

A logical next step would be to extend the requirement to only allow calls from apps being deployed into certain orgs and spaces. This can no longer be achieved by a setup “external” to CF given that apps from all types of orgs and spaces are usually deployed on the very same runners with calls being “indistinguishable” once a call has left the runner.

As alternatives, we came up with the following potential options:

1. Avoiding the notion of “private” services altogether, making sure that each application is secured by - for example - a JWT token carrying a shared secret.
2. Using messaging via a backing service instead of HTTP-based communication.
3. Recently suggested/implemented CF functionality like “container networking for applications” [2] (what’s the status on this one, actually?) and “Context Path Routing” [3] (this was discussed on the mailing list some time back, even though I’m not sure how you would actually use it to solve the above scenario). Is any of that allowing to implement the above requirement?

Any thoughts, opinions, suggestions are welcome.

Thanks in advance,
Bernd

[1] https://blog.starkandwayne.com/2014/10/31/public-and-private-microservices-on-the-same-cloud-foundry
[2] https://docs.google.com/document/d/1zQJqIEk4ldHH5iE5zat_oKIK8Ejogkgd_lySpg_oV_s/edit?usp=sharing
[3] http://berlin2015.cfsummit.com/sites/berlin2015.cfsummit.com/files//pages/files/summit-berlin.pdf

Bernd Krannich
Products & Innovation Technology
SAP SE
Dietmar-Hopp-Allee 16
69190 Walldorf Germany
mailto:bernd.krannich(a)sap.com
www.sap.com

Pflichtangaben/Mandatory Disclosure Statements: http://www.sap.com/company/legal/impressum.epx

Diese E-Mail kann Betriebs- oder Geschäftsgeheimnisse oder sonstige vertrauliche Informationen enthalten. Sollten Sie diese E-Mail irrtümlich erhalten haben, ist Ihnen eine Kenntnisnahme des Inhalts, eine Vervielfältigung oder Weitergabe der E-Mail ausdrücklich untersagt. Bitte benachrichtigen Sie uns und vernichten Sie die empfangene E-Mail. Vielen Dank.

This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.


dynamic networks

Vik R <vagcom.ben@...>
 

Are dynamic networks supported in the cf releases?

Ben


Re: Failed to upgrade cf from 212 to 233 (etcd_z1/1 failed)

王小锋 <zzuwxf at gmail.com...>
 

Many thanks !!

2016-04-07 9:55 GMT+08:00 Amit Gupta <agupta(a)pivotal.io>:

Hi there,

Since you've also asked this question on the cf-bosh mailing list:


https://lists.cloudfoundry.org/archives/list/cf-bosh(a)lists.cloudfoundry.org/thread/KPJNZSVAAOQEFLME4BERHL26DDBTDNXJ/

and have provided additional details there, I will ask the core team
responsible for etcd-release to answer your question on the cf-bosh mailing
list.

Cheers,
Amit

On Tue, Apr 5, 2016 at 12:26 AM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

I am trying to upgrade cf from version 212 to latest version 233 using
bosh, I am using spiff to generate the new manifest file. During upgrade,
it failed when updating job etcd_z1/1.

I login to etcd_z1/1 VM and found the following error message in etcd
error log.
Anyone meet the same issue? your help is greatly appreciated !!

2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/snap/00000000000000d6-0000000003712a07.snap
2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/wal/000000000000168e-0000000003712a08.wal
2016/04/05 06:46:09 rafthttp: server streaming to d7734b019d58ad5c at
term 214 has been stopped
2016/04/05 06:46:09 etcdserver: removed member d7734b019d58ad5c from
cluster f61148e36337417a
2016/04/05 06:46:10 etcdserver: reject message from removed member
d7734b019d58ad5c
2016/04/05 06:46:18 etcdserver: added member e3682036054bdae0 [
http://10.10.16.20:7001] to cluster f61148e36337417a
2016/04/05 06:46:18 sender: error posting to e3682036054bdae0: dial tcp
10.10.16.20:7001: connection refused
2016/04/05 06:46:18 sender: the connection with e3682036054bdae0 became
inactive
2016/04/05 06:46:20 sender: the connection with e3682036054bdae0 became
active
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 raft: ddea47ef60bbcdca received msgApp
rejection(lastindex: 0) from e3682036054bdae0 for index 57802518
2016/04/05 06:46:20 raft: ddea47ef60bbcdca decreased progress of
e3682036054bdae0 to [next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 raft: ddea47ef60bbcdca [firstindex: 57797981, commit:
57802541] sent snapshot[index: 57797980, term: 214] to e3682036054bdae0
[next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 rafthttp: server streaming to e3682036054bdae0 at
term 0 has been stopped
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed


Re: Failed to upgrade cf from 212 to 233 (etcd_z1/1 failed)

Amit Kumar Gupta
 

Hi there,

Since you've also asked this question on the cf-bosh mailing list:

https://lists.cloudfoundry.org/archives/list/cf-bosh(a)lists.cloudfoundry.org/thread/KPJNZSVAAOQEFLME4BERHL26DDBTDNXJ/

and have provided additional details there, I will ask the core team
responsible for etcd-release to answer your question on the cf-bosh mailing
list.

Cheers,
Amit

On Tue, Apr 5, 2016 at 12:26 AM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

I am trying to upgrade cf from version 212 to latest version 233 using
bosh, I am using spiff to generate the new manifest file. During upgrade,
it failed when updating job etcd_z1/1.

I login to etcd_z1/1 VM and found the following error message in etcd
error log.
Anyone meet the same issue? your help is greatly appreciated !!

2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/snap/00000000000000d6-0000000003712a07.snap
2016/04/05 06:36:16 filePurge: successfully removed file
/var/vcap/store/etcd/member/wal/000000000000168e-0000000003712a08.wal
2016/04/05 06:46:09 rafthttp: server streaming to d7734b019d58ad5c at term
214 has been stopped
2016/04/05 06:46:09 etcdserver: removed member d7734b019d58ad5c from
cluster f61148e36337417a
2016/04/05 06:46:10 etcdserver: reject message from removed member
d7734b019d58ad5c
2016/04/05 06:46:18 etcdserver: added member e3682036054bdae0 [
http://10.10.16.20:7001] to cluster f61148e36337417a
2016/04/05 06:46:18 sender: error posting to e3682036054bdae0: dial tcp
10.10.16.20:7001: connection refused
2016/04/05 06:46:18 sender: the connection with e3682036054bdae0 became
inactive
2016/04/05 06:46:20 sender: the connection with e3682036054bdae0 became
active
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 raft: ddea47ef60bbcdca received msgApp
rejection(lastindex: 0) from e3682036054bdae0 for index 57802518
2016/04/05 06:46:20 raft: ddea47ef60bbcdca decreased progress of
e3682036054bdae0 to [next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 raft: ddea47ef60bbcdca [firstindex: 57797981, commit:
57802541] sent snapshot[index: 57797980, term: 214] to e3682036054bdae0
[next = 1, match = 0, wait = 0]
2016/04/05 06:46:20 rafthttp: server streaming to e3682036054bdae0 at term
0 has been stopped
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed
2016/04/05 06:46:20 rafthttp: path msgapp/e3682036054bdae0 cannot be parsed
2016/04/05 06:46:20 rafthttp: path message/e3682036054bdae0 cannot be
parsed


Proposal for Service Discovery within Elastic Clusters

Amit Kumar Gupta
 

Hi all,

If you are familiar with the proposed Elastic Clusters feature narrative
(or even if you're not), I welcome you to take a look at the below proposal
for Consul-based service discovery for components within an Elastic
Clusters configuration.

https://docs.google.com/document/d/1aMpIXsPpB6O_oJsoazOl7kGajc3sRnlrflQFCtysfx8/edit

Feedback is eagerly solicited.

Links to background information on Elastic Clusters can be found within the
above doc.

Thanks,
Amit, CF Infrastructure team PM


Re: cf v233 api_z1/api_z2 failing

Ranga Rajagopalan
 

I tried v231. Unfortunately, same issue.


Re: 404 not found: Requested route ('app.domain') does not exist. -- Worked fine this morning!

Tom Sherrod <tom.sherrod@...>
 

Tracked down the issue, route_emitter instance had disappeared.
Reviewing https://docs.cloudfoundry.org/concepts/diego/diego-components.html
bosh vms of diego deployment, did not list a route_emitter. The downloaded
manifest matched the deployment manifest, which included a route_emitter.
bosh deploy started creating the missing vm, failed because route_emitter's
ip address was taken. Deleted the instance. bosh deploy completed
successfully.
Routes restored! Time to get logging in place for this environment.

Any thoughts on how/why/debugging, appreciated.

On Wed, Apr 6, 2016 at 4:10 PM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

CF 230

Developer get error above for an existing application.
I check another application, same error. I deploy a new application
successfully, same error.
bosh vms look fine. bosh cck looks fine. cf routes, yes routes are there.
No errors.

I created a new shared domain successfully. Deployed another app
specifying that domain, successful. Pull up the url and get the error in
the subject.

What happened to the existing routes? Check of the router logs show 404.
Where to look next to find out where the routes have gone??

(This is a different environment than my other issue. )

Confused,
Tom


April CAB call next week on Wednesday April 13th, 2016

Michael Maximilien
 

Hi, all,

Quick reminder of the CAB call next Wednesday, April 13th @ 8a PDT. All
info in link:

https://docs.google.com/document/d/1SCOlAquyUmNM-AQnekCOXiwhLs6gveTxAcduvDcW_xI/edit#heading=h.o44xhgvum2we


Remember, no more status update but rather discussions, so come ready with
your questions.

Join the slack.cloudfoundry.org and join the #CAB channel for previous and
future discussions.

Talk to you all next week. We'll send one more reminder on this list.

Best,

Chip, James, and Max

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone

4921 - 4940 of 9425