Date   

sporadic connection resets between login and uaa

Sievers, Jan <jan.sievers@...>
 

Hi,

while running the CF 207 smoke and acceptance tests repeatedly, we noticed sporadic connection resets during 'cf login'
(see log snippet from login log below).

The connection reset is happening on the login machine when it's doing an HTTP POST to

http://uaa.cf.<DOMAIN>/authenticate

(via load balancer, and getting a connection reset from the load balancer).
This is happening ~ 1 out of 5 times if we run the smoke tests every 5 minutes.

We found that adding

-Dhttp.keepAlive=false

to JAVA_OPTS in /var/vcap/jobs/login/bin/login_ctl

works around the problem. Otherwise, by default there is a pool of 5 connections being kept alive and reused.

We use an F5 BigIP load balancer with 300 seconds socket idle timeout configured.

Could this be a bug with stale connections being reused by the HTTP client on the login machine?

Best Regards,
Jan


--- log snippet from login machine ---

[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- DispatcherServlet: DispatcherServlet with name 'spring' processing POST request for [/error500]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- RequestMappingHandlerMapping: Looking up handler method for path /error500
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- RequestMappingHandlerMapping: Returning handler method [public java.lang.String org.cloudfoundry.identity.uaa.login.HomeController.error500(org.springframework.ui.Model,javax.servlet.http.HttpServletRequest)]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... ERROR --- HomeController: Internal error
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://uaa.cf.<DOMAIN>/authenticate":Connection reset; nested exception is java.net.SocketException: Connection reset
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:567)
at org.springframework.security.oauth2.client.OAuth2RestTemplate.doExecute(OAuth2RestTemplate.java:128)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:512)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:454)
at org.cloudfoundry.identity.uaa.login.RemoteUaaAuthenticationManager.authenticate(RemoteUaaAuthenticationManager.java:137)
at org.cloudfoundry.identity.uaa.authentication.AuthzAuthenticationFilter.doFilter(AuthzAuthenticationFilter.java:138)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [37/1995]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
at com.sun.proxy.$Proxy45.receiveResponseHeader(Unknown Source)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:91)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:551)
... 33 more


Re: Cloudfoundry UAA / Questions

Daniel Jones
 


For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

+1 for password expiry; that'd be really handy to have.

On Sun, May 31, 2015 at 2:43 AM, Frans Thamura <frans(a)meruvian.org> wrote:

fyi, we use UAA for our social login , take a look www.merv.id

F
--
Frans Thamura (曽志胜)
Java Champion
Shadow Master and Lead Investor
Meruvian.
Integrated Hypermedia Java Solution Provider.

Mobile: +628557888699
Blog: http://blogs.mervpolis.com/roller/flatburger (id)

FB: http://www.facebook.com/meruvian
TW: http://www.twitter.com/meruvian / @meruvian
Website: http://www.meruvian.org

"We grow because we share the same belief."


On Sun, May 31, 2015 at 1:11 AM, Filip Hanik <fhanik(a)pivotal.io> wrote:
For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry
failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it
to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

Is there any pluggable mechanism for user creation in UAA that we could
use
to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one
of
these authentication mechanism, a shadow account will be created in the
UAA.
These users will only be able to authenticate against their respective
identity providers.

Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants
identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

Other than notion of Zones/Multi-tenants are there any advantages of
using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <
satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry
failures etc..?
Is there any pluggable mechanism for user creation in UAA that we could
use to create them say in AD – instead of in UAA user database?
Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?
Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants
identity provider’s HA/DR capability?
Other than notion of Zones/Multi-tenants are there any advantages of
using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Regards,

Daniel Jones
EngineerBetter.com


R: Re: Monitor all outbound connections from apps in warden

Michael Grifalconi <michael.grifalconi@...>
 

Hello, thank you for the hint!

I'd prefer to do something at application level, like a shell script to run in parallel to the application that every X seconds prints the output of netstat, as the standard output is taken as a log on CF apps. Is it possible?

(I'm really sorry and embarrassed about the spam after my email signature, this is due to my University and I can't avoid it :/ )

Thank you,
Michael

Il 29/05/15 20:06, Dieu Cao <dcao(a)pivotal.io> ha scritto:


You could set up a security group that logs all outbound connections. These are logged on the DEAs.
You would then need to correlate the warden handle with the application.

I'm working with the docs team on getting this feature properly documented.


Relevant stories where this feature was added.
[1] https://www.pivotaltracker.com/story/show/73905126

[2] https://www.pivotaltracker.com/story/show/90078842



I don't know how you would do this via buildpacks.

-Dieu
CF Runtime PM






On Fri, May 29, 2015 at 6:59 AM, Michael Grifalconi <michael.grifalconi(a)studenti.unimi.it> wrote:

Hello all, 

How can I monitor (and log) all the outbound connection made from an application?

I would like to do by editing buildpacks:

edit the buildpack to run a netstat command every 10 sec and send a log of the estabilished connections..



I would also be able to sniff the traffic, is it possible to run a tcpdump with some filters and send logs with the result? All by editing the buildpack. I think the process will not have the necessary privileges..



Any hint is appreciated!

Thank you!

Michael



(http://www.unimi.it/13084.htm?utm_source=firmaMail&utm_medium=email&utm_content=linkFirmaEmail&utm_campaign=5xmille" target="1">
La tua firma per la sua idea. Per tutti noi
Per destinare il 5x1000 all'Università degli Studi di Milano: indicare nella dichiarazione dei redditi il codice fiscale 80012650158.



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


be there for IPM

Filip Hanik
 

i may miss standup but I'll be there for IPM


Re: Cloudfoundry UAA / Questions

Frans Thamura
 

fyi, we use UAA for our social login , take a look www.merv.id

F
--
Frans Thamura (曽志胜)
Java Champion
Shadow Master and Lead Investor
Meruvian.
Integrated Hypermedia Java Solution Provider.

Mobile: +628557888699
Blog: http://blogs.mervpolis.com/roller/flatburger (id)

FB: http://www.facebook.com/meruvian
TW: http://www.twitter.com/meruvian / @meruvian
Website: http://www.meruvian.org

"We grow because we share the same belief."

On Sun, May 31, 2015 at 1:11 AM, Filip Hanik <fhanik(a)pivotal.io> wrote:
For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated retry
failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

Is there any pluggable mechanism for user creation in UAA that we could use
to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one of
these authentication mechanism, a shadow account will be created in the UAA.
These users will only be able to authenticate against their respective
identity providers.

Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any tenants
identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

Other than notion of Zones/Multi-tenants are there any advantages of using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated retry
failures etc..?
Is there any pluggable mechanism for user creation in UAA that we could
use to create them say in AD – instead of in UAA user database?
Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?
Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any tenants
identity provider’s HA/DR capability?
Other than notion of Zones/Multi-tenants are there any advantages of using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloudfoundry UAA / Questions

Filip Hanik
 

1. For users created in UAA database, are there any policies we could
apply regarding password expiry/strength of the password/lockout on
repeated retry failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

2. Is there any pluggable mechanism for user creation in UAA that we
could use to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one
of these authentication mechanism, a shadow account will be created in the
UAA. These users will only be able to authenticate against their respective
identity providers.

3. Is there any work/pocs done on UAA integration with Shibboleth
Identity provider to have federated identity? I.e. Integration with
identity providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

4. Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

5. Other than notion of Zones/Multi-tenants are there any advantages of
using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

1. For users created in UAA database, are there any policies we could
apply regarding password expiry/strength of the password/lockout on
repeated retry failures etc..?
2. Is there any pluggable mechanism for user creation in UAA that we
could use to create them say in AD – instead of in UAA user database?
3. Is there any work/pocs done on UAA integration with Shibboleth
Identity provider to have federated identity? I.e. Integration with
identity providers behind firewalls?
4. Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants identity provider’s HA/DR capability?
5. Other than notion of Zones/Multi-tenants are there any advantages
of using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Cloudfoundry UAA / Questions

Satyapal Reddy
 

Looking into using UAA and have couple of questions:

1. For users created in UAA database, are there any policies we could apply regarding password expiry/strength of the password/lockout on repeated retry failures etc..?
2. Is there any pluggable mechanism for user creation in UAA that we could use to create them say in AD – instead of in UAA user database?
3. Is there any work/pocs done on UAA integration with Shibboleth Identity provider to have federated identity? I.e. Integration with identity providers behind firewalls?
4. Is UAA HA/DR capable if the underlying user database is replicated? Basically does it boil down to underlying UAA database HA/DR and any tenants identity provider’s HA/DR capability?
5. Other than notion of Zones/Multi-tenants are there any advantages of using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya


Re: Cloud Foundry install documentation

Tom Sherrod <tom.sherrod@...>
 

Kim and James, thanks for responding and asking clarifying questions.

Regarding latest, the end manifest ended up with lucid stemcell references.
I changed them to trusty and then the deploy failed with cloud property
object issues, like range missing. It went downhill from there.

Kim, thanks for the pointer to github docs. I will start back from a clean
slate and issue/update where possible. The first example issue was name
missing from the resulting deploy manifest. I need to line up the questions
and possible edits with the correct page, along with re-testing.

Best,
Tom

On Fri, May 29, 2015 at 8:35 PM, Kim Hoffman <khoffman(a)pivotal.io> wrote:

Hi Tom,

The docs team was also not aware that this document isn't working. It
looks like there have been various tweaks to this doc over time, including
pretty recently. Do you have any more details you could share?

If you do find that you know what's wrong with the topic, we welcome pull
requests or git issues against our documentation. You can find this
specific doc here:
https://github.com/cloudfoundry/docs-deploying-cf/blob/master/openstack/install_cf_openstack.html.md

Thanks!
Kim

On Fri, May 29, 2015 at 4:28 PM, James Bayer <jbayer(a)pivotal.io> wrote:

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What version of Cloud Foundry works with

http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Gorouter throughput

Simon Johansson <simon@...>
 

Great writeup Dieu, thanks!

On Fri, May 29, 2015 at 10:50 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Yes, we recently ran some performance tests with the gorouter.
You can find the results in this doc [1] and raw results and additional
graphs [2]
Related stories in tracker [3][4]

[image: Inline image 1]
[image: Inline image 2]

[1] Google doc:
https://docs.google.com/document/d/18rrh0MNjCljd1Kt4L2mZuV2GPvO-SZZ2rk2eE769JZY/edit?usp=sharing
[2] Excel sheet:
https://docs.google.com/spreadsheets/d/1uulkoXtlV7haH0oroEKz7zQ5hxeeY8eYA3D7I7jAX9g/edit?usp=sharing

[3] https://www.pivotaltracker.com/story/show/92895056
[4] https://www.pivotaltracker.com/story/show/93362206


On Fri, May 29, 2015 at 1:14 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Is there any perf test data on the gorouter? Number of parallel
connections possible given a specific VM size would help us in our sizing
efforts.

Thanks

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloud Foundry install documentation

Kim Hoffman <khoffman@...>
 

Hi Tom,

The docs team was also not aware that this document isn't working. It looks
like there have been various tweaks to this doc over time, including pretty
recently. Do you have any more details you could share?

If you do find that you know what's wrong with the topic, we welcome pull
requests or git issues against our documentation. You can find this
specific doc here:
https://github.com/cloudfoundry/docs-deploying-cf/blob/master/openstack/install_cf_openstack.html.md

Thanks!
Kim

On Fri, May 29, 2015 at 4:28 PM, James Bayer <jbayer(a)pivotal.io> wrote:

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What version of Cloud Foundry works with
http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloud Foundry install documentation

James Bayer
 

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

What version of Cloud Foundry works with
http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Thank you,

James Bayer


Re: Gorouter throughput

Dieu Cao <dcao@...>
 

Yes, we recently ran some performance tests with the gorouter.
You can find the results in this doc [1] and raw results and additional
graphs [2]
Related stories in tracker [3][4]

[image: Inline image 1]
[image: Inline image 2]

[1] Google doc:
https://docs.google.com/document/d/18rrh0MNjCljd1Kt4L2mZuV2GPvO-SZZ2rk2eE769JZY/edit?usp=sharing
[2] Excel sheet:
https://docs.google.com/spreadsheets/d/1uulkoXtlV7haH0oroEKz7zQ5hxeeY8eYA3D7I7jAX9g/edit?usp=sharing

[3] https://www.pivotaltracker.com/story/show/92895056
[4] https://www.pivotaltracker.com/story/show/93362206


On Fri, May 29, 2015 at 1:14 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Is there any perf test data on the gorouter? Number of parallel
connections possible given a specific VM size would help us in our sizing
efforts.

Thanks

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Gorouter throughput

john mcteague <john.mcteague@...>
 

Is there any perf test data on the gorouter? Number of parallel connections
possible given a specific VM size would help us in our sizing efforts.

Thanks


Re: Setting Org Manager via API

Daniel Jones
 

Hi Dieu,

Thanks for the response. Yep, that answers my question, and addresses some
of the assumptions I'd been making.

Good to hear the changes will be coming in v3.

Have a good weekend!

On Fri, May 29, 2015 at 6:37 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Yes, we'd recommend using only the role methods on the organization
resource as a general rule.

Those roles are org specific so yes the association only exists to
facillitate granting of org roles. I'm not sure if that answers your
question?

The /v2 end points have a certain amount of difficult to change meta
magic. It's a known issue that deletes are returning 201's but it is
actually surprisingly hard to fix.
We'll address this as things move over to /v3 end points.

-Dieu

On Fri, May 29, 2015 at 1:59 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Thanks for the clarification.

Would you recommend using only the role methods on the organization
resource (as opposed to on /users/) as a general rule?

Just for my own understanding, would it be correct to say the org-user
association exists under the hood to facilitate the granting of org roles?
Does this association get used anywhere other than determining roles on
orgs? I know it sounds like a bit of a daft question, but I'd always
assumed users and orgs were linked *only* if they had a role.

Is it intentional that Remove Auditor from the Organization
<http://apidocs.cloudfoundry.org/210/organizations/remove_auditor_from_the_organization.html> returns
a "201 Created" for a delete request?

On Fri, May 29, 2015 at 12:20 AM, CF Runtime <cfruntime(a)gmail.com> wrote:

Hi Daniel,

The purpose of the Associate User with the Organization endpoint is to
add a user to an organization. This endpoint needs to be called before a
user can be added to the organization's spaces. This is why the CLI
automatically adds the user to the organization's user list.

Associate Managed Organization with the User is the same call as Associate
Manager with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_manager_with_the_organization.html>,
but it uses the relation starting from the user rather than the
organization. We don't recommend using this endpoint, as only admins have
full access to the users list, and they may not be able to look the user up
this way. We have stories in our backlog to address this.

The CF Runtime Team,
Utako && Dan


On Thu, May 28, 2015 at 12:38 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

I'm working on some automation for my client to declaratively configure
orgs and spaces across multiple Cloud Foundry instances (hopefully they'll
permit open-sourcing this).

I erroneously tried to set a user as an OrgManager by first calling Associate
Managed Organization with the User
<http://apidocs.cloudfoundry.org/210/users/associate_managed_organization_with_the_user.html>;
after getting InvalidRelation errors I used CF_TRACE to spy on the CLI, and
realised that it instead uses Associate Manager with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_manager_with_the_organization.html>
.

I've got a few questions:

- What's the purpose of the Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html> CC
API call?


- If I don't call Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html>,
what effects can I expect to see?


- Is Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html> something
that only exists for the benefit of the Pivotal console app?


- What's the correct usage of Associate Managed Organization with
the User
<http://apidocs.cloudfoundry.org/210/users/associate_managed_organization_with_the_user.html>
?




Admin
Adding a user to an org
Adding user as manager to org
Adding user to manager list

Not an admin







Many thanks in advance.

--
Regards,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Regards,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Regards,

Daniel Jones
EngineerBetter.com


Re: Is CF etcd job using the latest etcd 2.0 release?

John Wong
 

Thank you Dieu.

This is great to know.

John

On Fri, May 29, 2015 at 1:11 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

In cf-release v210 [1] we upgraded etcd to v2.0.1.
We're currently running it with 3 nodes in our CI and Prod environments.
We'll be looking at upgrading to v2.0.11 in the next couple of weeks.

-Dieu

[1] https://github.com/cloudfoundry/cf-release/releases/tag/v210

On Fri, May 29, 2015 at 8:08 AM, John Wong <gokoproject(a)gmail.com> wrote:

I remember in the older version of CF release, CF was using an older
version of etcd (v.0,4.x), which has issue running etcd in cluster mode
from time to time, and there was a discussion that we would move to v0.5.x
which has been renamed to v2.X.

Have we upgraded to the latest release?

Previous discussion:
https://groups.google.com/a/cloudfoundry.org/forum/#!searchin/vcap-dev/etcd$20Raft/vcap-dev/fQ0PD3oPh90/joW4fHBx5H0J



Thanks.

John

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Monitor all outbound connections from apps in warden

Dieu Cao <dcao@...>
 

You could set up a security group that logs all outbound connections.
These are logged on the DEAs.
You would then need to correlate the warden handle with the application.

I'm working with the docs team on getting this feature properly documented.

Relevant stories where this feature was added.
[1] https://www.pivotaltracker.com/story/show/73905126
[2] https://www.pivotaltracker.com/story/show/90078842

I don't know how you would do this via buildpacks.

-Dieu
CF Runtime PM


On Fri, May 29, 2015 at 6:59 AM, Michael Grifalconi <
michael.grifalconi(a)studenti.unimi.it> wrote:

Hello all,

How can I monitor (and log) all the outbound connection made from an
application?

I would like to do by editing buildpacks:

edit the buildpack to run a netstat command every 10 sec and send a log of
the estabilished connections..


I would also be able to sniff the traffic, is it possible to run a tcpdump
with some filters and send logs with the result? All by editing the
buildpack. I think the process will not have the necessary privileges..


Any hint is appreciated!

Thank you!

Michael
------------------------------

[image: 5xmilleUniMi]
<http://www.unimi.it/13084.htm?utm_source=firmaMail&utm_medium=email&utm_content=linkFirmaEmail&utm_campaign=5xmille>


* La tua firma per la sua idea. Per tutti noi Per destinare il 5x1000
all'Università degli Studi di Milano: indicare nella dichiarazione dei
redditi il codice fiscale 80012650158. *
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Setting Org Manager via API

Dieu Cao <dcao@...>
 

Yes, we'd recommend using only the role methods on the organization
resource as a general rule.

Those roles are org specific so yes the association only exists to
facillitate granting of org roles. I'm not sure if that answers your
question?

The /v2 end points have a certain amount of difficult to change meta
magic. It's a known issue that deletes are returning 201's but it is
actually surprisingly hard to fix.
We'll address this as things move over to /v3 end points.

-Dieu

On Fri, May 29, 2015 at 1:59 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Thanks for the clarification.

Would you recommend using only the role methods on the organization
resource (as opposed to on /users/) as a general rule?

Just for my own understanding, would it be correct to say the org-user
association exists under the hood to facilitate the granting of org roles?
Does this association get used anywhere other than determining roles on
orgs? I know it sounds like a bit of a daft question, but I'd always
assumed users and orgs were linked *only* if they had a role.

Is it intentional that Remove Auditor from the Organization
<http://apidocs.cloudfoundry.org/210/organizations/remove_auditor_from_the_organization.html> returns
a "201 Created" for a delete request?

On Fri, May 29, 2015 at 12:20 AM, CF Runtime <cfruntime(a)gmail.com> wrote:

Hi Daniel,

The purpose of the Associate User with the Organization endpoint is to
add a user to an organization. This endpoint needs to be called before a
user can be added to the organization's spaces. This is why the CLI
automatically adds the user to the organization's user list.

Associate Managed Organization with the User is the same call as Associate
Manager with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_manager_with_the_organization.html>,
but it uses the relation starting from the user rather than the
organization. We don't recommend using this endpoint, as only admins have
full access to the users list, and they may not be able to look the user up
this way. We have stories in our backlog to address this.

The CF Runtime Team,
Utako && Dan


On Thu, May 28, 2015 at 12:38 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

I'm working on some automation for my client to declaratively configure
orgs and spaces across multiple Cloud Foundry instances (hopefully they'll
permit open-sourcing this).

I erroneously tried to set a user as an OrgManager by first calling Associate
Managed Organization with the User
<http://apidocs.cloudfoundry.org/210/users/associate_managed_organization_with_the_user.html>;
after getting InvalidRelation errors I used CF_TRACE to spy on the CLI, and
realised that it instead uses Associate Manager with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_manager_with_the_organization.html>
.

I've got a few questions:

- What's the purpose of the Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html> CC
API call?


- If I don't call Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html>,
what effects can I expect to see?


- Is Associate User with the Organization
<http://apidocs.cloudfoundry.org/210/organizations/associate_user_with_the_organization.html> something
that only exists for the benefit of the Pivotal console app?


- What's the correct usage of Associate Managed Organization with
the User
<http://apidocs.cloudfoundry.org/210/users/associate_managed_organization_with_the_user.html>
?




Admin
Adding a user to an org
Adding user as manager to org
Adding user to manager list

Not an admin







Many thanks in advance.

--
Regards,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Regards,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Is CF etcd job using the latest etcd 2.0 release?

Dieu Cao <dcao@...>
 

In cf-release v210 [1] we upgraded etcd to v2.0.1.
We're currently running it with 3 nodes in our CI and Prod environments.
We'll be looking at upgrading to v2.0.11 in the next couple of weeks.

-Dieu

[1] https://github.com/cloudfoundry/cf-release/releases/tag/v210

On Fri, May 29, 2015 at 8:08 AM, John Wong <gokoproject(a)gmail.com> wrote:

I remember in the older version of CF release, CF was using an older
version of etcd (v.0,4.x), which has issue running etcd in cluster mode
from time to time, and there was a discussion that we would move to v0.5.x
which has been renamed to v2.X.

Have we upgraded to the latest release?

Previous discussion:
https://groups.google.com/a/cloudfoundry.org/forum/#!searchin/vcap-dev/etcd$20Raft/vcap-dev/fQ0PD3oPh90/joW4fHBx5H0J



Thanks.

John

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Is CF etcd job using the latest etcd 2.0 release?

John Wong
 

I remember in the older version of CF release, CF was using an older
version of etcd (v.0,4.x), which has issue running etcd in cluster mode
from time to time, and there was a discussion that we would move to v0.5.x
which has been renamed to v2.X.

Have we upgraded to the latest release?

Previous discussion:
https://groups.google.com/a/cloudfoundry.org/forum/#!searchin/vcap-dev/etcd$20Raft/vcap-dev/fQ0PD3oPh90/joW4fHBx5H0J



Thanks.

John


Re: api and api_worker jobs fail to bosh update, but monit start OK

Guillaume Berche
 

For similar further diagnostics, I've updated docs with a working way to be
able to invoke steno-prettify to get cc_ng logs as timestamps, cf
https://github.com/cloudfoundry/docs-running-cf/pull/7/files

BTW, has anyone tried a log conversion for cc_ng logs collected into a
syslog endpoint (such as a logstash filter that invokes the steno-prettify
command or an equivalent ) ?

Besides, I ran into the bin/console pry shell that seems to open
interesting diagnostic/db interaction opportunities. If there are example
of common cases using it, I'd be happy to test and report them in cf docs,
to refine [2]

[1] https://github.com/cloudfoundry/docs-running-cf/pull/7/files
[2]
https://github.com/Orange-OpenSource/docs-running-cf/commit/cc46acc26eff50988a652acc3582c7573d6c8c8e


On Thu, May 28, 2015 at 7:57 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

We discovered this fix out of sheer luck. So, no help there sorry. :)

Mike

On Thu, May 28, 2015 at 1:56 AM, Guillaume Berche <bercheg(a)gmail.com>
wrote:

Thanks a lot Mike and Dieu. Indeed moving the nfs_mounter last seemed
indeed to fix the issue in v207. If ever this reproduces on master and can
help, I submitted https://github.com/cloudfoundry/cf-release/pull/689
against develop branch.

Out of curiosity, and for improving my next diagnostic task, how was the
root cause diagnosed? I was not observing any faulty output traces into
jobs outputs:
[...]/cloud_controller_worker_ctl.log, /var/vcap/sys/log/cloud_controller_ng_ctl.err.log
or [...]/cloud_controller_ng/cloud_controller_ng.log