Date   

Re: What ports will be needed to support hm and loggregator

MaggieMeng
 

I deployed my CF on vshpere server.

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Lev Berman
Sent: 2015年6月2日 18:30
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] What ports will be needed to support hm and loggregator

You have posted your Application Security Groups - http://docs.pivotal.io/pivotalcf/adminguide/app-sec-groups.html. This groups are created and managed by Cloud Foundry.
But the issue here is with security groups configured in your infrastructure - AWS, OpenStack, etc. Which one is your CF deployed on?

On Tue, Jun 2, 2015 at 1:23 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote:
Hi, Lev

Would you please let me know what exactly I should add to my security group? Following are the current configuration.

- name: public_networks
rules:
- protocol: all
destination: 0.0.0.0-9.255.255.255
- protocol: all
destination: 11.0.0.0-169.253.255.255
- protocol: all
destination: 169.255.0.0-172.15.255.255
- protocol: all
destination: 172.32.0.0-192.167.255.255
- protocol: all
destination: 192.169.0.0-255.255.255.255
- name: dns
rules:
- protocol: tcp
destination: 0.0.0.0/0<http://0.0.0.0/0>
ports: '53'
- protocol: udp
destination: 0.0.0.0/0<http://0.0.0.0/0>
ports: '53'
default_running_security_groups:
- public_networks
- dns
default_staging_security_groups:
- public_networks
- dns

Thanks,
Maggie

From: cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org> [mailto:cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org>] On Behalf Of Lev Berman
Sent: 2015年6月2日 18:16
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] What ports will be needed to support hm and loggregator

Hi,
At least for loggregator to successflly talk to metron agents, you need to add a rule to a security group for your private subnet allowing the ingress UDP traffic through ports 3456 and 3457 from all hosts (0.0.0.0/0<http://0.0.0.0/0>). See more about security group rules needed for CF here - http://docs.cloudfoundry.org/deploying/common/security_groups.html.




On Tue, Jun 2, 2015 at 1:04 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote:
Hi,

I am updating my cf env from 172 to 197. But I found some issues after upgrade is done. I couldn’t get the correct running application instance number:

CF_TRACE=true cf apps

"running_instances": -1,

application started ?/3

Another issue is I can’t get log information from loggregator. “cf logs” showed nothing after I restarted my application.

I think this may be related to our firewall configuration. Because in another environment where no firewall is configured, hm and loggregator work perfectly well. We have firewalls for deas, routers and all other components separately(three firewalls). So would anyone please tell me what ports we should open for deas, routers or other components?

Thanks,
Maggie



--
Lev Berman
Altoros - Cloud Foundry deployment, training and integration

Github: https://github.com/ldmberman



--
Lev Berman
Altoros - Cloud Foundry deployment, training and integration

Github: https://github.com/ldmberman


Re: What ports will be needed to support hm and loggregator

Lev Berman <lev.berman@...>
 

You have posted your Application Security Groups -
http://docs.pivotal.io/pivotalcf/adminguide/app-sec-groups.html. This
groups are created and managed by Cloud Foundry.

But the issue here is with security groups configured in your
infrastructure - AWS, OpenStack, etc. Which one is your CF deployed on?

On Tue, Jun 2, 2015 at 1:23 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:

Hi, Lev



Would you please let me know what exactly I should add to my security
group? Following are the current configuration.



- name: public_networks

rules:

- protocol: all

destination: 0.0.0.0-9.255.255.255

- protocol: all

destination: 11.0.0.0-169.253.255.255

- protocol: all

destination: 169.255.0.0-172.15.255.255

- protocol: all

destination: 172.32.0.0-192.167.255.255

- protocol: all

destination: 192.169.0.0-255.255.255.255

- name: dns

rules:

- protocol: tcp

destination: 0.0.0.0/0

ports: '53'

- protocol: udp

destination: 0.0.0.0/0

ports: '53'

default_running_security_groups:

- public_networks

- dns

default_staging_security_groups:

- public_networks

- dns



Thanks,

Maggie



*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto:
cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Lev Berman
*Sent:* 2015年6月2日 18:16
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* Re: [cf-dev] What ports will be needed to support hm and
loggregator



Hi,

At least for loggregator to successflly talk to metron agents, you need to
add a rule to a security group for your private subnet allowing the ingress
UDP traffic through ports 3456 and 3457 from all hosts (0.0.0.0/0). See
more about security group rules needed for CF here -
http://docs.cloudfoundry.org/deploying/common/security_groups.html.






On Tue, Jun 2, 2015 at 1:04 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi,



I am updating my cf env from 172 to 197. But I found some issues after
upgrade is done. I couldn’t get the correct running application instance
number:



CF_TRACE=true cf apps



"running_instances": -1,



application started ?/3



Another issue is I can’t get log information from loggregator. “cf logs”
showed nothing after I restarted my application.



I think this may be related to our firewall configuration. Because in
another environment where no firewall is configured, hm and loggregator
work perfectly well. We have firewalls for deas, routers and all other
components separately(three firewalls). So would anyone please tell me what
ports we should open for deas, routers or other components?



Thanks,

Maggie





--

Lev Berman

Altoros - Cloud Foundry deployment, training and integration



Github*: *https://github.com/ldmberman
--
Lev Berman

Altoros - Cloud Foundry deployment, training and integration

Github
*: https://github.com/ldmberman <https://github.com/ldmberman>*


Re: What ports will be needed to support hm and loggregator

MaggieMeng
 

Hi, Lev

Would you please let me know what exactly I should add to my security group? Following are the current configuration.

- name: public_networks
rules:
- protocol: all
destination: 0.0.0.0-9.255.255.255
- protocol: all
destination: 11.0.0.0-169.253.255.255
- protocol: all
destination: 169.255.0.0-172.15.255.255
- protocol: all
destination: 172.32.0.0-192.167.255.255
- protocol: all
destination: 192.169.0.0-255.255.255.255
- name: dns
rules:
- protocol: tcp
destination: 0.0.0.0/0
ports: '53'
- protocol: udp
destination: 0.0.0.0/0
ports: '53'
default_running_security_groups:
- public_networks
- dns
default_staging_security_groups:
- public_networks
- dns

Thanks,
Maggie

From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Lev Berman
Sent: 2015年6月2日 18:16
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] What ports will be needed to support hm and loggregator

Hi,
At least for loggregator to successflly talk to metron agents, you need to add a rule to a security group for your private subnet allowing the ingress UDP traffic through ports 3456 and 3457 from all hosts (0.0.0.0/0<http://0.0.0.0/0>). See more about security group rules needed for CF here - http://docs.cloudfoundry.org/deploying/common/security_groups.html.





On Tue, Jun 2, 2015 at 1:04 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote:
Hi,

I am updating my cf env from 172 to 197. But I found some issues after upgrade is done. I couldn’t get the correct running application instance number:

CF_TRACE=true cf apps

"running_instances": -1,

application started ?/3

Another issue is I can’t get log information from loggregator. “cf logs” showed nothing after I restarted my application.

I think this may be related to our firewall configuration. Because in another environment where no firewall is configured, hm and loggregator work perfectly well. We have firewalls for deas, routers and all other components separately(three firewalls). So would anyone please tell me what ports we should open for deas, routers or other components?

Thanks,
Maggie



--
Lev Berman
Altoros - Cloud Foundry deployment, training and integration

Github: https://github.com/ldmberman


Re: What ports will be needed to support hm and loggregator

Lev Berman <lev.berman@...>
 

Hi,

At least for loggregator to successflly talk to metron agents, you need to
add a rule to a security group for your private subnet allowing the ingress
UDP traffic through ports 3456 and 3457 from all hosts (0.0.0.0/0). See
more about security group rules needed for CF here -
http://docs.cloudfoundry.org/deploying/common/security_groups.html.

On Tue, Jun 2, 2015 at 1:04 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:

Hi,



I am updating my cf env from 172 to 197. But I found some issues after
upgrade is done. I couldn’t get the correct running application instance
number:



CF_TRACE=true cf apps



"running_instances": -1,



application started ?/3



Another issue is I can’t get log information from loggregator. “cf logs”
showed nothing after I restarted my application.



I think this may be related to our firewall configuration. Because in
another environment where no firewall is configured, hm and loggregator
work perfectly well. We have firewalls for deas, routers and all other
components separately(three firewalls). So would anyone please tell me what
ports we should open for deas, routers or other components?



Thanks,

Maggie
--
Lev Berman

Altoros - Cloud Foundry deployment, training and integration

Github
*: https://github.com/ldmberman <https://github.com/ldmberman>*


What ports will be needed to support hm and loggregator

MaggieMeng
 

Hi,

I am updating my cf env from 172 to 197. But I found some issues after upgrade is done. I couldn't get the correct running application instance number:

CF_TRACE=true cf apps
...
"running_instances": -1,
...
application started ?/3

Another issue is I can't get log information from loggregator. "cf logs" showed nothing after I restarted my application.

I think this may be related to our firewall configuration. Because in another environment where no firewall is configured, hm and loggregator work perfectly well. We have firewalls for deas, routers and all other components separately(three firewalls). So would anyone please tell me what ports we should open for deas, routers or other components?

Thanks,
Maggie


Re: sporadic connection resets between login and uaa

Sievers, Jan <jan.sievers@...>
 

Am I right this problem is obsolete since the login-uaa merge in CF 208 [1]?

Regards,
Jan

[1] http://lists.cloudfoundry.org/pipermail/cf-dev/2015-May/000087.html

-----Original Message-----
From: Sievers, Jan
Sent: Montag, 1. Juni 2015 11:31
To: 'cf-dev(a)lists.cloudfoundry.org'
Subject: sporadic connection resets between login and uaa

Hi,

while running the CF 207 smoke and acceptance tests repeatedly, we noticed
sporadic connection resets during 'cf login'
(see log snippet from login log below).

The connection reset is happening on the login machine when it's doing an
HTTP POST to

http://uaa.cf.<DOMAIN>/authenticate

(via load balancer, and getting a connection reset from the load balancer).
This is happening ~ 1 out of 5 times if we run the smoke tests every 5
minutes.

We found that adding

-Dhttp.keepAlive=false

to JAVA_OPTS in /var/vcap/jobs/login/bin/login_ctl

works around the problem. Otherwise, by default there is a pool of 5
connections being kept alive and reused.

We use an F5 BigIP load balancer with 300 seconds socket idle timeout
configured.

Could this be a bug with stale connections being reused by the HTTP client on
the login machine?

Best Regards,
Jan


--- log snippet from login machine ---

[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG ---
DispatcherServlet: DispatcherServlet with name 'spring' processing POST
request for [/error500]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG ---
RequestMappingHandlerMapping: Looking up handler method for path /error500
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG ---
RequestMappingHandlerMapping: Returning handler method [public
java.lang.String
org.cloudfoundry.identity.uaa.login.HomeController.error500(org.springframewo
rk.ui.Model,javax.servlet.http.HttpServletRequest)]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... ERROR ---
HomeController: Internal error
org.springframework.web.client.ResourceAccessException: I/O error on POST
request for "http://uaa.cf.<DOMAIN>/authenticate":Connection reset; nested
exception is java.net.SocketException: Connection reset
at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:567)
at
org.springframework.security.oauth2.client.OAuth2RestTemplate.doExecute(OAuth
2RestTemplate.java:128)
at
org.springframework.web.client.RestTemplate.execute(RestTemplate.java:512)
at
org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:454)
at
org.cloudfoundry.identity.uaa.login.RemoteUaaAuthenticationManager.authentica
te(RemoteUaaAuthenticationManager.java:137)
at
org.cloudfoundry.identity.uaa.authentication.AuthzAuthenticationFilter.doFilt
er(AuthzAuthenticationFilter.java:138)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter
(FilterChainProxy.java:342)
at
org.springframework.security.web.context.request.async.WebAsyncManagerIntegra
tionFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFi
lter.java:107)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter
(FilterChainProxy.java:342)
at
org.springframework.security.web.context.SecurityContextPersistenceFilter.doF
ilter(SecurityContextPersistenceFilter.java:87)
at
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter
(FilterChainProxy.java:342)
at
org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChai
nProxy.java:192)
at
org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.j
ava:160)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(Delegatin
gFilterProxy.java:344)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilte
rProxy.java:261)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationF
ilterChain.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCha
in.java:208)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.jav
a:220)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.jav
a:122)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.
java:501)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at
org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
116)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:
116) [37/1995]
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Proces
sor.java:1070)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(Abstract
Protocol.java:611)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:3
14)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:114
5)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:61
5)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.jav
a:61)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferI
mpl.java:136)
at
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferI
mpl.java:152)
at
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImp
l.java:270)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResp
onseParser.java:140)
at
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResp
onseParser.java:57)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.jav
a:260)
at
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(Defau
ltBHttpClientConnection.java:161)
at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.
java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
at com.sun.proxy.$Proxy45.receiveResponseHeader(Unknown Source)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExe
cutor.java:271)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java
:123)
at
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254
)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.j
ava:186)
at
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.j
ava:82)
at
org.springframework.http.client.HttpComponentsClientHttpRequest.executeIntern
al(HttpComponentsClientHttpRequest.java:91)
at
org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInt
ernal(AbstractBufferingClientHttpRequest.java:48)
at
org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractCli
entHttpRequest.java:53)
at
org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:551)
... 33 more


Memory Leak in doppler and metron_agent?

libnux <libnux.me@...>
 

Hi,

I'm running a v205 CF deployment with ~350 application instances.

I just found that the doppler process on the doppler node and the
metron_agent process on the gorouter node used too much memory(90.9% and
13.3 %), as below.

Acutually, the memory usage of metron_agent was also >90%, and I
restarted it yesterday, and just after 14 hours, the usage went up to
13.3%.


There are two doppler nodes and three gorouter nodes.

Process 'doppler'
============
status Running
monitoring status Monitored
pid 21648
parent pid 1
uptime 2d 11h 10m
children 0
memory kilobytes 14948028
memory kilobytes total 14948028
memory percent 90.9%
memory percent total 90.9%
cpu percent 12.5%
cpu percent total 12.5%

Process 'metron_agent'
==================
status Running
monitoring status Monitored
pid 28995
parent pid 1
uptime 14h 10m
children 2
memory kilobytes 2195608
memory kilobytes total 2195608
memory percent 13.3%
memory percent total 13.3%
cpu percent 6.8%
cpu percent total 6.8%


Are you having problems upgrading cf-release postgres?

CF Runtime
 

Good afternoon,

Last week, we merged a branch into cf-release that upgraded the postgres
instance if you are running one.

We saw this fail on two of our environments and reverted these changes on
Friday. We have since committed a fix and pushed that, too.

If you are using the develop branch of cf-release and your database did not
successfully upgrade, you should get the latest version, and do the
following on your postgres VM:

monit stop postgres
rm -rf /var/vcap/store/postgres-9.4.2
rm /var/vcap/store/FLAG_POSTGRES_UPGRADE

At that point you should be able to deploy the new version and it will
upgrade cleanly. Please let us know if you have any problems upgrading.


Dan Wendorf and Utako,
CF Runtime Team


sporadic connection resets between login and uaa

Sievers, Jan <jan.sievers@...>
 

Hi,

while running the CF 207 smoke and acceptance tests repeatedly, we noticed sporadic connection resets during 'cf login'
(see log snippet from login log below).

The connection reset is happening on the login machine when it's doing an HTTP POST to

http://uaa.cf.<DOMAIN>/authenticate

(via load balancer, and getting a connection reset from the load balancer).
This is happening ~ 1 out of 5 times if we run the smoke tests every 5 minutes.

We found that adding

-Dhttp.keepAlive=false

to JAVA_OPTS in /var/vcap/jobs/login/bin/login_ctl

works around the problem. Otherwise, by default there is a pool of 5 connections being kept alive and reused.

We use an F5 BigIP load balancer with 300 seconds socket idle timeout configured.

Could this be a bug with stale connections being reused by the HTTP client on the login machine?

Best Regards,
Jan


--- log snippet from login machine ---

[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- DispatcherServlet: DispatcherServlet with name 'spring' processing POST request for [/error500]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- RequestMappingHandlerMapping: Looking up handler method for path /error500
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... DEBUG --- RequestMappingHandlerMapping: Returning handler method [public java.lang.String org.cloudfoundry.identity.uaa.login.HomeController.error500(org.springframework.ui.Model,javax.servlet.http.HttpServletRequest)]
[2015-05-08 08:07:52.787] login - 9054 [http-bio-8080-exec-2] .... ERROR --- HomeController: Internal error
org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://uaa.cf.<DOMAIN>/authenticate":Connection reset; nested exception is java.net.SocketException: Connection reset
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:567)
at org.springframework.security.oauth2.client.OAuth2RestTemplate.doExecute(OAuth2RestTemplate.java:128)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:512)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:454)
at org.cloudfoundry.identity.uaa.login.RemoteUaaAuthenticationManager.authenticate(RemoteUaaAuthenticationManager.java:137)
at org.cloudfoundry.identity.uaa.authentication.AuthzAuthenticationFilter.doFilter(AuthzAuthenticationFilter.java:138)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:50)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:344)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:261)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:501)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:683)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) [37/1995]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1070)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
at sun.reflect.GeneratedMethodAccessor121.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
at com.sun.proxy.$Proxy45.receiveResponseHeader(Unknown Source)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.springframework.http.client.HttpComponentsClientHttpRequest.executeInternal(HttpComponentsClientHttpRequest.java:91)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:551)
... 33 more


Re: Cloudfoundry UAA / Questions

Daniel Jones
 


For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

+1 for password expiry; that'd be really handy to have.

On Sun, May 31, 2015 at 2:43 AM, Frans Thamura <frans(a)meruvian.org> wrote:

fyi, we use UAA for our social login , take a look www.merv.id

F
--
Frans Thamura (曽志胜)
Java Champion
Shadow Master and Lead Investor
Meruvian.
Integrated Hypermedia Java Solution Provider.

Mobile: +628557888699
Blog: http://blogs.mervpolis.com/roller/flatburger (id)

FB: http://www.facebook.com/meruvian
TW: http://www.twitter.com/meruvian / @meruvian
Website: http://www.meruvian.org

"We grow because we share the same belief."


On Sun, May 31, 2015 at 1:11 AM, Filip Hanik <fhanik(a)pivotal.io> wrote:
For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry
failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it
to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

Is there any pluggable mechanism for user creation in UAA that we could
use
to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one
of
these authentication mechanism, a shadow account will be created in the
UAA.
These users will only be able to authenticate against their respective
identity providers.

Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants
identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

Other than notion of Zones/Multi-tenants are there any advantages of
using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <
satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated
retry
failures etc..?
Is there any pluggable mechanism for user creation in UAA that we could
use to create them say in AD – instead of in UAA user database?
Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?
Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants
identity provider’s HA/DR capability?
Other than notion of Zones/Multi-tenants are there any advantages of
using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Regards,

Daniel Jones
EngineerBetter.com


R: Re: Monitor all outbound connections from apps in warden

Michael Grifalconi <michael.grifalconi@...>
 

Hello, thank you for the hint!

I'd prefer to do something at application level, like a shell script to run in parallel to the application that every X seconds prints the output of netstat, as the standard output is taken as a log on CF apps. Is it possible?

(I'm really sorry and embarrassed about the spam after my email signature, this is due to my University and I can't avoid it :/ )

Thank you,
Michael

Il 29/05/15 20:06, Dieu Cao <dcao(a)pivotal.io> ha scritto:


You could set up a security group that logs all outbound connections. These are logged on the DEAs.
You would then need to correlate the warden handle with the application.

I'm working with the docs team on getting this feature properly documented.


Relevant stories where this feature was added.
[1] https://www.pivotaltracker.com/story/show/73905126

[2] https://www.pivotaltracker.com/story/show/90078842



I don't know how you would do this via buildpacks.

-Dieu
CF Runtime PM






On Fri, May 29, 2015 at 6:59 AM, Michael Grifalconi <michael.grifalconi(a)studenti.unimi.it> wrote:

Hello all, 

How can I monitor (and log) all the outbound connection made from an application?

I would like to do by editing buildpacks:

edit the buildpack to run a netstat command every 10 sec and send a log of the estabilished connections..



I would also be able to sniff the traffic, is it possible to run a tcpdump with some filters and send logs with the result? All by editing the buildpack. I think the process will not have the necessary privileges..



Any hint is appreciated!

Thank you!

Michael



(http://www.unimi.it/13084.htm?utm_source=firmaMail&utm_medium=email&utm_content=linkFirmaEmail&utm_campaign=5xmille" target="1">
La tua firma per la sua idea. Per tutti noi
Per destinare il 5x1000 all'Università degli Studi di Milano: indicare nella dichiarazione dei redditi il codice fiscale 80012650158.



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


be there for IPM

Filip Hanik
 

i may miss standup but I'll be there for IPM


Re: Cloudfoundry UAA / Questions

Frans Thamura
 

fyi, we use UAA for our social login , take a look www.merv.id

F
--
Frans Thamura (曽志胜)
Java Champion
Shadow Master and Lead Investor
Meruvian.
Integrated Hypermedia Java Solution Provider.

Mobile: +628557888699
Blog: http://blogs.mervpolis.com/roller/flatburger (id)

FB: http://www.facebook.com/meruvian
TW: http://www.twitter.com/meruvian / @meruvian
Website: http://www.meruvian.org

"We grow because we share the same belief."

On Sun, May 31, 2015 at 1:11 AM, Filip Hanik <fhanik(a)pivotal.io> wrote:
For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated retry
failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

Is there any pluggable mechanism for user creation in UAA that we could use
to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one of
these authentication mechanism, a shadow account will be created in the UAA.
These users will only be able to authenticate against their respective
identity providers.

Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any tenants
identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

Other than notion of Zones/Multi-tenants are there any advantages of using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

For users created in UAA database, are there any policies we could apply
regarding password expiry/strength of the password/lockout on repeated retry
failures etc..?
Is there any pluggable mechanism for user creation in UAA that we could
use to create them say in AD – instead of in UAA user database?
Is there any work/pocs done on UAA integration with Shibboleth Identity
provider to have federated identity? I.e. Integration with identity
providers behind firewalls?
Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any tenants
identity provider’s HA/DR capability?
Other than notion of Zones/Multi-tenants are there any advantages of using
UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloudfoundry UAA / Questions

Filip Hanik
 

1. For users created in UAA database, are there any policies we could
apply regarding password expiry/strength of the password/lockout on
repeated retry failures etc..?

Currently there is a password score calculator. There is a feature being
implemented for a more clearly configurable password strength. Expect it to
be in the next release. Lockout is implemented, and will also be
configurable in the next release.

2. Is there any pluggable mechanism for user creation in UAA that we
could use to create them say in AD – instead of in UAA user database?

The UAA can integrate with LDAP (AD) or with SAML IDPs. When you use one
of these authentication mechanism, a shadow account will be created in the
UAA. These users will only be able to authenticate against their respective
identity providers.

3. Is there any work/pocs done on UAA integration with Shibboleth
Identity provider to have federated identity? I.e. Integration with
identity providers behind firewalls?

I believe Shibboleth is a SAML v2 provider, so it should be able to be
configured like any other provider.

4. Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants identity provider’s HA/DR capability?

Yes, that is how we run our UAA in production. It's backed by a HA/DR
database.

5. Other than notion of Zones/Multi-tenants are there any advantages of
using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Yes, most of the work has already been done for you.


On Sat, May 30, 2015 at 11:58 AM, Reddy, Satyapal <satyapal.reddy(a)emc.com>
wrote:

Looking into using UAA and have couple of questions:

1. For users created in UAA database, are there any policies we could
apply regarding password expiry/strength of the password/lockout on
repeated retry failures etc..?
2. Is there any pluggable mechanism for user creation in UAA that we
could use to create them say in AD – instead of in UAA user database?
3. Is there any work/pocs done on UAA integration with Shibboleth
Identity provider to have federated identity? I.e. Integration with
identity providers behind firewalls?
4. Is UAA HA/DR capable if the underlying user database is replicated?
Basically does it boil down to underlying UAA database HA/DR and any
tenants identity provider’s HA/DR capability?
5. Other than notion of Zones/Multi-tenants are there any advantages
of using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Cloudfoundry UAA / Questions

Satyapal Reddy
 

Looking into using UAA and have couple of questions:

1. For users created in UAA database, are there any policies we could apply regarding password expiry/strength of the password/lockout on repeated retry failures etc..?
2. Is there any pluggable mechanism for user creation in UAA that we could use to create them say in AD – instead of in UAA user database?
3. Is there any work/pocs done on UAA integration with Shibboleth Identity provider to have federated identity? I.e. Integration with identity providers behind firewalls?
4. Is UAA HA/DR capable if the underlying user database is replicated? Basically does it boil down to underlying UAA database HA/DR and any tenants identity provider’s HA/DR capability?
5. Other than notion of Zones/Multi-tenants are there any advantages of using UAA over plain Spring Security OAuth2/Spring Cloud Security?

Thanks
Satya


Re: Cloud Foundry install documentation

Tom Sherrod <tom.sherrod@...>
 

Kim and James, thanks for responding and asking clarifying questions.

Regarding latest, the end manifest ended up with lucid stemcell references.
I changed them to trusty and then the deploy failed with cloud property
object issues, like range missing. It went downhill from there.

Kim, thanks for the pointer to github docs. I will start back from a clean
slate and issue/update where possible. The first example issue was name
missing from the resulting deploy manifest. I need to line up the questions
and possible edits with the correct page, along with re-testing.

Best,
Tom

On Fri, May 29, 2015 at 8:35 PM, Kim Hoffman <khoffman(a)pivotal.io> wrote:

Hi Tom,

The docs team was also not aware that this document isn't working. It
looks like there have been various tweaks to this doc over time, including
pretty recently. Do you have any more details you could share?

If you do find that you know what's wrong with the topic, we welcome pull
requests or git issues against our documentation. You can find this
specific doc here:
https://github.com/cloudfoundry/docs-deploying-cf/blob/master/openstack/install_cf_openstack.html.md

Thanks!
Kim

On Fri, May 29, 2015 at 4:28 PM, James Bayer <jbayer(a)pivotal.io> wrote:

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What version of Cloud Foundry works with

http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Gorouter throughput

Simon Johansson <simon@...>
 

Great writeup Dieu, thanks!

On Fri, May 29, 2015 at 10:50 PM, Dieu Cao <dcao(a)pivotal.io> wrote:

Yes, we recently ran some performance tests with the gorouter.
You can find the results in this doc [1] and raw results and additional
graphs [2]
Related stories in tracker [3][4]

[image: Inline image 1]
[image: Inline image 2]

[1] Google doc:
https://docs.google.com/document/d/18rrh0MNjCljd1Kt4L2mZuV2GPvO-SZZ2rk2eE769JZY/edit?usp=sharing
[2] Excel sheet:
https://docs.google.com/spreadsheets/d/1uulkoXtlV7haH0oroEKz7zQ5hxeeY8eYA3D7I7jAX9g/edit?usp=sharing

[3] https://www.pivotaltracker.com/story/show/92895056
[4] https://www.pivotaltracker.com/story/show/93362206


On Fri, May 29, 2015 at 1:14 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Is there any perf test data on the gorouter? Number of parallel
connections possible given a specific VM size would help us in our sizing
efforts.

Thanks

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloud Foundry install documentation

Kim Hoffman <khoffman@...>
 

Hi Tom,

The docs team was also not aware that this document isn't working. It looks
like there have been various tweaks to this doc over time, including pretty
recently. Do you have any more details you could share?

If you do find that you know what's wrong with the topic, we welcome pull
requests or git issues against our documentation. You can find this
specific doc here:
https://github.com/cloudfoundry/docs-deploying-cf/blob/master/openstack/install_cf_openstack.html.md

Thanks!
Kim

On Fri, May 29, 2015 at 4:28 PM, James Bayer <jbayer(a)pivotal.io> wrote:

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

What version of Cloud Foundry works with
http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Cloud Foundry install documentation

James Bayer
 

i was not aware that the documentation was not working for the latest
cf-release versions. did you find that the documentation was out of date?

On Fri, May 29, 2015 at 7:30 AM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

What version of Cloud Foundry works with
http://docs.cloudfoundry.org/deploying/openstack/install_cf_openstack.html

Any pointers for the latest versions?
How can we get this doc updated?

Tom

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Thank you,

James Bayer


Re: Gorouter throughput

Dieu Cao <dcao@...>
 

Yes, we recently ran some performance tests with the gorouter.
You can find the results in this doc [1] and raw results and additional
graphs [2]
Related stories in tracker [3][4]

[image: Inline image 1]
[image: Inline image 2]

[1] Google doc:
https://docs.google.com/document/d/18rrh0MNjCljd1Kt4L2mZuV2GPvO-SZZ2rk2eE769JZY/edit?usp=sharing
[2] Excel sheet:
https://docs.google.com/spreadsheets/d/1uulkoXtlV7haH0oroEKz7zQ5hxeeY8eYA3D7I7jAX9g/edit?usp=sharing

[3] https://www.pivotaltracker.com/story/show/92895056
[4] https://www.pivotaltracker.com/story/show/93362206


On Fri, May 29, 2015 at 1:14 PM, john mcteague <john.mcteague(a)gmail.com>
wrote:

Is there any perf test data on the gorouter? Number of parallel
connections possible given a specific VM size would help us in our sizing
efforts.

Thanks

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

9121 - 9140 of 9409