Re: Proposal: Improving Security for HTTP Ingress to CFAR Application Containers

Mike Youngstrom

Got it.
Not quite: in the initial form of this more secured configuration, neither CF SSH nor TCP routing will work at all, as their gateway/front/edge routers would not have the network pathway into the container that they currently recognize. The Diego team is actually done at this point with the diego-release features required to enable that initial secured configuration, and will soon contribute an experimental operations file to opt into it to cf-deployment shortly and then focus on our approach to make CF SSH work again in this secure mode. We don't expect the current TCP routing tier ever to work with this configuration, though, as the Routing team is instead focused on the Istio integration effort as a longer-term plan to replace both the HTTP and TCP routing tiers. So I'm interested in knowing whether you'd be able to enable this extra security in any of your CF environments if either (a) CF SSH doesn't work as a result (short-term obstacle, resolved in a month or so) or (b) TCP routing doesn't work (longer-term obstacle, resolved only with Istio integration).

We would probably not deploy the improved http ingress until tcp and ssh are both working.  Our priority from the improvements in http ingress have been more on the reliable side and less on the security side.  We haven't run into any NATS mis-routing requests issues for a while and we do have TCP customers.  So, we would probably prefer to keep riding our current NATS good luck streak than disrupt our TCP customers.
Great, thanks for the feedback! The CF_INSTANCE_IP environment variable will continue to contain the cell VM's IP inside the container environment, and it'll likewise still be present in the response from the CC stats endpoint, so it sounds like those network-debugging activities would be unaffected. It'd of course be great to hear the specifics of how having that cell VM IP has been useful to your developers or to you as the platform operators in resolving those network-related issues, though.

That's perfect.  We don't use the port.

Some examples where cell ip address has come in handy:
* Mostly firewall debugging.  Our firewall situation is a mess.  Lots of manual work and issues to debug where sometimes knowing the specific cell ip address can help in debugging a problem.
* Occasionally customers want to do tcpdumps from a destination server.  The ip of the cell hosting the source app instance helps reduce the tcpdump scope.  Unfortunately, in tcpdump situations the CF operations team usually gets involved anyway to grab a tcpdump from the cell side since last time I checked we couldn't take tcpdumps from in a container.  So, these scenarios are usually not very self service anyway.

Hope that helps.


Join to automatically receive all group messages.