We would probably not deploy the improved http ingress until tcp and ssh are both working.  Our priority from the improvements in http ingress have been more on the reliable side and less on the security side.  We haven't run into any NATS mis-routing requests issues for a while and we do have TCP customers.  So, we would probably prefer to keep riding our current NATS good luck streak than disrupt our TCP customers.

Great, thanks for letting me know. Just to clarify, route integrity on its own (enabled via https://github.com/cloudfoundry/cf-deployment/blob/master/operations/experimental/enable-routing-integrity.yml) is intended primarily to improve HTTP routing reliability in the face of NATS and other component failures, and is still compatible with both CF SSH and TCP routing. This proposed more secure mode is an opt-in configuration that depends on but is separate from that "basic" route integrity.

Also, we on the Diego team think that we've finally finished ironing out a minor edge case around making sure incoming requests are handled correctly when app instances are being shutdown gracefully, so we expect to work with Release Integration in the next month or so to promote that route-integrity ops file to be a stable one and then later to be the default configuration in cf-deployment.

Not quite: in the initial form of this more secured configuration, neither CF SSH nor TCP routing will work at all, as their gateway/front/edge routers would not have the network pathway into the container that they currently recognize. The Diego team is actually done at this point with the diego-release features required to enable that initial secured configuration, and will soon contribute an experimental operations file to opt into it to cf-deployment shortly and then focus on our approach to make CF SSH work again in this secure mode. We don't expect the current TCP routing tier ever to work with this configuration, though, as the Routing team is instead focused on the Istio integration effort as a longer-term plan to replace both the HTTP and TCP routing tiers. So I'm interested in knowing whether you'd be able to enable this extra security in any of your CF environments if either (a) CF SSH doesn't work as a result (short-term obstacle, resolved in a month or so) or (b) TCP routing doesn't work (longer-term obstacle, resolved only with Istio integration).

Great, thanks for the feedback! The CF_INSTANCE_IP environment variable will continue to contain the cell VM's IP inside the container environment, and it'll likewise still be present in the response from the CC stats endpoint, so it sounds like those network-debugging activities would be unaffected. It'd of course be great to hear the specifics of how having that cell VM IP has been useful to your developers or to you as the platform operators in resolving those network-related issues, though.

- This secured configuration would initially be incompatible with CF SSH, and would likely never be compatible with TCP routing, as the Routing team has focused its efforts on replacing both the Gorouter and the TCP routers with Istio-configured gateway Envoy proxies. Would those incompatibilities prohibit you as platform operators from opting into this improved security in environments where you would particularly like to enforce it?

- As part of enforcing this more secure configuration, the Diego cell components no longer map ports on their host VM directly to application ports inside the container. Each app instance currently receives the value of its host-side port in its CF_INSTANCE_PORT environment variable, though, and it is also exposed in the response from CC's app stats endpoint. For a variety of reasons (primarily the general availability of container networking and default app-security-group policies), we expect that these values are no longer useful for applications, and so we would like to deprecate them as part of this work and not to supply them in this optional, more secure configuration. Before we do so, we would like to know whether your applications, libraries, or other CF-related tools currently use this information and, if so, to what end.