Hi Cloud Foundry,

We've had a recurring issue brought to our attention regarding a Cloud
Foundry deployment using a system_domain that is in the list of
app_domains. When the system domain is in the list of app domains, a Shared
Domain is created for the system domain. This is problematic because it
allows users to create routes on the system domain, see this [1] recent
issue as an example.

[1] https://github.com/cloudfoundry/cloud_controller_ng/issues/568

I'd like to propose two solutions and get some feedback regarding which the
community would prefer. Please respond with your preferred solution and a
brief reason why.

*Solution 1 - Require a Unique System Domain*

Instead of recommending a unique system domain, we would enforce this in
the Cloud Controller. The proposed change is as follows:

1. REQUIRE system_domain to NOT be in the list of app_domains
2. REQUIRE a system_domain_organization

This will create a Private Domain for the system domain. Failure to
configure correctly would not allow the Cloud Controller to start.

If we decide to implement this, an operator should ensure their deployment
uses a unique system domain and a system_domain_organization and correct
DNS entries before upgrading.

Example for BOSH-lite

- app_domains: [bosh-lite.com]
- system_domain: system.bosh-lite.com
- system_domain_organization: system


- api endpoint: api.system.bosh-lite.com
- sample app endpoint: dora.bosh-lite.com

Example for a PaaS:

- app_domains: [yuge-paas-apps.io]
- system_domain: yuge-pass.com
- system_domain_organization: yuge-system-org


- api endpoint: api.yuge-paas.com
- sample app endpoint: dora.yuge-paas-apps.io

*Pro: *Cloud Controller now enforces what was previously the recommended
configuration for separate system and apps domains.
Con: Second SSL cert for the system domain and possibly a second DNS record
if system domain is not covered by the current wildcard record.

*Solution 2 - Cloud Controller Seeds System Routes*

To prevent a non-system app from requesting a hostname on a shared system
domain, the Cloud Controller will take a list of hostnames and seed the
database with routes. As routes are associated with a space, we will
require a system_organization and system_space. An operator could choose to
omit hostnames as desired.

cc.system_hostnames:
description: List of hostnames for which routes will be created on the
system domain.
default: [api,uaa,login,doppler,loggregator,hm9000]
cc.system_space:
description: Space where system routes will be created.
default: system

*Pro:* Significantly less change for operators running Cloud Foundry with
matching system and apps domains.
*Con: *Cloud Controller has knowledge of unrelated system components and
the list of defaults needs to be maintained as we add and remove components.


Thanks,

-Nick
--
Nicholas Calugar
CAPI Product Manager
Pivotal Software, Inc.