Re: Brokered route services only receiving traffic for routes mapped to started apps

Shannon Coen


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Mon, May 16, 2016 at 3:29 PM, Guillaume Berche <bercheg(a)> wrote:

Thanks a lot Shannon for your detailed response and sharing the routing
architecture plans. I realize the priorization of such effort remains a

Out of curiosity, in step #6, how would CC be notified of LRP current
state change, as to perform route unregister/updates ? Would CC be
registering to BBS external event client [1] through server-side-events, or
rather diego notifying CC through HTTP callbacks ?
CC would not be involved with route registration. CC would send the route
and the app process ID (received from Diego) to the Routing API. The
route-emitter would adds/remove backends for this process ID based on Diego
server-sent-events, and periodic batch fetches (as it does now).

I wonder whether this architecture could also enable fully-brokered route
services implementations to fetch the routing table from the routing-api,
and perform direct routing to apps (an alternative discussed in [2]),
enabling more advanced features (such as custom load balancing). I
understand this currently would require granting
scope to route services. Granting them a routing.route.<bound_route_guid>.read
oauth scope at SB route binding time would remove such a need for "admin
Yes, we imagine the Routing API providing the route/backend mapping as a
service, which will enable bring-your-own router, as well as direct routing
from Route Services. This assumes your route service has access to the same
private network as the interfaces for the Cell VMs. We may to consider how
to partition this data so that your route service or router only receives
routing data it should know about.

Thanks again,

section "Other proposals that we considered"


On Fri, May 13, 2016 at 9:10 PM, Shannon Coen <scoen(a)> wrote:

Hi Guillaume,

It would be great to have a chance to talk more about this at summit.

In summary, I believe supporting your use case is a large effort, and
yours in the only evidence I've heard in support of it. This makes
prioritization a challenge. However, I believe our current plan for
architectural changes to routing will eventually satisfy your requirements
as well.

Currently, CC sends route registration to Diego when an app is started.
Routes do not land in the router's routing table until the app is started,
as Diego doesn't know anything about stopped apps (LRPs are deleted when a
user requests stop). Since Diego will have no information about the LRP,
the router-emitter has no way of discovering that a route should be

Our plan is to move routing info out Diego. I believe it will fulfill
your use case, and the Diego team very much wants this also.

The plan looks like this:

1. Update Routing API endpoints for HTTP route registration to be
consistent with the TCP endpoints we've been focused on
2. Update the Route-Registrar job used by system components, service
brokers, etc. to register HTTP, to point at the Routing API, instead of
3. Update the route-emitter to register HTTP routes for apps on Diego
with the Routing API. *At this point, we believe we will have
removed the need for NATS in CF*
4. Update the Routing API to support route reservation, independent
of whether there are backends or not. *At this point, an independent
client could conceivably register a route with a route_service_url, and
without backends*
5. We may need to update Route-Registrar job to support reservation
of a route without backends, and association of backends with the route
6. Update CC to register app routes with Routing API, instead of
sending this data to Diego with createLRP, and update the route-emitter to
significantly change its behavior: instead of calculating the routing table
and sending it to Routing API, it will ask Diego for backends associated
with routes in the Routing API (linked by the process ID, most likely). *At
this point, a developer could conceivably use CLI to create a route, bind
it to a Route Service, and without mapping the route to an app, the router
would forward requests for the route to the Route Service.*


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Fri, May 13, 2016 at 1:31 AM, Guillaume Berche <bercheg(a)>


What are your current thoughts on "maintaining routes with no backends
in the routing table" ? I quickly scanned the routing backlog few days ago
without yet finding trace of it.

I wish we could have used the opportunity of the cf summit "project
office hours" routing session [1] to have interactive exchanges around
these use cases. Unfortunately, my autosleep session [2] is scheduled at
the exact same timeslot.
If the cf foundation organizers were able to swap sessions that would be
great. I'll send a separate email to events(a), is there
are other community members suffering from the same conflict.

Thanks in advance,




On Sun, May 1, 2016 at 12:03 AM, Stefan Mayr <stefan(a)>


Am 28.04.2016 um 23:08 schrieb Mike Youngstrom:

Here is another minor use case. My users are often confused that a
stopped app returns a 404 instead of a 503. So, we implement that
functionality for the user using an app mapped to wildcard routes that
constantly asks the CC for valid routes. This works for wildcard
domains but not one off domains.

It might be better if the router returned a 503. At least for routes
bound to apps. Not sure if this should extend to routes not bound to
+1 for that proposal. A 404 also causes issues when crawler remove
pages from their index. A 503 has less side effects. I would also prefer a
503 service unavailable when a route is not bound - because there is no
service for this route. IMHO the meaning is much closer to what has



On Thu, Apr 28, 2016 at 1:32 PM, Shannon Coen <scoen(a)
<mailto:scoen(a)>> wrote:

Hello Guillaume,

Thank you for sharing your thoughts on these use cases. I can see
how having
a route service field requests for an app, whether the app is up
on not,
could be useful.

However, enabling this would significantly change how routes are
for apps on Cloud Foundry, and how the router handles the route
Routes are not currently enabled in the routing tier unless they
to an app, and only when the app is determined healthy.

You are proposing the router maintains routes which have no
backends, and
instead of a failed lookup determining whether a 404 is returned,
the router
should figure out whether a route has any backends or a route

I'll chew on your use case and keep my ear out for additional use
cases for
maintaining routes with no backends in the routing table.


View this message in context:
Sent from the CF Dev mailing list archive at

Join to automatically receive all group messages.