Feedback requested: Sticky sessions: If request includes VCAP_ID cookie, always include it in the response
Recently, the following feature request was made for gorouter:
#267: Add support for sticky sessions when CF-deployed reverse proxy is used by setting VCAP_ID to backend instance ID instead of proxy instance ID
The suggestion for implementing this feature in gorouter is relatively simple and we have validated that it works: If the request includes a VCAP_ID cookie, then always include a VCAP_ID in the response.
While the cookie in the response is redundant in most cases, we think this is a reasonable change to make. It would only impact applications that use sticky sessions.
That said, we wanted to bring this up to a wider audience before proceeding. Please respond here or on the issue if you can share any reasons why we should not make this change.
Currently, when an application sets a cookie that matches one of the sticky session cookie names configured for gorouter, the gorouter will add an additional Set-Cookie header with the name __VCAP_ID__ with the value being the instance ID of the backend that handled that request. On subsequent requests, as the client includes this cookie in the request, the gorouter will route the request to the same instance. If it succeeds, it does not include the instance ID in the response (as this was assumed to be redundant information). The only exceptions currently are 1) if the backend sets the session cookie (most don't do this in every response for the same reason - it's usually redundant), or 2) if the backend instance was not found (e.g. because that instance failed or was rescheduled to another cell).
The current implementation described above works in most cases, but fails in the following way when there is a cf-deployed proxy in front of the backend.
In that case, the request path looks like this:
client -> gorouter -> proxy -> gorouter -> backend
In this scenario, the first gorouter in front of the proxy fails to find the instance ID of the backend, as it only looks for instances of proxy. Because it fails to match, in its response, it sets the cookie to an instance of the proxy app instead, and the subsequent request gets routed to a random backend. It ends up in a pattern where 2 requests go to the same backend at a time, and then flipping to a different one.
A workaround in the current implementation of the gorouter is to always have the backend application set the session cookie. This works, but requires changes to application code.
Engineer on #networking