Re: SSH access to CF app instances on Diego

James Myers

I have to agree with Matt on this one. I feel that the recycling of
containers is a very anti-developer default. When you approach Cloud
Foundry from the perspective of running production applications the recycle
policy makes complete sense. However, I feel that this misses out on one of
the massive benefits/use cases of Cloud Foundry, what it offers to the
development process.

From a security stand point, if you can ssh into a container, it means you
have write access to the application in CloudFoundry. Thus you can already
push new bits/change the application in question. All of the "papertrail"
functionality around pushing/changing applications exists for SSH as well
(we record events, output log lines, make it visible to users that action
was taken on the application), and thus concerned operators would be able
to determine if someone modifying the application in question.

Therefore I'm lost on how this is truly the most secure default. If we are
really going by the idea that all defaults should be the most secure, ssh
should be disabled by default.

As a developer, I can see many times in which I would want to be able to
ssh into my container and change my application as part of a
troubleshooting process. Using BOSH as an example, CF Devs constantly ssh
into VMs and change the processes running on them in order to facilitate
development. BOSH does not reap the VM and redeploy a new instance when you
have closed the SSH session. Once again this is largely due to the fact
that if you have SSH access, you can already perform the necessary actions
to change the application through different means.

Another huge hindrance to development, is that the recycling policy is
controlled by administrators. It is not something that normal users can
control, even though we allow the granularity of enabling/disabling SSH
completely to the end user. This seems counterintuitive.

I feel that a better solution would be to provide the user with some
knowledge of which instances may be tainted, and then allowing them to opt
into a policy which will reap tainted containers. This provides users with
clear insight that their application instance may be a snowflake (and that
they may want to take action), while also allowing normal behavior with
regards to SSH access to containers.

To summarize, by enabling the recycling policy by default we not only
produce extremely unusual behavior / workflows for developers, we are also
minimizing the developer-friendliness of CF in general. This mixed with the
fact that as a user I cannot even control this policy, leads me to believe
that as a default recycling should be turned off as it provides the most
cohesive and friendly user experience.

On Mon, Jun 29, 2015 at 9:14 AM, John Wong <gokoproject(a)> wrote:

after executing a command, concluding an interactive session, or copying
a file into an instance, that instance will be restarted.

How does it monitor the behavior? Is there a list of commands whitelisted?
I am curious because I am trying to find out what the whitelist contain.
Also is it at the end of the bosh ssh APP_NAME session? What if two users
are there simultaneously?


On Mon, Jun 29, 2015 at 5:49 AM, Dieu Cao <dcao(a)> wrote:

I think with the CLI we could add clarifying messaging when using ssh
what the current policy around recycling is.
Eric, what do you think about calling it the "recycling" policy, enabled
by default? =D


On Sat, Jun 27, 2015 at 3:42 AM, Matthew Sykes <matthew.sykes(a)>

Depends on your role and where your app is in the deployment pipeline.
Most of the scenarios I envisioned were for the tail end of development
where you need to poke around to debug and figure out those last few

For example, Ryan Morgan was saying that the Cloud Foundry plugin for
eclipse is going to be using the ssh support in diego to enable debug of
application instances in the context of a buildpack deployed app. This is
aligned with other requirements I've heard from people working on dev tools.

As apps reach production, I would hope that interactive ssh is disabled
entirely on the prod space leaving only scp in source mode as an option
(something the proxy can do).

Between dev and prod, there's a spectrum, but in general, I either
expect access to be enabled or disabled - not enabled with a suicidal

On Thu, Jun 25, 2015 at 10:53 PM, Benjamin Black <bblack(a)>


could you elaborate a bit on what you believe ssh access to instances
is for?


On Thu, Jun 25, 2015 at 9:29 PM, Matthew Sykes <matthew.sykes(a)
My concern is the default behavior.

When I first prototyped this support in February, I never expected
that merely accessing a container would cause it to be terminated. As we
can see from Jan's response, it's completely unexpected; many others have
the same reaction.

I do not believe that this behavior should be part of the default
configuration and I do believe the control needs to be at the space level.
I have have already expressed this opinion during Diego retros and at the
runtime PMC meeting.

I honestly believe that if we were talking about applying this
behavior to `bosh ssh` and `bosh scp`, few would even consider running in a
'kill on taint mode' because of how useful it is. We should learn from that.

If this behavior becomes the default, I think our platform will be
seen as moving from opinionated to parochial. That would be unfortunate.

On Thu, Jun 25, 2015 at 6:05 PM, James Bayer <jbayer(a)>

you can turn the "restart tainted containers" feature off with
configuration if you are authorized to do so. then using scp to write files
into a container would be persisted for the lifetime of the container even
after the ssh session ends.

On Thu, Jun 25, 2015 at 5:50 PM, Jan Dubois <jand(a)>

On Thu, Jun 25, 2015 at 5:36 PM, Eric Malm <emalm(a)> wrote:
after executing a command, concluding an
interactive session, or copying a file into an instance, that
instance will
be restarted.
What is the purpose of being able to copy a file into an instance if
the instance is restarted as soon as the file has been received?

cf-dev mailing list

Thank you,

James Bayer

cf-dev mailing list

Matthew Sykes

cf-dev mailing list

cf-dev mailing list

Matthew Sykes

cf-dev mailing list

cf-dev mailing list

cf-dev mailing list

Join { to automatically receive all group messages.