Removing FUSE support from CF


Onsi Fakhouri <ofakhouri@...>
 

Hey CF-Dev,

The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!

Diego currently runs applications in *privileged* containers. These lack
the security features outlined above and we are planning on switching to
launch all CF applications in *unprivileged* containers.

Unfortunately, it has proved difficult to support mounting FUSE filesystems
from within unprivileged containers. We believe the security benefits
outweigh the features that FUSE give us and* are planning on removing
support for FUSE in favor of better securing our containers.*
If/when FUSE support in unprivileged containers becomes possible we may add
it back to the platform.

Thoughts? Concerns?

Thanks!

Onsi


Lerenc, Vedran <vedran.lerenc@...>
 

Hi Onsi,


Ø Thoughts? Concerns?

Well, that’s bad news.

We, and I assume many others as well (like the folks from Stackato who do it in the public), have used SSHFS + FUSE to implement a persistent file system for old-fashioned apps/apps that are not Cloud-native. I don’t want to fight an ideological battle here, it’s just that these apps do still exist (in majority) and a file system service is an important service/feature for them.

So if you remove FUSE (which we thought is not going away/was added to stay), it’s pretty bad for us/many apps.

Best regards, Vedran



From: Onsi Fakhouri
Reply-To: "Discussions about Cloud Foundry projects and the system overall."
Date: Saturday 11 July 2015 01:10
To: cf-dev
Subject: [cf-dev] Removing FUSE support from CF

Hey CF-Dev,

The Garden team has been hard at work substantially improving Garden-Linux's security features. Garden-Linux now employs user namespaces and drops capabilities when creating unprivileged containers - we're excited to bring both of these features to the platform!

Diego currently runs applications in privileged containers. These lack the security features outlined above and we are planning on switching to launch all CF applications in unprivileged containers.

Unfortunately, it has proved difficult to support mounting FUSE filesystems from within unprivileged containers. We believe the security benefits outweigh the features that FUSE give us and are planning on removing support for FUSE in favor of better securing our containers. If/when FUSE support in unprivileged containers becomes possible we may add it back to the platform.

Thoughts? Concerns?

Thanks!

Onsi


Daniel Mikusa
 

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com>
wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who do
it in the public), have used SSHFS + FUSE to implement a persistent file
system for old-fashioned apps/apps that are not Cloud-native. I don’t want
to fight an ideological battle here, it’s just that these apps do still
exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to
stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very helpful
for running apps that depend on a persistent FS, like Wordpress. Perhaps
this use case of mounting a remote SSHFS could be supported in some other
way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These lack
the security features outlined above and we are planning on switching to
launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers becomes
possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Onsi Fakhouri <ofakhouri@...>
 

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com>
wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who do
it in the public), have used SSHFS + FUSE to implement a persistent file
system for old-fashioned apps/apps that are not Cloud-native. I don’t want
to fight an ideological battle here, it’s just that these apps do still
exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to
stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very helpful
for running apps that depend on a persistent FS, like Wordpress. Perhaps
this use case of mounting a remote SSHFS could be supported in some other
way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These
lack the security features outlined above and we are planning on switching
to launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers becomes
possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Matt Cowger
 

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com>
wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who
do it in the public), have used SSHFS + FUSE to implement a persistent file
system for old-fashioned apps/apps that are not Cloud-native. I don’t want
to fight an ideological battle here, it’s just that these apps do still
exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to
stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very helpful
for running apps that depend on a persistent FS, like Wordpress. Perhaps
this use case of mounting a remote SSHFS could be supported in some other
way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These
lack the security features outlined above and we are planning on switching
to launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers becomes
possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
-- Matt


Onsi Fakhouri <ofakhouri@...>
 

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi

On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com>
wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who
do it in the public), have used SSHFS + FUSE to implement a persistent file
system for old-fashioned apps/apps that are not Cloud-native. I don’t want
to fight an ideological battle here, it’s just that these apps do still
exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to
stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These
lack the security features outlined above and we are planning on switching
to launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers becomes
possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Matt Cowger
 

We're wary of adding too many knobs to the platform and exposing them all
the way down to app developers increases the cognitive load for folks using
the platform. Enabling/disabling it on a per-installation level, and -
maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I too
amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com>
wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who
do it in the public), have used SSHFS + FUSE to implement a persistent file
system for old-fashioned apps/apps that are not Cloud-native. I don’t want
to fight an ideological battle here, it’s just that these apps do still
exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to
stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These
lack the security features outlined above and we are planning on switching
to launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers becomes
possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt


Jack Cai
 

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing them
all the way down to app developers increases the cognitive load for folks
using the platform. Enabling/disabling it on a per-installation level, and
- maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I too
amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com
wrote:
Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato
who do it in the public), have used SSHFS + FUSE to implement a persistent
file system for old-fashioned apps/apps that are not Cloud-native. I don’t
want to fight an ideological battle here, it’s just that these apps do
still exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added
to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the system
overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers. These
lack the security features outlined above and we are planning on switching
to launch all CF applications in *unprivileged* containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Guillaume Berche
 

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user
namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the
container has a user namespace and the root
// user in the container is mapped to a non-root user in the host.
Defaults to false.
Privileged bool <http://godoc.org/builtin#bool>
`json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using the
docker file USER directive (specially root): then the container will run as
the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing them
all the way down to app developers increases the cognitive load for folks
using the platform. Enabling/disabling it on a per-installation level, and
- maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I too
amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato
who do it in the public), have used SSHFS + FUSE to implement a persistent
file system for old-fashioned apps/apps that are not Cloud-native. I don’t
want to fight an ideological battle here, it’s just that these apps do
still exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added
to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Julz Friedman
 

Hi Guillaume, I'd put it like this: running containers with 'privileged: false' makes them safe /even if/ a user gets root. With a docker image this is essential, because getting root is trivial. With a buildpack image this is less essential, but it means /even if/ a root escalation exploit is found (these do exist, there was an escalation via overlayfs patched a while ago) you're still safe. 'Privileged: false' turns on user namespacing and turns off various capabilities: it's what the Garden team recommend most containers use. 'Privileged: true' relies on the security of your rootfs and users never getting root; if your use case requires it you'll of course need to make a judgement, but you're trading off quite a lot of security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged container (as a workaround for the lack of fuse support within user namespace), when pushing an app that goes through the buildpack staging process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see
// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool `json:"privileged,omitempty"`

I understand the risk is specific to running a docker image built using the docker file USER directive (specially root): then the container will run as the host root ?

BTW, is the ability to run as a different user in staging and running currently considered as discussed into https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ ?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:
+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:
We're wary of adding too many knobs to the platform and exposing them all the way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise?
Agreed. I'd argue this this probably not be a 'per-app' thing, as I too amy way of the knobs that developers like to turn. I think a per space level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:
That's still very much open for discussion. Obviously, someone with administrative privileges should be in charge of this particular piece of configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a deploy-time config (e.g. part of the CC config written out by BOSH) would make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the platform and exposing them all the way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:
Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:
Hey all,

Based on the feedback we got and the relatively low cost to maintain privileged support we'd like to propose making running privileged containers on the platform configurable - we will recommend this be turned off when running untrusted workloads and it will likely default to off. We have longer term plans to support mounting persistent volumes in Diego at which point support for mounting FUSE in unprivileged containers can become a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:
On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com> wrote:
Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who do it in the public), have used SSHFS + FUSE to implement a persistent file system for old-fashioned apps/apps that are not Cloud-native. I don’t want to fight an ideological battle here, it’s just that these apps do still exist (in majority) and a file system service is an important service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very helpful for running apps that depend on a persistent FS, like Wordpress. Perhaps this use case of mounting a remote SSHFS could be supported in some other way?

Dan







From: Onsi Fakhouri
Reply-To: "Discussions about Cloud Foundry projects and the system overall."
Date: Saturday 11 July 2015 01:10
To: cf-dev
Subject: [cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving Garden-Linux's security features. Garden-Linux now employs user namespaces and drops capabilities when creating unprivileged containers - we're excited to bring both of these features to the platform!



Diego currently runs applications in privileged containers. These lack the security features outlined above and we are planning on switching to launch all CF applications in unprivileged containers.



Unfortunately, it has proved difficult to support mounting FUSE filesystems from within unprivileged containers. We believe the security benefits outweigh the features that FUSE give us and are planning on removing support for FUSE in favor of better securing our containers. If/when FUSE support in unprivileged containers becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Onsi Fakhouri <ofakhouri@...>
 

To that I would add that privileged: true is the behavior of the existing platform and warden. Sticking with privileged: true essentially opts you out off gardens new security features.

So running privileged containers isn't a step backwards. It's simply a choice not to take a step forwards.

Onsi

On Jul 30, 2015, at 9:34 AM, Julian Friedman <julz.friedman(a)gmail.com> wrote:

Hi Guillaume, I'd put it like this: running containers with 'privileged: false' makes them safe /even if/ a user gets root. With a docker image this is essential, because getting root is trivial. With a buildpack image this is less essential, but it means /even if/ a root escalation exploit is found (these do exist, there was an escalation via overlayfs patched a while ago) you're still safe. 'Privileged: false' turns on user namespacing and turns off various capabilities: it's what the Garden team recommend most containers use. 'Privileged: true' relies on the security of your rootfs and users never getting root; if your use case requires it you'll of course need to make a judgement, but you're trading off quite a lot of security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged container (as a workaround for the lack of fuse support within user namespace), when pushing an app that goes through the buildpack staging process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see
// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool `json:"privileged,omitempty"`

I understand the risk is specific to running a docker image built using the docker file USER directive (specially root): then the container will run as the host root ?

BTW, is the ability to run as a different user in staging and running currently considered as discussed into https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ ?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:
+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:
We're wary of adding too many knobs to the platform and exposing them all the way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise?
Agreed. I'd argue this this probably not be a 'per-app' thing, as I too amy way of the knobs that developers like to turn. I think a per space level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:
That's still very much open for discussion. Obviously, someone with administrative privileges should be in charge of this particular piece of configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a deploy-time config (e.g. part of the CC config written out by BOSH) would make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the platform and exposing them all the way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:
Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:
Hey all,

Based on the feedback we got and the relatively low cost to maintain privileged support we'd like to propose making running privileged containers on the platform configurable - we will recommend this be turned off when running untrusted workloads and it will likely default to off. We have longer term plans to support mounting persistent volumes in Diego at which point support for mounting FUSE in unprivileged containers can become a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:
On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <vedran.lerenc(a)sap.com> wrote:
Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato who do it in the public), have used SSHFS + FUSE to implement a persistent file system for old-fashioned apps/apps that are not Cloud-native. I don’t want to fight an ideological battle here, it’s just that these apps do still exist (in majority) and a file system service is an important service/feature for them.



So if you remove FUSE (which we thought is not going away/was added to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very helpful for running apps that depend on a persistent FS, like Wordpress. Perhaps this use case of mounting a remote SSHFS could be supported in some other way?

Dan







From: Onsi Fakhouri
Reply-To: "Discussions about Cloud Foundry projects and the system overall."
Date: Saturday 11 July 2015 01:10
To: cf-dev
Subject: [cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving Garden-Linux's security features. Garden-Linux now employs user namespaces and drops capabilities when creating unprivileged containers - we're excited to bring both of these features to the platform!



Diego currently runs applications in privileged containers. These lack the security features outlined above and we are planning on switching to launch all CF applications in unprivileged containers.



Unfortunately, it has proved difficult to support mounting FUSE filesystems from within unprivileged containers. We believe the security benefits outweigh the features that FUSE give us and are planning on removing support for FUSE in favor of better securing our containers. If/when FUSE support in unprivileged containers becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Guillaume Berche
 

Thanks Julian and Onsi for the additional details. Much clearer now.

Guillaume.

On Thu, Jul 30, 2015 at 6:40 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

To that I would add that privileged: true is the behavior of the existing
platform and warden. Sticking with privileged: true essentially opts you
out off gardens new security features.

So running privileged containers isn't a step backwards. It's simply a
choice not to take a step forwards.

Onsi



On Jul 30, 2015, at 9:34 AM, Julian Friedman <julz.friedman(a)gmail.com>
wrote:

Hi Guillaume, I'd put it like this: running containers with 'privileged:
false' makes them safe /even if/ a user gets root. With a docker image this
is essential, because getting root is trivial. With a buildpack image this
is less essential, but it means /even if/ a root escalation exploit is
found (these do exist, there was an escalation via overlayfs patched a
while ago) you're still safe. 'Privileged: false' turns on user namespacing
and turns off various capabilities: it's what the Garden team recommend
most containers use. 'Privileged: true' relies on the security of your
rootfs and users never getting root; if your use case requires it you'll of
course need to make a judgement, but you're trading off quite a lot of
security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool <http://godoc.org/builtin#bool> `json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using
the docker file USER directive (specially root): then the container will
run as the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing them
all the way down to app developers increases the cognitive load for folks
using the platform. Enabling/disabling it on a per-installation level, and
- maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I too
amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato
who do it in the public), have used SSHFS + FUSE to implement a persistent
file system for old-fashioned apps/apps that are not Cloud-native. I don’t
want to fight an ideological battle here, it’s just that these apps do
still exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added
to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Mike Youngstrom <youngm@...>
 

Good description. Count us as one who does not use FUSE and would very
much like to run docker images in privileged mode.

Perhaps it would be appropriate to force privileged mode for docker apps
but allow running non privileged for non docker applications until FUSE can
be removed?

Mike



On Thu, Jul 30, 2015 at 10:34 AM, Julian Friedman <julz.friedman(a)gmail.com>
wrote:

Hi Guillaume, I'd put it like this: running containers with 'privileged:
false' makes them safe /even if/ a user gets root. With a docker image this
is essential, because getting root is trivial. With a buildpack image this
is less essential, but it means /even if/ a root escalation exploit is
found (these do exist, there was an escalation via overlayfs patched a
while ago) you're still safe. 'Privileged: false' turns on user namespacing
and turns off various capabilities: it's what the Garden team recommend
most containers use. 'Privileged: true' relies on the security of your
rootfs and users never getting root; if your use case requires it you'll of
course need to make a judgement, but you're trading off quite a lot of
security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool <http://godoc.org/builtin#bool> `json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using
the docker file USER directive (specially root): then the container will
run as the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing them
all the way down to app developers increases the cognitive load for folks
using the platform. Enabling/disabling it on a per-installation level, and
- maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I too
amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato
who do it in the public), have used SSHFS + FUSE to implement a persistent
file system for old-fashioned apps/apps that are not Cloud-native. I don’t
want to fight an ideological battle here, it’s just that these apps do
still exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was added
to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Onsi Fakhouri <ofakhouri@...>
 

Hey Mike,

Just to be clear, I think you have a (consistent) sign error throughout
your e-mail?

Cloud Controller's current behavior is to request *un*privileged (i.e.
"more secure") containers for Docker images and privileged (i.e. "less
secure") containers for buildpack apps.

Our proposal is to make the privileged flag for buildpack apps configurable
(and it sounds like folks are leaning towards the per-space approach).

I think we want to continue to enforce *un*privileged containers for docker
images as the attack surface is substantially higher with a docker image.

Onsi

On Thu, Jul 30, 2015 at 12:39 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Good description. Count us as one who does not use FUSE and would very
much like to run docker images in privileged mode.

Perhaps it would be appropriate to force privileged mode for docker apps
but allow running non privileged for non docker applications until FUSE can
be removed?

Mike



On Thu, Jul 30, 2015 at 10:34 AM, Julian Friedman <julz.friedman(a)gmail.com
wrote:
Hi Guillaume, I'd put it like this: running containers with 'privileged:
false' makes them safe /even if/ a user gets root. With a docker image this
is essential, because getting root is trivial. With a buildpack image this
is less essential, but it means /even if/ a root escalation exploit is
found (these do exist, there was an escalation via overlayfs patched a
while ago) you're still safe. 'Privileged: false' turns on user namespacing
and turns off various capabilities: it's what the Garden team recommend
most containers use. 'Privileged: true' relies on the security of your
rootfs and users never getting root; if your use case requires it you'll of
course need to make a judgement, but you're trading off quite a lot of
security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool <http://godoc.org/builtin#bool> `json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using
the docker file USER directive (specially root): then the container will
run as the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing them
all the way down to app developers increases the cognitive load for folks
using the platform. Enabling/disabling it on a per-installation level, and
- maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I
too amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey all,

Based on the feedback we got and the relatively low cost to maintain
privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from Stackato
who do it in the public), have used SSHFS + FUSE to implement a persistent
file system for old-fashioned apps/apps that are not Cloud-native. I don’t
want to fight an ideological battle here, it’s just that these apps do
still exist (in majority) and a file system service is an important
service/feature for them.



So if you remove FUSE (which we thought is not going away/was
added to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Mike Youngstrom <youngm@...>
 

Right. So to restate this discussion only applies to buildpack containers
docker containers currently are and will continue to be run in unprivileged
mode by default. Correct?

On Thu, Jul 30, 2015 at 2:47 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:

Hey Mike,

Just to be clear, I think you have a (consistent) sign error throughout
your e-mail?

Cloud Controller's current behavior is to request *un*privileged (i.e.
"more secure") containers for Docker images and privileged (i.e. "less
secure") containers for buildpack apps.

Our proposal is to make the privileged flag for buildpack apps
configurable (and it sounds like folks are leaning towards the per-space
approach).

I think we want to continue to enforce *un*privileged containers for
docker images as the attack surface is substantially higher with a docker
image.

Onsi



On Thu, Jul 30, 2015 at 12:39 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

Good description. Count us as one who does not use FUSE and would very
much like to run docker images in privileged mode.

Perhaps it would be appropriate to force privileged mode for docker apps
but allow running non privileged for non docker applications until FUSE can
be removed?

Mike



On Thu, Jul 30, 2015 at 10:34 AM, Julian Friedman <
julz.friedman(a)gmail.com> wrote:

Hi Guillaume, I'd put it like this: running containers with 'privileged:
false' makes them safe /even if/ a user gets root. With a docker image this
is essential, because getting root is trivial. With a buildpack image this
is less essential, but it means /even if/ a root escalation exploit is
found (these do exist, there was an escalation via overlayfs patched a
while ago) you're still safe. 'Privileged: false' turns on user namespacing
and turns off various capabilities: it's what the Garden team recommend
most containers use. 'Privileged: true' relies on the security of your
rootfs and users never getting root; if your use case requires it you'll of
course need to make a judgement, but you're trading off quite a lot of
security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool <http://godoc.org/builtin#bool> `json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using
the docker file USER directive (specially root): then the container will
run as the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing
them all the way down to app developers increases the cognitive load for
folks using the platform. Enabling/disabling it on a per-installation
level, and - maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I
too amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to a
deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io
wrote:
Hey all,

Based on the feedback we got and the relatively low cost to
maintain privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from
Stackato who do it in the public), have used SSHFS + FUSE to implement a
persistent file system for old-fashioned apps/apps that are not
Cloud-native. I don’t want to fight an ideological battle here, it’s just
that these apps do still exist (in majority) and a file system service is
an important service/feature for them.



So if you remove FUSE (which we thought is not going away/was
added to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Onsi Fakhouri <ofakhouri@...>
 

Yes, and perhaps I wasn't very clear about that. Thanks for teasing that
out Mike.

Onsi

On Thu, Jul 30, 2015 at 2:27 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Right. So to restate this discussion only applies to buildpack containers
docker containers currently are and will continue to be run in unprivileged
mode by default. Correct?

On Thu, Jul 30, 2015 at 2:47 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io>
wrote:

Hey Mike,

Just to be clear, I think you have a (consistent) sign error throughout
your e-mail?

Cloud Controller's current behavior is to request *un*privileged (i.e.
"more secure") containers for Docker images and privileged (i.e. "less
secure") containers for buildpack apps.

Our proposal is to make the privileged flag for buildpack apps
configurable (and it sounds like folks are leaning towards the per-space
approach).

I think we want to continue to enforce *un*privileged containers for
docker images as the attack surface is substantially higher with a docker
image.

Onsi



On Thu, Jul 30, 2015 at 12:39 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

Good description. Count us as one who does not use FUSE and would very
much like to run docker images in privileged mode.

Perhaps it would be appropriate to force privileged mode for docker apps
but allow running non privileged for non docker applications until FUSE can
be removed?

Mike



On Thu, Jul 30, 2015 at 10:34 AM, Julian Friedman <
julz.friedman(a)gmail.com> wrote:

Hi Guillaume, I'd put it like this: running containers with
'privileged: false' makes them safe /even if/ a user gets root. With a
docker image this is essential, because getting root is trivial. With a
buildpack image this is less essential, but it means /even if/ a root
escalation exploit is found (these do exist, there was an escalation via
overlayfs patched a while ago) you're still safe. 'Privileged: false' turns
on user namespacing and turns off various capabilities: it's what the
Garden team recommend most containers use. 'Privileged: true' relies on the
security of your rootfs and users never getting root; if your use case
requires it you'll of course need to make a judgement, but you're trading
off quite a lot of security in my opinion.

Sent from my iPhone

On 30 Jul 2015, at 17:12, Guillaume Berche <bercheg(a)gmail.com> wrote:

Thanks Onsi. Being able to use FUSE is quite important to us too.

Can you clarify the security risk associated with running a privileged
container (as a workaround for the lack of fuse support within user
namespace), when pushing an app that goes through the buildpack staging
process (hence running as vcap user) ?

From http://godoc.org/github.com/cloudfoundry-incubator/garden I see

// If Privileged is true the container does not have a user namespace and the root user in the container
// is the same as the root user in the host. Otherwise, the container has a user namespace and the root
// user in the container is mapped to a non-root user in the host. Defaults to false.
Privileged bool <http://godoc.org/builtin#bool> `json:"privileged,omitempty"`


I understand the risk is specific to running a docker image built using
the docker file USER directive (specially root): then the container will
run as the host root ?

BTW, is the ability to run as a different user in staging and running
currently considered as discussed into
https://groups.google.com/a/cloudfoundry.org/d/msg/vcap-dev/ZlC-2DVOSHo/uRrF6Io52mEJ
?

Thanks,

Guillaume.

On Thu, Jul 30, 2015 at 3:27 AM, Jack Cai <greensight(a)gmail.com> wrote:

+1 for space-level configuration.

Jack

On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:

We're wary of adding too many knobs to the platform and exposing
them all the way down to app developers increases the cognitive load for
folks using the platform. Enabling/disabling it on a per-installation
level, and - maybe - a per-space level, might be a decent compromise?

Agreed. I'd argue this this probably not be a 'per-app' thing, as I
too amy way of the knobs that developers like to turn. I think a per space
level is just the right level.

On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io
wrote:
That's still very much open for discussion. Obviously, someone with
administrative privileges should be in charge of this particular piece of
configuration.

Also making this a runtime config (e.g. feature flag) as opposed to
a deploy-time config (e.g. part of the CC config written out by BOSH) would
make the different behaviors more testable.

Thoughts? Preferences? We're wary of adding too many knobs to the
platform and exposing them all the way down to app developers increases the
cognitive load for folks using the platform. Enabling/disabling it on a
per-installation level, and - maybe - a per-space level, might be a decent
compromise?

Onsi



On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:

Once - configurable on a per-app, per space, or per deployment
basis?

On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <
ofakhouri(a)pivotal.io> wrote:

Hey all,

Based on the feedback we got and the relatively low cost to
maintain privileged support we'd like to propose making running privileged
containers on the platform configurable - we will recommend this be turned
off when running untrusted workloads and it will likely default to off. We
have longer term plans to support mounting persistent volumes in Diego at
which point support for mounting FUSE in unprivileged containers can become
a reality.

Thoughts?

Onsi

On Mon, Jul 13, 2015 at 4:42 AM, Daniel Mikusa <dmikusa(a)pivotal.io
wrote:
On Mon, Jul 13, 2015 at 2:48 AM, Lerenc, Vedran <
vedran.lerenc(a)sap.com> wrote:

Hi Onsi,



Ø Thoughts? Concerns?



Well, that’s bad news.



We, and I assume many others as well (like the folks from
Stackato who do it in the public), have used SSHFS + FUSE to implement a
persistent file system for old-fashioned apps/apps that are not
Cloud-native. I don’t want to fight an ideological battle here, it’s just
that these apps do still exist (in majority) and a file system service is
an important service/feature for them.



So if you remove FUSE (which we thought is not going away/was
added to stay), it’s pretty bad for us/many apps.



Best regards, Vedran
+1 - It would be sad to see FUSE support go away. It's been very
helpful for running apps that depend on a persistent FS, like Wordpress.
Perhaps this use case of mounting a remote SSHFS could be supported in some
other way?

Dan








*From: *Onsi Fakhouri
*Reply-To: *"Discussions about Cloud Foundry projects and the
system overall."
*Date: *Saturday 11 July 2015 01:10
*To: *cf-dev
*Subject: *[cf-dev] Removing FUSE support from CF



Hey CF-Dev,



The Garden team has been hard at work substantially improving
Garden-Linux's security features. Garden-Linux now employs user namespaces
and drops capabilities when creating unprivileged containers - we're
excited to bring both of these features to the platform!



Diego currently runs applications in *privileged* containers.
These lack the security features outlined above and we are planning on
switching to launch all CF applications in *unprivileged*
containers.



Unfortunately, it has proved difficult to support
mounting FUSE filesystems from within unprivileged containers. We believe
the security benefits outweigh the features that FUSE give us and* are
planning on removing support for FUSE in favor of better securing our
containers.* If/when FUSE support in unprivileged containers
becomes possible we may add it back to the platform.



Thoughts? Concerns?



Thanks!



Onsi

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
-- Matt

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev