Diego: Permission denied when starting application with startup command


Matthew Sykes <matthew.sykes@...>
 

Looking again, it seems the cli *tries* to set the mode but, for some
reason, the package that gets sent up doesn't have the permissions.

https://github.com/cloudfoundry/cli/blob/master/cf/app_files/zipper.go#L294

Not sure where it's getting lost.

On Wed, Feb 17, 2016 at 8:48 AM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

I can confirm that the problem is with how the cli handles zip entries in
archives. When the cli extracts the zip into a temporary directory, it does
not set the file mode on the files it creates. [1] It simply relies on the
behavior of os.Create() [2]

Looks like an issue should be raised against the cli but a simple
workaround is to simply expand the zip before pushing.

[1]:
https://github.com/cloudfoundry/cli/blob/master/cf/app_files/zipper.go#L283-L292
[2]: https://golang.org/pkg/os/#Create

On Wed, Feb 17, 2016 at 8:18 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Wed, Feb 17, 2016 at 4:30 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi, Daniel



CF:230 Diego: 0.1441.0 Garden-linux: 0.327.0



I ran “chmod a+x *” under my application’s “bin” folder and then zip all
files. The command for starting application is “bin/start_ngis.sh -m dev”.




I downloaded the droplet from CC after staging on diego, the file
permission under “bin” folder was modified to “-rw-r--r--”.



How would diego decide which file’s permission should be preserved and
which one should be modified.
My understanding is that it doesn't. Diego just lets thing pass through
unchanged, whereas the DEA would previously force a specific set of
permissions.

In this case, it might be the cf cli that is the problem. You're
creating the JAR/WAR with the proper permissions on your script (I'm
assuming you've unzipped to verify the permission is retained, if not do
that and confirm). The next step is for the cf cli to extract your files
and upload them. The cf cli doesn't upload your JAR / WAR file whole. It
unzips it and uploads files individually. This is how it can skip certain
parts of your application that have been already uploaded. This would be
the next phase where there could possibly be an issue.



I found a very strange thing. Part of jar files under “bundles” folder
were modified to “-rwxr--r--”. But part of them were not changed.
Interesting. What OS are you running locally? Linux, Mac, Windows,
Cygwin? Also, what is your version of cf? `cf -v`?

Dan





*From:* Daniel Mikusa [mailto:dmikusa(a)pivotal.io]
*Sent:* 2016年2月16日 23:26
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Diego: Permission denied when starting
application with startup command



On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi,



Our application is started by a shell script. So we pushed our
application with –c option. It works fine with dea. Application could be
started successfully. But when I pushed the application into diego, I got
“bash bin/start.sh permission denied”. I also found if I pushed and started
the application into dea first and enabled diego later, the error was
gone. I guess that is because dea will update file permission during
staging.



I suspect you're running into this:




https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes





I also tried to grant permission when zipping the applciation. But diego
totally ignore the setting. Could someone help me to solve this problem?



What version of cf are you using? What is your OS? What do you mean by
"grant permission when zipping the application", what commands are you
running to do that?



Dan





--
Matthew Sykes
matthew.sykes(a)gmail.com


--
Matthew Sykes
matthew.sykes(a)gmail.com


Daniel Mikusa
 

Or set your start up command to `chmod 755 bin/start_ngis.sh &&
bin/start_ngis.sh -m dev`.

Dan


On Wed, Feb 17, 2016 at 8:48 AM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

I can confirm that the problem is with how the cli handles zip entries in
archives. When the cli extracts the zip into a temporary directory, it does
not set the file mode on the files it creates. [1] It simply relies on the
behavior of os.Create() [2]

Looks like an issue should be raised against the cli but a simple
workaround is to simply expand the zip before pushing.

[1]:
https://github.com/cloudfoundry/cli/blob/master/cf/app_files/zipper.go#L283-L292
[2]: https://golang.org/pkg/os/#Create

On Wed, Feb 17, 2016 at 8:18 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Wed, Feb 17, 2016 at 4:30 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi, Daniel



CF:230 Diego: 0.1441.0 Garden-linux: 0.327.0



I ran “chmod a+x *” under my application’s “bin” folder and then zip all
files. The command for starting application is “bin/start_ngis.sh -m dev”.




I downloaded the droplet from CC after staging on diego, the file
permission under “bin” folder was modified to “-rw-r--r--”.



How would diego decide which file’s permission should be preserved and
which one should be modified.
My understanding is that it doesn't. Diego just lets thing pass through
unchanged, whereas the DEA would previously force a specific set of
permissions.

In this case, it might be the cf cli that is the problem. You're
creating the JAR/WAR with the proper permissions on your script (I'm
assuming you've unzipped to verify the permission is retained, if not do
that and confirm). The next step is for the cf cli to extract your files
and upload them. The cf cli doesn't upload your JAR / WAR file whole. It
unzips it and uploads files individually. This is how it can skip certain
parts of your application that have been already uploaded. This would be
the next phase where there could possibly be an issue.



I found a very strange thing. Part of jar files under “bundles” folder
were modified to “-rwxr--r--”. But part of them were not changed.
Interesting. What OS are you running locally? Linux, Mac, Windows,
Cygwin? Also, what is your version of cf? `cf -v`?

Dan





*From:* Daniel Mikusa [mailto:dmikusa(a)pivotal.io]
*Sent:* 2016年2月16日 23:26
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Diego: Permission denied when starting
application with startup command



On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi,



Our application is started by a shell script. So we pushed our
application with –c option. It works fine with dea. Application could be
started successfully. But when I pushed the application into diego, I got
“bash bin/start.sh permission denied”. I also found if I pushed and started
the application into dea first and enabled diego later, the error was
gone. I guess that is because dea will update file permission during
staging.



I suspect you're running into this:




https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes





I also tried to grant permission when zipping the applciation. But diego
totally ignore the setting. Could someone help me to solve this problem?



What version of cf are you using? What is your OS? What do you mean by
"grant permission when zipping the application", what commands are you
running to do that?



Dan





--
Matthew Sykes
matthew.sykes(a)gmail.com


Matthew Sykes <matthew.sykes@...>
 

I can confirm that the problem is with how the cli handles zip entries in
archives. When the cli extracts the zip into a temporary directory, it does
not set the file mode on the files it creates. [1] It simply relies on the
behavior of os.Create() [2]

Looks like an issue should be raised against the cli but a simple
workaround is to simply expand the zip before pushing.

[1]:
https://github.com/cloudfoundry/cli/blob/master/cf/app_files/zipper.go#L283-L292
[2]: https://golang.org/pkg/os/#Create

On Wed, Feb 17, 2016 at 8:18 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Wed, Feb 17, 2016 at 4:30 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi, Daniel



CF:230 Diego: 0.1441.0 Garden-linux: 0.327.0



I ran “chmod a+x *” under my application’s “bin” folder and then zip all
files. The command for starting application is “bin/start_ngis.sh -m dev”.




I downloaded the droplet from CC after staging on diego, the file
permission under “bin” folder was modified to “-rw-r--r--”.



How would diego decide which file’s permission should be preserved and
which one should be modified.
My understanding is that it doesn't. Diego just lets thing pass through
unchanged, whereas the DEA would previously force a specific set of
permissions.

In this case, it might be the cf cli that is the problem. You're creating
the JAR/WAR with the proper permissions on your script (I'm assuming you've
unzipped to verify the permission is retained, if not do that and
confirm). The next step is for the cf cli to extract your files and upload
them. The cf cli doesn't upload your JAR / WAR file whole. It unzips it
and uploads files individually. This is how it can skip certain parts of
your application that have been already uploaded. This would be the next
phase where there could possibly be an issue.



I found a very strange thing. Part of jar files under “bundles” folder
were modified to “-rwxr--r--”. But part of them were not changed.
Interesting. What OS are you running locally? Linux, Mac, Windows,
Cygwin? Also, what is your version of cf? `cf -v`?

Dan





*From:* Daniel Mikusa [mailto:dmikusa(a)pivotal.io]
*Sent:* 2016年2月16日 23:26
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Diego: Permission denied when starting
application with startup command



On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi,



Our application is started by a shell script. So we pushed our
application with –c option. It works fine with dea. Application could be
started successfully. But when I pushed the application into diego, I got
“bash bin/start.sh permission denied”. I also found if I pushed and started
the application into dea first and enabled diego later, the error was
gone. I guess that is because dea will update file permission during
staging.



I suspect you're running into this:




https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes





I also tried to grant permission when zipping the applciation. But diego
totally ignore the setting. Could someone help me to solve this problem?



What version of cf are you using? What is your OS? What do you mean by
"grant permission when zipping the application", what commands are you
running to do that?



Dan




--
Matthew Sykes
matthew.sykes(a)gmail.com


Daniel Mikusa
 

On Wed, Feb 17, 2016 at 4:30 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:

Hi, Daniel



CF:230 Diego: 0.1441.0 Garden-linux: 0.327.0



I ran “chmod a+x *” under my application’s “bin” folder and then zip all
files. The command for starting application is “bin/start_ngis.sh -m dev”.




I downloaded the droplet from CC after staging on diego, the file
permission under “bin” folder was modified to “-rw-r--r--”.



How would diego decide which file’s permission should be preserved and
which one should be modified.
My understanding is that it doesn't. Diego just lets thing pass through
unchanged, whereas the DEA would previously force a specific set of
permissions.

In this case, it might be the cf cli that is the problem. You're creating
the JAR/WAR with the proper permissions on your script (I'm assuming you've
unzipped to verify the permission is retained, if not do that and
confirm). The next step is for the cf cli to extract your files and upload
them. The cf cli doesn't upload your JAR / WAR file whole. It unzips it
and uploads files individually. This is how it can skip certain parts of
your application that have been already uploaded. This would be the next
phase where there could possibly be an issue.



I found a very strange thing. Part of jar files under “bundles” folder
were modified to “-rwxr--r--”. But part of them were not changed.
Interesting. What OS are you running locally? Linux, Mac, Windows,
Cygwin? Also, what is your version of cf? `cf -v`?

Dan





*From:* Daniel Mikusa [mailto:dmikusa(a)pivotal.io]
*Sent:* 2016年2月16日 23:26
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Diego: Permission denied when starting
application with startup command



On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com>
wrote:

Hi,



Our application is started by a shell script. So we pushed our application
with –c option. It works fine with dea. Application could be started
successfully. But when I pushed the application into diego, I got “bash
bin/start.sh permission denied”. I also found if I pushed and started the
application into dea first and enabled diego later, the error was gone. I
guess that is because dea will update file permission during staging.



I suspect you're running into this:




https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes





I also tried to grant permission when zipping the applciation. But diego
totally ignore the setting. Could someone help me to solve this problem?



What version of cf are you using? What is your OS? What do you mean by
"grant permission when zipping the application", what commands are you
running to do that?



Dan





MaggieMeng
 

Hi, Daniel

CF:230 Diego: 0.1441.0 Garden-linux: 0.327.0

I ran “chmod a+x *” under my application’s “bin” folder and then zip all files. The command for starting application is “bin/start_ngis.sh -m dev”.

I downloaded the droplet from CC after staging on diego, the file permission under “bin” folder was modified to “-rw-r--r--”.

How would diego decide which file’s permission should be preserved and which one should be modified. I found a very strange thing. Part of jar files under “bundles” folder were modified to “-rwxr--r--”. But part of them were not changed.

Thanks,
Maggie

From: Daniel Mikusa [mailto:dmikusa(a)pivotal.io]
Sent: 2016年2月16日 23:26
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Diego: Permission denied when starting application with startup command

On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote:
Hi,

Our application is started by a shell script. So we pushed our application with –c option. It works fine with dea. Application could be started successfully. But when I pushed the application into diego, I got “bash bin/start.sh permission denied”. I also found if I pushed and started the application into dea first and enabled diego later, the error was gone. I guess that is because dea will update file permission during staging.

I suspect you're running into this:

https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes


I also tried to grant permission when zipping the applciation. But diego totally ignore the setting. Could someone help me to solve this problem?

What version of cf are you using? What is your OS? What do you mean by "grant permission when zipping the application", what commands are you running to do that?

Dan


Daniel Mikusa
 

On Tue, Feb 16, 2016 at 4:21 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:

Hi,



Our application is started by a shell script. So we pushed our application
with –c option. It works fine with dea. Application could be started
successfully. But when I pushed the application into diego, I got “bash
bin/start.sh permission denied”. I also found if I pushed and started the
application into dea first and enabled diego later, the error was gone. I
guess that is because dea will update file permission during staging.
I suspect you're running into this:

https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md#file-permission-modes




I also tried to grant permission when zipping the applciation. But diego
totally ignore the setting. Could someone help me to solve this problem?
What version of cf are you using? What is your OS? What do you mean by
"grant permission when zipping the application", what commands are you
running to do that?

Dan



MaggieMeng
 

Hi,

Our application is started by a shell script. So we pushed our application with -c option. It works fine with dea. Application could be started successfully. But when I pushed the application into diego, I got "bash bin/start.sh permission denied". I also found if I pushed and started the application into dea first and enabled diego later, the error was gone. I guess that is because dea will update file permission during staging.

I also tried to grant permission when zipping the applciation. But diego totally ignore the setting. Could someone help me to solve this problem?

Thanks,
Maggie