Re: Debug failed to create container, Failed to stage application: staging failed?


Tom Sherrod <tom.sherrod@...>
 

Thanks for following up.

I recently found a mention of TMPDIR issue and deployed the latest diego
with v0.330.0. Appears to have resolved the issue.

Tom

On Fri, Jan 22, 2016 at 6:49 AM, Will Pragnell <wpragnell(a)pivotal.io> wrote:

In fact, I'd strongly recommend upgrading to Garden-Linux-Release v0.330.0
[1], as that release fixed a nasty security issue (CVE-2015-5350 [2]).

Best,
Will

[1]:
https://github.com/cloudfoundry-incubator/garden-linux-release/releases/tag/v0.330.0
[2]:
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/LSU7KECXU3X52RLWZPJQIBBVLPEOJSWR/

On 22 January 2016 at 11:45, Will Pragnell <wpragnell(a)pivotal.io> wrote:

Hi Tom,

Sorry, didn't spot this one (thanks Amit for bringing it to our
attention). This was a known issue relating to the way BOSH sets the
$TMPDIR environment variable by default. We fixed the issue in
Garden-Linux-Release v0.328.0 [1] so I'd recommend upgrading to that
release (or later) if possible.

Best,
Will

[1]:
https://github.com/cloudfoundry-incubator/garden-linux-release/releases/tag/v0.328.0

On 22 January 2016 at 01:57, Amit Gupta <agupta(a)pivotal.io> wrote:

Let me cc the Garden team, hopefully someone there can help.

Best,
Amit

On Sat, Jan 16, 2016 at 8:17 PM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

Thanks for the pointer.

Now the problem:

garden.stdout.log:
{"timestamp":"1453002747.400245667","source":"garden-linux","message":"garden-linux.pool.acquire.provide-rootfs-
failed","log_level":2,"data":{"error":"write /tmp/687182776: no space
left on
device","handle":"089ad5d6-4457-4184-bbda-ef918fb6676a-90065023-86c3-4fa2-8915-2508001bb91b-1e2fb2e4-f985-40f2-70f2-71bd8b0cdd12","session":"8.7"}}

/tmp is pointing to /dev/loop0 (120M) and full. How can this problem be
corrected?

Tom

On Thu, Jan 14, 2016 at 10:49 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

It's saying "failed to create container" so you could get logs from
the Diego cells and see what errors you see there. If you've configured
syslog to drain to an aggregation service, you can just watch the logs
there (especially look for errors in rep and garden logs). Otherwise, I
would simulcast to multiple terminal sessions, SSH to each cell, and tail
the logs of the rep and gardens to catch the error as it happens.

On Thu, Jan 14, 2016 at 11:04 AM, Tom Sherrod <tom.sherrod(a)gmail.com>
wrote:

CF: 225
Diego: 0.1441.0
Default to diego.

I deploy a sample app, no problems. I attempt to deploy another app
that fails, because of application issues. A deploy of any other app fails
with:
2016-01-14T18:35:54.47+0000 [STG/0] ERR Failed to create
container
2016-01-14T18:35:54.73+0000 [API/0] ERR Failed to stage
application: staging failed

I redeploy the sample app as a test and it fails. Any app deployment
fails.
CC, stager.stdout.log says staging failed.

Pointers of where to look next to find out why staging is failing?

Thanks,
Tom

Join {cf-dev@lists.cloudfoundry.org to automatically receive all group messages.