Hi, there

After I upgrade cf deployment to cf 237 with diego 0.1472.0 recently, cpu
usage for apps became abnormal, it became extremely high (exceeds 100% *
cpu cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory disk
details
#0 running 2016-06-22 02:35:43 AM * 2334.7%* 1.5G of 2G 190.2M
of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of 2G 190.2M
of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of 2G 190.2M
of 2G
#3 running 2016-06-22 02:56:51 AM * 2240.2% * 1.6G of 2G 190.2M
of 2G

The backend cell has 16 cores, not sure why the cpu usage execceeds 1600%.
I checked the cpu usage from aws console, the cpu usage is about 2%. Does
cf 237 changed the cpu usage algorithm? I am wondering if the cli displays
Wrong cpu usage?

The app is running on diego and I am able to ssh it. "uptime" shows the
cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016 _x86_64_ (16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvda 0.58 1.00 5.09 73889 375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700

Hi,

None of the CPU usage computations have changed in Diego recently. What
version of the CLI are you using? What does the CC stats endpoint,
accessible via `cf curl /v2/apps/$(cf app --guid APP_NAME)/stats`, report
for the CPU usage of the instances of this app?

Thanks,
Eric, CF Runtime Diego PM

On Tue, Jun 21, 2016 at 11:17 PM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

After I upgrade cf deployment to cf 237 with diego 0.1472.0 recently, cpu
usage for apps became abnormal, it became extremely high (exceeds 100% *
cpu cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory disk
details
#0 running 2016-06-22 02:35:43 AM * 2334.7%* 1.5G of 2G 190.2M
of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of 2G 190.2M
of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of 2G 190.2M
of 2G
#3 running 2016-06-22 02:56:51 AM * 2240.2% * 1.6G of 2G 190.2M
of 2G

The backend cell has 16 cores, not sure why the cpu usage execceeds
1600%. I checked the cpu usage from aws console, the cpu usage is about 2%.
Does cf 237 changed the cpu usage algorithm? I am wondering if the cli
displays Wrong cpu usage?

The app is running on diego and I am able to ssh it. "uptime" shows the
cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016 _x86_64_ (16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvda 0.58 1.00 5.09 73889 375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700

I am seeing this issue with the DEAs as well on ver 237 in out environment.

On Tue, Jun 21, 2016 at 11:47 PM, Eric Malm <emalm(a)pivotal.io> wrote:

Hi,

None of the CPU usage computations have changed in Diego recently. What
version of the CLI are you using? What does the CC stats endpoint,
accessible via `cf curl /v2/apps/$(cf app --guid APP_NAME)/stats`, report
for the CPU usage of the instances of this app?

Thanks,
Eric, CF Runtime Diego PM

On Tue, Jun 21, 2016 at 11:17 PM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

After I upgrade cf deployment to cf 237 with diego 0.1472.0 recently,
cpu usage for apps became abnormal, it became extremely high (exceeds 100%
* cpu cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory disk
details
#0 running 2016-06-22 02:35:43 AM * 2334.7%* 1.5G of 2G 190.2M
of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of 2G 190.2M
of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of 2G 190.2M
of 2G
#3 running 2016-06-22 02:56:51 AM * 2240.2% * 1.6G of 2G 190.2M
of 2G

The backend cell has 16 cores, not sure why the cpu usage execceeds
1600%. I checked the cpu usage from aws console, the cpu usage is about 2%.
Does cf 237 changed the cpu usage algorithm? I am wondering if the cli
displays Wrong cpu usage?

The app is running on diego and I am able to ssh it. "uptime" shows the
cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016 _x86_64_ (16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
xvda 0.58 1.00 5.09 73889 375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700

--
http://www.twitter.com/sivabalans

I am seeing this issue with the DEAs as well on ver 237 in out
environment.

On Tue, Jun 21, 2016 at 11:47 PM, Eric Malm <emalm(a)pivotal.io
<mailto:emalm(a)pivotal.io>> wrote:

Hi,

None of the CPU usage computations have changed in Diego recently.
What version of the CLI are you using? What does the CC stats
endpoint, accessible via `cf curl /v2/apps/$(cf app --guid
APP_NAME)/stats`, report for the CPU usage of the instances of
this app?

Thanks,
Eric, CF Runtime Diego PM

On Tue, Jun 21, 2016 at 11:17 PM, 王小锋 <zzuwxf(a)gmail.com
<mailto:zzuwxf(a)gmail.com>> wrote:

Hi, there

After I upgrade cf deployment to cf 237 with diego 0.1472.0
recently, cpu usage for apps became abnormal, it became
extremely high (exceeds 100% * cpu cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory disk details
#0 running 2016-06-22 02:35:43 AM *2334.7%* 1.5G of 2G
190.2M of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of 2G
190.2M of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of 2G
190.2M of 2G
#3 running 2016-06-22 02:56:51 AM *2240.2% * 1.6G of 2G
190.2M of 2G

The backend cell has 16 cores, not sure why the cpu usage
execceeds 1600%. I checked the cpu usage from aws console, the
cpu usage is about 2%. Does cf 237 changed the cpu usage
algorithm? I am wondering if the cli displays Wrong cpu usage?

The app is running on diego and I am able to ssh it. "uptime"
shows the cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016 _x86_64_(16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read
kB_wrtn
xvda 0.58 1.00 5.09 73889 375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700





--
http://www.twitter.com/sivabalans

Was this fixed in Cf-238? I cannot see a reference in the release
notes. Is there a pivotal tracker reference?

Thanks,
John

On Wed, Jun 22, 2016 at 7:45 PM, Michael Fraenkel
<michael.fraenkel(a)gmail.com <mailto:michael.fraenkel(a)gmail.com>> wrote:

This is a known issue that is fixed which may come in the next
release. It is just off by 100.

- Michael


On 6/22/16 11:25 AM, Siva Balan wrote:

I am seeing this issue with the DEAs as well on ver 237 in out
environment.

On Tue, Jun 21, 2016 at 11:47 PM, Eric Malm <emalm(a)pivotal.io
<mailto:emalm(a)pivotal.io>> wrote:

Hi,

None of the CPU usage computations have changed in Diego
recently. What version of the CLI are you using? What does
the CC stats endpoint, accessible via `cf curl /v2/apps/$(cf
app --guid APP_NAME)/stats`, report for the CPU usage of the
instances of this app?

Thanks,
Eric, CF Runtime Diego PM

On Tue, Jun 21, 2016 at 11:17 PM, 王小锋 <zzuwxf(a)gmail.com
<mailto:zzuwxf(a)gmail.com>> wrote:

Hi, there

After I upgrade cf deployment to cf 237 with
diego 0.1472.0 recently, cpu usage for apps became
abnormal, it became extremely high (exceeds 100% * cpu
cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory
disk details
#0 running 2016-06-22 02:35:43 AM *2334.7%* 1.5G of
2G 190.2M of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of
2G 190.2M of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of
2G 190.2M of 2G
#3 running 2016-06-22 02:56:51 AM *2240.2% * 1.6G of
2G 190.2M of 2G

The backend cell has 16 cores, not sure why the cpu usage
execceeds 1600%. I checked the cpu usage from aws
console, the cpu usage is about 2%. Does cf 237 changed
the cpu usage algorithm? I am wondering if the cli
displays Wrong cpu usage?

The app is running on diego and I am able to ssh it.
"uptime" shows the cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016
_x86_64_(16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read
kB_wrtn
xvda 0.58 1.00 5.09 73889
375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700





--
http://www.twitter.com/sivabalans