Re: CPU usage for apps became extremely high when upgrade to cf 237 with diego release 0.1472.0


Michael Fraenkel <michael.fraenkel@...>
 

This is a known issue that is fixed which may come in the next release.
It is just off by 100.

- Michael

On 6/22/16 11:25 AM, Siva Balan wrote:
I am seeing this issue with the DEAs as well on ver 237 in out
environment.

On Tue, Jun 21, 2016 at 11:47 PM, Eric Malm <emalm(a)pivotal.io
<mailto:emalm(a)pivotal.io>> wrote:

Hi,

None of the CPU usage computations have changed in Diego recently.
What version of the CLI are you using? What does the CC stats
endpoint, accessible via `cf curl /v2/apps/$(cf app --guid
APP_NAME)/stats`, report for the CPU usage of the instances of
this app?

Thanks,
Eric, CF Runtime Diego PM

On Tue, Jun 21, 2016 at 11:17 PM, 王小锋 <zzuwxf(a)gmail.com
<mailto:zzuwxf(a)gmail.com>> wrote:

Hi, there

After I upgrade cf deployment to cf 237 with diego 0.1472.0
recently, cpu usage for apps became abnormal, it became
extremely high (exceeds 100% * cpu cores) as below:

last uploaded: Wed Jun 22 02:34:58 UTC 2016
stack: cflinuxfs2
buildpack: java_buildpack

state since cpu memory disk details
#0 running 2016-06-22 02:35:43 AM *2334.7%* 1.5G of 2G
190.2M of 2G
#1 running 2016-06-22 02:56:50 AM *3478.9%* 1.5G of 2G
190.2M of 2G
#2 running 2016-06-22 02:56:50 AM *2819.1% * 1.5G of 2G
190.2M of 2G
#3 running 2016-06-22 02:56:51 AM *2240.2% * 1.6G of 2G
190.2M of 2G

The backend cell has 16 cores, not sure why the cpu usage
execceeds 1600%. I checked the cpu usage from aws console, the
cpu usage is about 2%. Does cf 237 changed the cpu usage
algorithm? I am wondering if the cli displays Wrong cpu usage?

The app is running on diego and I am able to ssh it. "uptime"
shows the cpu load is not high:

vcap(a)mgg62798lgm:~$ uptime
06:15:09 up 20:30, 0 users, load average: 0.33, 0.25, 0.25


"iostat" shows:

vcap(a)mgg62798lgm:~$ iostat
Linux 3.19.0-58-generic (mgg62798lgm) 06/22/2016 _x86_64_(16 CPU)

avg-cpu: %user %nice %system %iowait %steal %idle
0.28 0.00 0.02 0.03 0.02 99.65

Device: tps kB_read/s kB_wrtn/s kB_read
kB_wrtn
xvda 0.58 1.00 5.09 73889 375904
xvdb 1.86 0.04 141.61 3071 10459235
xvdf 0.20 0.03 23.61 1925 1743700





--
http://www.twitter.com/sivabalans

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.