Date   

Announcing support for TCP Routing

Shannon Coen
 

On behalf of the CF Routing team I invite you to bring your non-http
workloads to Cloud Foundry.

By deploying the Routing Release [1] alongside Cloud Foundry and the Diego
runtime backend, operators can enable developers to create TCP routes based
on reservable ports.

The developer UX for pushing an app and mapping a TCP route is as simple as
this:

cf p myapp -d tcp.bosh-lite.com --random-route

The response includes a port associated with the TCP route. Client requests
to these ports will be routed to applications running on CF through a
layer-4 protocol-agnostic routing tier.

The Routing Release is still very much a work in progress. We are focusing
on mitigating stale routing information in the event of network partition
with system components, and on streamlining the deployment workflow.

Please review our README for deployment instructions, give it a go. We are
looking forward to your feedback.

Thank you!

[1] https://github.com/cloudfoundry-incubator/cf-routing-release

Note:
- UDP protocols are not supported.
- Both HTTP and TCP routes will be directed to the same application port,
identified by environment variable $PORT. Support for multiple application
ports is coming soon.


Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.


CF CLI v6.18.0 Released Today

Koper, Dies <diesk@...>
 

The CF CLI team just cut 6.18.0. Binaries and link to release notes are available at:

https://github.com/cloudfoundry/cli#downloads

This is likely our last CLI release this month as our velocity slows around the CF Summit.
I'll be attending the Summit, please approach me with your CLI experiences when you see me!
Also note the talk on CLI plugins from former CLI team members Simon and Jonathan:
https://cfsummit2016.sched.org/event/6Zlt/cli-plugin-to-enhance-your-cloud-simon-leunhellip

Org Quota for Reserved Route Ports

Org quota related commands have been enhanced to allow CF admins to set, update and retrieve the limit on the total number of reserved ports that can be used to create routes within an organization. This can be used to distribute a limited number of available ports over multiple organizations for use in TCP routes.
This feature requires the target CF release to be v236 (CC API v2.55.0) or higher.

Color Output Enabled by Default on Windows

CLI output coloring on Windows, introduced in the previous release, is now enabled by default (it can be toggled with cf config --color true|false), and resolves a bug where ansi codes were displayed in cf help and trace output.

Updated Commands

* create-quota, update-quota, quotas and quota now take an argument to set/update the reserved route port quota, or display it.
* quotas, space-quotas now display shorter table headers in most locales to reduce the total width of the table.
* delete-orphaned-routes now displays the port of TCP routes that are about to be deleted
Enjoy!

Regards,
Dies Koper
Cloud Foundry CLI PM


Re: Loggregator has updated protobufs definitions and compiler for dropsonde

Mike Youngstrom
 

Thanks for the heads up Jim,

I use wire (https://github.com/square/wire) for a number of my projects.
Unfortunately, it appears the map<string, string> syntax is too new for
wire. Looks like I have some rewriting to do. :(

Mike

On Tue, May 10, 2016 at 3:59 PM, Jim CF Campbell <jcampbell(a)pivotal.io>
wrote:

Hi cf-dev,

In support of the epic to add tagging to CF metrics
<https://www.pivotaltracker.com/epic/show/2362529>, we recently added a
map to the dropsonde-protocol
<https://github.com/cloudfoundry/dropsonde-protocol> envelope type. This
is non-breaking to metric users. However if you compile in dropsonde, this
message applies to you. This change forced us to update to a newer protobuf
compiler. If you are using the .proto definitions directly you will need to
update to the new compiler as well. You can find the latest protobuf
compiler release on github <https://github.com/google/protobuf/releases>.

Thanks, The Loggregator Team
--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963


Loggregator has updated protobufs definitions and compiler for dropsonde

Jim CF Campbell
 

Hi cf-dev,

In support of the epic to add tagging to CF metrics
<https://www.pivotaltracker.com/epic/show/2362529>, we recently added a map
to the dropsonde-protocol
<https://github.com/cloudfoundry/dropsonde-protocol> envelope type. This is
non-breaking to metric users. However if you compile in dropsonde, this
message applies to you. This change forced us to update to a newer protobuf
compiler. If you are using the .proto definitions directly you will need to
update to the new compiler as well. You can find the latest protobuf
compiler release on github <https://github.com/google/protobuf/releases>.

Thanks, The Loggregator Team
--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963


Re: HTTP request status text is changed

CF Runtime
 

In order to debug this, could you provide a sample Dockerfile for an app that demonstrates this behavior, and can you push that sample app to dockerhub? Also, if you're overriding anything like the start command or the ports, can you let us know that too?

Thanks,
Rob Dimsdale & Alvaro Perez-Shirley
CF Release Integration
Pivotal


Re: Ubuntu Xenial stemcell and rootfs plans

Mike Youngstrom
 

I may not have anything that qualifies as compelling. But, here are some
of the reasons I've got:

* If skipping Xenial that give at the most 1 year to transition from trusty
to a 2018.04 based rootfs. Lets say it takes 6 months to get the new
rootFS into our customers hands and for everyone to be comfortable enough
with it to make it the default. I don't think 6 months is enough time for
my users to naturally transition all of their applications via pushes and
restages to the new rootfs. The more time we have with the new rootfs as
the default the less I will need to bother my customers to test before I
force them to change.

* Xenial uses OpenSSL 1.0.2. Improving security by not statically
compiling OpenSSL into Node would be nice.

* With the lucid rootfs after a while it became difficult to find pre-built
libraries for Lucid. This put increased burden on me to identify and
provide lucid compatible builds for some common tools. One example of this
is wkhtmltopdf a commonly used tool in my organization.

I think the biggest thing for me is that the move from Lucid to Trusty was
a nightmare for me and my customers. Though better planning and adding a
couple of more months to the process would help, giving my users a couple
of years to migrate would be better. :)

Mike

On Mon, May 9, 2016 at 2:05 PM, Danny Rosen <drosen(a)pivotal.io> wrote:

Hey Mike,

Thanks for reaching out. We've discussed supporting Xenial recently but
have had trouble identifying compelling reasons to do so. Our current
version of the rootfs is supported until April 2019 [1] and while we do not
plan on waiting until March 2019 :) we want to understand compelling
reasons to go forward with the work sooner than later.


On Mon, May 9, 2016 at 12:47 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Ubuntu Xenial Xerus was released a few weeks ago. Any plans to
incorporate Xenial into the platform? Stemcells and/or new root fs?

The recent lucid to trusty rootfs fire drill was frustrating to my
customers. I'm hoping that this year we can get a Xenial rootfs out
loooong before trusty support ends so I don't have to put another tight
deadline on my customers to test and move.

Thoughts?

Thanks,
Mike


Re: In Diego (maybe DEA also) why is $HOME /home/vcap/app and not /home/vcap?

Mike Youngstrom
 

Thanks for doing that research Eric. So, it appears to be some requirement
for Heroku compatibility. So, I guess I have 2 issues I'd like to dig
further on that might help other from repeating the issue I experienced
above:

1. Should ssh home be /home/vcap/app also to match application running
environment?
2. Why is Java's "user.home" system property "/home/vcap". I would think
that java's user.home should always match the HOME environment variable.

Mike

On Mon, May 9, 2016 at 2:38 PM, Eric Malm <emalm(a)pivotal.io> wrote:

Hey, Mike,

HOME is also `/home/vcap/app` on the DEAs, and Diego has replicated that
execution contract for buildpack apps (the buildpack-app-lifecycle's
launcher sets HOME to that value before exec'ing the start command).
Spelunking through the DEA codebase, it seems to have been that way since
buildpack support was introduced:
https://github.com/cloudfoundry/dea_ng/commit/43ea42bcbff09175cafa65c2e84bd36fd505be07
is the earliest reference I could find to it. Since /home/vcap/app is also
symlinked to /app, this may be to keep compatibility with Heroku
conventions around the HOME env var. Someone who was around in March 2013
(or who knows Heroku better! :) ) might be able to answer that more
definitively, though.

Best,
Eric

On Thu, May 5, 2016 at 10:00 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

This burned me today because I was attempting to put fonts into the home
directory identified by the java system property "user.home" (which is
/home/vcap) instead of the environment variable $HOME which is
/var/vcap/app.

This is especially confusing because SSH home is "/home/vcap".

Anyone have any context into why $HOME for the app is /var/vcap/app and
not /var/vcap?

Thanks,
Mike


Re: Unnaccounted memory in Java application running in Cloud Foundry

Lorenzo Jorquera <lorenzo.jorquera@...>
 

cap(a)hqcquhb1smb:~$ /home/vcap/app/.java-buildpack/open_jdk_jre/bin/jcmd 37
VM.native_memory summary
37:

Native Memory Tracking:

Total: reserved=2128154KB, committed=836614KB
- Java Heap (reserved=626688KB, committed=584192KB)
(mmap: reserved=626688KB, committed=584192KB)

- Class (reserved=1129321KB, committed=91881KB)
(classes #17000)
(malloc=7017KB #22067)
(mmap: reserved=1122304KB, committed=84864KB)

- Thread (reserved=52845KB, committed=52845KB)
(thread #52)
(stack: reserved=52428KB, committed=52428KB)
(malloc=166KB #261)
(arena=252KB #102)

- Code (reserved=257053KB, committed=45609KB)
(malloc=7453KB #9660)
(mmap: reserved=249600KB, committed=38156KB)

- GC (reserved=28679KB, committed=28519KB)
(malloc=5779KB #353)
(mmap: reserved=22900KB, committed=22740KB)

- Compiler (reserved=240KB, committed=240KB)
(malloc=109KB #300)
(arena=131KB #3)

- Internal (reserved=10528KB, committed=10528KB)
(malloc=10496KB #20352)
(mmap: reserved=32KB, committed=32KB)

- Symbol (reserved=18889KB, committed=18889KB)
(malloc=15748KB #175967)
(arena=3141KB #1)

- Native Memory Tracking (reserved=3595KB, committed=3595KB)
(malloc=12KB #140)
(tracking overhead=3583KB)

- Arena Chunk (reserved=316KB, committed=316KB)
(malloc=316KB)

On Tue, May 10, 2016 at 9:14 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

Try running `jcmd <pid> VM.native_memory summary` to get a full summary of
the memory being used by the JVM. Looks like this.

```
$ jcmd 72082 VM.native_memory summary
72082:

Native Memory Tracking:

Total: reserved=3532459KB, committed=271627KB
- Java Heap (reserved=2097152KB, committed=131072KB)
(mmap: reserved=2097152KB, committed=131072KB)

- Class (reserved=1065321KB, committed=18537KB)
(classes #2667)
(malloc=361KB #3182)
(mmap: reserved=1064960KB, committed=18176KB)

- Thread (reserved=30846KB, committed=30846KB)
(thread #31)
(stack: reserved=30720KB, committed=30720KB)
(malloc=91KB #163)
(arena=35KB #60)

- Code (reserved=251209KB, committed=9669KB)
(malloc=1609KB #2458)
(mmap: reserved=249600KB, committed=8060KB)

- GC (reserved=26595KB, committed=20167KB)
(malloc=19727KB #205)
(mmap: reserved=6868KB, committed=440KB)

- Compiler (reserved=141KB, committed=141KB)
(malloc=10KB #97)
(arena=131KB #3)

- Internal (reserved=630KB, committed=630KB)
(malloc=598KB #4186)
(mmap: reserved=32KB, committed=32KB)

- Symbol (reserved=4603KB, committed=4603KB)
(malloc=3124KB #23113)
(arena=1479KB #1)

- Native Memory Tracking (reserved=530KB, committed=530KB)
(malloc=5KB #63)
(tracking overhead=525KB)

- Arena Chunk (reserved=196KB, committed=196KB)
(malloc=196KB)

- Unknown (reserved=55236KB, committed=55236KB)
(mmap: reserved=55236KB, committed=55236KB)
```

Dan

On Mon, May 9, 2016 at 5:54 PM, Lorenzo Jorquera <
lorenzo.jorquera(a)servicemax.com> wrote:

Hello,

I am running a Java application in Cloud Foundry V3. The manifest
indicates 2000Mb of memory. The Java memory parameters (obtained via jcmd)
are as follow:

-XX:CICompilerCount=3 -XX:+HeapDumpOnOutOfMemoryError
-XX:InitialHeapSize=524288000 -XX:MaxHeapSize=641728512
-XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=213909504
-XX:MetaspaceSize=268435456 -XX:MinHeapDeltaBytes=524288
-XX:NativeMemoryTracking=summary -XX:NewSize=174587904
-XX:OldSize=349700096 -XX:ThreadStackSize=1024
-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC

So, the application has a heap size of ~ 600 Mb and a Metaspace size of ~
256 Mb.

However, if I execute top, I get:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+
COMMAND
34 vcap 0 -20 5756056 1.574g 28632 S 0.3 10.7 3:12.94
java


So for the operating system the process is using nearly 1.6 Gb, while the
Heap and Metaspace are configured to only use 856Mb.

I thought that it was possible that the difference would be in Native
Memory, but the native memory reported usage is also much less than 1.6 Gb:


cap(a)hqcquhb1si9:~$ /home/vcap/app/.java-buildpack/open_jdk_jre/bin/jcmd
34 VM.native_memory
34:

Native Memory Tracking:
Total: reserved=2203649KB, committed=960861KB
- Java Heap (reserved=626688KB, committed=609280KB)
(mmap: reserved=626688KB, committed=609280KB)


So, my doubt is, what are this extra 700Mb?



--
Lorenzo Jorquera
--
Lorenzo Jorquera


Re: Unnaccounted memory in Java application running in Cloud Foundry

Daniel Mikusa
 

Try running `jcmd <pid> VM.native_memory summary` to get a full summary of
the memory being used by the JVM. Looks like this.

```
$ jcmd 72082 VM.native_memory summary
72082:

Native Memory Tracking:

Total: reserved=3532459KB, committed=271627KB
- Java Heap (reserved=2097152KB, committed=131072KB)
(mmap: reserved=2097152KB, committed=131072KB)

- Class (reserved=1065321KB, committed=18537KB)
(classes #2667)
(malloc=361KB #3182)
(mmap: reserved=1064960KB, committed=18176KB)

- Thread (reserved=30846KB, committed=30846KB)
(thread #31)
(stack: reserved=30720KB, committed=30720KB)
(malloc=91KB #163)
(arena=35KB #60)

- Code (reserved=251209KB, committed=9669KB)
(malloc=1609KB #2458)
(mmap: reserved=249600KB, committed=8060KB)

- GC (reserved=26595KB, committed=20167KB)
(malloc=19727KB #205)
(mmap: reserved=6868KB, committed=440KB)

- Compiler (reserved=141KB, committed=141KB)
(malloc=10KB #97)
(arena=131KB #3)

- Internal (reserved=630KB, committed=630KB)
(malloc=598KB #4186)
(mmap: reserved=32KB, committed=32KB)

- Symbol (reserved=4603KB, committed=4603KB)
(malloc=3124KB #23113)
(arena=1479KB #1)

- Native Memory Tracking (reserved=530KB, committed=530KB)
(malloc=5KB #63)
(tracking overhead=525KB)

- Arena Chunk (reserved=196KB, committed=196KB)
(malloc=196KB)

- Unknown (reserved=55236KB, committed=55236KB)
(mmap: reserved=55236KB, committed=55236KB)
```

Dan

On Mon, May 9, 2016 at 5:54 PM, Lorenzo Jorquera <
lorenzo.jorquera(a)servicemax.com> wrote:

Hello,

I am running a Java application in Cloud Foundry V3. The manifest
indicates 2000Mb of memory. The Java memory parameters (obtained via jcmd)
are as follow:

-XX:CICompilerCount=3 -XX:+HeapDumpOnOutOfMemoryError
-XX:InitialHeapSize=524288000 -XX:MaxHeapSize=641728512
-XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=213909504
-XX:MetaspaceSize=268435456 -XX:MinHeapDeltaBytes=524288
-XX:NativeMemoryTracking=summary -XX:NewSize=174587904
-XX:OldSize=349700096 -XX:ThreadStackSize=1024
-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC

So, the application has a heap size of ~ 600 Mb and a Metaspace size of ~
256 Mb.

However, if I execute top, I get:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
34 vcap 0 -20 5756056 1.574g 28632 S 0.3 10.7 3:12.94
java


So for the operating system the process is using nearly 1.6 Gb, while the
Heap and Metaspace are configured to only use 856Mb.

I thought that it was possible that the difference would be in Native
Memory, but the native memory reported usage is also much less than 1.6 Gb:


cap(a)hqcquhb1si9:~$ /home/vcap/app/.java-buildpack/open_jdk_jre/bin/jcmd
34 VM.native_memory
34:

Native Memory Tracking:
Total: reserved=2203649KB, committed=960861KB
- Java Heap (reserved=626688KB, committed=609280KB)
(mmap: reserved=626688KB, committed=609280KB)


So, my doubt is, what are this extra 700Mb?



--
Lorenzo Jorquera


Re: Resource Conflict error in RabbitMQ broker.

MaggieMeng
 

Hi,

We had found that it was due to a mistake in rabbitmq deployment manifest. The administrator user names are not consistent between server and broker.

Thank everyone who helped to take a look at this problem.

Regards,
Maggie

From: Guruprakash S [mailto:prakash.guru4(a)gmail.com]
Sent: 2016年5月9日 5:03
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>
Subject: [cf-dev] Re: Re: Resource Conflict error in RabbitMQ broker.

William - Thanks for looking at this.

We deleted the old rabbit cluster and deployed this new release of Pivotal. So there would not be any orphan instances. The below command was tried as well and database tables were checked to see if any service instances, but we found nothing there.

cf purge-service-offering -f p-rabbitmq

On Sun, May 8, 2016 at 1:27 PM, Guruprakash S <prakash.guru4(a)gmail.com<mailto:prakash.guru4(a)gmail.com>> wrote:
When trying to register a service against the RabbitMQ broker using cf create-service

Example:
cloudfoundry(a)micro-deploy-uspl:~/persis-store/deploy/initial$ cf create-service p-rabbitmq standard ngis-rabbit-platformngis(a)emc.com<mailto:ngis-rabbit-platformngis(a)emc.com>
Creating service ngis-rabbit-platformngis(a)emc.com<mailto:ngis-rabbit-platformngis(a)emc.com> in org EMC / space ngisspace as admin...
FAILED
Server error, status code: 409, error code: 10001, message: Resource conflict: http://192.168.225.200:4567/v2/service_instances/8e038a12-a409-4f04-96ce-6948dd009ee7?accepts_incomplete=true

We narrowed down the problem by circumventing the Cloud Controller and trying to register against the RabbitMQ Broker directly.

The broker seems to be the culprit – we tried registering the service against it directly using the following on the broker VM itself:

curl http://administrator:password(a)192.168.225.200:4567/v2/service_instances/rabbitmq-test -d '{"organization_guid": "50c0e92e-0ab7-4888-904e-cd043597972d","plan_id": "178083d7-17fe-494b-b9ed-8f4dec16b11d","service_id": "25a2f684-b1b6-4ff4-9082-7afdc38d4f86","space_guid”: "7e27e716-24da-44cb-8f60-65506efb7df2"}' -X PUT -H "X-Broker-API-Version: 2.6" -H "Content-Type: application/json”

It still resulted in throwing errors about the service already existing:

2016-May-07 22:15:30 +0000 localhost INFO [io.pivotal.pcf.rabbitmq.server] - Asked to provision a service: rabbitmq-test

2016-May-07 22:15:30 +0000 localhost WARN [io.pivotal.pcf.rabbitmq.server] - Vhost rabbitmq-test already exists

2016-May-07 22:15:30 +0000 localhost INFO [io.pivotal.pcf.rabbitmq.server] - PUT /v2/service_instances/rabbitmq-test 409 2 (in 32 ms)
Essentially we need to understand why we can’t register any services against the RabbitMQ broker.



On Sun, May 8, 2016 at 7:20 AM, William Martin <wmartin(a)pivotal.io<mailto:wmartin(a)pivotal.io>> wrote:
You can find the relevant code for this at:

https://github.com/pivotal-cf/cf-rabbitmq-release/blob/master/src/rabbitmq-broker/src/clojure/io/pivotal/pcf/rabbitmq/server.clj#L140-L165

Is it possible you have an orphaned service instance (a vhost on your rabbit cluster). I'd be surprised to see this error from a `cf create-service`, how are you trying to provision?

On Sat, May 7, 2016 at 10:44 PM, Guruprakash S <prakash.guru4(a)gmail.com<mailto:prakash.guru4(a)gmail.com>> wrote:
Hi,

We are seeing this error while creating a service instance against a latest RabbitMQ deployment which we just deployed:

Server error, status code: 409, error code: 10001, message: Resource conflict:

We checked the CC database table service_instances to see if there are any service_instances related to RMQ and we don't see any there.

Request your help.

Thanks,
Guru.


Re: May CAB call next Wednesday May 11, 2016 @ 8a PDT

Michael Maximilien
 

Final reminder. CAB call this Wednesday @ 8a PDT. Best,

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone

On May 4, 2016, at 2:28 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote:


Hi, all,

Quick reminder of the CAB call next Wednesday, May 11th @ 8a PDT. All info in link:

https://docs.google.com/document/d/1SCOlAquyUmNM-AQnekCOXiwhLs6gveTxAcduvDcW_xI/edit#heading=h.o44xhgvum2we

Remember, no more status update but rather discussions, so come ready with your questions. If you are a PM, please update your project's section and join the call for QAs.

Also, everyone, please join the slack.cloudfoundry.org and join the #CAB channel for previous and future discussions.

This is the call before summit coming end of month in Santa Clara. Register today (https://www.cloudfoundry.org/community/summits/program/about/?summitId=10016) if you have not already.

Talk to you all next week. We'll send one more reminder on this list.

Best,

Chip, James, and Max

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


Admin Buildpack Location Feature Narrative

Dubois, Jan <jan.dubois@...>
 

I have written up a feature narrative about making admin buildpacks available to meta-buildpacks like the heroku-buildpack-multi, which delegates staging to other buildpacks. The narrative also includes a link to some exploratory code that shows how this feature could be used by the multi-buildpack:

https://docs.google.com/document/d/1APcx1EbdRDtGiUfXEbiM2-7Sq-wxMGS85a5ICt65HTc/edit#

Please leave any comments inside the document. This kind of feedback is an important indicator to the CF Foundation to show if there is community interest in a particular feature, or not!

Cheers,
-Jan


Unnaccounted memory in Java application running in Cloud Foundry

Lorenzo Jorquera <lorenzo.jorquera@...>
 

Hello,

I am running a Java application in Cloud Foundry V3. The manifest
indicates 2000Mb of memory. The Java memory parameters (obtained via jcmd)
are as follow:

-XX:CICompilerCount=3 -XX:+HeapDumpOnOutOfMemoryError
-XX:InitialHeapSize=524288000 -XX:MaxHeapSize=641728512
-XX:MaxMetaspaceSize=268435456 -XX:MaxNewSize=213909504
-XX:MetaspaceSize=268435456 -XX:MinHeapDeltaBytes=524288
-XX:NativeMemoryTracking=summary -XX:NewSize=174587904
-XX:OldSize=349700096 -XX:ThreadStackSize=1024
-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC

So, the application has a heap size of ~ 600 Mb and a Metaspace size of ~
256 Mb.

However, if I execute top, I get:

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
34 vcap 0 -20 5756056 1.574g 28632 S 0.3 10.7 3:12.94
java


So for the operating system the process is using nearly 1.6 Gb, while the
Heap and Metaspace are configured to only use 856Mb.

I thought that it was possible that the difference would be in Native
Memory, but the native memory reported usage is also much less than 1.6 Gb:


cap(a)hqcquhb1si9:~$ /home/vcap/app/.java-buildpack/open_jdk_jre/bin/jcmd 34
VM.native_memory
34:

Native Memory Tracking:
Total: reserved=2203649KB, committed=960861KB
- Java Heap (reserved=626688KB, committed=609280KB)
(mmap: reserved=626688KB, committed=609280KB)


So, my doubt is, what are this extra 700Mb?



--
Lorenzo Jorquera


Re: In Diego (maybe DEA also) why is $HOME /home/vcap/app and not /home/vcap?

Eric Malm <emalm@...>
 

Hey, Mike,

HOME is also `/home/vcap/app` on the DEAs, and Diego has replicated that
execution contract for buildpack apps (the buildpack-app-lifecycle's
launcher sets HOME to that value before exec'ing the start command).
Spelunking through the DEA codebase, it seems to have been that way since
buildpack support was introduced:
https://github.com/cloudfoundry/dea_ng/commit/43ea42bcbff09175cafa65c2e84bd36fd505be07
is the earliest reference I could find to it. Since /home/vcap/app is also
symlinked to /app, this may be to keep compatibility with Heroku
conventions around the HOME env var. Someone who was around in March 2013
(or who knows Heroku better! :) ) might be able to answer that more
definitively, though.

Best,
Eric

On Thu, May 5, 2016 at 10:00 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

This burned me today because I was attempting to put fonts into the home
directory identified by the java system property "user.home" (which is
/home/vcap) instead of the environment variable $HOME which is
/var/vcap/app.

This is especially confusing because SSH home is "/home/vcap".

Anyone have any context into why $HOME for the app is /var/vcap/app and
not /var/vcap?

Thanks,
Mike


Re: Ubuntu Xenial stemcell and rootfs plans

Danny Rosen
 

Hey Mike,

Thanks for reaching out. We've discussed supporting Xenial recently but
have had trouble identifying compelling reasons to do so. Our current
version of the rootfs is supported until April 2019 [1] and while we do not
plan on waiting until March 2019 :) we want to understand compelling
reasons to go forward with the work sooner than later.

On Mon, May 9, 2016 at 12:47 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Ubuntu Xenial Xerus was released a few weeks ago. Any plans to
incorporate Xenial into the platform? Stemcells and/or new root fs?

The recent lucid to trusty rootfs fire drill was frustrating to my
customers. I'm hoping that this year we can get a Xenial rootfs out
loooong before trusty support ends so I don't have to put another tight
deadline on my customers to test and move.

Thoughts?

Thanks,
Mike


Cloud Foundry Foundation Contributor Survey

Stormy
 

The Cloud Foundry Foundation would like to better know those that
contribute to Cloud Foundry so that we can support this work in the best
way possible.

If you contribute to Cloud Foundry in any way (write code or docs, answer
questions, organize events, design logos, etc), please help us by filling
out the following survey.

It should take about 10 minutes of your time.

https://www.surveymonkey.com/r/N38WDHT
Thanks for your help!

Stormy


Ubuntu Xenial stemcell and rootfs plans

Mike Youngstrom
 

Ubuntu Xenial Xerus was released a few weeks ago. Any plans to incorporate
Xenial into the platform? Stemcells and/or new root fs?

The recent lucid to trusty rootfs fire drill was frustrating to my
customers. I'm hoping that this year we can get a Xenial rootfs out
loooong before trusty support ends so I don't have to put another tight
deadline on my customers to test and move.

Thoughts?

Thanks,
Mike


Re: CR Push failure

shah c
 

Thank you Dan for such a wonderful review of my post.

The credentials in my post are morphed...so that's taken care :).

The application that I am trying is simple Node.js app. Will try out all points that you mentioned in your post and post back the results here.

Thanks,
shahc


Re: CR Push failure

Dan Wendorf
 

Hi shahc,

Dan's answer covers most of the details. Cloud Foundry applications are
intended to be long-lived (endless until you explicitly stop them)
processes that respond to web requests. The health-check is attempting to
confirm this by connecting to the application via the HTTP port provided to
the application by the PORT variable.

Because you've set your start command to a non-web command (a MySQL task),
it will not respond on that port and Cloud Foundry considers that to be a
failure. If you want to perform one-off tasks during app start (e.g.
seeding your database), that's best done as part of your application's
startup process. A common pattern in pseudo-code is something like:

```
class MyApplicationBootstrap {
function start() {
if (migrationsNeeded()) {
runMigrations();
}

MyApplication.new().start();
}
}
```

There will be support for one-off tasks (what you are attempting to do) in
the near-future (see the current documentation for tasks
<http://v3-apidocs.cloudfoundry.org/version/release-candidate/index.html#tasks>
in
the experimental V3 API — note that it is subject to change and not yet
stable), but until then, the in-app startup bootstrap is the best solution.

Additionally, take care to not copy-paste credentials when posting in a
public forum. I recommend cycling the credentials for your service as soon
as possible.


Hope this helps,
(the other) Dan

On Mon, May 9, 2016 at 4:56 AM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Mon, May 9, 2016 at 2:41 AM, Stanley Shen <meteorping(a)gmail.com> wrote:

From the message "failed to accept connections within health check
timeout"
You can try to disable health check and have another try to see if it
works

cf set-health-check APP_NAME 'none'
Don't be led astray here. With the DEA when an app crashes, you will
pretty much always be told "failed to accept connection within health check
timeout" (Diego is better and gives more helpful error messages). The DEA
isn't necessarily wrong, since the app in question isn't running and
listening for connections. It just lacks any sort helpful details about
why the app isn't running and listening for connections.

Possible causes:

- app not listening on the assigned port (i.e. $PORT)
- app has exited / crashes for some reason (look at exit_code in crash
message and application logs for clues)
- app is starting too slow (look at timestamp from DEA for `Starting
app instance` and subtract it from timestamp from DEA for crash message, if
the difference is right around the timeout then this is the problem)

Also, when you hit the start up timeout, you should *not* disable the
health check. You should first look to see why your application is taking
so long to start, especially if you're not expecting it to take a long time
to start. Check the application logs and see what it's doing. Three
common problems are when an app is having connection problems to the DB (or
other services), how Java apps can pause while getting random data (when
there isn't enough entropy on the system) and when APM / monitoring
solutions are enabled as they add overhead which can be particularly bad
during a busy startup.

At any rate, if you can fix the problem do. That's the best solution. If
slow startup is expected then increase the timeout with the `-t` option to
`cf push`. By default, you can increase it from 60s to 180 seconds.
Operators can increase both the default and max timeout for their platform.

Lastly, `cf set-health-check` is for Diego. To disable the health check
for DEAs, you would just use `cf push --no-route`.

Dan


Re: restart APP launched by docker still need to access remote docker image?

Daniel Mikusa
 

On Mon, May 9, 2016 at 8:38 AM, Stanley Shen <meteorping(a)gmail.com> wrote:

OK, thanks for information, I created a docker image and pushed as an APP.
And I can see it's pretty soon in staging process since we had nothing to
stage.
But the whole process still takes about 3 minutes, and the most time is in
"starting the instance"

I want to know what is happening in this "0 of 1 instances running, 1
starting" phrase?
When did the docker image download from the docker hub?
Start running `cf logs` in a second terminal. The log output should show
more details about what is happening while the app is starting. Hopefully
that will show what is taking up the time.

Dan



As we know, the docker image should can be started in seconds.
If we create our own docker register, can we reduce the time?


=========================================================================
Mon May 9 19:50:55 CST 2016
Using manifest file /Users/test/stanley/0506/test/manifest.yml

Creating app test in org stanley / space stanley as admin...
OK

Using route stanley.test.io
Binding stanley.test.io to test...
OK

Starting app test in org stanley / space stanley as admin...
Creating container
Successfully created container
Staging...
Staging process started ...
Staging process finished
Exit status 0
Staging Complete

0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
0 of 1 instances running, 1 starting
1 of 1 instances running

App started

OK

App test was started using this command `/docker-entrypoint.bash java
-Djava.io.tmpdir=/tmp/jetty -jar /usr/local/jetty/start.jar`
Showing health and status for app test in org stanley / space stanley as
admin...
OK

requested state: started
instances: 1/1
usage: 5G x 1 instances
urls: stanley.test.io
last uploaded: Mon May 9 11:51:00 UTC 2016
stack: cflinuxfs2
buildpack: unknown

state since cpu memory disk
details
#0 running 2016-05-09 07:53:46 PM 210.9% 3G of 5G 109.1M of 5G
Mon May 9 19:54:00 CST 2016

4581 - 4600 of 9425