Date   

Service Broker API - updating instances

Mike Heath
 

One of the really nice features of the V2 Service Broker API is the ability
to broker multiple services from a single process. Each service provides
its unique id in the broker catalog. The Cloud Controller then includes
that unique id with requests to the broker making it easy for the broker to
route the request to the correct service.

The problem I'm currently having is that the API added in version 2.4 for
updating a service instances does not include the service's unique id [1].
It only provides the service instance's id and the new plan id. This makes
it very difficult to route the update request to the correct service
provider within a process brokering multiple services as it would require
querying each service provider to determine if it brokered the provided
instance.

I would like to propose adding a service_id field to the update request,
the same as every other request in the service broker API. I believe that
the main issue with this is that it implies that an instance's service
provider could be changed by the broker. I think proper documentation would
alleviate that issue though.

Thoughts?

-Mike

1 - http://docs.cloudfoundry.org/services/api.html#updating_service_instance


How random is Metron's Doppler selection?

Mike Heath
 

Metron's documentation [1] says "All Metron traffic is randomly distributed
across available Dopplers." How random is this? Based on observation, it
appears that logs for an individual application instance are consistently
sent to the same Doppler instance. The consistency aspect is very important
for us so that our Syslog forwarder can consolidate stack traces into a
single logging event.

How random is this distribution really for an application instance's logs?

-Mike

1 - https://github.com/cloudfoundry/loggregator/tree/develop/src/metron


Re: TCP Router VS NoRouter

Mike Youngstrom
 

Thanks for your interest. We are close to supporting all current routing
experimental features but we recently experienced some personnel issues
which will caused us to lower NoRouter priority for a few months. :( We'll
update the list when we think it is production ready.

Mike

On Thu, Jun 11, 2015 at 1:22 PM, Mohamed, Owais <Owais.Mohamed(a)covisint.com>
wrote:

Makes perfect sense. I really like the NoRouter because of its simplicity.
I really hope CloudFoundry adopts it and makes it part of its official
release so that NoRouter stays in step with GoRouter updates.

Thanks,
Owais





Thinking a little more about your question Mohamed. In his presentation
Mike might have made reference to people making adhoc changes to F5. In
our organization today we have a handfull of engineers that have LTM
access. And some of our existing applications that we're moving to Cloud
Foundry do require some specific LTM config or behaviour since they
required that behaviour previously. One of the benfits of NoRouter is it
simplifies the porting of these applications to Cloud Foundry since they
can make the same or similar changes to the LTM that they needed before.
We definitly view these use cases as a hacks though and are using them to
compile a list of official Service Brokers we need to eventually provide
to
formalize this functionality in NoRouter. Other orgs may not have the
requirement and would prefer that nobody touches the LTM. That type of
org
would work well with NoRouter as well.

Hope that helps.

Mike

On Thu, Jun 11, 2015 at 9:50 AM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

Thanks for responding Mohamed.

Help me understand what you mean when you refer to opening up the LTM to
developers. NoRouter itself must have access to the LTM to update app
pool
members and such but end users/developers themselves will not actually
have
direct access to the LTM.

Some of the features Mike mentioned in his presentation were more a
comment on what is possible if we enable a feature to do so. When/If
NoRouter were to expose some of that functionality it would most likely
by
done using a Service Broker. For example, if an application wishes to
provide a custom OOS page we wouldn't give the developer direct access
to
the LTM to configure an OOS page, instead, I think we'd provide an OOS
Page
Service Broker. When created and bound to an application it would
prompt
NoRouter to configure the virtual server to use that Custom OOS page for
all of the routers mapped to that application. Keeping the details
sufficiently removed from the developer.

Does that help answer your question?

Mike

On Thu, Jun 11, 2015 at 9:04 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Apologize for restarting this thread after a few weeks.

That?s right Mike. Once the LTM opens up to the developers they might
start putting in too much functionality into the Load Balancer like
filters, interceptors. By using NoRouter the LTM will have to be
opened up
to the developer team.

I know that there has to be discipline and access control. But was
just
putting it out as con in the NoRouter Section.

Hoping that Chip does not have concerns with discussing NoRouter here.
Let me know and I can take this discussion offline.

From: Mike Youngstrom <youngm(a)gmail.com>
Date: Monday, May 18, 2015 at 3:22 PM
To: Chip Childers <cchilders(a)cloudfoundry.org>
Cc: Owais Mohamed <owais.mohamed(a)covisint.com>, "
cf-dev(a)lists.cloudfoundry.org" <cf-dev(a)lists.cloudfoundry.org>
Subject: Re: [cf-dev] TCP Router VS NoRouter

Hi Owais,

What are you referring to when you say you're concerned about the LTM
becoming a Monolith? Too much functionality in one component? Too
much of
the system depending on one component?

Chip,

Hopefully it's ok to discuss NoRouter here even though it isn't an
official CF project. Let us know if it is not.

Mike

On Fri, May 15, 2015 at 10:55 AM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

The "norouter", while interesting, isn't the official CF project
approach to HTTP traffic routing. The TCP router is being built to
support
TCP routing as a general solution, with the goal of it becoming an
official
part of the CF release when ready.

-chip

Chip Childers | Technology Chief of Staff | Cloud Foundry Foundation

On Fri, May 15, 2015 at 8:58 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Hi,

I attended sessions on both TCP Router (Cloud Foundry and IOT
protocol support by Atul Kshirsagar) and NoRouter (Norouter: Running
Cloud Foundry without the Gorouter by Mike Heath).

I just wanted to start a discussion on the pros and cons of each
approach.

As personal opinion I think NoRouter is a simpler approach and can
definitely be made to support IOT protocols. The main drawback I see
with
the NoRouter is the danger of the LTM becoming a Monolith.

Any suggestions\ideas?

Regards,
Owais


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

*************************************
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Key Rotation Strategies

Mike Youngstrom
 

On Thu, Jun 11, 2015 at 12:33 PM, CF Runtime <cfruntime(a)gmail.com> wrote:

as well as info on other keys that need attention.
That is a good question. Looking through a manifest it is difficult to
pick out which keys/passwords should be rotated. A quick perusal found
these candidates:

* UAA/CC DB passwords can probably be rotated without issue.
* Might initially think Nats should be rotated. However, since etcd isn't
even password protected we're probably ok relying on network firewall for
nats and etcd.
* cc.bulk_api_password should probably be rotated and could cause downtime
if the components that rely upon it don't have their config changed the
exact same time the CC is changed
* doppler_endpoint.shared_secret will probably cause the loss of messages
if rotated
* uaa.cc.token_secret would probably cause everyone to get logged out
right? Probably not optimal.
* uaa.cc.client_secret might cause connectivity issues between UAA and CC
while rotating could cause some downtime

Any others?

Mike


Re: TCP Router VS NoRouter

Owais Mohamed
 

Makes perfect sense. I really like the NoRouter because of its simplicity.
I really hope CloudFoundry adopts it and makes it part of its official
release so that NoRouter stays in step with GoRouter updates.

Thanks,
Owais


Thinking a little more about your question Mohamed. In his presentation
Mike might have made reference to people making adhoc changes to F5. In
our organization today we have a handfull of engineers that have LTM
access. And some of our existing applications that we're moving to Cloud
Foundry do require some specific LTM config or behaviour since they
required that behaviour previously. One of the benfits of NoRouter is it
simplifies the porting of these applications to Cloud Foundry since they
can make the same or similar changes to the LTM that they needed before.
We definitly view these use cases as a hacks though and are using them to
compile a list of official Service Brokers we need to eventually provide
to
formalize this functionality in NoRouter. Other orgs may not have the
requirement and would prefer that nobody touches the LTM. That type of
org
would work well with NoRouter as well.

Hope that helps.

Mike

On Thu, Jun 11, 2015 at 9:50 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Thanks for responding Mohamed.

Help me understand what you mean when you refer to opening up the LTM to
developers. NoRouter itself must have access to the LTM to update app
pool
members and such but end users/developers themselves will not actually
have
direct access to the LTM.

Some of the features Mike mentioned in his presentation were more a
comment on what is possible if we enable a feature to do so. When/If
NoRouter were to expose some of that functionality it would most likely
by
done using a Service Broker. For example, if an application wishes to
provide a custom OOS page we wouldn't give the developer direct access
to
the LTM to configure an OOS page, instead, I think we'd provide an OOS
Page
Service Broker. When created and bound to an application it would
prompt
NoRouter to configure the virtual server to use that Custom OOS page for
all of the routers mapped to that application. Keeping the details
sufficiently removed from the developer.

Does that help answer your question?

Mike

On Thu, Jun 11, 2015 at 9:04 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Apologize for restarting this thread after a few weeks.

That?s right Mike. Once the LTM opens up to the developers they might
start putting in too much functionality into the Load Balancer like
filters, interceptors. By using NoRouter the LTM will have to be
opened up
to the developer team.

I know that there has to be discipline and access control. But was
just
putting it out as con in the NoRouter Section.

Hoping that Chip does not have concerns with discussing NoRouter here.
Let me know and I can take this discussion offline.

From: Mike Youngstrom <youngm(a)gmail.com>
Date: Monday, May 18, 2015 at 3:22 PM
To: Chip Childers <cchilders(a)cloudfoundry.org>
Cc: Owais Mohamed <owais.mohamed(a)covisint.com>, "
cf-dev(a)lists.cloudfoundry.org" <cf-dev(a)lists.cloudfoundry.org>
Subject: Re: [cf-dev] TCP Router VS NoRouter

Hi Owais,

What are you referring to when you say you're concerned about the LTM
becoming a Monolith? Too much functionality in one component? Too
much of
the system depending on one component?

Chip,

Hopefully it's ok to discuss NoRouter here even though it isn't an
official CF project. Let us know if it is not.

Mike

On Fri, May 15, 2015 at 10:55 AM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

The "norouter", while interesting, isn't the official CF project
approach to HTTP traffic routing. The TCP router is being built to
support
TCP routing as a general solution, with the goal of it becoming an
official
part of the CF release when ready.

-chip

Chip Childers | Technology Chief of Staff | Cloud Foundry Foundation

On Fri, May 15, 2015 at 8:58 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Hi,

I attended sessions on both TCP Router (Cloud Foundry and IOT
protocol support by Atul Kshirsagar) and NoRouter (Norouter: Running
Cloud Foundry without the Gorouter by Mike Heath).

I just wanted to start a discussion on the pros and cons of each
approach.

As personal opinion I think NoRouter is a simpler approach and can
definitely be made to support IOT protocols. The main drawback I see
with
the NoRouter is the danger of the LTM becoming a Monolith.

Any suggestions\ideas?

Regards,
Owais


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

*************************************


Re: Staging error: no available stagers (status code: 400, error code: 170001)

CF Runtime
 

The issue was likely the disk. The staging_disk_limit_mb tells the cloud
controller how much disk space is needed for staging an app (unless the app
asks for more). This is normally much higher than the size of the app
itself because you have to accommodate all of the buildpacks on the DEA in
addition to the app itself.

So the staging_disk_limit_mb was 6144 mb, but the DEA was set to advertise
2048 mb.

Joseph Palermo
CF Runtime Team

On Thu, Jun 4, 2015 at 9:12 PM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

Finally, it works after I changed the configuration related to memory and
disk for dea.

< disk_mb: 2048
---
disk_mb: 10000
1025c1029
< memory_overcommit_factor: 3
---
memory_overcommit_factor: 8
1028c1032
< staging_disk_limit_mb: 6144
---
staging_disk_limit_mb: 4096
[#1] Received on [staging.advertise] :
'{"id":"0-2b2f83b4755749aba3c31cc58a69a306","stacks":["lucid64","cflinuxfs2"],"available_memory":8192,"available_disk":20000}'
[#2] Received on [staging.advertise] :
'{"id":"0-2b2f83b4755749aba3c31cc58a69a306","stacks":["lucid64","cflinuxfs2"],"available_memory":8192,"available_disk":20000}'

After I deployed a simple php-demo application, they are changed to

[#8] Received on [staging.advertise] :
'{"id":"0-2b2f83b4755749aba3c31cc58a69a306","stacks":["lucid64","cflinuxfs2"],"available_memory":8064,"available_disk":18976}'
[#9] Received on [staging.advertise] :
'{"id":"0-2b2f83b4755749aba3c31cc58a69a306","stacks":["lucid64","cflinuxfs2"],"available_memory":8064,"available_disk":18976}'


However, I still cannot understand why my previous configuration led to
"Staging error: no available stagers" as the nats messages below said it
has enough resource. My php application is only consuming 128M memory and
1G disk. Who can share some insights?

[#41] Received on [staging.advertise] :
'{"id":"0-05b732df21c54f9cab3ac42869b4be64","stacks":["lucid64","cflinuxfs2"],"available_memory":3072,"available_disk":4096}'


ubuntu(a)boshclivm:~/apps/cf-php-demo$ cat manifest.yml
---
applications:
- name: cf-php-demo
memory: 128M
instances: 1
host: cf-php-demo
path: .

ubuntu(a)boshclivm:~/apps/cf-php-demo$ cf apps
name requested state instances memory disk urls
cf-php-demo started 1/1 128M 1G
cf-php-demo.runmyapp.io



On Thu, Jun 4, 2015 at 5:08 PM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

I got the nats message on 'staging.advertise'. It has the enough
resource. but it seems not correct. And it also cannot explain the error -
Server error, status code: 400, error code: 170001, message: Staging error:
no available stagers.

[#41] Received on [staging.advertise] :
'{"id":"0-05b732df21c54f9cab3ac42869b4be64","stacks":["lucid64","cflinuxfs2"],"available_memory":3072,"available_disk":4096}'
[#42] Received on [staging.advertise] :
'{"id":"0-05b732df21c54f9cab3ac42869b4be64","stacks":["lucid64","cflinuxfs2"],"available_memory":3072,"available_disk":4096}'
[#43] Received on [staging.advertise] :
'{"id":"0-05b732df21c54f9cab3ac42869b4be64","stacks":["lucid64","cflinuxfs2"],"available_memory":3072,"available_disk":4096}'
[#44] Received on [staging.advertise] :
'{"id":"0-05b732df21c54f9cab3ac42869b4be64","stacks":["lucid64","cflinuxfs2"],"available_memory":3072,"available_disk":4096}'



+------------------------------------+---------+---------------+---------------+
| Job/index | State | Resource Pool |
IPs |

+------------------------------------+---------+---------------+---------------+
| api_worker_z1/0 | running | small_z1 |
100.64.0.23 |
| api_z1/0 | running | medium_z1 |
100.64.0.21 |
| clock_global/0 | running | medium_z1 |
100.64.0.22 |
| etcd_z1/0 | running | medium_z1 |
100.64.1.8 |
| ha_proxy_z1/0 | running | router_z1 |
100.64.1.0 |
| | | |
137.172.74.90 |
| hm9000_z1/0 | running | medium_z1 |
100.64.0.24 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 |
100.64.0.27 |
| loggregator_z1/0 | running | medium_z1 |
100.64.0.26 |
| login_z1/0 | running | medium_z1 |
100.64.0.20 |
| nats_z1/0 | running | medium_z1 |
100.64.1.2 |
| nfs_z1/0 | running | medium_z1 |
100.64.1.3 |
| postgres_z1/0 | running | medium_z1 |
100.64.1.4 |
| router_z1/0 | running | router_z1 |
100.64.1.5 |
| runner_z1/0 | running | runner_z1 |
100.64.0.25 |
| stats_z1/0 | running | small_z1 |
100.64.0.18 |
| uaa_z1/0 | running | medium_z1 |
100.64.0.19 |

+------------------------------------+---------+---------------+---------------+


- 100.64.0.25

m1.large | 8GB RAM | 4 VCPU | 20.0GB Disk

92cf66ec-f2e1-4505-bd25-28c02e991535 | m1.large | 8192 |
20 | 20 | | 4 | 1.0 | True


On Thu, Jun 4, 2015 at 11:57 AM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:


From the source code
/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/lib/cloud_controller/dea/app_stager_task.rb:26,
it seems there is no enough for memory or disk.

def stage(&completion_callback)
@stager_id = @stager_pool.find_stager(@app.stack.name,
staging_task_memory_mb, staging_task_disk_mb)
raise Errors::ApiError.new_from_details('StagingError', 'no
available stagers') unless @stager_id


However, this is my first app. It should be light. The DEA is using
m1.large which is
m1.large | 4096 | 20

Anyone has the same error? and any suggestion on manifest or debug tips?

Another question, I want to add more debug information in
cloud_controller_ng.log. I tried to add some code in
/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/lib/cloud_controller/dea/app_stager_task.rb,
but it did not show in the log. How to do?


On Thu, Jun 4, 2015 at 10:14 AM, Guangcai Wang <guangcai.wang(a)gmail.com>
wrote:

attached the deployment manifest. This is generated by spiff and then I
modified it.

On Thu, Jun 4, 2015 at 12:47 AM, Takeshi Morikawa <moog0814(a)gmail.com>
wrote:

Please check the 'staging.advertise' of nats message
https://github.com/cloudfoundry/dea_ng#staging

sample command:
bundle exec nats-sub -s
nats://[nats.user]:[nats.password]@[nats_ipaddress]:[nats.port]
'staging.advertise'


I have one additional request
Can you share your bosh deployment manifest?

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Key Rotation Strategies

CF Runtime
 

There is currently no way for users to rotate the cc.db_encryption_key.
We're going to schedule some work to look into ways to solve the problem
without downtime. Any input would be great, as well as info on other keys
that need attention.

Joseph Palermo
CF Runtime Team

On Thu, Jun 11, 2015 at 10:44 AM, Christopher B Ferris <chrisfer(a)us.ibm.com>
wrote:

We are also very interested in pursuing this capability.

Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Open Cloud
IBM Software Group, Open Technologies
email: chrisfer(a)us.ibm.com
twitter: @christo4ferris
blog: http://thoughtsoncloud.com/index.php/author/cferris/
phone: +1 508 667 0402

[image: Inactive hide details for Mike Youngstrom ---06/11/2015 01:31:45
PM---There are a lot of Keys in my CF deployment manifest. I']Mike
Youngstrom ---06/11/2015 01:31:45 PM---There are a lot of Keys in my CF
deployment manifest. I'd like to be able to rotate them. Most of

From: Mike Youngstrom <youngm(a)gmail.com>
To: CF Developers Mailing List <cf-dev(a)lists.cloudfoundry.org>
Date: 06/11/2015 01:31 PM
Subject: [cf-dev] Key Rotation Strategies
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org
------------------------------



There are a lot of Keys in my CF deployment manifest. I'd like to be able
to rotate them. Most of the keys I could probably just change in a
deployment but would cause some downtime or a service disruption. Others
like "cc.db_encryption_key" I have no idea how I'd rotate.

Any thoughts on key rotation for a CF deployment?

Mike
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Key Rotation Strategies

Christopher B Ferris <chrisfer@...>
 

We are also very interested in pursuing this capability.

Cheers,

Christopher Ferris
IBM Distinguished Engineer, CTO Open Cloud
IBM Software Group, Open Technologies
email: chrisfer(a)us.ibm.com
twitter: @christo4ferris
blog: http://thoughtsoncloud.com/index.php/author/cferris/
phone: +1 508 667 0402



From: Mike Youngstrom <youngm(a)gmail.com>
To: CF Developers Mailing List <cf-dev(a)lists.cloudfoundry.org>
Date: 06/11/2015 01:31 PM
Subject: [cf-dev] Key Rotation Strategies
Sent by: cf-dev-bounces(a)lists.cloudfoundry.org



There are a lot of Keys in my CF deployment manifest.  I'd like to be able
to rotate them.  Most of the keys I could probably just change in a
deployment but would cause some downtime or a service disruption.  Others
like "cc.db_encryption_key" I have no idea how I'd rotate.

Any thoughts on key rotation for a CF deployment?

Mike
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Key Rotation Strategies

Mike Youngstrom
 

There are a lot of Keys in my CF deployment manifest. I'd like to be able
to rotate them. Most of the keys I could probably just change in a
deployment but would cause some downtime or a service disruption. Others
like "cc.db_encryption_key" I have no idea how I'd rotate.

Any thoughts on key rotation for a CF deployment?

Mike


Re: HM9000 metrics

CF Runtime
 

I believe the full crashed Warden container is kept around for an hour.

The DEA keeps the Warden handle to the container. The Warden grace time
only applies after all handles have been released.

Joseph Palermo
CF Runtime Team

On Thu, Jun 11, 2015 at 8:55 AM, Pablo Alonso Rodriguez <palonsoro(a)gmail.com
wrote:
Ok. Just a question: When you say "the DEA will keep its container carcass
around for an hour", you mean that the DEA does not remove the container
files. However, if Warden grace time is configured at 300 seconds (5
minutes), the container is actually destroyed after that time (although its
files remain). Is this right?

Thank you very much.

2015-06-11 17:27 GMT+02:00 Dieu Cao <dcao(a)pivotal.io>:

Hi Pablo,

Ops Metrics is a PCF product and questions about that should be directed
to Pivotal customer support.

Regarding your second question, about the difference between crashed
indices and crashed indexes.

The NumberOfCrashedInstances metric is usually about 4 times the
NumberOfCrashedIndices metric. First, NumberOfCrashedInstances is the total
number of crashed containers that remain on the DEAs, while
NumberOfCrashedIndices is the number of app-index pairs which have only
crashed instances.

If an app has a droplet that crashes on startup, HM9000 will eventually
settle on restarting an instance at each of its indices every 16
minutes. When the instance crashes, the DEA will keep its container
carcass around for an hour (to allow the space developers to inspect its
files via the files API if they have the instance guid). So on average,
there will be 60/16 = 3.75 crashed instances in the system per crashed
index. That should account for most of the indices and instances that
are crashed in the system.

Hope that helps.

-Dieu
CF Runtime PM



On Thu, Jun 11, 2015 at 4:48 AM, Pablo Alonso Rodriguez <
palonsoro(a)gmail.com> wrote:

Good morning.

Recently, I have been revising metrics emitted by CF components. In
order to understand HM9000 metrics, I have been reading the metrics
documentation (at https://github.com/cloudfoundry/hm9000#metricsserver)

I post this message because I have two questions.

First question:

Not all the metrics retrieved via Ops Metrics are documented there. Is
there any additional documentation? If not, could you please explain my
what do the following metrics mean?

- StartEvacuating, StartCrashed, StartMissing
- StopDuplicate, StopEvacuationComplete, StopExtra

I have some guesses about some of them, but I am not completely sure
about them.

Second question:

I do not fully understand the difference between the concepts of
"instances" and "indices" at metrics like "NumberOfCrashedIndices" and
"NumberOfCrashedInstances".

For example, I have one crashed app in my CF instance, and
"NumberOfCrashedIndices" reports '1' and "NumberOfCrashedInstances" reports
'3'. If I have a look at `cf app myapp`, I see one single crashed instance
(this was expected). If I have a look at hm9000 dump, I see the following
about my crashed app (UUIDs have been replaced by false ones):

Guid: 7ef08c44-102d-11e5-9c0d-0fb30c2610f7 | Version:
8e16b09a-102d-11e5-b6ce-27f9445313f8
Desired: [1] instances, (STARTED, STAGED)
Heartbeats:
[0 CRASHED] a42a7236102d11e5813abfab583ad850 on 1-abc
[0 CRASHED] b35b9f1e102d11e5ad29cfc4c2c4e3ea on 2-ac3
[0 CRASHED] bbd37658102d11e5ba8e2b98d1fd1793 on 4-a67
CrashCounts: [0]:7499
Pending Starts:
[0] priority:1.00 send:2m34.628437793s

So, what does all this mean? I do not understand why do I get 3
heartbeats while
I only was trying to start a single instance.

Thank you in advance




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: TCP Router VS NoRouter

Mike Youngstrom
 

Thinking a little more about your question Mohamed. In his presentation
Mike might have made reference to people making adhoc changes to F5. In
our organization today we have a handfull of engineers that have LTM
access. And some of our existing applications that we're moving to Cloud
Foundry do require some specific LTM config or behaviour since they
required that behaviour previously. One of the benfits of NoRouter is it
simplifies the porting of these applications to Cloud Foundry since they
can make the same or similar changes to the LTM that they needed before.
We definitly view these use cases as a hacks though and are using them to
compile a list of official Service Brokers we need to eventually provide to
formalize this functionality in NoRouter. Other orgs may not have the
requirement and would prefer that nobody touches the LTM. That type of org
would work well with NoRouter as well.

Hope that helps.

Mike

On Thu, Jun 11, 2015 at 9:50 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

Thanks for responding Mohamed.

Help me understand what you mean when you refer to opening up the LTM to
developers. NoRouter itself must have access to the LTM to update app pool
members and such but end users/developers themselves will not actually have
direct access to the LTM.

Some of the features Mike mentioned in his presentation were more a
comment on what is possible if we enable a feature to do so. When/If
NoRouter were to expose some of that functionality it would most likely by
done using a Service Broker. For example, if an application wishes to
provide a custom OOS page we wouldn't give the developer direct access to
the LTM to configure an OOS page, instead, I think we'd provide an OOS Page
Service Broker. When created and bound to an application it would prompt
NoRouter to configure the virtual server to use that Custom OOS page for
all of the routers mapped to that application. Keeping the details
sufficiently removed from the developer.

Does that help answer your question?

Mike

On Thu, Jun 11, 2015 at 9:04 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Apologize for restarting this thread after a few weeks.

That’s right Mike. Once the LTM opens up to the developers they might
start putting in too much functionality into the Load Balancer like
filters, interceptors. By using NoRouter the LTM will have to be opened up
to the developer team.

I know that there has to be discipline and access control. But was just
putting it out as con in the NoRouter Section.

Hoping that Chip does not have concerns with discussing NoRouter here.
Let me know and I can take this discussion offline.

From: Mike Youngstrom <youngm(a)gmail.com>
Date: Monday, May 18, 2015 at 3:22 PM
To: Chip Childers <cchilders(a)cloudfoundry.org>
Cc: Owais Mohamed <owais.mohamed(a)covisint.com>, "
cf-dev(a)lists.cloudfoundry.org" <cf-dev(a)lists.cloudfoundry.org>
Subject: Re: [cf-dev] TCP Router VS NoRouter

Hi Owais,

What are you referring to when you say you're concerned about the LTM
becoming a Monolith? Too much functionality in one component? Too much of
the system depending on one component?

Chip,

Hopefully it's ok to discuss NoRouter here even though it isn't an
official CF project. Let us know if it is not.

Mike

On Fri, May 15, 2015 at 10:55 AM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

The "norouter", while interesting, isn't the official CF project
approach to HTTP traffic routing. The TCP router is being built to support
TCP routing as a general solution, with the goal of it becoming an official
part of the CF release when ready.

-chip

Chip Childers | Technology Chief of Staff | Cloud Foundry Foundation

On Fri, May 15, 2015 at 8:58 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Hi,

I attended sessions on both TCP Router (Cloud Foundry and IOT
protocol support by Atul Kshirsagar) and NoRouter (Norouter: Running
Cloud Foundry without the Gorouter by Mike Heath).

I just wanted to start a discussion on the pros and cons of each
approach.

As personal opinion I think NoRouter is a simpler approach and can
definitely be made to support IOT protocols. The main drawback I see with
the NoRouter is the danger of the LTM becoming a Monolith.

Any suggestions\ideas?

Regards,
Owais


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Syslog Drain to Logstash Problems

John Tuley <jtuley@...>
 

Mike,

Thanks for finding that. I've filed a bug
<https://www.pivotaltracker.com/story/show/96801752> to get the README
fixed.

– John Tuley

On Thu, Jun 11, 2015 at 10:22 AM, Mike Jacobi <jacobi(a)adobe.com> wrote:

We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at
https://github.com/cloudfoundry/loggregator which mentions only the
doppler template. Looking at
https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we
later determined that we also needed the syslog_drain_binder
and metron_agent templates for a complete loggregator deployment.

-Mike



On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com>
wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com <http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Syslog Drain to Logstash Problems

John Tuley <jtuley@...>
 

I don't have an answer, so here are some diagnostic questions to help you
debug:

Have you set doppler.blacklisted_syslog_ranges to something that includes
your destination? (This would be bad.)
Do you have syslog_drain_binder running? (It should be colocated with
doppler on the loggregator VMs.) Does it seem healthy?
Can you use the HTTP API to etcd to look at the keys in
/loggregator/services. You should see an entry per (app ID, drain URL)
pair. If those aren't present, then the syslog drain binder isn't working,
and doppler won't get bindings.
If you have the collector enabled, look in your metrics for one containing
"DopplerServer.messageRouter.numberOfSyslogSinks". If that number seems
accurate, then it's likely networking (which you have already ruled out) or
something really weird in the code.



– John Tuley

On Wed, Jun 10, 2015 at 10:35 AM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other
thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required
restarting the app. If you're post-v198, it *should* Just Work.

However, one of the things that could be in your way is network
security. In order to forward logs to your drain, your loggregator servers
must be able to access that server. This is the most common cause we see of
systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <
steve.wall(a)primetimesoftware.com> wrote:

Hello,
We are having problems draining log messages to Logstash. The drain is
setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more
specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were
successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: Syslog Drain to Logstash Problems

MJ
 

We had the same problem due to missing templates in our manifest.

We initially used the example manifest snippet shown at https://github.com/cloudfoundry/loggregator which mentions only the doppler template. Looking at https://github.com/cloudfoundry/loggregator/blob/develop/manifest-templates/cf-lamb.yml we later determined that we also needed the syslog_drain_binder and metron_agent templates for a complete loggregator deployment.

-Mike

On Jun 10, 2015, at 9:35 AM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:

I was able submit a log entry from the loggregator VM using -

nc -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"

and to test UDP

nc -u -w0 10.xx.xx.xx 5000 <<< "logging from loggregator"


Which leads me to believe the networking is working properly. Any other thoughts?
Thanks!
Steve

On Wed, Jun 3, 2015 at 6:14 PM, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:
We’ll check that, thanks!

Josh Ghiloni
Senior Consultant
303.932.2202<tel:303.932.2202> o | 303.590.5427<tel:303.590.5427> m | 303.565.2794<tel:303.565.2794> f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ecsteam.com/>





On Jun 3, 2015, at 15:41, John Tuley <jtuley(a)pivotal.io<mailto:jtuley(a)pivotal.io>> wrote:

Steve,

Until recently (cf-release v198), binding a syslog service required restarting the app. If you're post-v198, it should Just Work.

However, one of the things that could be in your way is network security. In order to forward logs to your drain, your loggregator servers must be able to access that server. This is the most common cause we see of systems failing to forward to syslog drains.

Please let us know if you have more questions.

– John Tuley

On Wed, Jun 3, 2015 at 12:37 PM, Steve Wall <steve.wall(a)primetimesoftware.com<mailto:steve.wall(a)primetimesoftware.com>> wrote:
Hello,
We are having problems draining log messages to Logstash. The drain is setup as a user provided service.

cf cups logstash-drain -l syslog://xx.xx.xx.xx:5000

And then bound to the service.

cf bind-service myapp logstash-drain

But no log messages are coming through to Logstash. Or more specifically, we are using ELK and the messages aren't seen through Kibana.

We were able to log into the DEA and using netcat (nc), messages were successfully submitted to the ELK stack.

nc -w0 -u xx.xx.xx.xx 5000 <<< "logging from remote"

Any suggestions on how to debug this further?
-Steve


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: HM9000 metrics

Pablo Alonso Rodriguez <palonsoro@...>
 

Ok. Just a question: When you say "the DEA will keep its container carcass
around for an hour", you mean that the DEA does not remove the container
files. However, if Warden grace time is configured at 300 seconds (5
minutes), the container is actually destroyed after that time (although its
files remain). Is this right?

Thank you very much.

2015-06-11 17:27 GMT+02:00 Dieu Cao <dcao(a)pivotal.io>:

Hi Pablo,

Ops Metrics is a PCF product and questions about that should be directed
to Pivotal customer support.

Regarding your second question, about the difference between crashed
indices and crashed indexes.

The NumberOfCrashedInstances metric is usually about 4 times the
NumberOfCrashedIndices metric. First, NumberOfCrashedInstances is the total
number of crashed containers that remain on the DEAs, while
NumberOfCrashedIndices is the number of app-index pairs which have only
crashed instances.

If an app has a droplet that crashes on startup, HM9000 will eventually
settle on restarting an instance at each of its indices every 16 minutes.
When the instance crashes, the DEA will keep its container carcass around
for an hour (to allow the space developers to inspect its files via the
files API if they have the instance guid). So on average, there will be
60/16 = 3.75 crashed instances in the system per crashed index. That
should account for most of the indices and instances that are crashed in
the system.

Hope that helps.

-Dieu
CF Runtime PM



On Thu, Jun 11, 2015 at 4:48 AM, Pablo Alonso Rodriguez <
palonsoro(a)gmail.com> wrote:

Good morning.

Recently, I have been revising metrics emitted by CF components. In order
to understand HM9000 metrics, I have been reading the metrics documentation
(at https://github.com/cloudfoundry/hm9000#metricsserver)

I post this message because I have two questions.

First question:

Not all the metrics retrieved via Ops Metrics are documented there. Is
there any additional documentation? If not, could you please explain my
what do the following metrics mean?

- StartEvacuating, StartCrashed, StartMissing
- StopDuplicate, StopEvacuationComplete, StopExtra

I have some guesses about some of them, but I am not completely sure
about them.

Second question:

I do not fully understand the difference between the concepts of
"instances" and "indices" at metrics like "NumberOfCrashedIndices" and
"NumberOfCrashedInstances".

For example, I have one crashed app in my CF instance, and
"NumberOfCrashedIndices" reports '1' and "NumberOfCrashedInstances" reports
'3'. If I have a look at `cf app myapp`, I see one single crashed instance
(this was expected). If I have a look at hm9000 dump, I see the following
about my crashed app (UUIDs have been replaced by false ones):

Guid: 7ef08c44-102d-11e5-9c0d-0fb30c2610f7 | Version:
8e16b09a-102d-11e5-b6ce-27f9445313f8
Desired: [1] instances, (STARTED, STAGED)
Heartbeats:
[0 CRASHED] a42a7236102d11e5813abfab583ad850 on 1-abc
[0 CRASHED] b35b9f1e102d11e5ad29cfc4c2c4e3ea on 2-ac3
[0 CRASHED] bbd37658102d11e5ba8e2b98d1fd1793 on 4-a67
CrashCounts: [0]:7499
Pending Starts:
[0] priority:1.00 send:2m34.628437793s

So, what does all this mean? I do not understand why do I get 3
heartbeats while
I only was trying to start a single instance.

Thank you in advance




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: TCP Router VS NoRouter

Mike Youngstrom
 

Thanks for responding Mohamed.

Help me understand what you mean when you refer to opening up the LTM to
developers. NoRouter itself must have access to the LTM to update app pool
members and such but end users/developers themselves will not actually have
direct access to the LTM.

Some of the features Mike mentioned in his presentation were more a comment
on what is possible if we enable a feature to do so. When/If NoRouter were
to expose some of that functionality it would most likely by done using a
Service Broker. For example, if an application wishes to provide a custom
OOS page we wouldn't give the developer direct access to the LTM to
configure an OOS page, instead, I think we'd provide an OOS Page Service
Broker. When created and bound to an application it would prompt NoRouter
to configure the virtual server to use that Custom OOS page for all of the
routers mapped to that application. Keeping the details sufficiently
removed from the developer.

Does that help answer your question?

Mike

On Thu, Jun 11, 2015 at 9:04 AM, Mohamed, Owais <Owais.Mohamed(a)covisint.com>
wrote:

Apologize for restarting this thread after a few weeks.

That’s right Mike. Once the LTM opens up to the developers they might
start putting in too much functionality into the Load Balancer like
filters, interceptors. By using NoRouter the LTM will have to be opened up
to the developer team.

I know that there has to be discipline and access control. But was just
putting it out as con in the NoRouter Section.

Hoping that Chip does not have concerns with discussing NoRouter here.
Let me know and I can take this discussion offline.

From: Mike Youngstrom <youngm(a)gmail.com>
Date: Monday, May 18, 2015 at 3:22 PM
To: Chip Childers <cchilders(a)cloudfoundry.org>
Cc: Owais Mohamed <owais.mohamed(a)covisint.com>, "
cf-dev(a)lists.cloudfoundry.org" <cf-dev(a)lists.cloudfoundry.org>
Subject: Re: [cf-dev] TCP Router VS NoRouter

Hi Owais,

What are you referring to when you say you're concerned about the LTM
becoming a Monolith? Too much functionality in one component? Too much of
the system depending on one component?

Chip,

Hopefully it's ok to discuss NoRouter here even though it isn't an
official CF project. Let us know if it is not.

Mike

On Fri, May 15, 2015 at 10:55 AM, Chip Childers <
cchilders(a)cloudfoundry.org> wrote:

The "norouter", while interesting, isn't the official CF project approach
to HTTP traffic routing. The TCP router is being built to support TCP
routing as a general solution, with the goal of it becoming an official
part of the CF release when ready.

-chip

Chip Childers | Technology Chief of Staff | Cloud Foundry Foundation

On Fri, May 15, 2015 at 8:58 AM, Mohamed, Owais <
Owais.Mohamed(a)covisint.com> wrote:

Hi,

I attended sessions on both TCP Router (Cloud Foundry and IOT
protocol support by Atul Kshirsagar) and NoRouter (Norouter: Running
Cloud Foundry without the Gorouter by Mike Heath).

I just wanted to start a discussion on the pros and cons of each
approach.

As personal opinion I think NoRouter is a simpler approach and can
definitely be made to support IOT protocols. The main drawback I see with
the NoRouter is the danger of the LTM becoming a Monolith.

Any suggestions\ideas?

Regards,
Owais


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


(No subject)

Daniel Mikusa
 

On Thu, Jun 11, 2015 at 11:37 AM, Arbi Akhina <arbi.akhina(a)gmail.com> wrote:

I removed static assets as well as many classes that were not used,
I left classes related to jetty, javax (managment, ..), apache commons io,
and the package (with all classes) where the main class is located.
Interesting. Do you get log output from the application when it's
working? I'm curious because you weren't seeing this when it was failing.
Trying to understand if the app is not logging properly or if your logs
were being lost somehow.



I didn't try on public cfs like PWS (what is this one?) and BlueMix.
Pivotal Web Services. Similar to Bluemix, it's a public CF that you can
sign up to use. There are others too. The idea here is just to test this
on a larger, more powerful system and see if the problems go away.

Dan




On Thu, Jun 11, 2015 at 4:42 PM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Thu, Jun 11, 2015 at 10:31 AM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

Hi, I had to reduce the size of the jar to ~ 51M to get the simple class
that runs jetty to work on bosh-lite!!!
Interesting. Out of curiosity what did you remove to make it smaller?
Static assets? Also have you tried a larger CF install like PWS / BlueMix,
just to see if this is an issue specific to bosh-lite.


Is there any kind of size restriction for executable jars??
The system can put a limit on how big of an application you can upload.
I don't know about bosh-lite specifically, but the default is 1G. Also,
when you exceed that you'll get a message from the cf cli saying your app
is too big. It won't silently fail, which seems to be what happened here.

Dan



Here is the system information of the warden container on which the jar
is running:
Java version : 1.8.0_45-
Java Spec version : 1.8
JVM version : 25.45-b02
JVM vendor : Oracle Corporation
JVM name : OpenJDK 64-Bit Server VM
Os Name : Linux
Os Architecture : amd64
Os version : 3.13.0-44-generic
Number of processors : 4
Max memory : 3087007744
Total memory : 3087007744
Available memory : 2958157616
Free work disk space : 17898352640
VM Args : [-Djava.io.tmpdir=/home/vcap/tmp,
-XX:OnOutOfMemoryError=/home/vcap/app/.java-buildpack/open_jdk_jre/bin/killjava.sh,
-Xss1M, -Xmx3G, -Xms3G, -XX:MaxMetaspaceSize=419430K,
-XX:MetaspaceSize=419430K]

On Tue, Jun 9, 2015 at 7:01 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

no I meant the source is for the company I work for, not my personal
project.

On Tue, Jun 9, 2015 at 6:58 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Tue, Jun 9, 2015 at 12:43 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

I don't own the source, but here is the jar file
http://tempsend.com/6D745B7B07/7033/modules.jar
you can try to run it locally with java -jar modules.jar then
connect to localhost:8080
Hmm, OK. I thought you had a minimal example from you last note. I'm
not really up for running untrusted binary code. If you can put a minimal
code sample together, I'll give it a try.



An here is the manifest.yml i'm using:
---
applications:
- name: modules
memory: 4G
disk_quota: 2G
timeout: 180
instances: 1
host: modules-${random-word}
path: modules.jar
buildpack: https://github.com/cloudfoundry/java-buildpack
command: sleep 2 &&
CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY com.heavenize.osmoze.kernel.HelloHandler
# env:
# JAVA_OPTS: "$JAVA_OPTS
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000"

I'm having both files in the same folder modules/ and I push with cf
push
That seems OK. Is this app designed to run on CF? By that, I mean is
it configure to log to STDOUT / STDERR and is it going to listen on the
right port (i.e $PORT)?

Dan




Thanks a lot

On Tue, Jun 9, 2015 at 6:25 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

I'd be happy to try the demo and see if it works for me. Can you
post a complete project on github so I can just `git clone` & build?

Dan

On Tue, Jun 9, 2015 at 12:21 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

I've tried with a simple app that just launches an embedded jetty
that listen on the vcap port and it worked (I can connect to the app url
from curl and chrome)
then I moved this single file into my modules.jar (the real app)
and configure it as main class, but here it doesn't work.

here is the class in question:
public class HelloHandler extends AbstractHandler
{
public static void main(String[] args)
{
int VCAP_APP_PORT =
Integer.parseInt((System.getenv("VCAP_APP_PORT") != null ?
System.getenv("VCAP_APP_PORT") : "8080"));

Server server = new Server(VCAP_APP_PORT);
server.setHandler(new HelloHandler());

try
{
server.start();
server.join();
}
catch (Exception exception)
{
System.out.println("Failed to start embedded jetty server.");
exception.printStackTrace();
}
}

final String greeting;
final String body;

public HelloHandler()
{
this("Hello World");
}

public HelloHandler( String greeting )
{
this(greeting, null);
}

public HelloHandler( String greeting, String body )
{
this.greeting = greeting;
this.body = body;
}

public void handle( String target,
Request baseRequest,
HttpServletRequest request,
HttpServletResponse response ) throws
IOException,
ServletException
{
response.setContentType("text/html; charset=utf-8");
response.setStatus(HttpServletResponse.SC_OK);

PrintWriter out = response.getWriter();

out.println("<h1>" + greeting + "</h1>");
out.println("<p>Current directory: " + getCurrentDirectory()
+ "</p>");
out.println("<p> Parent directory: " + getParentDirectory() +
"</p>");
for(File file: getFiles()) {
out.println("<p>" + file.getAbsolutePath() + "</p>");
}

if (body != null)
{
out.println(body);
}

baseRequest.setHandled(true);
}
private List<File> getFiles() {
return Arrays.asList(new File(".").listFiles());
}

private String getCurrentDirectory()
{
String directory = null;
try
{
directory = new File(".").getCanonicalPath();
}
catch (Exception e)
{
e.printStackTrace();
}
return directory;
}

private String getParentDirectory()
{
String directory = null;
try
{
directory = new File("..").getCanonicalPath();
}
catch (Exception e)
{
e.printStackTrace();
}
return directory;
}
}


On Tue, Jun 9, 2015 at 5:34 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

Double check that your app is configured to log to STDOUT /
STDER. If it's logging to a file, you won't see the messages since the
system only captures STDOUT & STDERR.

Also, you might try a demo app like spring-music, just to make
sure your install is working OK.

https://github.com/cloudfoundry-samples/spring-music

Dan

On Tue, Jun 9, 2015 at 10:40 AM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

Same logs I still can't see what went wrong!

On Tue, Jun 9, 2015 at 3:58 PM, Daniel Mikusa <dmikusa(a)pivotal.io
wrote:
And so you'd want to set `command` to `sleep 2 &&
CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/
bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m.. -memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java
-cp $PWD/. -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.
java-buildpack/open_jdk_jre/bin/killjava.sh $CALCULATED_MEMORY
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000
com.heavenize.osmoze.kernel.HelloHandler`.

Dan

On Tue, Jun 9, 2015 at 9:29 AM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

Here is the content of *detected_start_command*:
$ CF_TRACE=true cf app modules | grep "detected_start_command"
"detected_start_command":
"CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -agentlib:jdwp=transport=dt_socket,address=
192.168.2.8:8000 com.heavenize.osmoze.kernel.HelloHandler",

{"guid":"2dabf3c6-1736-4eb4-9bb7-40dc58bce246","name":"modules","routes":[{"guid":"014f77ed-e450-4209-be23-40e4a8257cfc","host":"modules-rhythmic-uvulatomy","domain":{"guid":"a77afa99-76d6-4f3f-9e07-27775597709f","name":"
10.244.0.34.xip.io
"}}],"running_instances":0,"services":[],"available_domains":[{"guid":"a77afa99-76d6-4f3f-9e07-27775597709f","name":"
10.244.0.34.xip.io
"}],"name":"modules","production":false,"space_guid":"381707f7-88d4-4f6e-bd7c-80d7f0699b0f","stack_guid":"3431865a-f165-4e75-9221-4f418e9de889","buildpack":"
https://github.com/cloudfoundry/java-buildpack","detected_buildpack":null,"environment_json":{"JAVA_OPTS":"$JAVA_OPTS
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000"},"memory":4096,"instances":1,"disk_quota":2048,"state":"STARTED","version":"e8f0e18e-7c09-4143-a17c-83a6e32eed84","command":null,"console":false,"debug":null,"staging_task_id":"1249276465c64c1884d401452d2365af","package_state":"STAGED","health_check_type":"port","health_check_timeout":180,"staging_failed_reason":null,"diego":false,"docker_image":null,"package_updated_at":"2015-06-08T07:56:44Z","detected_start_command":"CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -agentlib:jdwp=transport=dt_socket,address=
192.168.2.8:8000 com.heavenize.osmoze.kernel.HelloHandler"}


On Tue, Jun 9, 2015 at 1:35 PM, Daniel Mikusa <
dmikusa(a)pivotal.io> wrote:

On Tue, Jun 9, 2015 at 5:05 AM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

Hi, thanks for the hint, I tried to add to the manifest the
following entry:
command: sleep 2 && java -jar modules.jar
The Java build pack does not put Java on the path, which is
what you're seeing in the error below. I will usually run `CF_TRACE=true
cf app <app-name> | grep "detected_start_command"` which shows the command
the build pack detected and then just copy & paste that into my custom
command.



but it looks like it's not appropriate as I see in the logs:
2015-06-08T05:34:03.71+0200 [App/0] ERR bash: java:
command not found

On Mon, Jun 8, 2015 at 6:32 PM, Daniel Mikusa <
dmikusa(a)pivotal.io> wrote:

On Mon, Jun 8, 2015 at 12:03 PM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

I'm trying to push an executable JAR to a bosh-lite
instance, on the logs I see CF trying many times to restart the app and
eventually fail.

I can't find out why the app crashes as there is no app
logs returned by CF. I tried to remotely debug the app (as described in
[1]) but nothing happens on eclipse. Any hint to solve this issue is
appreciated.
There's a known issue that occurs when an app starts and
fails in rapid succession and results in the log entries being missed. If
you add a couple second pause into the app, it will give the system enough
time to attach the logger and you'll see the output generated by the
crashing app.

You can do this with a custom start command `cf push -c
'sleep 2 && <normal-cmd>'` or with a `.profile.d` script that sleeps for a
couple seconds.

Dan




1. I'm launching the app with:
*cf push -t 180*

2. Here is the manifest.yml content:
---
applications:
- name: modules
memory: 4G
instances: 1
host: modules-${random-word}
path: modules.jar
buildpack: https://github.com/cloudfoundry/java-buildpack
env:
JAVA_OPTS: "$JAVA_OPTS
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000
"

3. Here is the log:
Connecté, le dumping journaux récents pour application
modules en org heavenize / espace dev comme admin...

2015-06-07T12:05:15.69+0200 [API] OUT Created app with
guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:05:16.10+0200 [API] OUT Updated app with
guid 1fbf3378-6512-46de-bae4-02ee30275464
({"route"=>"68e27d8d-4ff6-443b-a3e0-416c40d325d3"})
2015-06-07T12:12:21.55+0200 [DEA] OUT Got staging
request for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:12:25.40+0200 [API] OUT Updated app with
guid 1fbf3378-6512-46de-bae4-02ee30275464 ({"state"=>"STARTED"})
2015-06-07T12:12:38.08+0200 [STG] OUT -----> Downloaded
app package (163M)
2015-06-07T12:13:40.93+0200 [STG] ERR Cloning into
'/tmp/buildpacks/java-buildpack'...
2015-06-07T12:14:01.67+0200 [STG] OUT -----> Java
Buildpack Version: c862ac8 |
https://github.com/cloudfoundry/java-buildpack#c862ac8
2015-06-07T12:14:13.89+0200 [STG] OUT ----->
Downloading Open Jdk JRE 1.8.0_45 from
https://download.run.pivotal.io/openjdk/lucid/x86_64/openjdk-1.8.0_45.tar.gz
(11.6s)
2015-06-07T12:14:15.34+0200 [STG] OUT Expanding
Open Jdk JRE to .java-buildpack/open_jdk_jre (1.4s)
2015-06-07T12:14:15.89+0200 [STG] OUT ----->
Downloading Open JDK Like Memory Calculator 1.1.1_RELEASE from
https://download.run.pivotal.io/memory-calculator/lucid/x86_64/memory-calculator-1.1.1_RELEASE
(0.5s)
2015-06-07T12:14:15.90+0200 [STG] OUT Memory
Settings: -XX:MaxMetaspaceSize=419430K -XX:MetaspaceSize=419430K -Xss1M
-Xmx3G -Xms3G
2015-06-07T12:15:43.44+0200 [STG] OUT -----> Uploading
droplet (151M)
2015-06-07T12:16:07.51+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:16:29.66+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"9d55c5f791324d358bffb4c961a4c7ee", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672189}
2015-06-07T12:17:14.18+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:31.10+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"2ae0c26f33864f40989ee870a1b9e3db", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672251}
2015-06-07T12:17:38.48+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:38.48+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:38.48+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:55.31+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:11.82+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"dc872d38f3324af481c82ba67f0e216c", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672291}
2015-06-07T12:18:18.69+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:18.69+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:18.69+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:34.06+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:50.98+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"623c3af7e3e84801b6fd44eeee9c0a12", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672330}
2015-06-07T12:18:58.80+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:58.80+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:58.80+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:34.08+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:50.36+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"40727eea293146948af197e13443843c", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672390}
2015-06-07T12:19:59.01+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:59.01+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:59.01+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:04.12+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:20.61+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"f7ffff55692a418c847f4f37be574ddf", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672480}
2015-06-07T12:21:29.43+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:29.43+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:29.47+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:34.16+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:49.97+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"4581e97c6b0f4504b8d64a5c69d6787b", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672629}
2015-06-07T12:23:50.29+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:50.29+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:50.29+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:14.24+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:29.82+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"f98749490a6743598f57d3848eb06177", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672909}
2015-06-07T12:28:31.73+0200 [DEA] OUT Removing crash
for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:31.73+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:31.73+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464



[1]
http://docs.cloudfoundry.org/buildpacks/java/java-tips.html#debugging



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


(No subject)

Arbi Akhina
 

I removed static assets as well as many classes that were not used,
I left classes related to jetty, javax (managment, ..), apache commons io,
and the package (with all classes) where the main class is located.

I didn't try on public cfs like PWS (what is this one?) and BlueMix.

On Thu, Jun 11, 2015 at 4:42 PM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Thu, Jun 11, 2015 at 10:31 AM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

Hi, I had to reduce the size of the jar to ~ 51M to get the simple class
that runs jetty to work on bosh-lite!!!
Interesting. Out of curiosity what did you remove to make it smaller?
Static assets? Also have you tried a larger CF install like PWS / BlueMix,
just to see if this is an issue specific to bosh-lite.


Is there any kind of size restriction for executable jars??
The system can put a limit on how big of an application you can upload. I
don't know about bosh-lite specifically, but the default is 1G. Also, when
you exceed that you'll get a message from the cf cli saying your app is too
big. It won't silently fail, which seems to be what happened here.

Dan



Here is the system information of the warden container on which the jar
is running:
Java version : 1.8.0_45-
Java Spec version : 1.8
JVM version : 25.45-b02
JVM vendor : Oracle Corporation
JVM name : OpenJDK 64-Bit Server VM
Os Name : Linux
Os Architecture : amd64
Os version : 3.13.0-44-generic
Number of processors : 4
Max memory : 3087007744
Total memory : 3087007744
Available memory : 2958157616
Free work disk space : 17898352640
VM Args : [-Djava.io.tmpdir=/home/vcap/tmp,
-XX:OnOutOfMemoryError=/home/vcap/app/.java-buildpack/open_jdk_jre/bin/killjava.sh,
-Xss1M, -Xmx3G, -Xms3G, -XX:MaxMetaspaceSize=419430K,
-XX:MetaspaceSize=419430K]

On Tue, Jun 9, 2015 at 7:01 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

no I meant the source is for the company I work for, not my personal
project.

On Tue, Jun 9, 2015 at 6:58 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

On Tue, Jun 9, 2015 at 12:43 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

I don't own the source, but here is the jar file
http://tempsend.com/6D745B7B07/7033/modules.jar
you can try to run it locally with java -jar modules.jar then connect
to localhost:8080
Hmm, OK. I thought you had a minimal example from you last note. I'm
not really up for running untrusted binary code. If you can put a minimal
code sample together, I'll give it a try.



An here is the manifest.yml i'm using:
---
applications:
- name: modules
memory: 4G
disk_quota: 2G
timeout: 180
instances: 1
host: modules-${random-word}
path: modules.jar
buildpack: https://github.com/cloudfoundry/java-buildpack
command: sleep 2 &&
CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY com.heavenize.osmoze.kernel.HelloHandler
# env:
# JAVA_OPTS: "$JAVA_OPTS -agentlib:jdwp=transport=dt_socket,address=
192.168.2.8:8000"

I'm having both files in the same folder modules/ and I push with cf
push
That seems OK. Is this app designed to run on CF? By that, I mean is
it configure to log to STDOUT / STDERR and is it going to listen on the
right port (i.e $PORT)?

Dan




Thanks a lot

On Tue, Jun 9, 2015 at 6:25 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

I'd be happy to try the demo and see if it works for me. Can you
post a complete project on github so I can just `git clone` & build?

Dan

On Tue, Jun 9, 2015 at 12:21 PM, Arbi Akhina <arbi.akhina(a)gmail.com>
wrote:

I've tried with a simple app that just launches an embedded jetty
that listen on the vcap port and it worked (I can connect to the app url
from curl and chrome)
then I moved this single file into my modules.jar (the real app) and
configure it as main class, but here it doesn't work.

here is the class in question:
public class HelloHandler extends AbstractHandler
{
public static void main(String[] args)
{
int VCAP_APP_PORT =
Integer.parseInt((System.getenv("VCAP_APP_PORT") != null ?
System.getenv("VCAP_APP_PORT") : "8080"));

Server server = new Server(VCAP_APP_PORT);
server.setHandler(new HelloHandler());

try
{
server.start();
server.join();
}
catch (Exception exception)
{
System.out.println("Failed to start embedded jetty server.");
exception.printStackTrace();
}
}

final String greeting;
final String body;

public HelloHandler()
{
this("Hello World");
}

public HelloHandler( String greeting )
{
this(greeting, null);
}

public HelloHandler( String greeting, String body )
{
this.greeting = greeting;
this.body = body;
}

public void handle( String target,
Request baseRequest,
HttpServletRequest request,
HttpServletResponse response ) throws
IOException,
ServletException
{
response.setContentType("text/html; charset=utf-8");
response.setStatus(HttpServletResponse.SC_OK);

PrintWriter out = response.getWriter();

out.println("<h1>" + greeting + "</h1>");
out.println("<p>Current directory: " + getCurrentDirectory() +
"</p>");
out.println("<p> Parent directory: " + getParentDirectory() +
"</p>");
for(File file: getFiles()) {
out.println("<p>" + file.getAbsolutePath() + "</p>");
}

if (body != null)
{
out.println(body);
}

baseRequest.setHandled(true);
}
private List<File> getFiles() {
return Arrays.asList(new File(".").listFiles());
}

private String getCurrentDirectory()
{
String directory = null;
try
{
directory = new File(".").getCanonicalPath();
}
catch (Exception e)
{
e.printStackTrace();
}
return directory;
}

private String getParentDirectory()
{
String directory = null;
try
{
directory = new File("..").getCanonicalPath();
}
catch (Exception e)
{
e.printStackTrace();
}
return directory;
}
}


On Tue, Jun 9, 2015 at 5:34 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

Double check that your app is configured to log to STDOUT / STDER.
If it's logging to a file, you won't see the messages since the system only
captures STDOUT & STDERR.

Also, you might try a demo app like spring-music, just to make sure
your install is working OK.

https://github.com/cloudfoundry-samples/spring-music

Dan

On Tue, Jun 9, 2015 at 10:40 AM, Arbi Akhina <arbi.akhina(a)gmail.com
wrote:
Same logs I still can't see what went wrong!

On Tue, Jun 9, 2015 at 3:58 PM, Daniel Mikusa <dmikusa(a)pivotal.io>
wrote:

And so you'd want to set `command` to `sleep 2 &&
CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/
bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m.. -memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java
-cp $PWD/. -Djava.io.tmpdir=$TMPDIR -XX:OnOutOfMemoryError=$PWD/.
java-buildpack/open_jdk_jre/bin/killjava.sh $CALCULATED_MEMORY
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000
com.heavenize.osmoze.kernel.HelloHandler`.

Dan

On Tue, Jun 9, 2015 at 9:29 AM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

Here is the content of *detected_start_command*:
$ CF_TRACE=true cf app modules | grep "detected_start_command"
"detected_start_command":
"CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -agentlib:jdwp=transport=dt_socket,address=
192.168.2.8:8000 com.heavenize.osmoze.kernel.HelloHandler",

{"guid":"2dabf3c6-1736-4eb4-9bb7-40dc58bce246","name":"modules","routes":[{"guid":"014f77ed-e450-4209-be23-40e4a8257cfc","host":"modules-rhythmic-uvulatomy","domain":{"guid":"a77afa99-76d6-4f3f-9e07-27775597709f","name":"
10.244.0.34.xip.io
"}}],"running_instances":0,"services":[],"available_domains":[{"guid":"a77afa99-76d6-4f3f-9e07-27775597709f","name":"
10.244.0.34.xip.io
"}],"name":"modules","production":false,"space_guid":"381707f7-88d4-4f6e-bd7c-80d7f0699b0f","stack_guid":"3431865a-f165-4e75-9221-4f418e9de889","buildpack":"
https://github.com/cloudfoundry/java-buildpack","detected_buildpack":null,"environment_json":{"JAVA_OPTS":"$JAVA_OPTS
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000"},"memory":4096,"instances":1,"disk_quota":2048,"state":"STARTED","version":"e8f0e18e-7c09-4143-a17c-83a6e32eed84","command":null,"console":false,"debug":null,"staging_task_id":"1249276465c64c1884d401452d2365af","package_state":"STAGED","health_check_type":"port","health_check_timeout":180,"staging_failed_reason":null,"diego":false,"docker_image":null,"package_updated_at":"2015-06-08T07:56:44Z","detected_start_command":"CALCULATED_MEMORY=$($PWD/.java-buildpack/open_jdk_jre/bin/java-buildpack-memory-calculator-1.1.1_RELEASE
-memorySizes=metaspace:64m..
-memoryWeights=heap:75,metaspace:10,stack:5,native:10
-totMemory=$MEMORY_LIMIT) && $PWD/.java-buildpack/open_jdk_jre/bin/java -cp
$PWD/. -Djava.io.tmpdir=$TMPDIR
-XX:OnOutOfMemoryError=$PWD/.java-buildpack/open_jdk_jre/bin/killjava.sh
$CALCULATED_MEMORY -agentlib:jdwp=transport=dt_socket,address=
192.168.2.8:8000 com.heavenize.osmoze.kernel.HelloHandler"}


On Tue, Jun 9, 2015 at 1:35 PM, Daniel Mikusa <
dmikusa(a)pivotal.io> wrote:

On Tue, Jun 9, 2015 at 5:05 AM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

Hi, thanks for the hint, I tried to add to the manifest the
following entry:
command: sleep 2 && java -jar modules.jar
The Java build pack does not put Java on the path, which is
what you're seeing in the error below. I will usually run `CF_TRACE=true
cf app <app-name> | grep "detected_start_command"` which shows the command
the build pack detected and then just copy & paste that into my custom
command.



but it looks like it's not appropriate as I see in the logs:
2015-06-08T05:34:03.71+0200 [App/0] ERR bash: java:
command not found

On Mon, Jun 8, 2015 at 6:32 PM, Daniel Mikusa <
dmikusa(a)pivotal.io> wrote:

On Mon, Jun 8, 2015 at 12:03 PM, Arbi Akhina <
arbi.akhina(a)gmail.com> wrote:

I'm trying to push an executable JAR to a bosh-lite
instance, on the logs I see CF trying many times to restart the app and
eventually fail.

I can't find out why the app crashes as there is no app logs
returned by CF. I tried to remotely debug the app (as described in [1]) but
nothing happens on eclipse. Any hint to solve this issue is appreciated.
There's a known issue that occurs when an app starts and
fails in rapid succession and results in the log entries being missed. If
you add a couple second pause into the app, it will give the system enough
time to attach the logger and you'll see the output generated by the
crashing app.

You can do this with a custom start command `cf push -c
'sleep 2 && <normal-cmd>'` or with a `.profile.d` script that sleeps for a
couple seconds.

Dan




1. I'm launching the app with:
*cf push -t 180*

2. Here is the manifest.yml content:
---
applications:
- name: modules
memory: 4G
instances: 1
host: modules-${random-word}
path: modules.jar
buildpack: https://github.com/cloudfoundry/java-buildpack
env:
JAVA_OPTS: "$JAVA_OPTS
-agentlib:jdwp=transport=dt_socket,address=192.168.2.8:8000"

3. Here is the log:
Connecté, le dumping journaux récents pour application
modules en org heavenize / espace dev comme admin...

2015-06-07T12:05:15.69+0200 [API] OUT Created app with
guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:05:16.10+0200 [API] OUT Updated app with
guid 1fbf3378-6512-46de-bae4-02ee30275464
({"route"=>"68e27d8d-4ff6-443b-a3e0-416c40d325d3"})
2015-06-07T12:12:21.55+0200 [DEA] OUT Got staging
request for app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:12:25.40+0200 [API] OUT Updated app with
guid 1fbf3378-6512-46de-bae4-02ee30275464 ({"state"=>"STARTED"})
2015-06-07T12:12:38.08+0200 [STG] OUT -----> Downloaded
app package (163M)
2015-06-07T12:13:40.93+0200 [STG] ERR Cloning into
'/tmp/buildpacks/java-buildpack'...
2015-06-07T12:14:01.67+0200 [STG] OUT -----> Java
Buildpack Version: c862ac8 |
https://github.com/cloudfoundry/java-buildpack#c862ac8
2015-06-07T12:14:13.89+0200 [STG] OUT -----> Downloading
Open Jdk JRE 1.8.0_45 from
https://download.run.pivotal.io/openjdk/lucid/x86_64/openjdk-1.8.0_45.tar.gz
(11.6s)
2015-06-07T12:14:15.34+0200 [STG] OUT Expanding
Open Jdk JRE to .java-buildpack/open_jdk_jre (1.4s)
2015-06-07T12:14:15.89+0200 [STG] OUT -----> Downloading
Open JDK Like Memory Calculator 1.1.1_RELEASE from
https://download.run.pivotal.io/memory-calculator/lucid/x86_64/memory-calculator-1.1.1_RELEASE
(0.5s)
2015-06-07T12:14:15.90+0200 [STG] OUT Memory
Settings: -XX:MaxMetaspaceSize=419430K -XX:MetaspaceSize=419430K -Xss1M
-Xmx3G -Xms3G
2015-06-07T12:15:43.44+0200 [STG] OUT -----> Uploading
droplet (151M)
2015-06-07T12:16:07.51+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:16:29.66+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"9d55c5f791324d358bffb4c961a4c7ee", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672189}
2015-06-07T12:17:14.18+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:31.10+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"2ae0c26f33864f40989ee870a1b9e3db", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672251}
2015-06-07T12:17:38.48+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:38.48+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:38.48+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:17:55.31+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:11.82+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"dc872d38f3324af481c82ba67f0e216c", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672291}
2015-06-07T12:18:18.69+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:18.69+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:18.69+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:34.06+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:50.98+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"623c3af7e3e84801b6fd44eeee9c0a12", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672330}
2015-06-07T12:18:58.80+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:58.80+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:18:58.80+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:34.08+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:50.36+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"40727eea293146948af197e13443843c", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672390}
2015-06-07T12:19:59.01+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:59.01+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:19:59.01+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:04.12+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:20.61+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"f7ffff55692a418c847f4f37be574ddf", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672480}
2015-06-07T12:21:29.43+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:29.43+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:21:29.47+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:34.16+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:49.97+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"4581e97c6b0f4504b8d64a5c69d6787b", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672629}
2015-06-07T12:23:50.29+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:50.29+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:23:50.29+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:14.24+0200 [DEA] OUT Starting app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:29.82+0200 [API] OUT App instance
exited with guid 1fbf3378-6512-46de-bae4-02ee30275464 payload:
{"cc_partition"=>"default",
"droplet"=>"1fbf3378-6512-46de-bae4-02ee30275464",
"version"=>"703f793c-e388-41ac-b1f1-c564b301ca70",
"instance"=>"f98749490a6743598f57d3848eb06177", "index"=>0,
"reason"=>"CRASHED", "exit_status"=>-1, "exit_description"=>"failed to
start", "crash_timestamp"=>1433672909}
2015-06-07T12:28:31.73+0200 [DEA] OUT Removing crash for
app with id 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:31.73+0200 [DEA] OUT Stopping app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464
2015-06-07T12:28:31.73+0200 [DEA] OUT Stopped app
instance (index 0) with guid 1fbf3378-6512-46de-bae4-02ee30275464



[1]
http://docs.cloudfoundry.org/buildpacks/java/java-tips.html#debugging



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: HM9000 metrics

Dieu Cao <dcao@...>
 

Hi Pablo,

Ops Metrics is a PCF product and questions about that should be directed to
Pivotal customer support.

Regarding your second question, about the difference between crashed
indices and crashed indexes.

The NumberOfCrashedInstances metric is usually about 4 times the
NumberOfCrashedIndices metric. First, NumberOfCrashedInstances is the total
number of crashed containers that remain on the DEAs, while
NumberOfCrashedIndices is the number of app-index pairs which have only
crashed instances.

If an app has a droplet that crashes on startup, HM9000 will eventually
settle on restarting an instance at each of its indices every 16 minutes.
When the instance crashes, the DEA will keep its container carcass around
for an hour (to allow the space developers to inspect its files via the
files API if they have the instance guid). So on average, there will be
60/16 = 3.75 crashed instances in the system per crashed index. That should
account for most of the indices and instances that are crashed in the
system.

Hope that helps.

-Dieu
CF Runtime PM

On Thu, Jun 11, 2015 at 4:48 AM, Pablo Alonso Rodriguez <palonsoro(a)gmail.com
wrote:
Good morning.

Recently, I have been revising metrics emitted by CF components. In order
to understand HM9000 metrics, I have been reading the metrics documentation
(at https://github.com/cloudfoundry/hm9000#metricsserver)

I post this message because I have two questions.

First question:

Not all the metrics retrieved via Ops Metrics are documented there. Is
there any additional documentation? If not, could you please explain my
what do the following metrics mean?

- StartEvacuating, StartCrashed, StartMissing
- StopDuplicate, StopEvacuationComplete, StopExtra

I have some guesses about some of them, but I am not completely sure about
them.

Second question:

I do not fully understand the difference between the concepts of
"instances" and "indices" at metrics like "NumberOfCrashedIndices" and
"NumberOfCrashedInstances".

For example, I have one crashed app in my CF instance, and
"NumberOfCrashedIndices" reports '1' and "NumberOfCrashedInstances" reports
'3'. If I have a look at `cf app myapp`, I see one single crashed instance
(this was expected). If I have a look at hm9000 dump, I see the following
about my crashed app (UUIDs have been replaced by false ones):

Guid: 7ef08c44-102d-11e5-9c0d-0fb30c2610f7 | Version:
8e16b09a-102d-11e5-b6ce-27f9445313f8
Desired: [1] instances, (STARTED, STAGED)
Heartbeats:
[0 CRASHED] a42a7236102d11e5813abfab583ad850 on 1-abc
[0 CRASHED] b35b9f1e102d11e5ad29cfc4c2c4e3ea on 2-ac3
[0 CRASHED] bbd37658102d11e5ba8e2b98d1fd1793 on 4-a67
CrashCounts: [0]:7499
Pending Starts:
[0] priority:1.00 send:2m34.628437793s

So, what does all this mean? I do not understand why do I get 3 heartbeats
while
I only was trying to start a single instance.

Thank you in advance




_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Re: TCP Router VS NoRouter

Owais Mohamed
 

Apologize for restarting this thread after a few weeks.

That's right Mike. Once the LTM opens up to the developers they might start putting in too much functionality into the Load Balancer like filters, interceptors. By using NoRouter the LTM will have to be opened up to the developer team.

I know that there has to be discipline and access control. But was just putting it out as con in the NoRouter Section.

Hoping that Chip does not have concerns with discussing NoRouter here. Let me know and I can take this discussion offline.

From: Mike Youngstrom <youngm(a)gmail.com<mailto:youngm(a)gmail.com>>
Date: Monday, May 18, 2015 at 3:22 PM
To: Chip Childers <cchilders(a)cloudfoundry.org<mailto:cchilders(a)cloudfoundry.org>>
Cc: Owais Mohamed <owais.mohamed(a)covisint.com<mailto:owais.mohamed(a)covisint.com>>, "cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>" <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: Re: [cf-dev] TCP Router VS NoRouter

Hi Owais,

What are you referring to when you say you're concerned about the LTM becoming a Monolith? Too much functionality in one component? Too much of the system depending on one component?

Chip,

Hopefully it's ok to discuss NoRouter here even though it isn't an official CF project. Let us know if it is not.

Mike

On Fri, May 15, 2015 at 10:55 AM, Chip Childers <cchilders(a)cloudfoundry.org<mailto:cchilders(a)cloudfoundry.org>> wrote:
The "norouter", while interesting, isn't the official CF project approach to HTTP traffic routing. The TCP router is being built to support TCP routing as a general solution, with the goal of it becoming an official part of the CF release when ready.

-chip

Chip Childers | Technology Chief of Staff | Cloud Foundry Foundation

On Fri, May 15, 2015 at 8:58 AM, Mohamed, Owais <Owais.Mohamed(a)covisint.com<mailto:Owais.Mohamed(a)covisint.com>> wrote:
Hi,

I attended sessions on both TCP Router (Cloud Foundry and IOT protocol support by Atul Kshirsagar) and NoRouter (Norouter: Running Cloud Foundry without the Gorouter by Mike Heath).

I just wanted to start a discussion on the pros and cons of each approach.

As personal opinion I think NoRouter is a simpler approach and can definitely be made to support IOT protocols. The main drawback I see with the NoRouter is the danger of the LTM becoming a Monolith.

Any suggestions\ideas?

Regards,
Owais


_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev



_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

9001 - 9020 of 9425