Take a look at the value of $TOKEN (many online decoders out there.
https://jwt.io is one) and see what scopes your token actually has.

Filip

As of cf-release v232 there will be some required changes to the blobstore configuration in the cf-release manifest.

The linked doc should explain what you will need to do depending on the type of blobstore being used by your CF deployment.

https://docs.google.com/document/d/1PDswakRCBdnQEbJYZa01Fo8vo3DC6h3rtP3sckcp5Eo/edit#heading=h.fett23163lm5

Thanks,
Zach & Eric

Thanks for letting us know

That's fantastic! Great to hear DNS hostname worked well for you, as
supporting it is a new feature in consul-release, so it's good to hear it's
working "in the wild".

Best,
Amit

On Tue, Mar 15, 2016 at 8:15 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Amit



Thank you very much for your support, as I am able to successfully deploy
with cf-231.

We use DNS hostname for all the jobs, with dynamic IP networks. So I used
cf-231 release today..

All the jobs are up and running..



Thanks for the pointers you have given..



Regards

Nithiyasri



*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Tuesday, March 15, 2016 8:00 AM

*To:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>
*Cc:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>
*Subject:* Re: Reg the minimal-openstack yml files



Try the following:



ssh onto the consul VM

run "sudo su -" to switch to root

run "monit stop consul_agent"

run "ps uwax | grep consul" to see if there are any errant consul-related
processes, and kill them

run "rm -rf /var/vcap/store/consul_agent"

run "monit start consul_agent"



This will blow away all the consul data, which is okay because it's all
ephemeral data that is continuously repopulated. In your case, the "rm -rf"
step may not be strictly necessary, but this "hard reset" strategy tends to
be the cleanest and avoids unforeseen edge cases.



This is all assuming you only have one instance of the consul server.



Best,

Amit





On Mon, Mar 14, 2016 at 7:22 PM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:



Hi Amit



Sorry if the colors, and bolding disturbs you ..

Will refrain from using that.



I tried giving the static ip in consul.agent.servers.lan:10.20.0.116. But
I get following error in cf-230



==> Error starting agent: Failed to start Consul server: Failed to start
RPC layer: listen tcp 10.20.0.116:8300: bind: address already in use





Regards

Nithiyasri





*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Monday, March 14, 2016 11:10 PM
*To:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>
*Cc:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>


*Subject:* Re: Reg the minimal-openstack yml files



Using the DNS hostname of consul instead of the IP will not work in CF
v230. It should work in v231.



Open source Cloud Foundry does not guarantee direct upgrades between
non-consecutive versions. I have no way to tell you if 205 to 230 will
work. Some proprietary distributions of CF from certified vendors do
support slower upgrade cadences.



I'm sorry, but it looks as though you're asking 5 or 6 different
questions, and with the change in color, boldness, underline, etc. I find
it very difficult to follow.



Best,

Amit



On Mon, Mar 14, 2016 at 9:32 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Amit



We are trying to upgrade cf-205 to cf-230. So I have checked out with the
commit id of cf-230.yml and I could generate the manifest using the
generate_deployment_manifest script for our stub.

While deploying , *consul_z1* job was not started, *though all the other
jobs are running.*



*Consul Logs below ::*



*consul_agent.stdout.log**::*



{"timestamp":"1457972549.805328369","source":"confab","message":"confab.agent-client.verify-joined.members.not-joined","log_level":2,"data":{"error":"no
expected members","members":["10.20.0.116"],"wan":false}}



*consul_agent.stderr.log**::*



++ tee -a /var/vcap/sys/log/consul_agent/consul_agent.stdout.log

error booting consul agent: timeout exceeded



*Properties in yml:*



consul:

agent:

log_level: INFO

servers:

lan:

- 0.consul-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh

require_ssl: false



*Please note that 10.20.0.116 will resolve to lan member dns name. As we
don’t use static ip, we have given the dns hostname of the consul and the
corresponding ip is picked by the consul job too..*



I also have included the consul_agent template in haproxy job as stated by
you in the below mail thread
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/FCWBMKVOZSI4D6RBTOHL3JL42JN2LCRS/#4OW3HWF55O7TQRRM4UEQFHT4OPCNV7NB







*I have another question:*



Pls let me know is it possible to upgrade a deployment of cf-205 to
cf-230, with the updated yml file. *With cf-230 release*, new jobs like
api_z1,uaa, hm9000, clock_global, etc * have been added* and
cloud_controller, dea, etc *have been removed*.. Will the upgrade,
remove the unwanted jobs and create new jobs ? or a fresh deployment of
cf-230 is only possible?



We do cf push few apps into dea vms in our existing deployment. With the
new cf-230, will metron agent accept cf push of apps?





*Thank you very much for your support.*



Regards

Nithiyasri







*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Tuesday, March 08, 2016 3:43 AM
*To:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>
*Cc:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>


*Subject:* Re: Reg the minimal-openstack yml files



You need some static IPs, because some components don't use service
discovery, so clients need to be configured up front to know those
components' IPs. For example, the route-registrar jobs talk to the NATS
cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers
must be assigned static IPs in the manifest and components like
route-registrar that talk to NATS need to have those IPs provided in their
config in the manifest.



This current requirement on static IPs exists for some other components as
well. Some other components however would be fine with dynamic networking.
A private vendor may be able to work with you to manage a Cloud Foundry
installation that suits your networking requirements, but said guidance is
beyond the scope of this mailing list. The deployments that the Foundation
tests continuously use static IPs for simplicity.



Cheers,

Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com> wrote:

Thanks Amit!



A minor correction in our case.

We are not using static IP. We are only using dynamic IP.

None of the Cloud Foundry components are using Floating IP.



What is Pivotal recommendation for the networking? Is it static or dynamic?

Given that we are using dynamic IPs, do you foresee any issues with our
current approach to generate the manifest for CF-229?



Regards

Jayaraj



*From: *Amit Gupta <agupta(a)pivotal.io>
*Date: *Monday, March 7, 2016 at 12:24 PM
*To: *"Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at
Cisco)" <ngnanase(a)cisco.com>
*Cc: *Rohit Kumar <rokumar(a)pivotal.io>, Jayarajan Ramapurath Kozhummal <
jayark(a)cisco.com>, "Discussions about Cloud Foundry projects and the
system overall." <cf-dev(a)lists.cloudfoundry.org>
*Subject: *Re: Reg the minimal-openstack yml files



We're not likely to be able to create and maintain a new minimal manifest
any time soon.



As for generating the manifest, you can look at these docs for what needs
to go into the OpenStack stub as of the latest release:



http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html



Combining this with updating your current template guided by the errors is
a good way to generate the manifest. This is not guaranteed to work for
100% of use cases, e.g. if you want to use all floating IPs instead of
static IPs. But it is a good starting point.



Cheers,

Amit



On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



We are trying to upgrade our deployment with the latest cloud-foundry,
from 205 to 230 release, as per your advice.



We could see minimal-aws.yml available in the GIT repo. *Can we have a
similar one available for openstack environment, with which we can deploy
the basic cloud foundry* and do our custom changes on top of it



Parallely we are updating our stub to match the template yml files guided
by the errors given by the generate_deployment_manifest script. Kindly let
us know if this is the correct way to generate the manifest.





Regards

Nithiyasri

This turned out to be a browser cache issue. Clearing out the browser cache and it worked as before. Figured I post in case anyone else sees a similar issue after upgrading.

-Rich

Hi Amit

Thank you very much for your support, as I am able to successfully deploy with cf-231.
We use DNS hostname for all the jobs, with dynamic IP networks. So I used cf-231 release today..
All the jobs are up and running..

Thanks for the pointers you have given..

Regards
Nithiyasri

From: Amit Gupta [mailto:agupta(a)pivotal.io]
Sent: Tuesday, March 15, 2016 8:00 AM
To: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com>
Cc: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com>
Subject: Re: Reg the minimal-openstack yml files

Try the following:

ssh onto the consul VM
run "sudo su -" to switch to root
run "monit stop consul_agent"
run "ps uwax | grep consul" to see if there are any errant consul-related processes, and kill them
run "rm -rf /var/vcap/store/consul_agent"
run "monit start consul_agent"

This will blow away all the consul data, which is okay because it's all ephemeral data that is continuously repopulated. In your case, the "rm -rf" step may not be strictly necessary, but this "hard reset" strategy tends to be the cleanest and avoids unforeseen edge cases.

This is all assuming you only have one instance of the consul server.

Best,
Amit


On Mon, Mar 14, 2016 at 7:22 PM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:

Hi Amit

Sorry if the colors, and bolding disturbs you ..
Will refrain from using that.

I tried giving the static ip in consul.agent.servers.lan:10.20.0.116. But I get following error in cf-230

==> Error starting agent: Failed to start Consul server: Failed to start RPC layer: listen tcp 10.20.0.116:8300<http://10.20.0.116:8300>: bind: address already in use


Regards
Nithiyasri


From: Amit Gupta [mailto:agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>]
Sent: Monday, March 14, 2016 11:10 PM
To: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>
Cc: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>

Subject: Re: Reg the minimal-openstack yml files

Using the DNS hostname of consul instead of the IP will not work in CF v230. It should work in v231.

Open source Cloud Foundry does not guarantee direct upgrades between non-consecutive versions. I have no way to tell you if 205 to 230 will work. Some proprietary distributions of CF from certified vendors do support slower upgrade cadences.

I'm sorry, but it looks as though you're asking 5 or 6 different questions, and with the change in color, boldness, underline, etc. I find it very difficult to follow.

Best,
Amit

On Mon, Mar 14, 2016 at 9:32 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi Amit

We are trying to upgrade cf-205 to cf-230. So I have checked out with the commit id of cf-230.yml and I could generate the manifest using the generate_deployment_manifest script for our stub.
While deploying , consul_z1 job was not started, though all the other jobs are running.

Consul Logs below ::

consul_agent.stdout.log::

{"timestamp":"1457972549.805328369","source":"confab","message":"confab.agent-client.verify-joined.members.not-joined","log_level":2,"data":{"error":"no expected members","members":["10.20.0.116"],"wan":false}}

consul_agent.stderr.log::

++ tee -a /var/vcap/sys/log/consul_agent/consul_agent.stdout.log
error booting consul agent: timeout exceeded

Properties in yml:

consul:
agent:
log_level: INFO
servers:
lan:
- 0.consul-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh
require_ssl: false

Please note that 10.20.0.116 will resolve to lan member dns name. As we don’t use static ip, we have given the dns hostname of the consul and the corresponding ip is picked by the consul job too..

I also have included the consul_agent template in haproxy job as stated by you in the below mail threadhttps://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/FCWBMKVOZSI4D6RBTOHL3JL42JN2LCRS/#4OW3HWF55O7TQRRM4UEQFHT4OPCNV7NB



I have another question:

Pls let me know is it possible to upgrade a deployment of cf-205 to cf-230, with the updated yml file. With cf-230 release, new jobs like api_z1,uaa, hm9000, clock_global, etc have been added and cloud_controller, dea, etc have been removed.. Will the upgrade, remove the unwanted jobs and create new jobs ? or a fresh deployment of cf-230 is only possible?

We do cf push few apps into dea vms in our existing deployment. With the new cf-230, will metron agent accept cf push of apps?


Thank you very much for your support.

Regards
Nithiyasri



From: Amit Gupta [mailto:agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>]
Sent: Tuesday, March 08, 2016 3:43 AM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Cc: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>

Subject: Re: Reg the minimal-openstack yml files

You need some static IPs, because some components don't use service discovery, so clients need to be configured up front to know those components' IPs. For example, the route-registrar jobs talk to the NATS cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers must be assigned static IPs in the manifest and components like route-registrar that talk to NATS need to have those IPs provided in their config in the manifest.

This current requirement on static IPs exists for some other components as well. Some other components however would be fine with dynamic networking. A private vendor may be able to work with you to manage a Cloud Foundry installation that suits your networking requirements, but said guidance is beyond the scope of this mailing list. The deployments that the Foundation tests continuously use static IPs for simplicity.

Cheers,
Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>> wrote:
Thanks Amit!

A minor correction in our case.
We are not using static IP. We are only using dynamic IP.
None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?
Given that we are using dynamic IPs, do you foresee any issues with our current approach to generate the manifest for CF-229?

Regards
Jayaraj

From: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Date: Monday, March 7, 2016 at 12:24 PM
To: "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>
Cc: Rohit Kumar <rokumar(a)pivotal.io<mailto:rokumar(a)pivotal.io>>, Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>, "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest any time soon.

As for generating the manifest, you can look at these docs for what needs to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is a good way to generate the manifest. This is not guaranteed to work for 100% of use cases, e.g. if you want to use all floating IPs instead of static IPs. But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi

We are trying to upgrade our deployment with the latest cloud-foundry, from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. Can we have a similar one available for openstack environment, with which we can deploy the basic cloud foundry and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided by the errors given by the generate_deployment_manifest script. Kindly let us know if this is the correct way to generate the manifest.


Regards
Nithiyasri

Hi, guys,

I wanna get the users email , so per the docs of UAA at
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst#query-for-information-get-users,
i create a client with following scopes, scim.userids cloud_controller.read
password.write cloud_controller.write openid scim.write scim.read
cloud_controller.admin and with grant types:
authorization_code,refresh_token,client_credentials,password

when using this client to login a user , the JWT of the token parsed
doesn't contain scim.read scopt, lead to fail calling /Users api.
But , when login the client using uaac and using uaac context to obtain the
token, the token has scim.read scope and success calling /Users api

Here's related infos

​#​

uaac client get myconsole

scope: cloud_controller.admin cloud_controller.read
cloud_controller.write openid password.write
​ ​
scim.read scim.userids scim.write uaa.user
client_id: myconsole
resource_ids: none
authorized_grant_types: authorization_code client_credentials password
refresh_token
autoapprove: true
action: none
authorities: scim.userids cloud_controller.read password.write
cloud_controller.write openid
​ ​
scim.write scim.read cloud_controller.admin
name: myconsole
lastmodified: 1458017396000

​login the user user1 using myconsole client​

curl -X POST -d"username=

​user1(a)abc.com​
&password=password&client_id=myconsole&client_secret=
​XXX
&grant_type=password" -u "myconsole:
​XXX
" http://uaa.
​XXX
.com/oauth/token

​
got the token
get the users

curl -v -X GET -H "Accept: application/json" -H "Authorization: basic
$TOKEN" http://uaa.
​XXX.
com/Users?attributes=userName​

failed with

{
"error": "insufficient_scope",
"error_description": "Insufficient scope for this resource",
"scope": "scim.read zones.uaa.admin"

}​


​But if replace token with uaac context returned, i could get the users​




--

Regards,

Yitao

Hey Stanley,

Yup, that /state endpoint is for the Cell's bookkeeping (it's essentially
implementing the overcommit factor of 1, aka no overcommit), but it doesn't
mean physical memory has been allocated. But yes, for your purposes, it
doesn't matter if actual memory has been allocated or if for bookkeeping
purposes that memory is "reserved" by the Cell, since either way you can't
add more work to the Cell when it's reported available memory is less than
the max limit you want to put on your new container.

There are no near-term plans I know of for those features Deepak mentioned
(resource reclamation/pre-emption, and predictive analytics), I'll let the
Diego PM know you're interested. I've also mentioned it to Pivotal's data
science team as a possible optimization to explore.

Best,
Amit

it looks like one of the vms is unresponsive. you can try uding bosh cck to recover.

Sent from my iPhone

Hi Rafal,

Packages are not stored in the database for the V2 API, that table is for
experimental V3 support and wouldn't have any rows unless you are pushing
apps with the V3 API.

Can you tell us how you have configured your package blobstore? Have you
made any changes around this configuration and can you verify you have any
packages in your blobstore? If you share any configuration, please be sure
to scrub credentials.


Thanks,

Nick

On Mon, Mar 14, 2016 at 8:27 AM Rafal Radecki <radecki.rafal(a)gmail.com>
wrote:

Hi All :)

I am during a domain change for my CF 212. High level overview:
1) redeployment of CF212 with domain changed in the manifest -> done, ok
2) update of service brokers:
- route within the new domain added -> done, ok
- cf update-service-broker with new url -> done, ok
3) update of user provided service instances using 'cf
update-user-provided-service' with a modified json -> done, ok
4) addition of routes within new domain for every app wit cf map-route ->
done, ok
5) restart or restage of apps (one at a time) always returns:

"Server error, status code: 404, error code: 150002, message: The app
package could not be found: app-guid"

I checked ccdb for this and found that for an example app:
a) in apps table I have a row with:

guid | c0193c17-bc96-4e4f-801a-6105021128ce
package_hash |
9c2668161749dfec21e2f1ee00d451de18bcec90
droplet_hash |
c2b9137823d8f4c8cb16e4447e619c2c49aa28a4

b) in droplets table I have a row with:

droplet_hash | c2b9137823d8f4c8cb16e4447e619c2c49aa28a4

c) but in packages table I DO NOT have a row with package_hash =
9c2668161749dfec21e2f1ee00d451de18bcec90

From my point of view as a consequence of one of the steps which I
performed today when changing the domain some entries from ccdb tables do
not correlate with each other and because of this when any app is started I
get in cloud controller logs:

{"timestamp":1457965007.4719954,"message":"dea-client.no-package-found","log_level":"error","source":"cc.dea.client","data":{"request_guid":"aefe46f4-5737-49e1-4d70-7e17289e37f9::09509713-8

746-46fd-9bec-1cf4138cee9f","guid":"c0193c17-bc96-4e4f-801a-6105021128ce"},"thread_id":70246967656660,"fiber_id":70246971588820,"process_id":13042,"file":"/var/vcap/packages/cloud_controlle

r_ng/cloud_controller_ng/lib/cloud_controller/dea/client.rb","lineno":133,"method":"start_instance_at_index"}

-> "message":"dea-client.no-package-found"

Does anyone know a solution for this other than deleting and pushing the
apps once again?

BR,
Rafal.

then, after this problem, when i ‘bosh deploy’, another problem came, how can i continue deploy ?


Deploying
---------


Director task 6259
Started preparing deployment
Started preparing deployment Binding deployment. Done (00:00:00)
Started preparing deployment Binding releases. Done (00:00:00)
Started preparing deployment Binding existing deployment. Failed: VM `5fe98f76-5207-471b-9286-204e1f855076' is out of sync: expected to be a part of deployment `cf' but is actually a part of deployment `' (00:00:00)


Error 400003: VM `5fe98f76-5207-471b-9286-204e1f855076' is out of sync: expected to be a part of deployment `cf' but is actually a part of deployment `'


Task 6259 error




于长江
15101057694


原始邮件
发件人:于长江yuchangjiang(a)cmss.chinamobile.com
收件人:cf-dev(a)lists.cloudfoundry.org; cf-bosh(a)lists.cloudfoundry.org
发送时间:2016年3月15日(周二) 11:11
主题:vm state change into 'unknown/unknown' after a while


hello everybody,
when ‘bosh deploy’ all vms looks well , after a few hours some vms state turn into ‘unknown/unknown’ , like this:


+------------------------------------+--------------------+-----------+--------------+
| VM | State | VM Type | IPs |
+------------------------------------+--------------------+-----------+--------------+
| unknown/unknown | unresponsive agent | | |
| unknown/unknown | running | | |
| consul_z1/0 | running | small_z1 | 10.120.1.53 |
| doppler_z1/0 | running | medium_z1 | 10.120.1.105 |
| etcd_z1/0 | running | medium_z1 | 10.120.1.49 |
| ha_proxy_z1/0 | running | router_z1 | 10.120.1.41 |
| | | | 10.133.0.233 |
| hm9000_z1/0 | running | medium_z1 | 10.120.1.103 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.120.1.106 |
| nats_z1/0 | running | medium_z1 | 10.120.1.43 |
| nfs_z1/0 | running | medium_z1 | 10.120.1.44 |
| router_z1/0 | running | router_z1 | 10.120.1.46 |
| runner_z1/0 | running | runner_z1 | 10.120.1.104 |
| uaa_z1/0 | running | medium_z1 | 10.120.1.101 |
+------------------------------------+--------------------+-----------+--------------+


VMs total: 13


---------------------------------------------------------------------------------------------
then i login into the unknown vm, ‘monit summary’ display none result, there is nothing in directory ‘/var/vcap/jobs/‘, logs below:


/var/vcap/bosh/etc/monitrc:8: Warning: include files not found '/var/vcap/monit/job/*.monitrc'
The Monit daemon 5.2.4 uptime: 1h 3m


System 'system_5ad45340-005b-4f74-9a63-524cbe627634’ running


# ls /var/vcap/jobs/
consul_agent dea_logging_agent dea_next metron_agent
---------------------------------------------------------------------------------------------


someone meet this problem ?


---------------------------------------------------------------------------------------------
bosh version:v250
bosh-openstack-cpi-release: v4






于长江
15101057694

hello everybody,
when ‘bosh deploy’ all vms looks well , after a few hours some vms state turn into ‘unknown/unknown’ , like this:


+------------------------------------+--------------------+-----------+--------------+
| VM | State | VM Type | IPs |
+------------------------------------+--------------------+-----------+--------------+
| unknown/unknown | unresponsive agent | | |
| unknown/unknown | running | | |
| consul_z1/0 | running | small_z1 | 10.120.1.53 |
| doppler_z1/0 | running | medium_z1 | 10.120.1.105 |
| etcd_z1/0 | running | medium_z1 | 10.120.1.49 |
| ha_proxy_z1/0 | running | router_z1 | 10.120.1.41 |
| | | | 10.133.0.233 |
| hm9000_z1/0 | running | medium_z1 | 10.120.1.103 |
| loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.120.1.106 |
| nats_z1/0 | running | medium_z1 | 10.120.1.43 |
| nfs_z1/0 | running | medium_z1 | 10.120.1.44 |
| router_z1/0 | running | router_z1 | 10.120.1.46 |
| runner_z1/0 | running | runner_z1 | 10.120.1.104 |
| uaa_z1/0 | running | medium_z1 | 10.120.1.101 |
+------------------------------------+--------------------+-----------+--------------+


VMs total: 13


---------------------------------------------------------------------------------------------
then i login into the unknown vm, ‘monit summary’ display none result, there is nothing in directory ‘/var/vcap/jobs/‘, logs below:


/var/vcap/bosh/etc/monitrc:8: Warning: include files not found '/var/vcap/monit/job/*.monitrc'
The Monit daemon 5.2.4 uptime: 1h 3m


System 'system_5ad45340-005b-4f74-9a63-524cbe627634’ running


# ls /var/vcap/jobs/
consul_agent dea_logging_agent dea_next metron_agent
---------------------------------------------------------------------------------------------


someone meet this problem ?


---------------------------------------------------------------------------------------------
bosh version:v250
bosh-openstack-cpi-release: v4






于长江
15101057694

Try the following:

ssh onto the consul VM
run "sudo su -" to switch to root
run "monit stop consul_agent"
run "ps uwax | grep consul" to see if there are any errant consul-related
processes, and kill them
run "rm -rf /var/vcap/store/consul_agent"
run "monit start consul_agent"

This will blow away all the consul data, which is okay because it's all
ephemeral data that is continuously repopulated. In your case, the "rm -rf"
step may not be strictly necessary, but this "hard reset" strategy tends to
be the cleanest and avoids unforeseen edge cases.

This is all assuming you only have one instance of the consul server.

Best,
Amit


On Mon, Mar 14, 2016 at 7:22 PM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Amit



Sorry if the colors, and bolding disturbs you ..

Will refrain from using that.



I tried giving the static ip in consul.agent.servers.lan:10.20.0.116. But
I get following error in cf-230



==> Error starting agent: Failed to start Consul server: Failed to start
RPC layer: listen tcp 10.20.0.116:8300: bind: address already in use





Regards

Nithiyasri





*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Monday, March 14, 2016 11:10 PM
*To:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>
*Cc:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>

*Subject:* Re: Reg the minimal-openstack yml files



Using the DNS hostname of consul instead of the IP will not work in CF
v230. It should work in v231.



Open source Cloud Foundry does not guarantee direct upgrades between
non-consecutive versions. I have no way to tell you if 205 to 230 will
work. Some proprietary distributions of CF from certified vendors do
support slower upgrade cadences.



I'm sorry, but it looks as though you're asking 5 or 6 different
questions, and with the change in color, boldness, underline, etc. I find
it very difficult to follow.



Best,

Amit



On Mon, Mar 14, 2016 at 9:32 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Amit



We are trying to upgrade cf-205 to cf-230. So I have checked out with the
commit id of cf-230.yml and I could generate the manifest using the
generate_deployment_manifest script for our stub.

While deploying , *consul_z1* job was not started, *though all the other
jobs are running.*



*Consul Logs below ::*



*consul_agent.stdout.log**::*



{"timestamp":"1457972549.805328369","source":"confab","message":"confab.agent-client.verify-joined.members.not-joined","log_level":2,"data":{"error":"no
expected members","members":["10.20.0.116"],"wan":false}}



*consul_agent.stderr.log**::*



++ tee -a /var/vcap/sys/log/consul_agent/consul_agent.stdout.log

error booting consul agent: timeout exceeded



*Properties in yml:*



consul:

agent:

log_level: INFO

servers:

lan:

- 0.consul-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh

require_ssl: false



*Please note that 10.20.0.116 will resolve to lan member dns name. As we
don’t use static ip, we have given the dns hostname of the consul and the
corresponding ip is picked by the consul job too..*



I also have included the consul_agent template in haproxy job as stated by
you in the below mail thread
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/FCWBMKVOZSI4D6RBTOHL3JL42JN2LCRS/#4OW3HWF55O7TQRRM4UEQFHT4OPCNV7NB







*I have another question:*



Pls let me know is it possible to upgrade a deployment of cf-205 to
cf-230, with the updated yml file. *With cf-230 release*, new jobs like
api_z1,uaa, hm9000, clock_global, etc * have been added* and
cloud_controller, dea, etc *have been removed*.. Will the upgrade,
remove the unwanted jobs and create new jobs ? or a fresh deployment of
cf-230 is only possible?



We do cf push few apps into dea vms in our existing deployment. With the
new cf-230, will metron agent accept cf push of apps?





*Thank you very much for your support.*



Regards

Nithiyasri







*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Tuesday, March 08, 2016 3:43 AM
*To:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>
*Cc:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>


*Subject:* Re: Reg the minimal-openstack yml files



You need some static IPs, because some components don't use service
discovery, so clients need to be configured up front to know those
components' IPs. For example, the route-registrar jobs talk to the NATS
cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers
must be assigned static IPs in the manifest and components like
route-registrar that talk to NATS need to have those IPs provided in their
config in the manifest.



This current requirement on static IPs exists for some other components as
well. Some other components however would be fine with dynamic networking.
A private vendor may be able to work with you to manage a Cloud Foundry
installation that suits your networking requirements, but said guidance is
beyond the scope of this mailing list. The deployments that the Foundation
tests continuously use static IPs for simplicity.



Cheers,

Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com> wrote:

Thanks Amit!



A minor correction in our case.

We are not using static IP. We are only using dynamic IP.

None of the Cloud Foundry components are using Floating IP.



What is Pivotal recommendation for the networking? Is it static or dynamic?

Given that we are using dynamic IPs, do you foresee any issues with our
current approach to generate the manifest for CF-229?



Regards

Jayaraj



*From: *Amit Gupta <agupta(a)pivotal.io>
*Date: *Monday, March 7, 2016 at 12:24 PM
*To: *"Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at
Cisco)" <ngnanase(a)cisco.com>
*Cc: *Rohit Kumar <rokumar(a)pivotal.io>, Jayarajan Ramapurath Kozhummal <
jayark(a)cisco.com>, "Discussions about Cloud Foundry projects and the
system overall." <cf-dev(a)lists.cloudfoundry.org>
*Subject: *Re: Reg the minimal-openstack yml files



We're not likely to be able to create and maintain a new minimal manifest
any time soon.



As for generating the manifest, you can look at these docs for what needs
to go into the OpenStack stub as of the latest release:



http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html



Combining this with updating your current template guided by the errors is
a good way to generate the manifest. This is not guaranteed to work for
100% of use cases, e.g. if you want to use all floating IPs instead of
static IPs. But it is a good starting point.



Cheers,

Amit



On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



We are trying to upgrade our deployment with the latest cloud-foundry,
from 205 to 230 release, as per your advice.



We could see minimal-aws.yml available in the GIT repo. *Can we have a
similar one available for openstack environment, with which we can deploy
the basic cloud foundry* and do our custom changes on top of it



Parallely we are updating our stub to match the template yml files guided
by the errors given by the generate_deployment_manifest script. Kindly let
us know if this is the correct way to generate the manifest.





Regards

Nithiyasri

Hi Amit

Sorry if the colors, and bolding disturbs you ..
Will refrain from using that.

I tried giving the static ip in consul.agent.servers.lan:10.20.0.116. But I get following error in cf-230

==> Error starting agent: Failed to start Consul server: Failed to start RPC layer: listen tcp 10.20.0.116:8300: bind: address already in use


Regards
Nithiyasri


From: Amit Gupta [mailto:agupta(a)pivotal.io]
Sent: Monday, March 14, 2016 11:10 PM
To: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com>
Cc: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com>
Subject: Re: Reg the minimal-openstack yml files

Using the DNS hostname of consul instead of the IP will not work in CF v230. It should work in v231.

Open source Cloud Foundry does not guarantee direct upgrades between non-consecutive versions. I have no way to tell you if 205 to 230 will work. Some proprietary distributions of CF from certified vendors do support slower upgrade cadences.

I'm sorry, but it looks as though you're asking 5 or 6 different questions, and with the change in color, boldness, underline, etc. I find it very difficult to follow.

Best,
Amit

On Mon, Mar 14, 2016 at 9:32 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi Amit

We are trying to upgrade cf-205 to cf-230. So I have checked out with the commit id of cf-230.yml and I could generate the manifest using the generate_deployment_manifest script for our stub.
While deploying , consul_z1 job was not started, though all the other jobs are running.

Consul Logs below ::

consul_agent.stdout.log::

{"timestamp":"1457972549.805328369","source":"confab","message":"confab.agent-client.verify-joined.members.not-joined","log_level":2,"data":{"error":"no expected members","members":["10.20.0.116"],"wan":false}}

consul_agent.stderr.log::

++ tee -a /var/vcap/sys/log/consul_agent/consul_agent.stdout.log
error booting consul agent: timeout exceeded

Properties in yml:

consul:
agent:
log_level: INFO
servers:
lan:
- 0.consul-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh
require_ssl: false

Please note that 10.20.0.116 will resolve to lan member dns name. As we don’t use static ip, we have given the dns hostname of the consul and the corresponding ip is picked by the consul job too..

I also have included the consul_agent template in haproxy job as stated by you in the below mail threadhttps://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/FCWBMKVOZSI4D6RBTOHL3JL42JN2LCRS/#4OW3HWF55O7TQRRM4UEQFHT4OPCNV7NB



I have another question:

Pls let me know is it possible to upgrade a deployment of cf-205 to cf-230, with the updated yml file. With cf-230 release, new jobs like api_z1,uaa, hm9000, clock_global, etc have been added and cloud_controller, dea, etc have been removed.. Will the upgrade, remove the unwanted jobs and create new jobs ? or a fresh deployment of cf-230 is only possible?

We do cf push few apps into dea vms in our existing deployment. With the new cf-230, will metron agent accept cf push of apps?


Thank you very much for your support.

Regards
Nithiyasri



From: Amit Gupta [mailto:agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>]
Sent: Tuesday, March 08, 2016 3:43 AM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Cc: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>

Subject: Re: Reg the minimal-openstack yml files

You need some static IPs, because some components don't use service discovery, so clients need to be configured up front to know those components' IPs. For example, the route-registrar jobs talk to the NATS cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers must be assigned static IPs in the manifest and components like route-registrar that talk to NATS need to have those IPs provided in their config in the manifest.

This current requirement on static IPs exists for some other components as well. Some other components however would be fine with dynamic networking. A private vendor may be able to work with you to manage a Cloud Foundry installation that suits your networking requirements, but said guidance is beyond the scope of this mailing list. The deployments that the Foundation tests continuously use static IPs for simplicity.

Cheers,
Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>> wrote:
Thanks Amit!

A minor correction in our case.
We are not using static IP. We are only using dynamic IP.
None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?
Given that we are using dynamic IPs, do you foresee any issues with our current approach to generate the manifest for CF-229?

Regards
Jayaraj

From: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Date: Monday, March 7, 2016 at 12:24 PM
To: "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>
Cc: Rohit Kumar <rokumar(a)pivotal.io<mailto:rokumar(a)pivotal.io>>, Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>, "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest any time soon.

As for generating the manifest, you can look at these docs for what needs to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is a good way to generate the manifest. This is not guaranteed to work for 100% of use cases, e.g. if you want to use all floating IPs instead of static IPs. But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi

We are trying to upgrade our deployment with the latest cloud-foundry, from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. Can we have a similar one available for openstack environment, with which we can deploy the basic cloud foundry and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided by the errors given by the generate_deployment_manifest script. Kindly let us know if this is the correct way to generate the manifest.


Regards
Nithiyasri

Yes, we do not recommend using HaProxy in production environments because
of this downtime issue. I have heard of some HaProxy load balancer
solutions using keepalived
<https://www.howtoforge.com/setting-up-a-high-availability-load-balancer-with-haproxy-keepalived-on-debian-lenny>
but
have not investigated/tried them.

Stephen, for health checking the GoRouter we configure it to check that it
can establish a TCP connections on port 80 (or whatever port it is
listening on) with additional configuration depending on your load
balancer's features. For example, on the Routing team we use AWS for our
test environments and configure it with a connection timeout, checking
interval, healthy threshold, and unhealthy threshold. We then rely on the
AWS ELB's load balancing algorithm to retry any failed connections.

One thing to note is that most load balancers, like the GoRouter and AWS
ELB, will only retry on connection errors, if a connection is established
any HTTP errors will be sent back to the client.

Chris and Iryna, CF Routing

Hi Dimitar,

Thank you for pointing this issue out to us. I've raised a story in the CF
Docs tracker to verify and correct this issue:
https://www.pivotaltracker.com/story/show/115647925

Thanks again!

Max Hufnagel

On Mon, Mar 14, 2016 at 11:01 AM, Dimitar Valov <dimitar.valov(a)gmail.com>
wrote:

Hi,

According to
https://docs.cloudfoundry.org/buildpacks/custom.html#release-script the
config_vars section of the release output should be taken into account when
starting applications. However this is not the case.

I've went through the of dea_ng (source and test cases) and I could not
find any traces of this being implements. Only in the fixutres:
fake_buildpacks\admin_buildpack\bin\release, there's
config_vars:
PATH: bin:/usr/local/bin:/usr/bin:/bin
FROM_BUILD_PACK: "yes"

I've found this https://github.com/cloudfoundry/java-buildpack/issues/3
from 2013... Why does the documentation mention them? Are there any plans
to implement them.

Best Regards,
Dimitar

Hi Jim,

So, to be clear what we're basically doing is using unicode newline
character to fool loggregator (which is looking for \n) into thinking that
it isn't a new log event right? Does \u2028 work as a new line character
when tailing logs in the CLI? Anyone tried this unicode new line character
in various consoles? IDE, xterm, etc? I'm wondering if developers will
need to have different config for development.

Mike

On Mon, Mar 14, 2016 at 12:17 PM, Jim CF Campbell <jcampbell(a)pivotal.io>
wrote:

Hi Mike and Alex,

Two things - for Java, we are working toward defining an enhanced metric
format that will support transport of Multi Lines.

The second is this workaround that David Laing suggested for Logstash.
Think you could use it for Splunk?

With the Java Logback library you can do this by adding
"%replace(%xException){'\n','\u2028'}%nopex" to your logging config[1] ,
and then use the following logstash conf.[2]
Replace the unicode newline character \u2028 with \n, which Kibana will
display as a new line.

mutate {

gsub => [ "[@message]", '\u2028', "

"]
^^^ Seems that passing a string with an actual newline in it is the only
way to make gsub work

}

to replace the token with a regular newline again so it displays
"properly" in Kibana.

[1] github.com/dpin...ication.yml#L12
<https://github.com/dpinto-pivotal/cf-SpringBootTrader-config/blob/master/application.yml#L12>

[2] github.com/logs...se.conf#L60-L64
<https://github.com/logsearch/logsearch-for-cloudfoundry/blob/master/src/logsearch-config/src/logstash-filters/snippets/firehose.conf#L60-L64>


On Mon, Mar 14, 2016 at 11:11 AM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

I'll let the Loggregator team respond formally. But, in my conversations
with the Loggregator team I think we're basically stuck not sure what the
right thing to do is on the client side. How does the client trigger in
loggregator that this is a multi line log message or what is the right way
for loggregator to detect that the client is trying to send a multi line
log message? Any ideas?

Mike

On Mon, Mar 14, 2016 at 10:25 AM, Aliaksandr Prysmakou <
prysmakou(a)gmail.com> wrote:

Hi guys,
Are there any updates about "Multiline Log Entry" issue? How correctly
deal with stacktraces?
Links to the tracker to read?
----
Alex Prysmakou / Altoros
Tel: (617) 841-2121 ext. 5161 | Toll free: 855-ALTOROS
Skype: aliaksandr.prysmakou
www.altoros.com | blog.altoros.com | twitter.com/altoros

--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963

Hi Jim,

Thank you for sharing it with us.

Two things - for Java, we are working toward defining an enhanced metric format that will support transport of Multi Lines.

Could you please share a link to track status of this work?
So there are "first class citizen"(Java) and others (Ruby, Python etc)?
Is it so rare case that we should use workarounds?

The second is this workaround that David Laing suggested for Logstash. Think you could use it for Splunk?

With the Java Logback library you can do this by adding "%replace(%xException){'\n','\u2028'}%nopex" to your logging config[1] , and then use the following logstash conf.[2]
Replace the unicode newline character \u2028 with \n, which Kibana will display as a new line.

mutate {
gsub => [ "[@message]", '\u2028', "
"]
^^^ Seems that passing a string with an actual newline in it is the only way to make gsub work

}
to replace the token with a regular newline again so it displays "properly" in Kibana.
[1] github.com/dpin...ication.yml#L12 <https://github.com/dpinto-pivotal/cf-SpringBootTrader-config/blob/master/application.yml#L12>
[2] github.com/logs...se.conf#L60-L64 <https://github.com/logsearch/logsearch-for-cloudfoundry/blob/master/src/logsearch-config/src/logstash-filters/snippets/firehose.conf#L60-L64>

On Mon, Mar 14, 2016 at 11:11 AM, Mike Youngstrom <youngm(a)gmail.com <mailto:youngm(a)gmail.com>> wrote:
I'll let the Loggregator team respond formally. But, in my conversations with the Loggregator team I think we're basically stuck not sure what the right thing to do is on the client side. How does the client trigger in loggregator that this is a multi line log message or what is the right way for loggregator to detect that the client is trying to send a multi line log message? Any ideas?

Mike

On Mon, Mar 14, 2016 at 10:25 AM, Aliaksandr Prysmakou <prysmakou(a)gmail.com <mailto:prysmakou(a)gmail.com>> wrote:
Hi guys,
Are there any updates about "Multiline Log Entry" issue? How correctly deal with stacktraces?
Links to the tracker to read?
----
Alex Prysmakou / Altoros
Tel: (617) 841-2121 ext. 5161 <tel:%28617%29%20841-2121%20ext.%205161> | Toll free: 855-ALTOROS
Skype: aliaksandr.prysmakou
www.altoros.com <http://www.altoros.com/> | blog.altoros.com <http://blog.altoros.com/> | twitter.com/altoros <http://twitter.com/altoros>




--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963

Hi Mike and Alex,

Two things - for Java, we are working toward defining an enhanced metric
format that will support transport of Multi Lines.

The second is this workaround that David Laing suggested for Logstash.
Think you could use it for Splunk?

With the Java Logback library you can do this by adding
"%replace(%xException){'\n','\u2028'}%nopex" to your logging config[1] ,
and then use the following logstash conf.[2]
Replace the unicode newline character \u2028 with \n, which Kibana will
display as a new line.

mutate {

gsub => [ "[@message]", '\u2028', "

"]
^^^ Seems that passing a string with an actual newline in it is the only
way to make gsub work

}

to replace the token with a regular newline again so it displays "properly"
in Kibana.

[1] github.com/dpin...ication.yml#L12
<https://github.com/dpinto-pivotal/cf-SpringBootTrader-config/blob/master/application.yml#L12>

[2] github.com/logs...se.conf#L60-L64
<https://github.com/logsearch/logsearch-for-cloudfoundry/blob/master/src/logsearch-config/src/logstash-filters/snippets/firehose.conf#L60-L64>

--
Jim Campbell | Product Manager | Cloud Foundry | Pivotal.io | 303.618.0963

Hi,

According to https://docs.cloudfoundry.org/buildpacks/custom.html#release-script the config_vars section of the release output should be taken into account when starting applications. However this is not the case.

I've went through the of dea_ng (source and test cases) and I could not find any traces of this being implements. Only in the fixutres: fake_buildpacks\admin_buildpack\bin\release, there's
config_vars:
PATH: bin:/usr/local/bin:/usr/bin:/bin
FROM_BUILD_PACK: "yes"

I've found this https://github.com/cloudfoundry/java-buildpack/issues/3 from 2013... Why does the documentation mention them? Are there any plans to implement them.

Best Regards,
Dimitar