Date   

Re: [abacus] Eureka vs gorouter

Saravanakumar A. Srinivasan
 


> Last few days, I am working on getting Hystrix Dashboard to use Turbine +  Eureka to monitor Abacus and will be updating the document with the steps needed to get this going.   

I have added steps [1] to monitor Abacus applications using Eureka and Turbine. @Hristo... Could you please try it out and update the corresponding Cloud Foundry installation steps at [2]?

[1] https://github.com/cloudfoundry-incubator/cf-abacus/commit/480e5603963be366e688361454754cb5e3eb4dd3

Thanks,
Saravanakumar Srinivasan (Assk),

-----Saravanakumar A Srinivasan/Burlingame/IBM wrote: -----
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...>
From: Saravanakumar A Srinivasan/Burlingame/IBM
Date: 11/12/2015 08:55PM
Subject: Re: [cf-dev] Re: [abacus] Eureka vs gorouter

I believe that Assk (@sasrin) has started to document the beginning of that monitoring setup as well in doc/monitor.md [1]
Yes...We have started to write down the steps to setup the Hystrix Dashboard to monitor Abacus and Thanks to @Hristo, we now have steps to configure Hystrix Dashboard using Cloud Foundry environment as well. 

There are several ways to set up Hystrix to monitor Cloud apps, but Eureka comes handy when you don't know their IP addresses ahead of time. The usual setup is then to use Eureka + Turbine + Hystrix (as described in [2]). You get > your apps to register with Eureka, set up Turbine to get their IPs from Eureka, and serve an aggregated performance data stream to your Hystrix dashboard for all your apps.

Last few days, I am working on getting Hystrix Dashboard to use Turbine +  Eureka to monitor Abacus and will be updating the document with the steps needed to get this going.   


Thanks,
Saravanakumar Srinivasan (Assk),




CF CLI v6.14.0 Released Today

Koper, Dies <diesk@...>
 

The CF CLI team just cut 6.14.0. Binaries and link to release notes are available at:

https://github.com/cloudfoundry/cli#downloads

Highlights of this release include:

RBAC for Org and Space Managers

With this release of the CLI, Org Managers can assign org and space roles to users. Also, Space Managers can assign space roles to users in their org, using the existing `cf set-org-role` and `cf set-space-role` (and equivalent 'unset') commands. This feature was previously only available to admin users.
Note that this feature requires the target CF release to be v219 (CC API v2.37.0) or higher, and the feature flags 'set_roles_by_username' and 'set_roles_by_username' to be enabled (use `cf feature-flags` to check).
Related to the above, as an Org Manager creating an org using `cf create-org`, you will now be assigned Org Manager role in it automatically so you can start managing it straight away.

Other Features:

* New command `cf purge-service-instance` for admins dealing with an unresponsive service instance, to delete it
* New command `cf router-groups` for admins to list the type of router groups configured. This command is in preparation of new Routing features to be rolled out in the near future, and relies on the Routing API endpoint made available in the targeted CF
* The version reported by `cf -v` is now SemVer<http://semver.org/> compliant and easier to understand by leaving out the build time component: cf version 6.14.0+2654a47-2015-11-18

Bug Fixes:

* Fixed issues<https://github.com/cloudfoundry/cli/pull/635> in help and usage of `cf stack` and `cf stacks`. [PR from SrinivasChilveri]
* Fixed issues<https://www.pivotaltracker.com/story/show/107628656> in help examples of `cf create-service`
* Fixed issue<https://github.com/cloudfoundry/cli/issues/653> with plugins with long command names
* `cf push` did not upload empty folders<https://www.pivotaltracker.com/story/show/107835158>, causing an issue with the Java buildpack not recognizing a valid Web application with an empty WEB-INF folder
* When pushing small apps with `cf push`, the 'bytes' unit was omitted<https://www.pivotaltracker.com/story/show/108280642> in the upload status message

Improved User Experience/Error Messages:

* Meaningful error message<https://www.pivotaltracker.com/story/show/66201062> when trying to create a user with `cf create-user` and passing in an invalid username
* Proper error message<https://github.com/cloudfoundry/cli/pull/638> when passing more arguments than the required app name argument to `cf ssh`. [PR from SrinivasChilveri]
* Fixed typo<https://github.com/cloudfoundry/cli/pull/672> in `cf space` help. [PR from Matthew M. Boedicker]
* Updated the bug report template<https://www.pivotaltracker.com/story/show/106902332> (both on the GitHub CLI repo page and in the CLI "panic" message) to request more details, such as trace output, and reproducibility on the latest CLI release

CLI Plugins Development:

* Tokens are now refreshed<https://github.com/cloudfoundry/cli/issues/597> automatically when the AccessToken plugin method is called.
* Added<https://www.pivotaltracker.com/story/show/107137824> documentation and coding examples around libraries available for TDD plugin development
* Plugin API was leaking file descriptors<https://www.pivotaltracker.com/story/show/108012132>
* MinCliVersion in PluginMetadata was not working<https://www.pivotaltracker.com/story/show/103468700> for certain CLI versions
* Field PackageUpdatedAt in GetApp response was not populated<https://www.pivotaltracker.com/story/show/107859114>

New Plugins:

* Manifest Generator Plugin https://github.com/ArthurHlt/plugin-cf-manifest-generator


Enjoy!

Regards,
Dies Koper
Cloud Foundry CLI PM


Re: cloud_controller_ng performance degrades slowly over time

Matt Cholick
 

So things got a little more interesting yesterday. We have some custom
Golang jobs for auditing and some other purposes. A recent change added the
consult_agent template to the box with these jobs and I saw similar
behavior: requests with dns resolution taking +5s consistently, fixable by
restarting the process. I wrote a small Golang app to try and reproduce it.
I got similar, but not identical behavior in the test app.

package main


import (
"net"
"fmt"
"time"
)

func main() {
for {
startOpen := time.Now()
conn, err := net.Dial("tcp", "api.useast.appfog.qa.ctl.io:http")
if err != nil {
fmt.Printf("%#v", err)
} else {
endOpen := time.Now()
conn.Close()
endClose := time.Now()
openDur := endOpen.Sub(startOpen) / 1e6
closeDur := endClose.Sub(endOpen) / 1e6

fmt.Printf("dns_open: %dms | dns_close: %dms\n", openDur, closeDur)
}
}
}

After some time, I saw:
dns_open: 14ms | dns_close: 0ms
dns_open: 5019ms | dns_close: 1ms
dns_open: 25ms | dns_close: 1ms
dns_open: 27ms | dns_close: 1ms
dns_open: 20ms | dns_close: 0ms
dns_open: 5015ms | dns_close: 0ms
dns_open: 14ms | dns_close: 0ms
dns_open: 5018ms | dns_close: 1ms
dns_open: 15ms | dns_close: 0ms
dns_open: 5018ms | dns_close: 1ms
dns_open: 19ms | dns_close: 0ms
dns_open: 5046ms | dns_close: 0ms

To reiterate, the behavior in the actual app seemed to be every single
request. The test app went into a similar mode where a roughly every other
request failed. Here's the strace for that section:

https://gist.github.com/cholick/428df93c112067f50558

On some other runs it was quite consistently alternating slow and fast
calls (for ~300 calls in a row alternating slow/fast). Just like the Ruby,
once things get into an unhappy state they stay that way until restarting
the process. (The process ran for 12 hours and never recovered).

I can capture a Ruby strace too if you'd like, but I figured this was more
useful as it added more novel information to the mix.

These Golang jobs have run in production for months without showing this
behavior and, within a few hours of including the consul template, the
issue manifested.


Re: CFScaler - CloudFoundry Auto Scaling

Harpreet Ghai
 

Hi,
Good work,
Could you please provide some documentation. Could you explain, how does it work and how it is to be executed.
I'm new to go, so your response will be a great help.

Regards


Unable to deploy application

Deepak Arn <arn.deepak1@...>
 

Hello,

I have configured local cloud foundry on openstack, its giving some staging
error while deploying web application. Please find the log file attached
below.

Thanks,

--
Regards
Deepak Kumar Aggarwal
Research Assitant,
Concordia University, Canada


Re: Private brokers - how to enable access?

Dr Nic Williams <drnicwilliams@...>
 

Ok, discovered that private brokers automatically show up in marketplace
and "just work"; no need for "cf enable-service-access"

/closed :)

On Wed, Nov 18, 2015 at 10:11 AM, Dr Nic Williams <drnicwilliams(a)gmail.com>
wrote:

I was experimenting with private service brokers today using v225 (via
PivotalWS) and was successfully able to register a service broker via the
API/cf curl, but cannot figure out how to enable access.

If I use the CLI I get an error:

Server error, status code: 400, error code: 260001, message: Service Plan
Visibility is invalid: service_plan is from a private broker

But I don't see anything in the API for specifying space_guid information
for plan visibility.
http://apidocs.cloudfoundry.org/225/service_plan_visibilities/creating_a_service_plan_visibility.html

What am I missing? Thanks!

WIP blog post is coming long

Nic

--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic


--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic


Private brokers - how to enable access?

Dr Nic Williams <drnicwilliams@...>
 

I was experimenting with private service brokers today using v225 (via
PivotalWS) and was successfully able to register a service broker via the
API/cf curl, but cannot figure out how to enable access.

If I use the CLI I get an error:

Server error, status code: 400, error code: 260001, message: Service Plan
Visibility is invalid: service_plan is from a private broker

But I don't see anything in the API for specifying space_guid information
for plan visibility.
http://apidocs.cloudfoundry.org/225/service_plan_visibilities/creating_a_service_plan_visibility.html

What am I missing? Thanks!

WIP blog post is coming long

Nic

--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic


tcpdump via diego ssh

Mike Youngstrom <youngm@...>
 

I have some customers who would like to be able to execute a tcpdump on
their container interface.

I made a quick attempt:

/usr/sbin$ ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode
DEFAULT group default
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
535: w4elk1d3ta31-1: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
state UP mode DEFAULT group default qlen 1
link/ether 72:d5:fa:30:96:29 brd ff:ff:ff:ff:ff:ff

./tcpdump -i w4elk1d3ta31-1
tcpdump: w4elk1d3ta31-1: You don't have permission to capture on that device
(socket: Operation not permitted)

Anyone know if this is something that is impossible within an unprivileged
container? Or is there by chance something that can be done to enable
tcpdump from within a container?

It appears it is possible to enable tcpdump to work without root but not
sure if it would apply to our situation:
http://peternixon.net/news/2012/01/28/configure-tcpdump-work-non-root-user-opensuse-using-file-system-capabilities/

Thoughts?

Mike


diego: disk filling up over time

Tom Sherrod <tom.sherrod@...>
 

diego release 0.1398.0

After a couple of weeks of dev, the cells end up filling their disks. Did I miss a clean up job somewhere?
Currently, once pushes start failing, I get bosh to recreate the machine.

Other options?

Thanks,
Tom


Re: regarding using public key to verify client

Noburou TANIGUCHI
 

ankit,

It's still unclear for me what you want for me to explain.

I want to put these in buildpack and specify it’s path in app’s manifest
file which is used while deploying the application on cloud foundry.
I think I have explained in this post [1] about where in java buildpack to
put cert keys and how to modify java buildpack to put them into the place
where an app can access when it is running. So my question, again, is "what
do you want to be explained more?" in the post [1].

[1]
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2732.html


ankit wrote
Hi

See basically what I need is to put some file in my custom java buildpack
and let my application use that file when it is deployed on the cloud
foundry whether it is those certificate key files or any other file. So, I
just don’t want to package these files with my app war file. I want to put
these in buildpack and specify it’s path in app’s manifest file which is
used while deploying the application on cloud foundry.

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:
ml-node+s70369n2741h92(a).nabble
]
Sent: Wednesday, November 18, 2015 7:36 AM
To: ., Ankit &lt;
ankit.ankit@
&gt;
Subject: RE: regarding using public key to verify client

Hi ankit,

Can you explain it a little bit or refer any document. That would be
helpful.
Would you please narrow the focus of your question?
What do you want to know about?

Thanks.

ankit wrote
Hi

Can you explain it a little bit or refer any document. That would be
helpful.

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2741&amp;i=0&gt;]
Sent: Tuesday, November 17, 2015 12:59 PM
To: ., Ankit <[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2741&amp;i=1&gt;>
Subject: RE: regarding using public key to verify client

Hello ankit,

But I just want to know that is there any way where I can put my key
files to java build pack with which I am deploying my application on
cloud foundry and let my application use those keys from build pack to
verify the signature. Because I don’t want to add those key files to my
app’s war.
Now I think I've understood what you want to do.

And one more thing that I am using custom java buildpack. (This thing I
forgot to mention earlier.)
So I think you can add cert keys anywhere in java buildpack you think
appropriate (`resources` directory, for example), and modify the buildpack
to copy them in the buildpack's compile phase, such like:

```diff
diff --git a/bin/compile b/bin/compile
index 53e0b8d..3d198c9 100755
--- a/bin/compile
+++ b/bin/compile
@@ -23,6 +23,10 @@ require 'java_buildpack/buildpack'

build_dir = ARGV[0]

+require('fileutils')
+cert_keys_path = File.expand_path('../../resources/.ssh', __FILE__)
+FileUtils.cp_r(cert_keys_path, build_dir, {:preserve => true})
+
JavaBuildpack::Buildpack.with_buildpack(build_dir, 'Compile failed with
exception %s') do |buildpack|
buildpack.compile
end
```

(the code above assumes that the cert keys in `resources/.ssh/` and copies
them to `.ssh` under the application directory)


ankit wrote
Hi Taniguchi,

Thank you for your response. You are right that my application is
responsible for verification of the signature. But I just want to know
that is there any way where I can put my key files to java build pack with
which I am deploying my application on cloud foundry and let my
application use those keys from build pack to verify the signature.
Because I don’t want to add those key files to my app’s war.

Just like we put cacerts in java buildpack which is used by application,
can’t we put the key files in buildpack and use those by our application
to verify signature.

And one more thing that I am using custom java buildpack. (This thing I
forgot to mention earlier.)

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2732&amp;i=0&gt;]
Sent: Monday, November 16, 2015 6:09 PM
To: ., Ankit <[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2732&amp;i=1&gt;>
Subject: Re: regarding using public key to verify client

Hi ankit,

First of all, do you think who is responsible to verify the signature?
Your application? Or (one of) the components of Cloud Foundry? I assume
the former is your answer. I think there is no functionality in Cloud
Foundry to verify client signature.

Then, if you use the Cloud Foundry java-buildpack to deploy your
application, I think there is the only one way to send key files with your
app on deployment. It is to add your key files to your app's war / jar /
zip file, primitively like:

```
jar uvf your-war-jar-zip-file path-to-your-key-files-or-directories
```

But you may add a maven / gradle task to do such a thing.

This is because the Cloud Foundry java-buildpack accepts only one
zip-format file on a deployment.

# Please correct this post if I am wrong. Thank you.

ankit wrote
Suppose my application is deployed on the cloud foundry and my client
sends a POST request that contains some message but that message is
digitally signed by client’s private key. So, I need client’s public
key(digital id of client) to verify my client for inbound calls in the
cloud foundry where application is running. So, can you tell me where can
I put these public keys(digital IDs of clients) in java build pack or any
other place.
Similarly, for outbound calls I want my message to be digitally signed and
for that I need private key to be used. So, where can I put that also?
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion
below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2719.html
To unsubscribe from regarding using public key to verify client, click
here&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e%3chttp://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e%3e&gt;.
NAML&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e%3chttp://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e%3e&gt;

________________________________
The information contained in this message may be confidential and legally
protected under applicable law. The message is intended solely for the
addressee(s). If you are not the intended recipient, you are hereby
notified that any use, forwarding, dissemination, or reproduction of this
message is strictly prohibited and may be unlawful. If you are not the
intended recipient, please contact the sender by return e-mail and destroy
all copies of the original message.
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion
below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2732.html
To unsubscribe from regarding using public key to verify client, click
here&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e&gt;.
NAML&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e&gt;
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion
below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2741.html
To unsubscribe from regarding using public key to verify client, click
here&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=&gt;.
NAML&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml&gt;




-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2746.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: regarding using public key to verify client

ankit <ankit.ankit@...>
 

Hi

See basically what I need is to put some file in my custom java buildpack and let my application use that file when it is deployed on the cloud foundry whether it is those certificate key files or any other file. So, I just don’t want to package these files with my app war file. I want to put these in buildpack and specify it’s path in app’s manifest file which is used while deploying the application on cloud foundry.

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:ml-node+s70369n2741h92(a)n6.nabble.com]
Sent: Wednesday, November 18, 2015 7:36 AM
To: ., Ankit <ankit.ankit(a)philips.com>
Subject: RE: regarding using public key to verify client

Hi ankit,

Can you explain it a little bit or refer any document. That would be helpful.
Would you please narrow the focus of your question?
What do you want to know about?

Thanks.

ankit wrote
Hi

Can you explain it a little bit or refer any document. That would be helpful.

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=2741&i=0>]
Sent: Tuesday, November 17, 2015 12:59 PM
To: ., Ankit <[hidden email]</user/SendEmail.jtp?type=node&node=2741&i=1>>
Subject: RE: regarding using public key to verify client

Hello ankit,

But I just want to know that is there any way where I can put my key files to java build pack with which I am deploying my application on cloud foundry and let my application use those keys from build pack to verify the signature. Because I don’t want to add those key files to my app’s war.
Now I think I've understood what you want to do.

And one more thing that I am using custom java buildpack. (This thing I forgot to mention earlier.)
So I think you can add cert keys anywhere in java buildpack you think appropriate (`resources` directory, for example), and modify the buildpack to copy them in the buildpack's compile phase, such like:

```diff
diff --git a/bin/compile b/bin/compile
index 53e0b8d..3d198c9 100755
--- a/bin/compile
+++ b/bin/compile
@@ -23,6 +23,10 @@ require 'java_buildpack/buildpack'

build_dir = ARGV[0]

+require('fileutils')
+cert_keys_path = File.expand_path('../../resources/.ssh', __FILE__)
+FileUtils.cp_r(cert_keys_path, build_dir, {:preserve => true})
+
JavaBuildpack::Buildpack.with_buildpack(build_dir, 'Compile failed with exception %s') do |buildpack|
buildpack.compile
end
```

(the code above assumes that the cert keys in `resources/.ssh/` and copies them to `.ssh` under the application directory)


ankit wrote
Hi Taniguchi,

Thank you for your response. You are right that my application is responsible for verification of the signature. But I just want to know that is there any way where I can put my key files to java build pack with which I am deploying my application on cloud foundry and let my application use those keys from build pack to verify the signature. Because I don’t want to add those key files to my app’s war.

Just like we put cacerts in java buildpack which is used by application, can’t we put the key files in buildpack and use those by our application to verify signature.

And one more thing that I am using custom java buildpack. (This thing I forgot to mention earlier.)

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:[hidden email]</user/SendEmail.jtp?type=node&node=2732&i=0>]
Sent: Monday, November 16, 2015 6:09 PM
To: ., Ankit <[hidden email]</user/SendEmail.jtp?type=node&node=2732&i=1>>
Subject: Re: regarding using public key to verify client

Hi ankit,

First of all, do you think who is responsible to verify the signature? Your application? Or (one of) the components of Cloud Foundry? I assume the former is your answer. I think there is no functionality in Cloud Foundry to verify client signature.

Then, if you use the Cloud Foundry java-buildpack to deploy your application, I think there is the only one way to send key files with your app on deployment. It is to add your key files to your app's war / jar / zip file, primitively like:

```
jar uvf your-war-jar-zip-file path-to-your-key-files-or-directories
```

But you may add a maven / gradle task to do such a thing.

This is because the Cloud Foundry java-buildpack accepts only one zip-format file on a deployment.

# Please correct this post if I am wrong. Thank you.

ankit wrote
Suppose my application is deployed on the cloud foundry and my client sends a POST request that contains some message but that message is digitally signed by client’s private key. So, I need client’s public key(digital id of client) to verify my client for inbound calls in the cloud foundry where application is running. So, can you tell me where can I put these public keys(digital IDs of clients) in java build pack or any other place.
Similarly, for outbound calls I want my message to be digitally signed and for that I need private key to be used. So, where can I put that also?
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2719.html
To unsubscribe from regarding using public key to verify client, click here<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e%3chttp://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e%3e>.
NAML<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e%3chttp://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e%3e>

________________________________
The information contained in this message may be confidential and legally protected under applicable law. The message is intended solely for the addressee(s). If you are not the intended recipient, you are hereby notified that any use, forwarding, dissemination, or reproduction of this message is strictly prohibited and may be unlawful. If you are not the intended recipient, please contact the sender by return e-mail and destroy all copies of the original message.
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2732.html
To unsubscribe from regarding using public key to verify client, click here<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e>.
NAML<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml><http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e>
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2741.html
To unsubscribe from regarding using public key to verify client, click here<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=2711&code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=>.
NAML<http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2745.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Also, the bosh vms command shows that api_z1/0 is not in running status
rather its in starting state always. Thanks..

On 18 November 2015 at 10:10, Parthiban Annadurai <senjiparthi(a)gmail.com>
wrote:

Hey Warren,
Thanks for your valuable suggestions. I have SSHed into
that VM. Monit Summary Command shows the following,

root(a)5c446a3d-3070-4d24-9f2e-1cff18218c07:/var/vcap/sys/log# monit summary
The Monit daemon 5.2.4 uptime: 20m

Process 'cloud_controller_ng' initializing
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' initializing
Process 'metron_agent' running
File 'nfs_mounter' Does not exist
System 'system_5c446a3d-3070-4d24-9f2e-1cff18218c07' running

Also I have checked for cloud_controller_ng_ctl.log, it has the following,

[2015-11-18 04:33:34+0000] ------------ STARTING cloud_controller_ng_ctl
at Wed Nov 18 04:32:53 UTC 2015 --------------
[2015-11-18 04:33:34+0000] Preparing local package directory
[2015-11-18 04:33:34+0000] Preparing local resource_pool directory
[2015-11-18 04:33:34+0000] Preparing local droplet directory
[2015-11-18 04:33:34+0000] Deprecated: Use -s or --insert-seed flag
[2015-11-18 04:33:34+0000] Killing
/var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 32522
[2015-11-18 04:33:34+0000] .Stopped


Then, nfs_mounter_ctl.log has the following,

[2015-11-18 04:27:20+0000] Found NFS mount, unmounting...
[2015-11-18 04:27:20+0000] NFS unmounted
[2015-11-18 04:27:20+0000] idmapd start/post-stop, process 25777
[2015-11-18 04:27:20+0000] NFS unmounted
[2015-11-18 04:27:20+0000] Mounting NFS...
[2015-11-18 04:27:20+0000] mount.nfs: timeout set for Wed Nov 18 04:29:20
2015
[2015-11-18 04:27:20+0000] mount.nfs: trying text-based options
'timeo=10,intr,lookupcache=positive,vers=4,addr=192.168.33.53,clientaddr=192.168.33.184'
[2015-11-18 04:27:20+0000] mount.nfs: trying text-based options
'timeo=10,intr,lookupcache=positive,addr=192.168.33.53'
[2015-11-18 04:27:20+0000] mount.nfs: prog 100003, trying vers=3, prot=6
[2015-11-18 04:27:20+0000] mount.nfs: prog 100005, trying vers=3, prot=17
[2015-11-18 04:27:20+0000] Failed to start: cannot write to NFS

I think the problem is with the NFS. Could you please help on this issue??
Thanks..

Regards

Parthiban A

On 18 November 2015 at 08:42, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Try ssh'ing onto the box to see what the logs say.

`bosh ssh api_z1 0`

Then,
`cd /var/vcap/sys/log/`

There are plenty of logs to look through, I'd start by running `monit
summary` (make sure to be root) to see if any process is failing. If there
is a process failing then look at its respective logs. Else start with
cloud_controller_ng log directory.


Re: CF-RELEASE v202 UPLOAD ERROR

Parthiban Annadurai <senjiparthi@...>
 

Hey Warren,
Thanks for your valuable suggestions. I have SSHed into
that VM. Monit Summary Command shows the following,

root(a)5c446a3d-3070-4d24-9f2e-1cff18218c07:/var/vcap/sys/log# monit summary
The Monit daemon 5.2.4 uptime: 20m

Process 'cloud_controller_ng' initializing
Process 'cloud_controller_worker_local_1' not monitored
Process 'cloud_controller_worker_local_2' not monitored
Process 'nginx_cc' initializing
Process 'metron_agent' running
File 'nfs_mounter' Does not exist
System 'system_5c446a3d-3070-4d24-9f2e-1cff18218c07' running

Also I have checked for cloud_controller_ng_ctl.log, it has the following,

[2015-11-18 04:33:34+0000] ------------ STARTING cloud_controller_ng_ctl at
Wed Nov 18 04:32:53 UTC 2015 --------------
[2015-11-18 04:33:34+0000] Preparing local package directory
[2015-11-18 04:33:34+0000] Preparing local resource_pool directory
[2015-11-18 04:33:34+0000] Preparing local droplet directory
[2015-11-18 04:33:34+0000] Deprecated: Use -s or --insert-seed flag
[2015-11-18 04:33:34+0000] Killing
/var/vcap/sys/run/cloud_controller_ng/cloud_controller_ng.pid: 32522
[2015-11-18 04:33:34+0000] .Stopped


Then, nfs_mounter_ctl.log has the following,

[2015-11-18 04:27:20+0000] Found NFS mount, unmounting...
[2015-11-18 04:27:20+0000] NFS unmounted
[2015-11-18 04:27:20+0000] idmapd start/post-stop, process 25777
[2015-11-18 04:27:20+0000] NFS unmounted
[2015-11-18 04:27:20+0000] Mounting NFS...
[2015-11-18 04:27:20+0000] mount.nfs: timeout set for Wed Nov 18 04:29:20
2015
[2015-11-18 04:27:20+0000] mount.nfs: trying text-based options
'timeo=10,intr,lookupcache=positive,vers=4,addr=192.168.33.53,clientaddr=192.168.33.184'
[2015-11-18 04:27:20+0000] mount.nfs: trying text-based options
'timeo=10,intr,lookupcache=positive,addr=192.168.33.53'
[2015-11-18 04:27:20+0000] mount.nfs: prog 100003, trying vers=3, prot=6
[2015-11-18 04:27:20+0000] mount.nfs: prog 100005, trying vers=3, prot=17
[2015-11-18 04:27:20+0000] Failed to start: cannot write to NFS

I think the problem is with the NFS. Could you please help on this issue??
Thanks..

Regards

Parthiban A

On 18 November 2015 at 08:42, Warren Fernandes <wfernandes(a)pivotal.io>
wrote:

Try ssh'ing onto the box to see what the logs say.

`bosh ssh api_z1 0`

Then,
`cd /var/vcap/sys/log/`

There are plenty of logs to look through, I'd start by running `monit
summary` (make sure to be root) to see if any process is failing. If there
is a process failing then look at its respective logs. Else start with
cloud_controller_ng log directory.


Re: CF-RELEASE v202 UPLOAD ERROR

Warren Fernandes
 

Try ssh'ing onto the box to see what the logs say.

`bosh ssh api_z1 0`

Then,
`cd /var/vcap/sys/log/`

There are plenty of logs to look through, I'd start by running `monit summary` (make sure to be root) to see if any process is failing. If there is a process failing then look at its respective logs. Else start with cloud_controller_ng log directory.


Re: regarding using public key to verify client

Noburou TANIGUCHI
 

Hi ankit,

Can you explain it a little bit or refer any document. That would be
helpful.
Would you please narrow the focus of your question?
What do you want to know about?

Thanks.



ankit wrote
Hi

Can you explain it a little bit or refer any document. That would be
helpful.

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:
ml-node+s70369n2732h13(a).nabble
]
Sent: Tuesday, November 17, 2015 12:59 PM
To: ., Ankit &lt;
ankit.ankit@
&gt;
Subject: RE: regarding using public key to verify client

Hello ankit,

But I just want to know that is there any way where I can put my key
files to java build pack with which I am deploying my application on
cloud foundry and let my application use those keys from build pack to
verify the signature. Because I don’t want to add those key files to my
app’s war.
Now I think I've understood what you want to do.

And one more thing that I am using custom java buildpack. (This thing I
forgot to mention earlier.)
So I think you can add cert keys anywhere in java buildpack you think
appropriate (`resources` directory, for example), and modify the buildpack
to copy them in the buildpack's compile phase, such like:

```diff
diff --git a/bin/compile b/bin/compile
index 53e0b8d..3d198c9 100755
--- a/bin/compile
+++ b/bin/compile
@@ -23,6 +23,10 @@ require 'java_buildpack/buildpack'

build_dir = ARGV[0]

+require('fileutils')
+cert_keys_path = File.expand_path('../../resources/.ssh', __FILE__)
+FileUtils.cp_r(cert_keys_path, build_dir, {:preserve => true})
+
JavaBuildpack::Buildpack.with_buildpack(build_dir, 'Compile failed with
exception %s') do |buildpack|
buildpack.compile
end
```

(the code above assumes that the cert keys in `resources/.ssh/` and copies
them to `.ssh` under the application directory)


ankit wrote
Hi Taniguchi,

Thank you for your response. You are right that my application is
responsible for verification of the signature. But I just want to know
that is there any way where I can put my key files to java build pack with
which I am deploying my application on cloud foundry and let my
application use those keys from build pack to verify the signature.
Because I don’t want to add those key files to my app’s war.

Just like we put cacerts in java buildpack which is used by application,
can’t we put the key files in buildpack and use those by our application
to verify signature.

And one more thing that I am using custom java buildpack. (This thing I
forgot to mention earlier.)

Thanks
Ankit


From: Noburou TANIGUCHI [via CF Dev] [mailto:[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2732&amp;i=0&gt;]
Sent: Monday, November 16, 2015 6:09 PM
To: ., Ankit <[hidden
email]&lt;/user/SendEmail.jtp?type=node&amp;node=2732&amp;i=1&gt;>
Subject: Re: regarding using public key to verify client

Hi ankit,

First of all, do you think who is responsible to verify the signature?
Your application? Or (one of) the components of Cloud Foundry? I assume
the former is your answer. I think there is no functionality in Cloud
Foundry to verify client signature.

Then, if you use the Cloud Foundry java-buildpack to deploy your
application, I think there is the only one way to send key files with your
app on deployment. It is to add your key files to your app's war / jar /
zip file, primitively like:

```
jar uvf your-war-jar-zip-file path-to-your-key-files-or-directories
```

But you may add a maven / gradle task to do such a thing.

This is because the Cloud Foundry java-buildpack accepts only one
zip-format file on a deployment.

# Please correct this post if I am wrong. Thank you.

ankit wrote
Suppose my application is deployed on the cloud foundry and my client
sends a POST request that contains some message but that message is
digitally signed by client’s private key. So, I need client’s public
key(digital id of client) to verify my client for inbound calls in the
cloud foundry where application is running. So, can you tell me where can
I put these public keys(digital IDs of clients) in java build pack or any
other place.
Similarly, for outbound calls I want my message to be digitally signed and
for that I need private key to be used. So, where can I put that also?
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion
below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2719.html
To unsubscribe from regarding using public key to verify client, click
here&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=%3e&gt;.
NAML&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml&gt;&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml%3e&gt;

________________________________
The information contained in this message may be confidential and legally
protected under applicable law. The message is intended solely for the
addressee(s). If you are not the intended recipient, you are hereby
notified that any use, forwarding, dissemination, or reproduction of this
message is strictly prohibited and may be unlawful. If you are not the
intended recipient, please contact the sender by return e-mail and destroy
all copies of the original message.
I'm not a ...
noburou taniguchi

________________________________
If you reply to this email, your message will be added to the discussion
below:
http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2732.html
To unsubscribe from regarding using public key to verify client, click
here&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&;node=2711&amp;code=YW5raXQuYW5raXRAcGhpbGlwcy5jb218MjcxMXw3MzcyNjkwNzY=&gt;.
NAML&lt;http://cf-dev.70369.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&;id=instant_html%21nabble%3Aemail.naml&amp;base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&amp;breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml&gt;




-----
I'm not a ...
noburou taniguchi
--
View this message in context: http://cf-dev.70369.x6.nabble.com/regarding-using-public-key-to-verify-client-tp2711p2741.html
Sent from the CF Dev mailing list archive at Nabble.com.


Re: cloud_controller_ng performance degrades slowly over time

Amit Kumar Gupta
 

Heya Matt,

I keep thinking about this puzzle every couple days to see if I can get any
more insight. It would be great to understand the root cause; in the mean
time we are going to try and setup Consul to recurse to other nameservers
(instead of just letting the system fall to the next namserver in
/etc/resolv.conf), and hopefully that'll make some of the weird Consul
behaviour we've seen in different environments better.

But back to this one, what I'm seeing is that there's a bunch of fast
iterations, then one weird one where it takes a few seconds, and then all
subsequent iterations are slow and hit the 5s timeout. In the fast
iteration, we see a sendmmsg sending multiple (2) messages, followed by 2
poll/ioctl/recvfroms. Each fast iteration does this first for 127.0.0.1,
then 8.8.8.8.

The first bad iteration, that takes about 3s, does the sendmmsg, and the 2
poll/ioctl/recvfroms for 127.0.0.1 just fine. Then after doing the
sendmmsg to 8.8.8.8, it gets a recv's a response for one of the messages,
but *times out (3s) polling for a response to the other message*. It then
appears to retry talking to 8.8.8.8, this time, and *all subsequent calls,
using sendto instead of sendmmsg* to send one message at a time:
sendto/poll/ioctl/recvfrom, sendto/poll/ioctl/recvfrom.

All subsequent iterations only use sendto. And now it's the first *sendto
to 127.0.0.1 that times out (5s)*. It immediately goes on to 8.8.8.8 after
that first sendto to 127.0.0.1 fails.

Some questions are: (1) why doesn't the response to one of the sendmmsg
messages arrive? (2) why do all system calls switch from sendmmsg to sendto
after that first failure? (3) why do requests to 127.0.0.1 in the form of
separate sendto calls timeout on the first sendto call, whereas when sent
simultaneously as two messages in sendmmsg, it seems to work fine?

I put these findings together and asked on serverfault.com:
http://serverfault.com/questions/736041/dns-lookups-eventually-become-consistently-slow-in-long-lived-ruby-process.
One suggestion I saw, which could shed a bit more light is to *call strace
with the additional -r -T* flags to get relative time between syscalls, and
the time spent within each syscall.

Care to give that a try when you have some time?

Cheers,
Amit

On Wed, Nov 4, 2015 at 1:54 PM, Matt Cholick <cholick(a)gmail.com> wrote:

Gotcha. Yeah, the rescue lets that test run; after 425k lookups, it never
got slow.

Here's a bit of the strace:
https://gist.github.com/cholick/88c756760faca77208f8


On Wed, Nov 4, 2015 at 11:59 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hey Matt,

I wanted to keep using the uaa.SYSTEM_DOMAIN domain, not the internal
domain, for that experiment. I do expect the TCPSocket.open to fail when
talking to 127.0.0.1, what I wanted to know is, in the presence of no other
nameservers, does it eventually start to fail slow again, or does this
behaviour happen only when there are other nameservers. I imagine the
TCPSocket.open is blowing up on the first iteration in the loop and exiting
the script? My bad, can you replace:

TCPSocket.open("--UAA-DOMAIN--", 80).close

with

TCPSocket.open("--UAA-DOMAIN--", 80).close rescue nil

for the experiment with only 127.0.0.1 listed amongst the nameservers?

Yes, something about the move from the first to second nameserver seems
weird. I have seen strace of one case where it times out polling the FD of
the socket it opened to talk to 127.0.0.1, but in one of your straces it
looked like the poll timeout was on polling the FD for the socket for
8.8.8.8. The fact that the problem persists is interesting too, it seems
like it's not just a one-off race condition where someone messed up with FD
it was supposed to be polling.

Thanks,
Amit

On Wed, Nov 4, 2015 at 11:41 AM, Matt Cholick <cholick(a)gmail.com> wrote:

Ah, I misunderstood.

Consul isn't configured as a recursive resolver, so for a test with only
the 127.0.0.1 in resolve.conf I changed the url in the ruby loop to
"uaa.service.cf.internal", which is what uaa is registering for in consul.

I ran through 225k lookups and it never got slow. Here's a bit of the
strace:
https://gist.github.com/cholick/38e02ce3f351847d5fa3

Bother versions of that test definitely pointing to the move from the
first to the second nameserver in ruby, when the first nameserver doesn't
know the address.


On Tue, Nov 3, 2015 at 11:43 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

I looked at the strace, I see you did indeed mean "loop without
resolver on localhost". If you try it with *only* a resolver on localhost,
do you get the eventually consistent DNS slowdown?

On Tue, Nov 3, 2015 at 8:33 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Thanks Matt!

When you say "the loop without the resolver on local host" did you
mean "the loop with only a resolver on local host"? Sorry if my setup
wasn't clear, but my intention was to only have 127.0.0.1 in
etc/resolv.conf.


On Tuesday, November 3, 2015, Matt Cholick <cholick(a)gmail.com> wrote:

Here are the results of the ruby loop with strace:
https://gist.github.com/cholick/e7e122e34b524cae5fa1

As expected, things eventually get slow. The bash version of the loop
with a new vm each time didn't get slow.

For the loop without a resolver on localhost, it never did get slow.
Though it's hard to prove with something so inconsistent, it hadn't
happened after 100k requests. Here's some of the strace:
https://gist.github.com/cholick/81e58f58e82bfe0a1489

On the final loop, with the SERVFAIL resolver, the issue did
manifest. Here's the trace of that run:
https://gist.github.com/cholick/bd2af46795911cb9f63c

Thanks for digging in on this.


On Mon, Nov 2, 2015 at 6:53 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Okay, interesting, hopefully we're narrowing in on something.
There's a couple variables I'd like to eliminate, so I wonder if you could
try the following. Also, feel free at any point to let me know if you are
not interesting in digging further.

Try all things as sudo, on one of the CCs.

1. It appears that the problem goes away when the CC process is
restarted, so it feels as though there's some sort of resource that the
ruby process is not able to GC, leading to this problem to show up
eventually, and then go away when restarted. I want to confirm this by
trying two different loops, one where the loop is in bash, spinning up a
new ruby process each time, and one where the loop is in ruby.

* bash loop:

while true; do time /var/vcap/packages/ruby-VERSION/bin/ruby
-r'net/protocol' -e 'TCPSocket.open("--UAA-DOMAIN--", 80).close'; done

* ruby loop

/var/vcap/packages/ruby-VERSION/bin/ruby -r'net/protocol' -e '1.step
do |i|; t = Time.now; TCPSocket.open("--UAA-DOMAIN--", 80).close; puts
"#{i}: #{(1000*(Time.now - t)).round}ms"; end'

For each loop, it might also be useful to run `strace -f -p PID >
SOME_FILE` to see what system calls are going on before and after.

2. Another variable is the interaction with the other nameservers.
For this experiment, I would do `monit stop all` to take one of
your CC's out of commission, so that the router doesn't load balance to it,
because it will likely fail requests given the following changes:

* monit stop all && watch monit summary # wait for all the
processes to be stopped, then ctrl+c to stop the watch
* monit start consul_agent && watch monit summary # wait for
consul_agent to be running, then ctrl+c to stop the watch
* Remove nameservers other than 127.0.0.1 from /etc/resolv.conf
* Run the "ruby loop", and see if it still eventually gets slow
* When it's all done, put the original nameservers back in
/etc/resolv.conf, and `monit restart all`

Again, strace-ing the ruby loop would be interesting here.

3. Finally, consul itself. Dmitriy (BOSH PM) has a little DNS
resolver that can be run instead of consul, that will always SERVFAIL (same
as what you see from consul when you nslookup something), so we can try
that:

* Modify `/var/vcap/bosh/etc/gemrc` to remove the `--local` flag
* Run `gem install rubydns`
* Dump the following into a file, say `/var/vcap/data/tmp/dns.rb`:

#!/usr/bin/env ruby

require "rubydns"

RubyDNS.run_server(listen: [[:udp, "0.0.0.0", 53], [:tcp, "0.0.0.0",
53]]) do
otherwise do |transaction|
transaction.fail!(:ServFail)
end
end

* monit stop all && watch monit summary # and again, wait for
everything to be stopped
* Run it with `ruby /var/vcap/data/tmp/dns.rb`. Note that this
command, and the previous `gem install`, use the system gem/ruby,
not the ruby package used by CC, so it maintains some separation. When
running this, it will spit out logs to the terminal, so one can keep an eye
on what it's doing, make sure it all looks reasonable
* Make sure the original nameservers are back in the
`/etc/resolv.conf` (i.e. ensure this experiment is independent of the
previous experiment).
* Run the "ruby loop" (in a separate shell session on the CC)
* After it's all done, add back `--local` to `
/var/vcap/bosh/etc/gemrc`, and `monit restart all`

Again, run strace on the ruby process.

What I hope we find out is that (1) only the ruby loop is affected,
so it has something to do with long running ruby processes, (2) the problem
is independent of the other nameservers listed in /etc/resolv.conf,
and (3) the problem remains when running Dmitriy's DNS-FAILSERVer instead
of consul on 127.0.0.1:53, to determine that the problem is not
specific to consul.

On Sun, Nov 1, 2015 at 5:18 PM, Matt Cholick <cholick(a)gmail.com>
wrote:

Amit,
It looks like consul isn't configured as a recursive resolver. When
running the above code, resolving fails on the first nameserver and the
script fails. resolv-replace's TCPSocket.open is different from the code
http.rb (and thus api) is using. http.rb is pulling in 'net/protocol'. I
changed the script, replacing the require for 'resolv-replace' to
'net/protocol' to match the cloud controller.

Results:

3286 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4 ms |
dns_close: 0 ms
3287 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3288 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 6 ms |
dns_close: 0 ms
3289 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3290 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3291 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3292 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3293 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 5 ms |
dns_close: 0 ms
3294 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 2008 ms |
dns_close: 0 ms
3295 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4010 ms |
dns_close: 0 ms
3296 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4010 ms |
dns_close: 0 ms
3297 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4006 ms |
dns_close: 0 ms
3298 -- ip_open: 2 ms | ip_close: 0 ms | dns_open: 4010 ms |
dns_close: 0 ms
3299 -- ip_open: 3 ms | ip_close: 0 ms | dns_open: 4011 ms |
dns_close: 0 ms
3300 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4010 ms |
dns_close: 0 ms
3301 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4011 ms |
dns_close: 0 ms
3302 -- ip_open: 1 ms | ip_close: 0 ms | dns_open: 4010 ms |
dns_close: 0 ms

And the consul logs, though there's nothing interesting there:
https://gist.github.com/cholick/03d74f7f012e54c50b56


On Fri, Oct 30, 2015 at 5:51 PM, Amit Gupta <agupta(a)pivotal.io>
wrote:

Yup, that's what I was suspecting. Can you try the following now:

1. Add something like the following to your cf manifest:

...
jobs:
...
- name: cloud_controller_z1
...
properties:
consul:
agent:
...
log_level: debug
...

This will set the debug level for the consul agents on your CC job
to debug, so we might be able to see more for its logs. It only sets it on
the job that matters, so when you redeploy, it won't have to roll the whole
deployment. It's okay if you can't/don't want to do this, I'm not sure how
much you want to play around with your environment, but it could be helpful.

2. Add the following line to the bottom of your /etc/resolv.conf

options timeout:4

Let's see if the slow DNS is on the order of 4000ms now, to pin
down where the 5s is exactly coming from.

3. Run the following script on your CC box:

require 'resolv-replace'

UAA_DOMAIN = '--CHANGE-ME--' # e.g. 'uaa.run.pivotal.io'
UAA_IP = '--CHANGE-ME-TOO--' # e.g. '52.21.135.158'

def dur(start_time, end_time)
"#{(1000*(end_time-start_time)).round} ms"
end

1.step do |i|
ip_start = Time.now
s = TCPSocket.open(UAA_IP, 80)
ip_open = Time.now
s.close
ip_close = Time.now

dns_start = Time.now
s = TCPSocket.open(UAA_DOMAIN, 80)
dns_open = Time.now
s.close
dns_close = Time.now

ip_open_dur = dur(ip_start, ip_open)
ip_close_dur = dur(ip_open, ip_close)
dns_open_dur = dur(dns_start, dns_open)
dns_close_dur = dur(dns_open, dns_close)

puts "#{"%04d" % i} -- ip_open: #{ip_open_dur} | ip_close:
#{ip_close_dur} | dns_open: #{dns_open_dur} | dns_close: #{dns_close_dur}"
end

You will need to first nslookup (or otherwise determine) the IP
that the UAA_DOMAIN resolves to (it will be some load balancer, possibly
the gorouter, ha_proxy, or your own upstream LB)

4. Grab the files in /var/vcap/sys/log/consul_agent/

Cheers,
Amit

On Fri, Oct 30, 2015 at 4:29 PM, Matt Cholick <cholick(a)gmail.com>
wrote:

Here's the results:

https://gist.github.com/cholick/1325fe0f592b1805eba5

The time all between opening connection and opened, with the
corresponding ruby source in http.rb's connect method:

D "opening connection to #{conn_address}:#{conn_port}..."

s = Timeout.timeout(@open_timeout, Net::OpenTimeout) {
TCPSocket.open(conn_address, conn_port, @local_host, @local_port)
}
s.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1)
D "opened"

I don't know much ruby, so that's as far I drilled down.

-Matt


Re: UAA Admin interface to register user/client?

Sree Tummidi
 

UAA Provides an Admin Tool UAAC (written in ruby) to manage Users, Clients
and Groups.
You can find more information here: https://github.com/cloudfoundry/cf-uaac


Thanks,
Sree Tummidi
Sr. Product Manager
Identity - Pivotal Cloud Foundry


On Tue, Nov 17, 2015 at 12:32 AM, Juan Antonio Breña Moral <
bren(a)juanantonio.info> wrote:

Good morning,

In my case, I have some Scripts for UAA here:

https://github.com/prosociallearnEU/cf-nodejs-client/blob/master/test/lib/model/uaa/UserUAATests.js

https://github.com/prosociallearnEU/cf-nodejs-client/blob/master/test/lib/model/cloudcontroller/OrganizationsTests.js

Cheers


Re: Documentation on creating and deploying windows applications diego-windows-release

James Bayer
 

this blog [1] shows an example windows app push:
cf push APPNAME -s windows2012R2 -b binary_buildpack

in this example, you'd want to take an app like this example [2] that has
been "published" from visual studio and push from that publish directory
(or specify it with -p).

web socket apps should work well provided your load balancer configuration
supports web socket.

[1]
https://blog.pivotal.io/pivotal-cloud-foundry/products/pivotal-cloud-foundry-1-6-technical-blog-new-runtime-services-net-more
[2] https://github.com/jbayer/windows-cf-demo

On Tue, Nov 17, 2015 at 6:44 AM, Vinay Vaidya <vvaidya(a)rediffmail.com>
wrote:

Deployed Diego components for Cloud Foundry per
https://github.com/cloudfoundry-incubator/diego-windows-release

How do I create and deploy windows apps on Diego? Specifically, is it
possible to deploy a native windows application that has multiple fiies and
directories besides the main executable? What about a .NET application that
is a client using a websocket connection to the server? Thanks.


--
Thank you,

James Bayer


Documentation on creating and deploying windows applications diego-windows-release

Vinay Vaidya
 

Deployed Diego components for Cloud Foundry per https://github.com/cloudfoundry-incubator/diego-windows-release

How do I create and deploy windows apps on Diego? Specifically, is it possible to deploy a native windows application that has multiple fiies and directories besides the main executable? What about a .NET application that is a client using a websocket connection to the server? Thanks.


Re: Deploying a shell script driven java application to cf

Ben Hale <bhale@...>
 

Dammina,

Unfortunately, Daniel’s suggestion won’t work. The buildpack expects the filesystem to look like an exploded JAR/WAR and will refuse to stage if it does not. Instead of trying to get your shell script to run inside of the container, you should actually be trying to run your application without a shell script. What exactly does your shell script do? I think you’ll find that the buildpack already does the same things removing the need for it.


-Ben Hale
Cloud Foundry Java Experience, Lead

On Nov 16, 2015, at 04:39, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

I haven't tried this, but I think it should work.

1.) Make a directory. In that directory put your JAR file, your start script and anything else the app needs to run.
2.) From that directory, run `cf push <app-name> -b java_buildpack -c '$PWD/start-script.sh'`. This will upload your script, the JAR file and everything else in the current directory. It will also tell CF that you specifically want to use the Java build pack (which will install Java) and that you want to use your script to start your app.

What could be tricky about this is your start script. It's going to need to reference JAVA_HOME as `/home/vcap/app/.java-buildpack/open_jdk_jre`, and `java` as `$JAVA_HOME/bin/java` since `java` is not going to be on the $PATH.

You're also going to need to handle some of the things that the JBP would normally do like set -Xmx and other JVM memory settings to keep the JVM from exceeding the containers MEMORY_LIMIT. Note, *all* memory needs to fit under the limit, not just the JVM's heap. In other words, setting -Xmx == MEMORY_LIMIT is 100% wrong.

Beyond that, you'd need to make sure the app is listening on $PORT or if it's not taking web requests, disable that health check (`cf push --no-route` & `cf set-health-check none`).

Dan

On Fri, Nov 13, 2015 at 12:47 AM, dammina sahabandu <dammina(a)adroitlogic.com> wrote:
Hi All,
I have a java application which is a composition of several jars in the running environment. The main jar is invoked by a shell script. Is it possible to push such application into cloud foundry? And if it is possible can you please help me on how to achieve that. A simple guide will be really helpful.

Thank you in advance,
Dammina