Date   

Script hangs when updating the cf-release

Qing Gong
 

When I ran the following command, I always got stuck when updating the etcd-incubator. Any idea? I tried on multiple machines and they all hang at the same place. Is it possible that this is a problem in github?

The console info is:
prompt> ./update

===> Uncommitted submodules changes will be clobbered <===


===> Unversioned changes will be clobbered <===

+ has_upstream
+ git rev-parse '@{u}'
+ git pull
Already up-to-date.
+ git submodule sync
+ git submodule foreach --recursive 'git submodule sync; git clean -d --force --force'
+ git submodule update --init --recursive
Submodule 'shared' (https://github.com/cloudfoundry/shared-release-packages.git) registered for path 'shared'
Submodule 'src/cloud_controller_ng' (https://github.com/cloudfoundry/cloud_controller_ng.git) registered for path 'src/cloud_controller_ng'
Submodule 'src/collector' (https://github.com/cloudfoundry/collector.git) registered for path 'src/collector'
Submodule 'src/dea_next' (https://github.com/cloudfoundry/dea_ng.git) registered for path 'src/dea_next'
Submodule 'src/etcd-metrics-server' (https://github.com/cloudfoundry-incubator/etcd-metrics-server.git) registered for path 'src/etcd-metrics-server'
Submodule 'src/etcd-release' (https://github.com/cloudfoundry-incubator/etcd-release.git) registered for path 'src/etcd-release'
Submodule 'src/github.com/cloudfoundry-incubator/routing-api' (https://github.com/cloudfoundry-incubator/routing-api.git) registered for path 'src/github.com/cloudfoundry-incubator/routing-api'
Submodule 'src/github.com/cloudfoundry/cf-acceptance-tests' (https://github.com/cloudfoundry/cf-acceptance-tests) registered for path 'src/github.com/cloudfoundry/cf-acceptance-tests'
Submodule 'src/github.com/cloudfoundry/gorouter' (https://github.com/cloudfoundry/gorouter) registered for path 'src/github.com/cloudfoundry/gorouter'
Submodule 'src/gnatsd' (https://github.com/apcera/gnatsd.git) registered for path 'src/gnatsd'
Submodule 'src/hm9000' (https://github.com/cloudfoundry/hm-workspace.git) registered for path 'src/hm9000'
Submodule 'src/loggregator' (https://github.com/cloudfoundry/loggregator) registered for path 'src/loggregator'
Submodule 'src/smoke-tests' (https://github.com/cloudfoundry/cf-smoke-tests) registered for path 'src/smoke-tests'
Submodule 'src/statsd-injector' (https://github.com/cloudfoundry/statsd-injector.git) registered for path 'src/statsd-injector'
Submodule 'src/uaa' (https://github.com/cloudfoundry/uaa.git) registered for path 'src/uaa'
Submodule 'src/warden' (https://github.com/cloudfoundry/warden.git) registered for path 'src/warden'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/shared/.git/
remote: Counting objects: 1560, done.
remote: Total 1560 (delta 0), reused 0 (delta 0), pack-reused 1560
Receiving objects: 100% (1560/1560), 380.81 KiB, done.
Resolving deltas: 100% (589/589), done.
Submodule path 'shared': checked out '87112bae91127792ffbed7fc6c76ac7088708ace'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/cloud_controller_ng/.git/
remote: Counting objects: 66064, done.
remote: Compressing objects: 100% (107/107), done.
remote: Total 66064 (delta 26), reused 0 (delta 0), pack-reused 65955
Receiving objects: 100% (66064/66064), 17.75 MiB | 4.32 MiB/s, done.
Resolving deltas: 100% (45227/45227), done.
Submodule path 'src/cloud_controller_ng': checked out 'ff442527f0bee06cb6ea6baa22175905a92fc718'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/collector/.git/
remote: Counting objects: 2488, done.
remote: Total 2488 (delta 0), reused 0 (delta 0), pack-reused 2488
Receiving objects: 100% (2488/2488), 9.19 MiB | 5.23 MiB/s, done.
Resolving deltas: 100% (1411/1411), done.
Submodule path 'src/collector': checked out 'c38732735b36ed3ee9e44ee9bf91dce07bedd63b'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/dea_next/.git/
remote: Counting objects: 12737, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 12737 (delta 0), reused 0 (delta 0), pack-reused 12734
Receiving objects: 100% (12737/12737), 15.25 MiB | 4.26 MiB/s, done.
Resolving deltas: 100% (7014/7014), done.
Submodule path 'src/dea_next': checked out 'b74390b2472a6a929807040f4439a30ecb46e699'
Submodule 'go/src/github.com/cloudfoundry/gosteno' (https://github.com/cloudfoundry/gosteno.git) registered for path 'go/src/github.com/cloudfoundry/gosteno'
Submodule 'go/src/github.com/howeyc/fsnotify' (https://github.com/howeyc/fsnotify.git) registered for path 'go/src/github.com/howeyc/fsnotify'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/dea_next/go/src/github.com/cloudfoundry/gosteno/.git/
remote: Counting objects: 436, done.
remote: Total 436 (delta 0), reused 0 (delta 0), pack-reused 436
Receiving objects: 100% (436/436), 105.20 KiB, done.
Resolving deltas: 100% (222/222), done.
Submodule path 'go/src/github.com/cloudfoundry/gosteno': checked out 'c6379cb7ef097850eec4dc61e1730bbb99a2a2a8'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/dea_next/go/src/github.com/howeyc/fsnotify/.git/
remote: Counting objects: 660, done.
remote: Total 660 (delta 0), reused 0 (delta 0), pack-reused 660
Receiving objects: 100% (660/660), 168.97 KiB, done.
Resolving deltas: 100% (382/382), done.
Submodule path 'go/src/github.com/howeyc/fsnotify': checked out '08040c5a90632bd721465eb8ad74a8e61bd7bf95'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/etcd-metrics-server/.git/
remote: Counting objects: 1658, done.
remote: Total 1658 (delta 0), reused 0 (delta 0), pack-reused 1658
Receiving objects: 100% (1658/1658), 591.04 KiB, done.
Resolving deltas: 100% (757/757), done.
Submodule path 'src/etcd-metrics-server': checked out '90c444c7f93cacb998e45c46f1e06ecf4c8eb9c4'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/etcd-release/.git/
remote: Counting objects: 1546, done.
remote: Total 1546 (delta 0), reused 0 (delta 0), pack-reused 1546
Receiving objects: 100% (1546/1546), 1.50 MiB | 2.22 MiB/s, done.
Resolving deltas: 100% (396/396), done.
Submodule path 'src/etcd-release': checked out '6722dd34e92e760c6e34134224cd3323215c7817'
Submodule 'src/etcd' (https://github.com/coreos/etcd.git) registered for path 'src/etcd'
Submodule 'src/github.com/cloudfoundry-incubator/cf-test-helpers' (http://github.com/cloudfoundry-incubator/cf-test-helpers.git) registered for path 'src/github.com/cloudfoundry-incubator/cf-test-helpers'
Submodule 'src/github.com/coreos/go-etcd' (http://github.com/coreos/go-etcd.git) registered for path 'src/github.com/coreos/go-etcd'
Submodule 'src/github.com/nu7hatch/gouuid' (http://github.com/nu7hatch/gouuid.git) registered for path 'src/github.com/nu7hatch/gouuid'
Submodule 'src/github.com/onsi/ginkgo' (http://github.com/onsi/ginkgo.git) registered for path 'src/github.com/onsi/ginkgo'
Submodule 'src/github.com/onsi/gomega' (http://github.com/onsi/gomega.git) registered for path 'src/github.com/onsi/gomega'
Submodule 'src/github.com/ugorji/go' (http://github.com/ugorji/go.git) registered for path 'src/github.com/ugorji/go'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/etcd-release/src/etcd/.git/
remote: Counting objects: 29815, done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 29815 (delta 6), reused 0 (delta 0), pack-reused 29783
Receiving objects: 100% (29815/29815), 17.07 MiB | 6.59 MiB/s, done.
Resolving deltas: 100% (18339/18339), done.
Submodule path 'src/etcd': checked out '6335fdc595ff03d27007db04e5b545189b9647c6'
Initialized empty Git repository in /local/install/users/cfg/workspace/cf-release/src/etcd-release/src/github.com/cloudfoundry-incubator/cf-test-helpers/.git/
(hang!)


Re: cannot access director, trying 4 more times...

Qing Gong
 

The problem was that I ran the "vagrant up" outside the bosh-lite directory so the virtual box was not bosh.

Thanks all!


Recommended way/place to configure uaa for CF runtime

Tom Sherrod <tom.sherrod@...>
 

I've deployed cf-release 214.
I now wish to configure uaa to use ldap(AD) for authentication.
I find documentation on how uaa works and the uaa.yml.
How does the configuring/updating of uaa.yml work in the cf-release deploy work flow?
I deployed with remote/download instead of create and upload. Edit directly on the server?
Is editing uaa.yml then create release and upload the recommended way?

Any additional pointers appreciated.

Regarding uaa functionality...any additional information on ldap groups and org permissions. https://groups.google.com/a/cloudfoundry.org/forum/#!topic/vcap-dev/X1OLss4zumQ

Thanks,
Tom


CF integration with logger and monitoring tools

Swatz bosh
 

I would like to know in CF, how to integrate with 3rd party app logging and monitoring like Graphite, Nagios, etc, what is the recommended approach. I found few articles where its mentioned firehose is the better option. Whereas I remember having collector (stats_z1/z2) job pointing to such monitoring server works well. So what steps I need to follow if I have to integrate such application monitoring tool using firehose? Do I need to write Nozzles for my monitoring tool like CloudCredo did for Graphite? https://github.com/CloudCredo/graphite-nozzle http://www.cloudcredo.com/how-to-integrate-graphite-with-cloud-foundry/
So if I have to integrate with Nagios, Wily, Splunk etc I would need nozzle for all of them? If not what changes I need to do in my configuration and in buildpack(not sure)?

I also found NOAA client https://github.com/cloudfoundry/noaa which I think consumes all logs from doppler and show them on console? How this Noaa client is using firehose, is not very clear in the document?


CF integration with logger and monitoring tools

Swatz bosh
 

Hi,

I would like to know in CF, how to integrate with 3rd party app logging and monitoring like Graphite, Nagios, etc, what is the recommended approach. I found few articles where its mentioned firehose is the better option. Whereas I remember having collector (stats_z1/z2) job pointing to such monitoring server works well. So what steps I need to follow if I have to integrate such application monitoring tool using firehose? Do I need to write Nozzles for my monitoring tool like CloudCredo did for Graphite? https://github.com/CloudCredo/graphite-nozzle http://www.cloudcredo.com/how-to-integrate-graphite-with-cloud-foundry/
So if I have to integrate with Nagios, Wily, Splunk etc I would need nozzle for all of them? If not what changes I need to do in my configuration and in buildpack(not sure)?

I also found NOAA client https://github.com/cloudfoundry/noaa which I think consumes all logs from doppler and show them on console? How this Noaa client is using firehose, is not very clear in the document?


Re: Missing routing logs from "cf logs app-name"

Simon Johansson <simon@...>
 

The http start/stop stuff is usually due to requests that take longer than
a minute (the starts get purged). I don't know if that makes sense for
these requests.
Ah, that is interesting, thanks!

Re my initial problem, turned out to be a scaling issue. Most of the CPU on the gorouters were consumed by metron agent, so adding more CPU and scaling out the routers fixed our problem.


Re: Missing routing logs from "cf logs app-name"

Matthew Sykes <matthew.sykes@...>
 

Simon, what level of cf-release are you using? We've seen some extremely
high log drop rates (~40% during log spurts) to log sinks with v209 and
v210. May or may not be related since we were focused on application logs...

In some cases it was doppler dropping the messages because the truncating
buffer was overrun, in some cases it was a bug where sinks were not picked
up by doppler because it's poller ran out of sockets due to a connection
leak, and in others still, we just never saw the message get to one of the
dopplers.

The http start/stop stuff is usually due to requests that take longer than
a minute (the starts get purged). I don't know if that makes sense for
these requests.

On Wed, Aug 12, 2015 at 11:19 AM, Simon Johansson <simon(a)simonjohansson.com>
wrote:

Gwenn, our buffer is set to 500(default is 100).


On Wed, Aug 12, 2015 at 3:24 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Simon not sure is related but don,t if you don't consume enough quickly
the message, the buffer will be exhausted.

Did you try to increase the buffer size ?


On Wed, Aug 12, 2015 at 5:47 AM, Simon Johansson <
simon(a)simonjohansson.com> wrote:

Howdie! Not really a dev related question, but since there is no cf-user
list I'm trying my luck here. :)

Im looking into an issue where we don't get enough RTR logs from cf logs
/ firehose-to-syslog
I'm using cf-env[1] as the test app, and Im hitting
app.domain.com/kehehehe.
For each request the app itself logs the the path requested and the
gorouters logs the same information but a bit more verbose.

$ cf logs cf-env-test
...
2015-08-11T11:43:41.24+0200 [RTR/0] OUT cf-env-test.domain.com -
[11/08/2015:09:43:41 +0000] "GET /kehehehe HTTP/1.1" 404 0 18 "-"
"curl/7.43.0" 10.230.15.4:59728 x_forwarded_for:"10.230.15.4"
vcap_request_id:4a186d9e-046a-47bc-74d0-f0c3a8cb1257
response_time:0.008503156 app_id:7f1944a2-2197-43b5-9334-0be7c8b9b40e
2015-08-11T11:43:41.31+0200 [App/0] ERR 10.230.15.8 - -
[11/Aug/2015 09:43:41] "GET /kehehehe HTTP/1.1" 404 18 0.0006

I shoot off 1000 requests and would expect to see 1000 of each log type.
But only the App logs are correct.

$ grep "App/" /tmp/cf-logs | wc -l # /tmp/cf-logs comes from cf logs
cf-env-test > /tmp/cf-logs
1000
$ grep "RTR/" /tmp/cf-logs | wc -l
145

Looking at the gorouters
nsaadmin(a)a99a5339-4308-43db-b3a4-59442169861d:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
484
nsaadmin(a)1d8ef2c2-2ec1-417d-8882-0234de251c60:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
516
So the gorouter process definitely logs the right amount of requests to
disk.

In metron_agent.stdout.log on the gorouters there are a lot of lines like

{"timestamp":1439286941.958369255,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:6001753075995010215
high:3404432699330643838
1}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}
{"timestamp":1439286941.959420443,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:8668459744461365080
high:13197025281073682796
2}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}
{"timestamp":1439286941.964927912,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:13205430422726438703
high:7208042014820494920
1}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}

Has anyone seen this before?

[1] https://github.com/cloudfoundry-community/cf-env

--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: Billing & Metering of app usage with Abacus

CF Runtime
 

You might get some answers by querying the events api.
http://apidocs.cloudfoundry.org/214/events/list_all_events.html

You should be able to query it where the actee equals the guid of the app.

Joseph
OSS Release Integration Team

On Wed, Aug 12, 2015 at 5:26 PM, Piotr Przybylski <piotrp(a)us.ibm.com> wrote:

Joseph,
thank you, for the timestamps I just pointed it out as something that
implementation must deal with, quite easily as the order is correct.

For the duplicate events, I cannot say what causes them. It used to be
simple to reproduce by (almost) simultaneously sending 'cf stop <app>' from
two terminal windows. This does not 'work' any more (CF 210), however I see
logged duplicate STARTED and STOPPED events - not too frequent, but still
there. Somewhat surprisingly, for the two I looked at, the time difference
is quite large - 2 minutes for STARTED and 15 minutes for STOPPED. Is there
a way to determine how that happened without access to the application ?

Piotr



[image: Inactive hide details for CF Runtime ---08/12/2015 03:45:55
PM---Piotr, The timestamps not being correct is a known limitation]CF
Runtime ---08/12/2015 03:45:55 PM---Piotr, The timestamps not being correct
is a known limitation of how events are



From:


CF Runtime <cfruntime(a)gmail.com>

To:


"Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>

Date:


08/12/2015 03:45 PM

Subject:


[cf-dev] Re: Re: Billing & Metering of app usage with Abacus
------------------------------



Piotr,

The timestamps not being correct is a known limitation of how events are
being generated, but as you said, order should be guaranteed (and
timestamps should hopefully be close).

Duplicate events are something I'm not aware of though. In theory only one
API instance should be able to get a database lock on an app, and should
not release it until it is done updating and has recorded the app usage
event. Do you have any details on what caused duplicate events?

Joseph
OSS Release Integration Team

On Wed, Aug 12, 2015 at 8:07 AM, Piotr Przybylski <*piotrp(a)us.ibm.com*
<piotrp(a)us.ibm.com>> wrote:

Hi, I am also looking at runtime submissions for Abacus, worked on it
for Bluemix, couple of points for discussion.

In addition to usage events (start followed by stop), the scaling and
duplicate events need to be handled. The former is a START followed by
START with memory or instance count changed, the latter can be STOP
followed by STOP.
We also encountered a situation where the ordering is correct - START
followed by STOP but the timestamp for the START is later than STOP.

For the running applications - your points #2 and #3 are a working
though it may generate fair amount of traffic, depending on frequency and
number of running applications. Eventually we may want to look at
alternatives, for example enhance metering to allow for time based
submissions. Instead of continually submitting usage, submit the state of
application - (STARTED, memory, instances), the metering could then
calculate usage for that application based on the time passed until
application is stopped.

I think handling some of above scenarios requires persistence, even if
to log CF events that were used for submission (or skipped). You may want
to persist state to recovery from application failure or restart, as well
as keep track of running/active applications.

Thanks,

Piotr


Hristo Iliev <*hsiliev(a)gmail.com* <hsiliev(a)gmail.com>> wrote on
08/07/2015 08:07:53 AM:

> We want to integrate the metrics provided by Cloud Foundry with
Abacus.

> We plan to create a billing/metering integration layer that:
> fetches the app usage events from CC
> inserts runtime usage data to Abacus
> The events should be processed to build usage data based on them.
> This is much like the idea outlined in Dr.Nic's blog. The
> integration layer can do this periodically.

> AFAIK Abacus provides usage reports for the current month only and
> not for arbitrary period of time. This implies some restrictions to
> what we can report when the application is:
> 1. started and stopped several times in the month
> 2. started in the current month but not stopped at all
> 3. started in a previous month and not stopped
> The first problem can be solved by iterating through the events and
> finding the respective start and end timestamps that have to be
> reported to Abacus.



> The second issue might be solved by reporting small amounts of
> usage, stating from the last start event and continuing to report on
> every poll of the integration layer. For example we can report
several usages:
> start: 1438945112; end: 1438946000 (current time for the billing
integration)
> start: 1438946000 (previous reporting cycle); end: 1438947000
> The third issue might be solved by finding the last start event and
> reporting in the same manner as with #2.

> Reporting usage in small steps might require persistence so we can
> store the end time of the previous reporting. We might use in-memory
> cache and reach to Abacus as primary storage. If Abacus can
> accumulate usage reporting we can even get rid of the persistence
and cache.


> Is such integration in the scope of Abacus project?
>
> Regards,
> Hristo Iliev



Re: Questions about service broker

王小锋 <zzuwxf at gmail.com...>
 

Thanks Daniel

That's what I need.

2015-08-12 20:22 GMT+08:00 Daniel Jones <daniel.jones(a)engineerbetter.com>:

Hi,

You might like to check out the forthcoming asynchronous additions to the
Service Broker API:
https://docs.cloudfoundry.org/services/asynchronous-operations.html

It sounds like exactly what you're after.

Alternatively you could pre-provision a load of databases, and have your
broker maintain a pool of available ones. When each instance is destroyed,
you then clean it and return it to the pool only when it is back in a state
ready for someone else to use.

On Wed, Aug 12, 2015 at 1:14 PM, 王小锋 <zzuwxf(a)gmail.com> wrote:

Hi, there

I am trying to develop a database service broker in CF, but have a
question regarding to provisioning rest api, if it takes long time for the
database to be ready, it will fail in binding process if the database is
not ready. Is there a method to get the database service instance's status
before binding to an app? Or is there any suggestions?

thanks


--
Regards,

Daniel Jones
EngineerBetter.com


Re: cannot access director, trying 4 more times...

Gwenn Etourneau
 

The add route script add the 10.X route the 192.X is added by
Vagrant/VirtualBox so that's mean you can reach the director without the
add-route script.

On Wed, Aug 12, 2015 at 8:00 PM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

Resending due to mailman issues:

Looks like you may not be doing what you think you're doing.

$ vagrant init hashicorp/precise64 (I tried both 32 and 64)
$ vagrant up --provider=virtualbox
I'm not sure where you got those commands from but they are unrelated to
bosh-lite. You're basically bringing up an Ubuntu 12.04 base box as
created by Hashicorp. The bosh-lite code isn't part of it; no director
runs in there.

You may have more luck with

$ vagrant init cloudfoundry/bosh-lite
$ vagrant up --provider virtualbox

Then you'll need to add the routes via the add-route script and use the
bosh cli to target the director in that box.

On Fri, Aug 7, 2015 at 8:21 PM, Qing Gong <qinggong(a)gmail.com> wrote:

I am very new to CF. Trying to set up one using the directions here:
https://github.com/cloudfoundry/bosh-lite

After running the following commands:

$ vagrant init hashicorp/precise64 (I tried both 32 and 64)
$ vagrant up --provider=virtualbox

The console shows:
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'hashicorp/precise64' is up to date...
==> default: Setting the name of the VM:
xxxxxx_default_1438889867528_29074
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
default:
default: Vagrant insecure key detected. Vagrant will automatically
replace
default: this with a newly generated keypair for better security.
default:
default: Inserting generated public key within guest...
default: Removing insecure key from the guest if it's present...
default: Key inserted! Disconnecting and reconnecting using new SSH
key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
default: The guest additions on this VM do not match the installed
version of
default: VirtualBox! In most cases this is fine, but in rare cases it
can
default: prevent things such as shared folders from working properly.
If you see
default: shared folder errors, please make sure the guest additions
within the
default: virtual machine match the version of VirtualBox you have
installed on
default: your host and reload your VM.
default:
default: Guest Additions Version: 4.2.0
default: VirtualBox Version: 5.0
==> default: Mounting shared folders...
default: /vagrant => /local/install/users/xxxxxx

Then, I tried to set the target for the bosh director but it was
unsuccessful. I am not sure what's wrong. Searched around online and still
could not figure out why.

I followed what's said below:
Target the BOSH Director and login with admin/admin.
# if behind a proxy, exclude both the VM's private IP and xip.io by
setting no_proxy (xip.io is introduced later)
$ export no_proxy=192.168.50.4,xip.io
(or export no_proxy=192.168.50.4.xip.io)

$ bosh target 192.168.50.4 lite
I always got the message shown in the subject,

cannot access director, trying 4 more times...
cannot access director, trying 3 more times...
cannot access director, trying 2 more times...

I wonder if the IP was not right. I tried 192.168.50.4 and a few other
IP. I also tried to customize the IP in the Vagrantfile file.

How can I debug what's going wrong? Really appreciate any help or
guidance.

TIA!

Qing


--
Matthew Sykes
matthew.sykes(a)gmail.com


Re: Service instance name is not passed from cloud controller to service broker

Mike Youngstrom
 

The service can also be renamed.

Mike

On Aug 12, 2015 5:48 PM, "Dieu Cao" <dcao(a)pivotal.io> wrote:

Hi Dave,

Could you elaborate on your use case for wanting to know the service
instance name? It is only unique to the space that the service instance
was provisioned in.

Dieu
CF CAPI PM

On Wednesday, August 12, 2015, Dave Chen <dave.chen80(a)gmail.com> wrote:

The service instance name is not passed from cloud controller to service
broker when we do "cf cs sample-service free my-service". We need to get
the service instance name "my-service" from the
CreateServiceInstanceRequest for our service broker when performing the
above CLI command. Is there a way to make service instance name is part of
the contract between CC and service brokers?

Thanks,
Dave


Re: Billing & Metering of app usage with Abacus

Piotr Przybylski <piotrp@...>
 

Joseph,
thank you, for the timestamps I just pointed it out as something that
implementation must deal with, quite easily as the order is correct.

For the duplicate events, I cannot say what causes them. It used to be
simple to reproduce by (almost) simultaneously sending 'cf stop <app>' from
two terminal windows. This does not 'work' any more (CF 210), however I see
logged duplicate STARTED and STOPPED events - not too frequent, but still
there. Somewhat surprisingly, for the two I looked at, the time difference
is quite large - 2 minutes for STARTED and 15 minutes for STOPPED. Is there
a way to determine how that happened without access to the application ?

Piotr




|------------>
| From: |
|------------>
>-----------------------------------------------------------------------------------------------------------------------------------------|
|CF Runtime <cfruntime(a)gmail.com> |
>-----------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| To: |
|------------>
>-----------------------------------------------------------------------------------------------------------------------------------------|
|"Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org> |
>-----------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Date: |
|------------>
>-----------------------------------------------------------------------------------------------------------------------------------------|
|08/12/2015 03:45 PM |
>-----------------------------------------------------------------------------------------------------------------------------------------|
|------------>
| Subject: |
|------------>
>-----------------------------------------------------------------------------------------------------------------------------------------|
|[cf-dev] Re: Re: Billing & Metering of app usage with Abacus |
>-----------------------------------------------------------------------------------------------------------------------------------------|





Piotr,

The timestamps not being correct is a known limitation of how events are
being generated, but as you said, order should be guaranteed (and
timestamps should hopefully be close).

Duplicate events are something I'm not aware of though. In theory only one
API instance should be able to get a database lock on an app, and should
not release it until it is done updating and has recorded the app usage
event. Do you have any details on what caused duplicate events?

Joseph
OSS Release Integration Team

On Wed, Aug 12, 2015 at 8:07 AM, Piotr Przybylski <piotrp(a)us.ibm.com>
wrote:
Hi, I am also looking at runtime submissions for Abacus, worked on it for
Bluemix, couple of points for discussion.

In addition to usage events (start followed by stop), the scaling and
duplicate events need to be handled. The former is a START followed by
START with memory or instance count changed, the latter can be STOP
followed by STOP.
We also encountered a situation where the ordering is correct - START
followed by STOP but the timestamp for the START is later than STOP.

For the running applications - your points #2 and #3 are a working though
it may generate fair amount of traffic, depending on frequency and number
of running applications. Eventually we may want to look at alternatives,
for example  enhance metering to allow for time based submissions.
Instead of continually submitting usage, submit the state of application
- (STARTED, memory, instances), the metering could then calculate usage
for that application based on the time passed until application is
stopped.

I think handling some of above scenarios requires persistence, even if to
log CF events that were used for submission (or skipped). You may want to
persist state to recovery from application failure or restart, as well as
keep track of running/active applications.

Thanks,

Piotr


Hristo Iliev <hsiliev(a)gmail.com> wrote on 08/07/2015 08:07:53 AM:

> We want to integrate the metrics provided by Cloud Foundry with Abacus.

> We plan to create a billing/metering integration layer that:
> fetches the app usage events from CC
> inserts runtime usage data to Abacus
> The events should be processed to build usage data based on them.
> This is much like the idea outlined in Dr.Nic's blog. The
> integration layer can do this periodically.

> AFAIK Abacus provides usage reports for the current month only and
> not for arbitrary period of time. This implies some restrictions to
> what we can report when the application is:
> 1. started and stopped several times in the month
> 2. started in the current month but not stopped at all
> 3. started in a previous month and not stopped
> The first problem can be solved by iterating through the events and
> finding the respective start and end timestamps that have to be
> reported to Abacus.



> The second issue might be solved by reporting small amounts of
> usage, stating from the last start event and continuing to report on
> every poll of the integration layer. For example we can report several
usages:
> start: 1438945112; end: 1438946000 (current time for the billing
integration)
> start: 1438946000 (previous reporting cycle); end: 1438947000
> The third issue might be solved by finding the last start event and
> reporting in the same manner as with #2.

> Reporting usage in small steps might require persistence so we can
> store the end time of the previous reporting. We might use in-memory
> cache and reach to Abacus as primary storage. If Abacus can
> accumulate usage reporting we can even get rid of the persistence and
cache.


> Is such integration in the scope of Abacus project?
>
> Regards,
> Hristo Iliev


Re: Service instance name is not passed from cloud controller to service broker

Dieu Cao <dcao@...>
 

Hi Dave,

Could you elaborate on your use case for wanting to know the service
instance name? It is only unique to the space that the service instance
was provisioned in.

Dieu
CF CAPI PM

On Wednesday, August 12, 2015, Dave Chen <dave.chen80(a)gmail.com> wrote:

The service instance name is not passed from cloud controller to service
broker when we do "cf cs sample-service free my-service". We need to get
the service instance name "my-service" from the
CreateServiceInstanceRequest for our service broker when performing the
above CLI command. Is there a way to make service instance name is part of
the contract between CC and service brokers?

Thanks,
Dave


Re: Questions about removal of the heartbeat message type from dropsonde-protocol

Mike Youngstrom
 

That makes sense. So, this change wasn't so much the result of a change in
direction but instead an acknowledgement that use of it wasn't really a
good direction so we might as well remove it.

That is useful info. Thanks for the explantation. Helps me feel more
confident in the approach my group has decided on.

Thanks,
Mike

On Tue, Aug 11, 2015 at 1:14 PM, Erik Jasiak <ejasiak(a)pivotal.io> wrote:

(resend #2)
Hi again Mike,

There were quite a few pros and cons that went into it; the high (low?)
lights from my notes are below. I'll have the rest of the team check in
if they have more info.

1) A ruby version of the dropsonde-protocol would require some amount
of maintaining state done by the consumer, which is more challenging in
ruby.
2) How to shoehorn a heartbeat mechanism into the statsd injector (by
its nature, statsd sends last known value; is a heartbeat binary yes/no, or
milliseconds uptime, and a component is dead when there's no increase?)
3) Whose job is it to maintain heartbeat state to begin with?
Metron's, as the aggregator of dropsonde counters? A Nozzle's?
4) Is the correct model to use heartbeats as the 'source of truth'
about a component being alive, regardless of the data being broadcast, or
does a component / developer prefer the non-statsd-model of wanting metric
updates to serve as a heartbeat? (We've leaned toward the statsd model of
'last update is valid', but then that implies everyone agrees a heartbeat
is really a running uptime counter or similar.)

We didn't have answers to all of these questions; what we did find was
that dropsonde-protocol heartbeats were rarely being used, and largely
being ignored. Because they were also in the way of figuring out a path
forward for things like dropsonde with ruby, we went for their removal
until we had a clearer use case and strategy, or we could handle them in a
cleaner, generally agreed upon way.

Hope that helps,
Erik

On Sat, Aug 8, 2015 at 11:32 AM, Mike Youngstrom <youngm(a)gmail.com> wrote:

I noticed that heartbeat messages are no longer a part of the
dropsonde-protocol.

Can I get a quick summary of the thinking behind this change?

Is there an assumption that we should be using the bosh health manager
and not the firehose for this type of thing?

I'm just like some background and help understanding the LAMB team's
monitoring mindset regarding the removal of this message.

Thanks,
Mike


Re: Billing & Metering of app usage with Abacus

CF Runtime
 

Piotr,

The timestamps not being correct is a known limitation of how events are
being generated, but as you said, order should be guaranteed (and
timestamps should hopefully be close).

Duplicate events are something I'm not aware of though. In theory only one
API instance should be able to get a database lock on an app, and should
not release it until it is done updating and has recorded the app usage
event. Do you have any details on what caused duplicate events?

Joseph
OSS Release Integration Team

On Wed, Aug 12, 2015 at 8:07 AM, Piotr Przybylski <piotrp(a)us.ibm.com> wrote:

Hi, I am also looking at runtime submissions for Abacus, worked on it for
Bluemix, couple of points for discussion.

In addition to usage events (start followed by stop), the scaling and
duplicate events need to be handled. The former is a START followed by
START with memory or instance count changed, the latter can be STOP
followed by STOP.
We also encountered a situation where the ordering is correct - START
followed by STOP but the timestamp for the START is later than STOP.

For the running applications - your points #2 and #3 are a working though
it may generate fair amount of traffic, depending on frequency and number
of running applications. Eventually we may want to look at alternatives,
for example enhance metering to allow for time based submissions. Instead
of continually submitting usage, submit the state of application -
(STARTED, memory, instances), the metering could then calculate usage for
that application based on the time passed until application is stopped.

I think handling some of above scenarios requires persistence, even if to
log CF events that were used for submission (or skipped). You may want to
persist state to recovery from application failure or restart, as well as
keep track of running/active applications.

Thanks,

Piotr


Hristo Iliev <hsiliev(a)gmail.com> wrote on 08/07/2015 08:07:53 AM:

We want to integrate the metrics provided by Cloud Foundry with Abacus.
We plan to create a billing/metering integration layer that:
fetches the app usage events from CC
inserts runtime usage data to Abacus
The events should be processed to build usage data based on them.
This is much like the idea outlined in Dr.Nic's blog. The
integration layer can do this periodically.
AFAIK Abacus provides usage reports for the current month only and
not for arbitrary period of time. This implies some restrictions to
what we can report when the application is:
1. started and stopped several times in the month
2. started in the current month but not stopped at all
3. started in a previous month and not stopped
The first problem can be solved by iterating through the events and
finding the respective start and end timestamps that have to be
reported to Abacus.


The second issue might be solved by reporting small amounts of
usage, stating from the last start event and continuing to report on
every poll of the integration layer. For example we can report several
usages:
start: 1438945112; end: 1438946000 (current time for the billing
integration)
start: 1438946000 (previous reporting cycle); end: 1438947000
The third issue might be solved by finding the last start event and
reporting in the same manner as with #2.
Reporting usage in small steps might require persistence so we can
store the end time of the previous reporting. We might use in-memory
cache and reach to Abacus as primary storage. If Abacus can
accumulate usage reporting we can even get rid of the persistence and
cache.


Is such integration in the scope of Abacus project?

Regards,
Hristo Iliev


Service instance name is not passed from cloud controller to service broker

Dave Chen
 

The service instance name is not passed from cloud controller to service broker when we do "cf cs sample-service free my-service". We need to get the service instance name "my-service" from the CreateServiceInstanceRequest for our service broker when performing the above CLI command. Is there a way to make service instance name is part of the contract between CC and service brokers?

Thanks,
Dave


Service instance name is not passed from cloud controller to service broker

Dave Chen
 

The service instance name is not passed from cloud controller to service broker when we do "cf cs sample-service free my-service". We need to get the service instance name "my-service" from the CreateServiceInstanceRequest for our service broker when performing the above CLI command. Is there a way to make service instance name is part of the contract between CC and service brokers?

Thanks,
Dave


Notifications for service provisioning

Vineet Banga
 

Is there any notification mechanism available in CF to listen on service broker create/update/delete calls? We are implementing multiple services exposed via Service Broker in the marketplace and would like to take certain common actions when services are being provisioned.

Vineet


Re: Missing routing logs from "cf logs app-name"

Simon Johansson <simon@...>
 

Gwenn, our buffer is set to 500(default is 100).


On Wed, Aug 12, 2015 at 3:24 AM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

Simon not sure is related but don,t if you don't consume enough quickly
the message, the buffer will be exhausted.

Did you try to increase the buffer size ?


On Wed, Aug 12, 2015 at 5:47 AM, Simon Johansson <simon(a)simonjohansson.com
wrote:
Howdie! Not really a dev related question, but since there is no cf-user
list I'm trying my luck here. :)

Im looking into an issue where we don't get enough RTR logs from cf logs
/ firehose-to-syslog
I'm using cf-env[1] as the test app, and Im hitting
app.domain.com/kehehehe.
For each request the app itself logs the the path requested and the
gorouters logs the same information but a bit more verbose.

$ cf logs cf-env-test
...
2015-08-11T11:43:41.24+0200 [RTR/0] OUT cf-env-test.domain.com -
[11/08/2015:09:43:41 +0000] "GET /kehehehe HTTP/1.1" 404 0 18 "-"
"curl/7.43.0" 10.230.15.4:59728 x_forwarded_for:"10.230.15.4"
vcap_request_id:4a186d9e-046a-47bc-74d0-f0c3a8cb1257
response_time:0.008503156 app_id:7f1944a2-2197-43b5-9334-0be7c8b9b40e
2015-08-11T11:43:41.31+0200 [App/0] ERR 10.230.15.8 - - [11/Aug/2015
09:43:41] "GET /kehehehe HTTP/1.1" 404 18 0.0006

I shoot off 1000 requests and would expect to see 1000 of each log type.
But only the App logs are correct.

$ grep "App/" /tmp/cf-logs | wc -l # /tmp/cf-logs comes from cf logs
cf-env-test > /tmp/cf-logs
1000
$ grep "RTR/" /tmp/cf-logs | wc -l
145

Looking at the gorouters
nsaadmin(a)a99a5339-4308-43db-b3a4-59442169861d:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
484
nsaadmin(a)1d8ef2c2-2ec1-417d-8882-0234de251c60:~$ grep "GET /kehehehe"
/var/vcap/sys/log/gorouter/access.log | wc -l
516
So the gorouter process definitely logs the right amount of requests to
disk.

In metron_agent.stdout.log on the gorouters there are a lot of lines like

{"timestamp":1439286941.958369255,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:6001753075995010215
high:3404432699330643838
1}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}
{"timestamp":1439286941.959420443,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:8668459744461365080
high:13197025281073682796
2}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}
{"timestamp":1439286941.964927912,"process_id":25749,"source":"metron","log_level":"warn","message":"no
matching HTTP start message found for {low:13205430422726438703
high:7208042014820494920
1}","data":null,"file":"/var/vcap/data/compile/metron_agent/loggregator/src/metron/messageaggregator/message_aggregator.go","line":97,"method":"metron/messageaggregator.(*MessageAggregator).handleHTTPStop"}

Has anyone seen this before?

[1] https://github.com/cloudfoundry-community/cf-env


Re: Billing & Metering of app usage with Abacus

Piotr Przybylski <piotrp@...>
 

Hi, I am also looking at runtime submissions for Abacus, worked on it for
Bluemix, couple of points for discussion.

In addition to usage events (start followed by stop), the scaling and
duplicate events need to be handled. The former is a START followed by
START with memory or instance count changed, the latter can be STOP
followed by STOP.
We also encountered a situation where the ordering is correct - START
followed by STOP but the timestamp for the START is later than STOP.

For the running applications - your points #2 and #3 are a working though
it may generate fair amount of traffic, depending on frequency and number
of running applications. Eventually we may want to look at alternatives,
for example enhance metering to allow for time based submissions. Instead
of continually submitting usage, submit the state of application -
(STARTED, memory, instances), the metering could then calculate usage for
that application based on the time passed until application is stopped.

I think handling some of above scenarios requires persistence, even if to
log CF events that were used for submission (or skipped). You may want to
persist state to recovery from application failure or restart, as well as
keep track of running/active applications.

Thanks,

Piotr


Hristo Iliev <hsiliev(a)gmail.com> wrote on 08/07/2015 08:07:53 AM:

We want to integrate the metrics provided by Cloud Foundry with Abacus.
We plan to create a billing/metering integration layer that:
fetches the app usage events from CC
inserts runtime usage data to Abacus
The events should be processed to build usage data based on them.
This is much like the idea outlined in Dr.Nic's blog. The
integration layer can do this periodically.
AFAIK Abacus provides usage reports for the current month only and
not for arbitrary period of time. This implies some restrictions to
what we can report when the application is:
1. started and stopped several times in the month
2. started in the current month but not stopped at all
3. started in a previous month and not stopped
The first problem can be solved by iterating through the events and
finding the respective start and end timestamps that have to be
reported to Abacus.


The second issue might be solved by reporting small amounts of
usage, stating from the last start event and continuing to report on
every poll of the integration layer. For example we can report several
usages:
start: 1438945112; end: 1438946000 (current time for the billing
integration)
start: 1438946000 (previous reporting cycle); end: 1438947000
The third issue might be solved by finding the last start event and
reporting in the same manner as with #2.
Reporting usage in small steps might require persistence so we can
store the end time of the previous reporting. We might use in-memory
cache and reach to Abacus as primary storage. If Abacus can
accumulate usage reporting we can even get rid of the persistence and
cache.


Is such integration in the scope of Abacus project?

Regards,
Hristo Iliev

8181 - 8200 of 9421