Date   

UAA 4.1.1 release maven central

Mike Roda
 

When will 4.1.1 appear on maven central?


Issue on querying service instances

Yitao Jiang
 

per the apidocs, the GET /v2/service_instances response should return
service instance including user_provided_service_instance, but actually it
doesn't return services of such type.

cf is V226, do you have any ideas?

​[1]:
http://apidocs.cloudfoundry.org/235/service_instances/list_all_service_instances.html​

--

Regards,

Yitao


Re: Looking for a buildpack to run JMeter on CF

Manglu Balasubramanian
 

Dan,

Sincere thanks.
This worked very well.
Special thanks for the detailed comments on how things work (with special mention towards - JBP_CONFIG_JAVA_MAIN)

Appreciate your help

Thanks
Manglu


CF-239 defaults to Unprivileged Containers on Diego

Nicholas Calugar
 

Hello Cloud Foundry,

As you may have noticed in the release notes for CF-239 [1], Cloud Foundry
now defaults to run containers on Diego in unprivileged mode. This greatly
improves security as root escalation inside the container is no longer a
threat to the host operating system. More information about unprivileged
containers can be found here [2].

Please note that this new default only applies to a newly requested process
on the Diego backend. Running applications must be restarted or otherwise
changed to switch to unprivileged containers.

One known incompatibility is running applications that use FUSE file system
support. Operators that would like to continue allowing privileged
containers on their Cloud Foundry deployment can use the two new deployment
manifest properties listed in the Job Spec Changes for CF-239 [3].


[1] https://github.com/cloudfoundry/cf-release/releases/tag/v239
[2]
https://linuxcontainers.org/lxc/getting-started/#creating-unprivileged-containers-as-a-user
[3]
https://github.com/cloudfoundry/cf-release/releases/tag/v239#job-spec-changes


Thanks,

Nick

--
Nicholas Calugar
Product Manager - Cloud Foundry API
Pivotal Software, Inc.


Re: Looking for a buildpack to run JMeter on CF

Daniel Mikusa
 

Here's what worked for me:

manifest.yml

```
---
applications:
- name: jmeter-test
memory: 512M
instances: 1
path: .
buildpack: java_buildpack
command: 'JAVA_HOME=.java-buildpack/open_jdk_jre
PATH=$PATH:$JAVA_HOME/bin ./bin/jmeter.sh -n -t test-google.jmx && echo
"Done" && sleep 600'
no-route: true
health-check-type: none
env:
JBP_CONFIG_JAVA_MAIN: '{java_main_class: "DoesNotMatter"}'
```

.cfignore

```
LICENSE
README
NOTICE
docs
licenses
printable_docs
jmeter.log
bin/*.bat
bin/*.cmd
extras
```

I put these two files into and pushed this from the root of the
apache-jmeter.2.11 directory. I also put my .jmx test plan into that
folder so it would get pushed up.

Notes on this:

1.) Set the name, instance and memory count to fit your needs.
2.) path should be `.` so that it will upload the jmeter files and the test
plan which are in the current directory.
3.) buildpack forces CF to use the java_buildpack (your name might vary)
4.) command tells CF how to run the app. We need to set JAVA_HOME & PATH
because the JBP doesn't. Then we just run jmeter like usual. I've added
the `&& echo 'Done' && sleep 600` to show when it's done and keep the app
up longer. When it exits, the system will interpret that as a crash.
There's other ways you could handle this see below.
5.) no-route & health-check-type disable the HTTP based functionality,
since we're not listening for web requests
6.) JBP_CONFIG_JAVA_MAIN is set to quiet the JBP. It get's cranky because
it doesn't understand the files we're pushing. Setting this value just
tells it we're running a Java main class, which it understands. The only
thing that matters is the presence of this setting to calm the JBP. The
command is specifying everything needed to run jmeter.
7.) I add a .cfignore so I don't push up all the docs and other junk.
That's optional. Ideally the build pack would download Jmeter for me, but
that would require customizations to the JBP.

Output looks like this:

```
2016-07-19T09:51:06.44-0400 [API/7] OUT Updated app with guid
b9452289-c8fc-4ac9-b68e-8f7e68c16dab ({"state"=>"STARTED"})
2016-07-19T09:51:06.78-0400 [STG/0] OUT Downloading java_buildpack...
2016-07-19T09:51:06.80-0400 [STG/0] OUT Downloaded java_buildpack
2016-07-19T09:51:06.80-0400 [STG/0] OUT Creating container
2016-07-19T09:51:07.74-0400 [STG/0] OUT Successfully created container
2016-07-19T09:51:07.75-0400 [STG/0] OUT Downloading app package...
2016-07-19T09:51:09.14-0400 [STG/0] OUT Downloaded app package (23.6M)
2016-07-19T09:51:09.14-0400 [STG/0] OUT Staging...
2016-07-19T09:51:10.06-0400 [STG/0] OUT -----> Java Buildpack Version:
v3.8.1 (offline) |
https://github.com/cloudfoundry/java-buildpack.git#29c79f2
2016-07-19T09:51:10.22-0400 [STG/0] OUT -----> Downloading Open Jdk
JRE 1.8.0_91-unlimited-crypto from
https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_91-unlimited-crypto.tar.gz
(found in cache)
2016-07-19T09:51:11.24-0400 [STG/0] OUT -----> Downloading Open JDK
Like Memory Calculator 2.0.2_RELEASE from
https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-2.0.2_RELEASE.tar.gz
(found in cache)
2016-07-19T09:51:11.24-0400 [STG/0] OUT Expanding Open Jdk JRE
to .java-buildpack/open_jdk_jre (1.0s)
2016-07-19T09:51:11.27-0400 [STG/0] OUT Memory Settings:
-Xmx317161K -XX:MaxMetaspaceSize=64M -Xss228K -Xms317161K
-XX:MetaspaceSize=64M
2016-07-19T09:51:11.27-0400 [STG/0] OUT -----> Downloading Spring Auto
Reconfiguration 1.10.0_RELEASE from
https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-1.10.0_RELEASE.jar
(found in cache)
2016-07-19T09:51:18.96-0400 [STG/0] OUT Exit status 0
2016-07-19T09:51:18.96-0400 [STG/0] OUT Staging complete
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading droplet, build
artifacts cache...
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading build artifacts
cache...
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading droplet...
2016-07-19T09:51:19.00-0400 [STG/0] OUT Uploaded build artifacts cache
(108B)
2016-07-19T09:51:25.50-0400 [STG/0] OUT Uploaded droplet (69.2M)
2016-07-19T09:51:25.51-0400 [STG/0] OUT Uploading complete
2016-07-19T09:51:26.24-0400 [CELL/0] OUT Creating container
2016-07-19T09:51:27.33-0400 [CELL/0] OUT Successfully created container
2016-07-19T09:51:31.00-0400 [APP/0] OUT Creating summariser <summary>
2016-07-19T09:51:31.01-0400 [APP/0] OUT Created the tree successfully
using test-google.jmx
2016-07-19T09:51:31.01-0400 [APP/0] OUT Starting the test @ Tue Jul 19
13:51:31 UTC 2016 (1468936291015)
2016-07-19T09:51:31.02-0400 [APP/0] OUT Waiting for possible shutdown
message on port 4445
2016-07-19T09:51:31.72-0400 [APP/0] OUT summary + 1 in 0.3s =
3.9/s Avg: 254 Min: 254 Max: 254 Err: 0 (0.00%) Active: 1
Started: 1 Finished: 0
2016-07-19T09:51:32.26-0400 [APP/0] OUT summary = 15 in 1s =
19.2/s Avg: 51 Min: 30 Max: 254 Err: 0 (0.00%)
2016-07-19T09:51:32.26-0400 [APP/0] OUT Tidying up ... @ Tue Jul 19
13:51:32 UTC 2016 (1468936292260)
2016-07-19T09:51:32.26-0400 [APP/0] OUT ... end of run
2016-07-19T09:51:32.58-0400 [APP/0] OUT Done
```

Another way that you might want to handle this would be to change the
command to something like `sleep 99999`. This would essentially make the
app run for a long time not doing anything. You could then run `cf ssh` to
get into the app container and manually run your jmeter scripts. This fits
better because of the fact that jmeter scripts don't generally run forever
(although they can). It would also allow you to save test results to a
file and `scp` those off of the container before it goes away.

Hope that helps!

Dan


On Mon, Jul 18, 2016 at 8:03 PM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi Dan,

Thanks for the various suggestions.

Thanks. I tried the "-c" option.

To make things simple, I created a hello-world java app and I get the
following message:

[Buildpack] ERROR Compile failed with exception
#<RuntimeError: No container can run this application. Please ensure that
you’ve pushed a valid JVM artifact or artifacts using the -p command line
argument or path manifest entry. Information about valid JVM artifacts can
be found at
https://github.com/cloudfoundry/java-buildpack#additional-documentation. >
No container can run this application. Please ensure that you’ve pushed a
valid JVM artifact or artifacts using the -p command line argument or path
manifest entry. Information about valid JVM artifacts can be found at
https://github.com/cloudfoundry/java-buildpack#additional-documentation.
Staging failed: Buildpack compilation step failed

I am planning to run JMeter in a headless mode (so will not use the GUI)
and pointing to the test plans. I am pushing both JMeter scripts, configs,
jars along with testplans.

I was thinking about the crashing issue (when the scripts are executed). I
was thinking that there might be a way around this. I wanted to get the
first step forward before tackling this issue.

My initial plan was to use Blazemeter service, however we have
connectivity issues hence the desire to run the JMeter within the CF
environment.

I will explore the "extending" section that you have referenced.


[ann] Dingo PostgreSQL v0.9

Dr Nic Williams <drnicwilliams@...>
 

Dingo PostgreSQL is a "batteries included" service broker which offers
dedicated HA clusters of PostgreSQL, and continuous streaming archives to
Amazon S3 with disaster recovery. Each PostgreSQL node runs inside a Docker
container, and nodes in a cluster are spread across different AZs.

Its the DB backend for https://www.starkandwayne.com/blog/ &
https://www.starkandwayne.com/videos on AWS, for example.

We cut v0.9 [1] which includes some bug fixes for node allocation (ensuring
clusters are spread across AZs and to emptier cells/vms).

We've also started adding advanced admin features: explicitly moving the
nodes of an existing cluster to new cells/vms with minimal downtime [2].

If your CF has 1000 apps, then it probably needs 1000 SQL databases. Our
focus is on making as easy to be the DBA of 1000 highly-available
PostgreSQL database clusters as it is to run one. Most future features are
guided by this vision.


[1]
https://github.com/dingotiles/dingo-postgresql-release/releases/tag/v0.9.0
[2]
https://github.com/dingotiles/dingo-postgresql-broker/#move-cluster-into-different-cells

For chit-chat and questions, we have a Slack chat room with a fancy
dedicated URL https://slack.dingotiles.com

(For Pivotal Ops Mgr users, v0.9 will be available at
https://network.pivotal.io/products/dingo-postgresql-for-pcf soon)

--
Dr Nic Williams
Stark & Wayne LLC - consultancy for Cloud Foundry users
http://drnicwilliams.com
http://starkandwayne.com
cell +1 (415) 860-2185
twitter @drnic


Re: Issue on nested shared domains

Tomoe Sugihara
 

Hi Carlo,

Can you try using Private Domains[1]? IIRC, that should allow what you
were trying to do.

[1]: https://docs.cloudfoundry.org/devguide/deploy-apps/routes-domains.html#private-domains

Thanks,
Tomoe

On Tue, Jul 19, 2016 at 1:15 PM, Carlo Alberto Ferraris
<carlo.ferraris(a)rakuten.com> wrote:
Due to internal regulations our users need to deploy apps on two wildcard domains, where the second is the a subdomain of the first one, e.g.:
- *.A.com
- *.B.A.com

We registered both as shared domains and they work fine, but we have run now into a surprising behavior. Turns out it's not allowed to register route B.A.com:

$ cf create-route my-space A.com --hostname B
Creating route B.A.com for org my-org / space my-space as my-user...
FAILED
Server error, status code: 400, error code: 210001, message: The route is invalid: host domain_conflict

I agree that using nested shared domains may not be the best of designs, but is the above behavior expected? At the DNS level this is allowed, and I don't really see a reason why it shouldn't be allowed in CF...

Carlo


Issue on nested shared domains

Carlo Alberto Ferraris
 

Due to internal regulations our users need to deploy apps on two wildcard domains, where the second is the a subdomain of the first one, e.g.:
- *.A.com
- *.B.A.com

We registered both as shared domains and they work fine, but we have run now into a surprising behavior. Turns out it's not allowed to register route B.A.com:

$ cf create-route my-space A.com --hostname B
Creating route B.A.com for org my-org / space my-space as my-user...
FAILED
Server error, status code: 400, error code: 210001, message: The route is invalid: host domain_conflict

I agree that using nested shared domains may not be the best of designs, but is the above behavior expected? At the DNS level this is allowed, and I don't really see a reason why it shouldn't be allowed in CF...

Carlo


Re: Looking for a buildpack to run JMeter on CF

Manglu Balasubramanian
 

Hi Dan,

Thanks for the various suggestions.

Thanks. I tried the "-c" option.

To make things simple, I created a hello-world java app and I get the following message:

[Buildpack] ERROR Compile failed with exception #<RuntimeError: No container can run this application. Please ensure that you’ve pushed a valid JVM artifact or artifacts using the -p command line argument or path manifest entry. Information about valid JVM artifacts can be found at https://github.com/cloudfoundry/java-buildpack#additional-documentation. >
No container can run this application. Please ensure that you’ve pushed a valid JVM artifact or artifacts using the -p command line argument or path manifest entry. Information about valid JVM artifacts can be found at https://github.com/cloudfoundry/java-buildpack#additional-documentation.
Staging failed: Buildpack compilation step failed

I am planning to run JMeter in a headless mode (so will not use the GUI) and pointing to the test plans. I am pushing both JMeter scripts, configs, jars along with testplans.

I was thinking about the crashing issue (when the scripts are executed). I was thinking that there might be a way around this. I wanted to get the first step forward before tackling this issue.

My initial plan was to use Blazemeter service, however we have connectivity issues hence the desire to run the JMeter within the CF environment.

I will explore the "extending" section that you have referenced.


Re: Looking for a buildpack to run JMeter on CF

Manglu Balasubramanian
 

Hi Daniel,

Thanks. I am on dedicated Bluemix PaaS and do not have access to a docker container. Bluemix is still on the previous version of CF.

Docker was my initial thought and it is not available on a dedicated Bluemix at the moment.

Thanks
Manglu


Re: Looking for a buildpack to run JMeter on CF

Daniel Mikusa
 

On Mon, Jul 18, 2016 at 2:42 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi Manglu,

Does your Cloud Foundry use Diego, and is Docker support
<https://docs.pivotal.io/pivotalcf/1-7/concepts/docker.html#push-docker>
enabled? For what I assume is a one-off task, creating a Docker image with
JMeter in will be a lot easier than crafting a buildpack.

I'm curious - what's your use-case for JMeter on Cloud Foundry?
+1


On Mon, Jul 18, 2016 at 1:04 AM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi

I haven't created buildpacks in the past (and only understand this
concept in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh.
Though JMeter is a java app and has a JAR which packages the artefacts, i
can't run it like a Java application and have this need to run jmeter.sh.
Why not? What happens when you try this? What fails.

Have you tried manually setting a command? You can set the `-c` arg to `cf
push` and it should run the command exactly as you tell it.



I tried to comprehend the Java Build pack to see how it runs the scripts
file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh
file.
What are you trying to push? The jmeter jars or are you trying to push a
test plan? It would probably be easier to push a test plan which is just a
small xml file and then have the build pack download the latest jmeter
binary for you.

THe compile script doesn't need to do anything fanciful

This could download the Jmeter binary for you.


while the release script should using the "jmeter.sh" for the start up of
the droplet.
Yes. While probably obvious I just want to point out there's no GUI, so
you'd need to run the cli only commands. You should also run jmeter in
some way that it keeps running forever. If jmeter exits, for example when
a test plan finishes, CF (the v2 API) will interpret this as the
application crashing and it will restart the app which may not be expected
/ wanted.

Appreciate if someone could point me to docs that could help me in
comprehending the Java Build pack better so that I can fork this and create
a build pack for me to run jmeter.
I think the "Extending" section here is probably a good start:
https://github.com/cloudfoundry/java-buildpack#additional-documentation

Dan


Re: Looking for a buildpack to run JMeter on CF

Daniel Jones
 

Hi Manglu,

Does your Cloud Foundry use Diego, and is Docker support
<https://docs.pivotal.io/pivotalcf/1-7/concepts/docker.html#push-docker>
enabled? For what I assume is a one-off task, creating a Docker image with
JMeter in will be a lot easier than crafting a buildpack.

I'm curious - what's your use-case for JMeter on Cloud Foundry?

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
@DanielJonesEB <https://twitter.com/DanielJonesEB>
*EngineerBetter* Ltd <http://www.engineerbetter.com> - UK Cloud Foundry
Specialists

On Mon, Jul 18, 2016 at 1:04 AM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi

I haven't created buildpacks in the past (and only understand this concept
in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh.
Though JMeter is a java app and has a JAR which packages the artefacts, i
can't run it like a Java application and have this need to run jmeter.sh.

I tried to comprehend the Java Build pack to see how it runs the scripts
file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh
file. THe compile script doesn't need to do anything fanciful while the
release script should using the "jmeter.sh" for the start up of the droplet.


Appreciate if someone could point me to docs that could help me in
comprehending the Java Build pack better so that I can fork this and create
a build pack for me to run jmeter.

Thanks


Re: Reg cf-235 release

Jayarajan Ramapurath Kozhummal (jayark) <jayark@...>
 

+Nithiya

Thanks Amit!

Regards
Jayaraj

From: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Date: Saturday, July 16, 2016 at 9:31 AM
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Cc: Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>, "Sowmya Chandrapati -X (sowmyach - PERSISTENT SYSTEMS INC at Cisco)" <sowmyach(a)cisco.com<mailto:sowmyach(a)cisco.com>>
Subject: Re: [cf-dev] Reg cf-235 release

Don't run the update script the first time. git gets confused because the remote URL for one of the recursive submodules has changed. I'm not sure if there's a non-manual way for the script to deal with that case, but it's easy to avoid:

$ rm -rf cf-release/
$ git clone git(a)github.com<mailto:git(a)github.com>:cloudfoundry/cf-release
$ cd cf-release/
$ git checkout -b branchcf edc3e3c9
$ ./scripts/update

On Fri, Jul 15, 2016 at 1:33 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi

We are trying to use cf-235 release.
Following are the steps we have used for creating the release:


1. sudo git clone https://github.com/cloudfoundry/cf-release

2. cd cf-release/scripts

3. ./update

4. sudo git checkout -b branchcf edc3e3c9

5. cd cf-release/scripts

6. ./update

We get the following error while the checkout from the branch, due to which I cannot create the cf-235 tar with some custom changes.

......................................................................................................
....................................................................................................
Submodule path 'src/loggregator/src/github.com/ugorji/go<http://github.com/ugorji/go>': checked out 'f1f1a805ed361a0e078bb537e4ea78cd37dcf065'
Submodule path 'src/smoke-tests': checked out 'f3092b9040bf93cb3a6a7670bf53be11b2d0f957'
Submodule path 'src/uaa-release': checked out '68cf1f16843392b4f115719047eaf61ef4b67375'
Submodule path 'src/uaa-release/src/uaa': checked out '616e8bcc58f9bc16f9d9ec806da1b0bf0f5dae85'
Failed to recurse into submodule path 'src/consul-release'

Regards
Nithiyasri


Looking for a buildpack to run JMeter on CF

Manglu Balasubramanian
 

Hi

I haven't created buildpacks in the past (and only understand this concept in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh. Though JMeter is a java app and has a JAR which packages the artefacts, i can't run it like a Java application and have this need to run jmeter.sh.

I tried to comprehend the Java Build pack to see how it runs the scripts file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh file. THe compile script doesn't need to do anything fanciful while the release script should using the "jmeter.sh" for the start up of the droplet.


Appreciate if someone could point me to docs that could help me in comprehending the Java Build pack better so that I can fork this and create a build pack for me to run jmeter.

Thanks


Re: Reg cf-235 release

Amit Kumar Gupta
 

Don't run the update script the first time. git gets confused because the
remote URL for one of the recursive submodules has changed. I'm not sure
if there's a non-manual way for the script to deal with that case, but it's
easy to avoid:

$ rm -rf cf-release/
$ git clone git(a)github.com:cloudfoundry/cf-release
$ cd cf-release/
$ git checkout -b branchcf edc3e3c9
$ ./scripts/update


On Fri, Jul 15, 2016 at 1:33 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



We are trying to use cf-235 release.

Following are the steps we have used for creating the release:



1. sudo git clone https://github.com/cloudfoundry/cf-release

2. cd cf-release/scripts

3. ./update

4. sudo git checkout -b branchcf edc3e3c9

5. cd cf-release/scripts

6. ./update



We get the following error while the checkout from the branch, due to
which I cannot create the cf-235 tar with some custom changes.



…………………………………………………………………………………………

……………………………………………………………………………………….

Submodule path 'src/loggregator/src/github.com/ugorji/go': checked out
'f1f1a805ed361a0e078bb537e4ea78cd37dcf065'

Submodule path 'src/smoke-tests': checked out
'f3092b9040bf93cb3a6a7670bf53be11b2d0f957'

Submodule path 'src/uaa-release': checked out
'68cf1f16843392b4f115719047eaf61ef4b67375'

Submodule path 'src/uaa-release/src/uaa': checked out
'616e8bcc58f9bc16f9d9ec806da1b0bf0f5dae85'

*Failed to recurse into submodule path 'src/consul-release'*



Regards

Nithiyasri


Deprecation of loggregator_consumer

Warren Fernandes
 

Hey cf-dev,

Loggregator Consumer <https://github.com/cloudfoundry/loggregator_consumer> has
been deprecated and is no longer supported.

Loggregator Consumer is a library that allows an application developer to
set up a connection to a loggregator server, and begin receiving log
messages from it. It includes the ability to tail logs as well as get the
recent logs

loggregator_consumer talks to the Traffic Controller via the endpoints /tail,
/dump, and /recent which will also be deprecated in the near future.

This library has been deprecated in favor of noaa
<https://github.com/cloudfoundry/noaa>. NOAA is a client library to consume
metric and log messages from Loggregator.

Sincerely,
CF Loggregator


Re: Reg the monit of the bosh vm in cloud foundry

Vik R <vagcom.ben@...>
 

See whether the following helps:

https://mmonit.com/monit/documentation/monit.html#SERVICE-POLL-TIME



On Thu, Jul 14, 2016 at 11:12 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



I am using cloud foundry -231 and have created a bosh release of a job,
which takes long time (more than 5 mins) to complete.



During bosh deploy, sometimes the job fails. On observing the monit log,
the bosh checks the pid every 30 secs and reports error and retries after
10 secs. After retrying 6/7 times, the job fails.

Just after the job has failed, the job is completed successfully and all
the process is running in the bosh vm.



So I increased the timeout of the monit process to 90 secs instead of
default 30sec. Now the bosh checks for the pid file (every 90 secs) but
only thrice(not 7 or 8 times as before) and reports failure, though .



Pls suggest me how can I control the monit of the bosh vm in cloud foundry
to wait until the job completes. Please note this issue doesnt happen
always..





Regards

Nithiyasri


Reg cf-235 release

Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM@Cisco) <ngnanase at cisco.com...>
 

Hi

We are trying to use cf-235 release.
Following are the steps we have used for creating the release:


1. sudo git clone https://github.com/cloudfoundry/cf-release

2. cd cf-release/scripts

3. ./update

4. sudo git checkout -b branchcf edc3e3c9

5. cd cf-release/scripts

6. ./update

We get the following error while the checkout from the branch, due to which I cannot create the cf-235 tar with some custom changes.

......................................................................................................
....................................................................................................
Submodule path 'src/loggregator/src/github.com/ugorji/go': checked out 'f1f1a805ed361a0e078bb537e4ea78cd37dcf065'
Submodule path 'src/smoke-tests': checked out 'f3092b9040bf93cb3a6a7670bf53be11b2d0f957'
Submodule path 'src/uaa-release': checked out '68cf1f16843392b4f115719047eaf61ef4b67375'
Submodule path 'src/uaa-release/src/uaa': checked out '616e8bcc58f9bc16f9d9ec806da1b0bf0f5dae85'
Failed to recurse into submodule path 'src/consul-release'

Regards
Nithiyasri


Re: Question regarding rate limit for app logs on CF

Mehran Saliminia
 

Thank you Jim!


Re: Wiping out data for consul/etcd with bosh drain script?

Tomoe Sugihara
 

Hi Amit,

On Thu, Jun 16, 2016 at 4:01 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Yup, the modification should be as simple as flipping these two lines:
https://github.com/cloudfoundry-incubator/etcd-release/blob/master/jobs/etcd/templates/etcd_bosh_utils.sh.erb#L153-L154.

It requires more setup to try the scenario with data, but it's more
informative. If you have the time, yeah, I'd go straight to testing the
with-data scenario. If not, even the more basic test (without data, just
check if the cluster comes back healthy) would be valuable info.
I flipped the lines as you suggested and did some testing against 3
job instances that collocate diego bbs and etcd in a CF deployment.

I did a CF system level testing where while running the following
script, I was monitoring `cf app some-app` as well as response for
http://some-app.app.domain.

---------
#!/bin/bash -x

for t in {0..5}; do
echo === $t, `date`
for i in {0..2} ; do bosh -n stop bbs-etcd $i ; done
for i in {0..2} ; do bosh -n start bbs-etcd $i ; done

done
---------

It looked like, as soon as the 1st job instance starts up, `cf app`
starts responding normally.

Even more interesting would be to see what happens if you have an actual out of sync cluster, and try this. This would be helpful input to have before we would get a chance to prioritize the implementation.
Haven't done this yet, but any suggestions on how to produce this out
of sync state?

Thanks,
Tomoe


On Wed, Jun 15, 2016 at 11:45 PM, Tomoe Sugihara <tsugihara(a)pivotal.io>
wrote:

Thanks Amit for sharing your insight. That was helpful. Some followup
questions inline:

On Thu, Jun 16, 2016 at 3:05 PM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hey Tomoe

Great question. I would also prefer disaster recovery to be possible via
"bosh stop" then "bosh start". This is almost possible in etcd, except for
the conditional you found. The reason for the conditional is that it allows
rolling a 1-node cluster without data loss. I'd like to entertain the idea
that etcd-release's SLA (currently embodied in the acceptance tests [0])
should drop the requirement of maintaining data for a 1-node cluster roll.
That reduced SLA will probably be fine with the community, and the improved
disaster recovery experience would be worth the reduced SLA, but I haven't
validated that.

consul-release is a little further removed from this possibility because
consul requires different orchestration logic, and currently the
implementation doesn't wipe out the data as aggressively. We have a story
[1] already to explore whether we could do that without reducing SLA [2].

By the way, have you tried this?


If you have a 3 node cluster with etcd (healthy), and bosh stop then bosh
start, does the cluster recover?

Just to confirm, with a modified drain script that *always* wipes out(
not conditionally as in current script) ?


If you have a 3 node cluster with data, and you do this, does the cluster
recover (with data loss, which is acceptable in this case)?

What's the difference between this and the first one above? "with data" or
not?



Even more interesting would be to see what happens if you have an
actual out of sync cluster, and try this. This would be helpful input to
have before we would get a chance to prioritize the implementation.


I'll find some time to test those scenarios so I can share my findings.

Thanks,
Tomoe




[0]
https://github.com/cloudfoundry-incubator/etcd-release/tree/master/src/acceptance-tests
[1] https://www.pivotaltracker.com/story/show/120648349
[2]
https://github.com/cloudfoundry-incubator/consul-release/tree/master/src/acceptance-tests

Best,
Amit

On Wed, Jun 15, 2016 at 10:00 PM, Tomoe Sugihara <tsugihara(a)pivotal.io>
wrote:

Hi there,

I have seen issues multiple times where consul and/or etcd nodes went
out of sync and needed to be clean-restarted as explained in the Disaster
Recovery instructions [1][2].

I am wondering if it makes sense to add those steps in bosh drain
script[3], That way, you can always get a clean start, and if something is
wrong, you can recover by "bosh stop" followed by "bosh start". FWIT, I
noticed that etcd does that conditionally[4][5].

Maybe there're some drawbacks, but I thought I'd start a thread to hear
from experts.

[1]:
https://github.com/cloudfoundry-incubator/etcd-release#disaster-recovery
[2]:
https://github.com/cloudfoundry-incubator/consul-release#disaster-recovery
[3]: https://bosh.io/docs/drain.html

[4]:
https://github.com/cloudfoundry-incubator/etcd-release/blob/e809b46202be89a24c7bfeebf270d24b17589260/jobs/etcd/templates/drain#L12
[5]:
https://github.com/cloudfoundry-incubator/etcd-release/blob/e809b46202be89a24c7bfeebf270d24b17589260/jobs/etcd/templates/etcd_bosh_utils.sh.erb#L147

Best,
Tomoe

4021 - 4040 of 9426