Looking for a buildpack to run JMeter on CF


Manglu Balasubramanian
 

Hi

I haven't created buildpacks in the past (and only understand this concept in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh. Though JMeter is a java app and has a JAR which packages the artefacts, i can't run it like a Java application and have this need to run jmeter.sh.

I tried to comprehend the Java Build pack to see how it runs the scripts file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh file. THe compile script doesn't need to do anything fanciful while the release script should using the "jmeter.sh" for the start up of the droplet.


Appreciate if someone could point me to docs that could help me in comprehending the Java Build pack better so that I can fork this and create a build pack for me to run jmeter.

Thanks


Daniel Jones
 

Hi Manglu,

Does your Cloud Foundry use Diego, and is Docker support
<https://docs.pivotal.io/pivotalcf/1-7/concepts/docker.html#push-docker>
enabled? For what I assume is a one-off task, creating a Docker image with
JMeter in will be a lot easier than crafting a buildpack.

I'm curious - what's your use-case for JMeter on Cloud Foundry?

Regards,
Daniel Jones - CTO
+44 (0)79 8000 9153
@DanielJonesEB <https://twitter.com/DanielJonesEB>
*EngineerBetter* Ltd <http://www.engineerbetter.com> - UK Cloud Foundry
Specialists

On Mon, Jul 18, 2016 at 1:04 AM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi

I haven't created buildpacks in the past (and only understand this concept
in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh.
Though JMeter is a java app and has a JAR which packages the artefacts, i
can't run it like a Java application and have this need to run jmeter.sh.

I tried to comprehend the Java Build pack to see how it runs the scripts
file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh
file. THe compile script doesn't need to do anything fanciful while the
release script should using the "jmeter.sh" for the start up of the droplet.


Appreciate if someone could point me to docs that could help me in
comprehending the Java Build pack better so that I can fork this and create
a build pack for me to run jmeter.

Thanks


Daniel Mikusa
 

On Mon, Jul 18, 2016 at 2:42 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi Manglu,

Does your Cloud Foundry use Diego, and is Docker support
<https://docs.pivotal.io/pivotalcf/1-7/concepts/docker.html#push-docker>
enabled? For what I assume is a one-off task, creating a Docker image with
JMeter in will be a lot easier than crafting a buildpack.

I'm curious - what's your use-case for JMeter on Cloud Foundry?
+1


On Mon, Jul 18, 2016 at 1:04 AM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi

I haven't created buildpacks in the past (and only understand this
concept in theory).

I wanted to run JMeter on CF and i have the need to run the jmeter.sh.
Though JMeter is a java app and has a JAR which packages the artefacts, i
can't run it like a Java application and have this need to run jmeter.sh.
Why not? What happens when you try this? What fails.

Have you tried manually setting a command? You can set the `-c` arg to `cf
push` and it should run the command exactly as you tell it.



I tried to comprehend the Java Build pack to see how it runs the scripts
file for Tomcat, however I am lost and need help in figuring things out.

Here is how, conceptully I think it should work.

In the detect script, I should be looking for the presence of jmeter.sh
file.
What are you trying to push? The jmeter jars or are you trying to push a
test plan? It would probably be easier to push a test plan which is just a
small xml file and then have the build pack download the latest jmeter
binary for you.

THe compile script doesn't need to do anything fanciful

This could download the Jmeter binary for you.


while the release script should using the "jmeter.sh" for the start up of
the droplet.
Yes. While probably obvious I just want to point out there's no GUI, so
you'd need to run the cli only commands. You should also run jmeter in
some way that it keeps running forever. If jmeter exits, for example when
a test plan finishes, CF (the v2 API) will interpret this as the
application crashing and it will restart the app which may not be expected
/ wanted.

Appreciate if someone could point me to docs that could help me in
comprehending the Java Build pack better so that I can fork this and create
a build pack for me to run jmeter.
I think the "Extending" section here is probably a good start:
https://github.com/cloudfoundry/java-buildpack#additional-documentation

Dan


Manglu Balasubramanian
 

Hi Daniel,

Thanks. I am on dedicated Bluemix PaaS and do not have access to a docker container. Bluemix is still on the previous version of CF.

Docker was my initial thought and it is not available on a dedicated Bluemix at the moment.

Thanks
Manglu


Manglu Balasubramanian
 

Hi Dan,

Thanks for the various suggestions.

Thanks. I tried the "-c" option.

To make things simple, I created a hello-world java app and I get the following message:

[Buildpack] ERROR Compile failed with exception #<RuntimeError: No container can run this application. Please ensure that you’ve pushed a valid JVM artifact or artifacts using the -p command line argument or path manifest entry. Information about valid JVM artifacts can be found at https://github.com/cloudfoundry/java-buildpack#additional-documentation. >
No container can run this application. Please ensure that you’ve pushed a valid JVM artifact or artifacts using the -p command line argument or path manifest entry. Information about valid JVM artifacts can be found at https://github.com/cloudfoundry/java-buildpack#additional-documentation.
Staging failed: Buildpack compilation step failed

I am planning to run JMeter in a headless mode (so will not use the GUI) and pointing to the test plans. I am pushing both JMeter scripts, configs, jars along with testplans.

I was thinking about the crashing issue (when the scripts are executed). I was thinking that there might be a way around this. I wanted to get the first step forward before tackling this issue.

My initial plan was to use Blazemeter service, however we have connectivity issues hence the desire to run the JMeter within the CF environment.

I will explore the "extending" section that you have referenced.


Daniel Mikusa
 

Here's what worked for me:

manifest.yml

```
---
applications:
- name: jmeter-test
memory: 512M
instances: 1
path: .
buildpack: java_buildpack
command: 'JAVA_HOME=.java-buildpack/open_jdk_jre
PATH=$PATH:$JAVA_HOME/bin ./bin/jmeter.sh -n -t test-google.jmx && echo
"Done" && sleep 600'
no-route: true
health-check-type: none
env:
JBP_CONFIG_JAVA_MAIN: '{java_main_class: "DoesNotMatter"}'
```

.cfignore

```
LICENSE
README
NOTICE
docs
licenses
printable_docs
jmeter.log
bin/*.bat
bin/*.cmd
extras
```

I put these two files into and pushed this from the root of the
apache-jmeter.2.11 directory. I also put my .jmx test plan into that
folder so it would get pushed up.

Notes on this:

1.) Set the name, instance and memory count to fit your needs.
2.) path should be `.` so that it will upload the jmeter files and the test
plan which are in the current directory.
3.) buildpack forces CF to use the java_buildpack (your name might vary)
4.) command tells CF how to run the app. We need to set JAVA_HOME & PATH
because the JBP doesn't. Then we just run jmeter like usual. I've added
the `&& echo 'Done' && sleep 600` to show when it's done and keep the app
up longer. When it exits, the system will interpret that as a crash.
There's other ways you could handle this see below.
5.) no-route & health-check-type disable the HTTP based functionality,
since we're not listening for web requests
6.) JBP_CONFIG_JAVA_MAIN is set to quiet the JBP. It get's cranky because
it doesn't understand the files we're pushing. Setting this value just
tells it we're running a Java main class, which it understands. The only
thing that matters is the presence of this setting to calm the JBP. The
command is specifying everything needed to run jmeter.
7.) I add a .cfignore so I don't push up all the docs and other junk.
That's optional. Ideally the build pack would download Jmeter for me, but
that would require customizations to the JBP.

Output looks like this:

```
2016-07-19T09:51:06.44-0400 [API/7] OUT Updated app with guid
b9452289-c8fc-4ac9-b68e-8f7e68c16dab ({"state"=>"STARTED"})
2016-07-19T09:51:06.78-0400 [STG/0] OUT Downloading java_buildpack...
2016-07-19T09:51:06.80-0400 [STG/0] OUT Downloaded java_buildpack
2016-07-19T09:51:06.80-0400 [STG/0] OUT Creating container
2016-07-19T09:51:07.74-0400 [STG/0] OUT Successfully created container
2016-07-19T09:51:07.75-0400 [STG/0] OUT Downloading app package...
2016-07-19T09:51:09.14-0400 [STG/0] OUT Downloaded app package (23.6M)
2016-07-19T09:51:09.14-0400 [STG/0] OUT Staging...
2016-07-19T09:51:10.06-0400 [STG/0] OUT -----> Java Buildpack Version:
v3.8.1 (offline) |
https://github.com/cloudfoundry/java-buildpack.git#29c79f2
2016-07-19T09:51:10.22-0400 [STG/0] OUT -----> Downloading Open Jdk
JRE 1.8.0_91-unlimited-crypto from
https://java-buildpack.cloudfoundry.org/openjdk/trusty/x86_64/openjdk-1.8.0_91-unlimited-crypto.tar.gz
(found in cache)
2016-07-19T09:51:11.24-0400 [STG/0] OUT -----> Downloading Open JDK
Like Memory Calculator 2.0.2_RELEASE from
https://java-buildpack.cloudfoundry.org/memory-calculator/trusty/x86_64/memory-calculator-2.0.2_RELEASE.tar.gz
(found in cache)
2016-07-19T09:51:11.24-0400 [STG/0] OUT Expanding Open Jdk JRE
to .java-buildpack/open_jdk_jre (1.0s)
2016-07-19T09:51:11.27-0400 [STG/0] OUT Memory Settings:
-Xmx317161K -XX:MaxMetaspaceSize=64M -Xss228K -Xms317161K
-XX:MetaspaceSize=64M
2016-07-19T09:51:11.27-0400 [STG/0] OUT -----> Downloading Spring Auto
Reconfiguration 1.10.0_RELEASE from
https://java-buildpack.cloudfoundry.org/auto-reconfiguration/auto-reconfiguration-1.10.0_RELEASE.jar
(found in cache)
2016-07-19T09:51:18.96-0400 [STG/0] OUT Exit status 0
2016-07-19T09:51:18.96-0400 [STG/0] OUT Staging complete
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading droplet, build
artifacts cache...
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading build artifacts
cache...
2016-07-19T09:51:18.96-0400 [STG/0] OUT Uploading droplet...
2016-07-19T09:51:19.00-0400 [STG/0] OUT Uploaded build artifacts cache
(108B)
2016-07-19T09:51:25.50-0400 [STG/0] OUT Uploaded droplet (69.2M)
2016-07-19T09:51:25.51-0400 [STG/0] OUT Uploading complete
2016-07-19T09:51:26.24-0400 [CELL/0] OUT Creating container
2016-07-19T09:51:27.33-0400 [CELL/0] OUT Successfully created container
2016-07-19T09:51:31.00-0400 [APP/0] OUT Creating summariser <summary>
2016-07-19T09:51:31.01-0400 [APP/0] OUT Created the tree successfully
using test-google.jmx
2016-07-19T09:51:31.01-0400 [APP/0] OUT Starting the test @ Tue Jul 19
13:51:31 UTC 2016 (1468936291015)
2016-07-19T09:51:31.02-0400 [APP/0] OUT Waiting for possible shutdown
message on port 4445
2016-07-19T09:51:31.72-0400 [APP/0] OUT summary + 1 in 0.3s =
3.9/s Avg: 254 Min: 254 Max: 254 Err: 0 (0.00%) Active: 1
Started: 1 Finished: 0
2016-07-19T09:51:32.26-0400 [APP/0] OUT summary = 15 in 1s =
19.2/s Avg: 51 Min: 30 Max: 254 Err: 0 (0.00%)
2016-07-19T09:51:32.26-0400 [APP/0] OUT Tidying up ... @ Tue Jul 19
13:51:32 UTC 2016 (1468936292260)
2016-07-19T09:51:32.26-0400 [APP/0] OUT ... end of run
2016-07-19T09:51:32.58-0400 [APP/0] OUT Done
```

Another way that you might want to handle this would be to change the
command to something like `sleep 99999`. This would essentially make the
app run for a long time not doing anything. You could then run `cf ssh` to
get into the app container and manually run your jmeter scripts. This fits
better because of the fact that jmeter scripts don't generally run forever
(although they can). It would also allow you to save test results to a
file and `scp` those off of the container before it goes away.

Hope that helps!

Dan


On Mon, Jul 18, 2016 at 8:03 PM, Manglu Balasubramanian <manglu(a)gmail.com>
wrote:

Hi Dan,

Thanks for the various suggestions.

Thanks. I tried the "-c" option.

To make things simple, I created a hello-world java app and I get the
following message:

[Buildpack] ERROR Compile failed with exception
#<RuntimeError: No container can run this application. Please ensure that
you’ve pushed a valid JVM artifact or artifacts using the -p command line
argument or path manifest entry. Information about valid JVM artifacts can
be found at
https://github.com/cloudfoundry/java-buildpack#additional-documentation. >
No container can run this application. Please ensure that you’ve pushed a
valid JVM artifact or artifacts using the -p command line argument or path
manifest entry. Information about valid JVM artifacts can be found at
https://github.com/cloudfoundry/java-buildpack#additional-documentation.
Staging failed: Buildpack compilation step failed

I am planning to run JMeter in a headless mode (so will not use the GUI)
and pointing to the test plans. I am pushing both JMeter scripts, configs,
jars along with testplans.

I was thinking about the crashing issue (when the scripts are executed). I
was thinking that there might be a way around this. I wanted to get the
first step forward before tackling this issue.

My initial plan was to use Blazemeter service, however we have
connectivity issues hence the desire to run the JMeter within the CF
environment.

I will explore the "extending" section that you have referenced.


Manglu Balasubramanian
 

Dan,

Sincere thanks.
This worked very well.
Special thanks for the detailed comments on how things work (with special mention towards - JBP_CONFIG_JAVA_MAIN)

Appreciate your help

Thanks
Manglu