Re: OpenSSL v0.9.8 in CF Ruby Buildpack 1.4


Daya Sharma <dsharma@...>
 

Hi Mike,

May be the following will help you reproduce the problem without having to
setup Jenkins

But to give you little more context. We have a Jenkins build slave which is
container instantiated from the stem cell
bosh-stemcell-2776-warden-boshlite-ubuntu-trusty-go_agent.tgz image with
ruby-buildpack
<https://github.com/cloudfoundry/ruby-buildpack/releases/tag/v1.4.1> . On
that slave a build is configured to run the unit tests specified in this
Ruby applicationhttps://github.com/daya/pong_matcher_rails

#1 ---> clone git repo https://github.com/daya/pong_matcher_rails.git

export PATH=$PATH:/var/vcap/packages/buildpacks/bin/:

echo "Ruby Version ---> $(ruby -v)"

cat << EOS > testing_openssl.rb
require 'net/http'
require 'uri'
uri = URI('https://www.google.com/')

Net::HTTP.start(uri.host, uri.port,
:use_ssl => uri.scheme == 'https') do |http|
request = Net::HTTP::Get.new uri
puts "port #{uri.port} host #{uri.host} before use_ssl? #{http.use_ssl?}"
response = http.request request # Net::HTTPResponse object
end

uri = URI.parse("https://www.google.com")
req = Net::HTTP::Get.new("/")
http = Net::HTTP.new(uri.host,uri.port)
puts "port #{uri.port} host #{uri.host} before use_ssl? #{http.use_ssl?}"
http.use_ssl=true
puts "after #{http.use_ssl?}"
resp = http.request(req)
puts resp.code

puts "OpenSSL::OPENSSL_VERSION = #{OpenSSL::OPENSSL_VERSION}
#{defined?(OpenSSL::SSL)} OpenSSL::SSL #{OpenSSL::SSL} "

EOS

ruby testing_openssl.rb

echo "sourcing .... cf_buildpack"

. cf_buildpack

echo "----> installing gem bundle"

bundle install --jobs=3 --retry=3

echo "----> installed gem bundle running tests"

bundle exec rake test:units

following is the output of the above build configuration script


Started by user admin
Building remotely on 10.244.2.6 (built-in) in workspace
/var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test
> /var/vcap/packages/git/bin/git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> /var/vcap/packages/git/bin/git config remote.origin.url
https://github.com/daya/pong_matcher_rails.git # timeout=10
Fetching upstream changes from https://github.com/daya/pong_matcher_rails.git
> /var/vcap/packages/git/bin/git --version # timeout=10
> /var/vcap/packages/git/bin/git fetch --tags --progress
https://github.com/daya/pong_matcher_rails.git
+refs/heads/*:refs/remotes/origin/*
> /var/vcap/packages/git/bin/git rev-parse
refs/remotes/origin/master^{commit} # timeout=10
> /var/vcap/packages/git/bin/git rev-parse
refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 826c37d3f3e5bf8920c4332e8f9bd62b80eae25b
(refs/remotes/origin/master)
> /var/vcap/packages/git/bin/git config core.sparsecheckout # timeout=10
> /var/vcap/packages/git/bin/git checkout -f
826c37d3f3e5bf8920c4332e8f9bd62b80eae25b
> /var/vcap/packages/git/bin/git rev-list
826c37d3f3e5bf8920c4332e8f9bd62b80eae25b # timeout=10
[ruby-buildpack-test] $ /bin/sh -xe
/var/vcap/data/tmp/hudson6623649704041168706.sh
+ export PATH=/var/vcap/packages/openjdk-8/bin:/var/vcap/packages/buildpacks/nodejs/bin:/var/vcap/packages/cloudfoundry-cli/bin:/var/vcap/packages/git/bin:/var/vcap/packages/maven/bin:/var/vcap/packages/ruby/bin:/var/vcap/packages/openjdk-8/bin:/var/vcap/packages/buildpacks/nodejs/bin:/var/vcap/packages/cloudfoundry-cli/bin:/var/vcap/packages/git/bin:/var/vcap/packages/maven/bin:/var/vcap/packages/ruby/bin:/var/vcap/packages/openjdk-8/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/var/vcap/packages/buildpacks/bin/:
+ ruby -v
+ echo Ruby Version ---> ruby 2.2.2p95 (2015-04-13 revision 50295)
[x86_64-linux]
Ruby Version ---> ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
+ cat
+ ruby testing_openssl.rb
port 443 host www.google.com before use_ssl? true
port 443 host www.google.com before use_ssl? false
after true
200
OpenSSL::OPENSSL_VERSION = OpenSSL 1.0.1f 6 Jan 2014 constant
OpenSSL::SSL OpenSSL::SSL
+ echo sourcing .... cf_buildpack
sourcing .... cf_buildpack
+ . cf_buildpack
+ set +xe
-------> Buildpack version 1.4.1
ln: failed to create symbolic link
‘/var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/ld_library_path/libreadline.so.5’:
File exists
-----> Compiling Ruby/Rails
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using 1.7.12
Running: bundle install --without none --path vendor/bundle
--binstubs vendor/bundle/bin -j4 --deployment
Could not load OpenSSL.
You must recompile Ruby with OpenSSL support or change the
sources in your
Gemfile from 'https' to 'http'. Instructions for compiling with
OpenSSL using
RVM are available at http://rvm.io/packages/openssl.
Bundler Output:
Could not load OpenSSL.
You must recompile Ruby with OpenSSL support or change the
sources in your
Gemfile from 'https' to 'http'. Instructions for compiling with
OpenSSL using
RVM are available at http://rvm.io/packages/openssl.
!
! Failed to install gems via Bundler.
!
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Debugging this further reveals the problem of OpenSSL version mismatch
between ruby-buildpacks (even the latest 1.4.1) and stem cell image. *If I
do install OpenSSL version 0.9.8, on the container slave machine then
everything works fine, even though the container already has higher version
OpenSSL 1.0.1f.*

*Seems like we may have to resort to packaging the blob of OpenSSL 0.9.8
which is what the ruby-buildpack 1.4.1 is expecting. What do you think?*

Thanks,

Daya Sharma


On Tue, May 12, 2015 at 3:55 PM, Mike Dalessio <mdalessio(a)pivotal.io> wrote:

Hi Daya,

I can't reproduce what you're seeing in a rootfs container -- I reproduced
your Gemfile and the app deploys without error. I'm assuming this means
that I'm not accurately reproducing what you're doing.

Can you help me reproduce this without having to set up Jenkins? It still
sounds to me like you're running these processes on a stemcell, and not in
a container rootfs. If that's the case, you probably shouldn't be using the
buildpack.

-m


On Tue, May 12, 2015 at 1:24 PM, Daya Sharma <dsharma(a)cloudbees.com>
wrote:

Hi Mike,

We are running this ruby app on the container rootfs not on bare VM.

Our use case is built around our need to run Jenkins
<http://jenkins-ci.org/> Master and Slaves on different containers. The
build job is configured on Jenkins Master, and the build is then run on
Jenkins Slave. In this specific case the build job is configured for a MRI
Ruby project, the first step of that build job is to download the
dependencies using ```bundle install``` from https://rubygems.org
followed by running automated tests

CF Ruby Build Pack is used so that end user configuring the build job
does not have to install Ruby platform on the build slaves to run their
ruby web app and tests against it.


Please let me know if further explanation is needed.

Thanks,
-daya

On Tue, May 12, 2015 at 3:05 PM, Mike Dalessio <mdalessio(a)pivotal.io>
wrote:

OK, that makes sense.

It sounds like you're installing the web app on the bare VM, and not in
a container rootfs. This isn't really a supported use case for the
Buildpack, which is going to make assumptions that you're running in either
the `lucid64` or `cflinuxfs` rootfs.

Can you explain a bit more about what you're trying to do, and why
you're using the ruby buildpack?



On Tue, May 12, 2015 at 1:03 PM, Daya Sharma <dsharma(a)cloudbees.com>
wrote:

Hi Mike,

the line

```gem 'jruby-openssl', :platforms => :jruby ```

means jruby-openssl gem will be downloaded by bundler if and only if
JRuby platform is detected, I am using MRI not JRuby.

The OpenSSL error happens even before any dependency is downloaded or
resolved, it happens because Gemfile has this line

```source 'https://rubygems.org'``` to fulfill https needs system
tries to load OpenSSL 0.9.8 library which is not found on the virtual
machine.

To further clarify the problem if I install OpenSSL 0.9.8 on the
virtual machine the problem disappears even though a higher version of
OpenSSL 1.0.1f is already installed as shown in the log output.

-daya




On Tue, May 12, 2015 at 2:45 PM, Mike Dalessio <mdalessio(a)pivotal.io>
wrote:

Hi Daya,

I'm a bit confused, as it looks from your Gemfile like you're trying
to load a JRuby openssl gem while running MRI ruby.

What happens if you remove your dependency on the `jruby-openssl` gem?

On Tue, May 12, 2015 at 12:24 PM, Daya Sharma <dsharma(a)cloudbees.com>
wrote:

testing_openssl.rb mentioned previously is here
https://gist.github.com/daya/71a589f9b83f9ff57608

CloudFoundry Ruby Build Pack 1.4
https://github.com/cloudfoundry/ruby-buildpack/releases/tag/v1.4.0

On Tue, May 12, 2015 at 1:32 AM, Daya Sharma <dsharma(a)cloudbees.com>
wrote:

Hi

There is a OpenSSL version/dependency mismatch as far as Ruby
BuildPack and Stemcells are concerned

Our bosh-lite environment is

*StemCell*:

Name: bosh-warden-boshlite-ubuntu-trusty-go_agent
Version: 2776
CID: 3241e75c-e0ae-4c81-4c3e-69ef5545dcfe
OpenSSL: 1.0.1f

*CF Ruby BuildPack*:

Version
​:​
1.4
OpenSSL: 0.9.8

*Gemfile*: here
<https://github.com/daya/active_merchant/blob/master/Gemfile>
refers to Ruby 2.2.2

*Problem*: When sourcing cf_buildpack followed by bundle install
with https://rubygems.orginvokes Ruby version 2.2.2 which is built
against OpenSSL version 0.9.8 but vm based on above mentioned StemCell has
OpenSSL version 1.0.0 causing *ruby app failures*.

Following are the relevant outputs from a jenkins CI job


-------> Buildpack version 1.4.0
-----> Compiling Ruby
-----> Using Ruby version: ruby-2.2.2
-----> Installing dependencies using 1.7.12
Running: bundle install --without none --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
Could not load OpenSSL.
You must recompile Ruby with OpenSSL support or change the sources in your
Gemfile from 'https' to 'http'. Instructions for compiling with OpenSSL using
RVM are available at http://rvm.io/packages/openssl.
Bundler Output:
Could not load OpenSSL.
You must recompile Ruby with OpenSSL support or change the sources in your
Gemfile from 'https' to 'http'. Instructions for compiling with OpenSSL using
RVM are available at http://rvm.io/packages/openssl.

Steps to Reproduce:

Using file testing_openssl.rb

On VM


root(a)21d7bdc5-cbd5-4cdb-80cc-25c654549d3e:~# /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/bin/ruby -v
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]
root(a)21d7bdc5-cbd5-4cdb-80cc-25c654549d3e:~# /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/bin/ruby -W2 /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/testing_openssl.rb
/var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/testing_openssl.rb:9: warning: assigned but unused variable - response
/var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require': libssl.so.0.9.8: cannot open shared object file: No such file or directory - /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/lib/ruby/2.2.0/x86_64-linux/openssl.so (LoadError)
from /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/lib/ruby/2.2.0/openssl.rb:17:in `<top (required)>'
from /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/vendor/ruby-2.2.2/lib/ruby/2.2.0/net/http.rb:574:in `start'
from /var/vcap/store/jenkins_slave/workspace/ruby-buildpack-test/testing_openssl.rb:5:in `<main>'

root(a)21d7bdc5-cbd5-4cdb-80cc-25c654549d3e:~# apt-cache showpkg openssl
Package: openssl
Versions:
1.0.1f-1ubuntu2.7

Any ideas on how to best solve this problem.

Thanks,

-daya


_______________________________________________
cf-bosh mailing list
cf-bosh(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-bosh

Join cf-bosh@lists.cloudfoundry.org to automatically receive all group messages.