Issues with offline ruby_buildpack, disconnected CF, and gems from Git


Josh Ghiloni
 

Thanks for the response Mike. I'll take a look this afternoon when I get a chance and get back to you.

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ECSTeam.com<mailto:rgarrett(a)ECSTeam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://www.ecsteam.com/> <http://www.ecsteam.com/>


________________________________
From: cf-dev-bounces(a)lists.cloudfoundry.org <cf-dev-bounces(a)lists.cloudfoundry.org> on behalf of Mike Dalessio <mdalessio(a)pivotal.io>
Sent: Friday, June 26, 2015 10:14 AM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: Re: [cf-dev] Issues with offline ruby_buildpack, disconnected CF, and gems from Git

Hi Josh,

I can't reproduce what you're describing, so in order to diagnose what's going on, we'll probably need more information.

Here's what I did to attempt to reproduce, so that you can compare what you're doing:

- I'm pushing the app at https://github.com/cloudfoundry/ruby-buildpack/tree/master/cf_spec/fixtures/sinatra_web_app

- I ran `bundle package` locally, and here's what the `vendor` directory looks like:

```
vendor/
└── bundle
├── bin
│ ├── rackup
│ └── tilt
└── ruby
└── 2.2.0
├── bin
│ ├── rackup
│ └── tilt
├── build_info
├── cache
│ ├── rack-1.5.2.gem
│ ├── rack-protection-1.5.2.gem
│ ├── sinatra-1.4.4.gem
│ └── tilt-1.4.1.gem
├── doc
├── extensions
├── gems
│ ├── rack-1.5.2
│ │ ├── COPYING
│ │ ├── KNOWN-ISSUES
│ │ ├── README.rdoc
│ │ ├── Rakefile
│ │ ├── SPEC
...
│ ├── sinatra-1.4.4
│ │ ├── AUTHORS
│ │ ├── CHANGES
│ │ ├── Gemfile
│ │ ├── LICENSE
│ │ ├── README.de.md<http://README.de.md>
...
└── specifications
├── rack-1.5.2.gemspec
├── rack-protection-1.5.2.gemspec
├── sinatra-1.4.4.gemspec
└── tilt-1.4.1.gemspec

70 directories, 410 files
```

- I made sure I didn't have a `.cfignore` file, which might prevent the `vendor` subdirectory from being uploaded on `cf push`. (You can read more about this lesser-known feature here: http://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#exclude)

- I pushed the app with ruby-buildpack v1.3.0, and amongst the log output I see:

```
-------> Buildpack version 1.3.0
-----> Compiling Ruby/Rack
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using 1.7.12
Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
Using bundler 1.7.12
Installing tilt 1.4.1
Installing rack 1.5.2
Installing rack-protection 1.5.2
Installing sinatra 1.4.4
Your bundle is complete!
Gems in the groups development and test were not installed.
It was installed into ./vendor/bundle
Bundle completed (0.54s)
Cleaning up the bundler cache.
```

Note that the `bundle` command is displayed. This command does not require any network connectivity if all the gems are in your vendor directory. You can test this yourself by disconnecting your dev machine from the network and running it:

```
$ bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
Using bundler 1.9.4
Installing tilt 1.4.1
Installing rack 1.5.2
Installing rack-protection 1.5.2
Installing sinatra 1.4.4
Bundle complete! 1 Gemfile dependency, 5 gems now installed.
Gems in the groups development and test were not installed.
Bundled gems are installed into ./vendor/bundle.
```


So, what are we doing differently?

-mike




On Thu, Jun 25, 2015 at 11:50 AM, Josh Ghiloni <jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>> wrote:
Hi all,

I’m currently working on a project where I have my gem source in our local git enterprise server, and I need to push an app that uses it to an instance of CF (v208) that is disconnected from the internet. The app has in its Gemfile a reference to that gem like this:

```
gem 'mygem', :git => 'https://github.myco.com/common/mygem.git'
```

When I run

```
bundle install
bundle package
```

That gem never shows up in my `vendor/cache` directory. However, if I use `bundle package --all`, it shows up in my `vendor/cache` directory as `vendor/cache/mygem-0.0.1-d1c2237f1ebca`. Unfortunately, when I `cf push`, the result is the same in both cases: my ruby app fails to find the require. I’m using the ruby buildpack v1.3, for what it’s worth. Also of note, this does run locally.

Is there something I can do to git ruby to recognize that this gem exists? I’m still rather new to Ruby and don’t fully understand the intricacies.

Josh Ghiloni
Senior Consultant
303.932.2202<tel:303.932.2202> o | 303.590.5427<tel:303.590.5427> m | 303.565.2794<tel:303.565.2794> f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ECSTeam.com>






_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Mike Dalessio
 

Hi Josh,

I can't reproduce what you're describing, so in order to diagnose what's
going on, we'll probably need more information.

Here's what I did to attempt to reproduce, so that you can compare what
you're doing:

- I'm pushing the app at
https://github.com/cloudfoundry/ruby-buildpack/tree/master/cf_spec/fixtures/sinatra_web_app

- I ran `bundle package` locally, and here's what the `vendor` directory
looks like:

```

vendor/
└── bundle
├── bin
│ ├── rackup
│ └── tilt
└── ruby
└── 2.2.0
├── bin
│ ├── rackup
│ └── tilt
├── build_info
├── cache
│ ├── rack-1.5.2.gem
│ ├── rack-protection-1.5.2.gem
│ ├── sinatra-1.4.4.gem
│ └── tilt-1.4.1.gem
├── doc
├── extensions
├── gems
│ ├── rack-1.5.2
│ │ ├── COPYING
│ │ ├── KNOWN-ISSUES
│ │ ├── README.rdoc
│ │ ├── Rakefile
│ │ ├── SPEC
...
│ ├── sinatra-1.4.4
│ │ ├── AUTHORS
│ │ ├── CHANGES
│ │ ├── Gemfile
│ │ ├── LICENSE
│ │ ├── README.de.md
...
└── specifications
├── rack-1.5.2.gemspec
├── rack-protection-1.5.2.gemspec
├── sinatra-1.4.4.gemspec
└── tilt-1.4.1.gemspec

70 directories, 410 files

```

- I made sure I didn't have a `.cfignore` file, which might prevent the
`vendor` subdirectory from being uploaded on `cf push`. (You can read more
about this lesser-known feature here:
http://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#exclude
)

- I pushed the app with ruby-buildpack v1.3.0, and amongst the log output I
see:

```

-------> Buildpack version 1.3.0
-----> Compiling Ruby/Rack
-----> Using Ruby version: ruby-2.0.0
-----> Installing dependencies using 1.7.12
Running: bundle install --without development:test --path
vendor/bundle --binstubs vendor/bundle/bin -j4 --deployment
Using bundler 1.7.12
Installing tilt 1.4.1
Installing rack 1.5.2
Installing rack-protection 1.5.2
Installing sinatra 1.4.4
Your bundle is complete!
Gems in the groups development and test were not installed.
It was installed into ./vendor/bundle
Bundle completed (0.54s)
Cleaning up the bundler cache.

```

Note that the `bundle` command is displayed. This command does not require
any network connectivity if all the gems are in your vendor directory. You
can test this yourself by disconnecting your dev machine from the network
and running it:

```

$ bundle install --without development:test --path vendor/bundle --binstubs
vendor/bundle/bin -j4 --deployment
Using bundler 1.9.4
Installing tilt 1.4.1
Installing rack 1.5.2
Installing rack-protection 1.5.2
Installing sinatra 1.4.4
Bundle complete! 1 Gemfile dependency, 5 gems now installed.
Gems in the groups development and test were not installed.
Bundled gems are installed into ./vendor/bundle.

```


So, what are we doing differently?

-mike

On Thu, Jun 25, 2015 at 11:50 AM, Josh Ghiloni <jghiloni(a)ecsteam.com> wrote:

Hi all,

I’m currently working on a project where I have my gem source in our
local git enterprise server, and I need to push an app that uses it to an
instance of CF (v208) that is disconnected from the internet. The app has
in its Gemfile a reference to that gem like this:

```
gem 'mygem', :git => 'https://github.myco.com/common/mygem.git'
```

When I run

```
bundle install
bundle package
```

That gem never shows up in my `vendor/cache` directory. However, if I
use `bundle package --all`, it shows up in my `vendor/cache` directory as
`vendor/cache/mygem-0.0.1-d1c2237f1ebca`. Unfortunately, when I `cf push`,
the result is the same in both cases: my ruby app fails to find the
require. I’m using the ruby buildpack v1.3, for what it’s worth. Also of
note, this does run locally.

Is there something I can do to git ruby to recognize that this gem
exists? I’m still rather new to Ruby and don’t fully understand the
intricacies.

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com

ECS Team
Technology Solutions Delivered
ECSTeam.com






_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Josh Ghiloni
 

Hi all,

I’m currently working on a project where I have my gem source in our local git enterprise server, and I need to push an app that uses it to an instance of CF (v208) that is disconnected from the internet. The app has in its Gemfile a reference to that gem like this:

```
gem 'mygem', :git => 'https://github.myco.com/common/mygem.git'
```

When I run

```
bundle install
bundle package
```

That gem never shows up in my `vendor/cache` directory. However, if I use `bundle package --all`, it shows up in my `vendor/cache` directory as `vendor/cache/mygem-0.0.1-d1c2237f1ebca`. Unfortunately, when I `cf push`, the result is the same in both cases: my ruby app fails to find the require. I’m using the ruby buildpack v1.3, for what it’s worth. Also of note, this does run locally.

Is there something I can do to git ruby to recognize that this gem exists? I’m still rather new to Ruby and don’t fully understand the intricacies.

Josh Ghiloni
Senior Consultant
303.932.2202 o | 303.590.5427 m | 303.565.2794 f
jghiloni(a)ecsteam.com<mailto:jghiloni(a)ecsteam.com>

ECS Team
Technology Solutions Delivered
ECSTeam.com<http://ECSTeam.com>