Java Buildpack: Stale Files Outside of WEB-INF


Daniel Jones
 

Hi all,

If one pushes a .war containing files outside of WEB-INF, deletes the app,
then pushes a new version where those files outside of WEB-INF have been
changed, these changes are not reflected in the running app. Is this
expected behaviour? The buildpack is forked, from commit 71ab82a
<https://github.com/cloudfoundry/java-buildpack/commit/71ab82af4cd95465dcbc35eb89f3ac9dc9431e9a>
.

I was just about to go about creating a repeatable test case and raising a
GitHub issue, but figured it'd be best to check that this isn't the way
it's *supposed* to work.

For the record I realise that resources should be in WEB-INF by convention,
but the users where I am are expecting feature parity with non-CF Tomcat
which unpacks the whole .war.

Many thanks in advance,

Daniel Jones
EngineerBetter.com


Daniel Mikusa
 

On Thu, Jun 11, 2015 at 7:42 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

If one pushes a .war containing files outside of WEB-INF, deletes the app,
then pushes a new version where those files outside of WEB-INF have been
changed, these changes are not reflected in the running app. Is this
expected behaviour? The buildpack is forked, from commit 71ab82a
<https://github.com/cloudfoundry/java-buildpack/commit/71ab82af4cd95465dcbc35eb89f3ac9dc9431e9a>
.
Seems odd. Every time you stage an app, you should get a copy of the
latest app bits (i.e. what was pushed up by cf). If you're not getting the
latest set of files, I don't think that would be a build pack problem. The
DEA should provide the build pack with the latest set of your files, if
it's not I don't think there's much the build pack can do about it.


I was just about to go about creating a repeatable test case and raising a
GitHub issue, but figured it'd be best to check that this isn't the way
it's *supposed* to work.
A test case would be a good idea. You could post it here, I'd be happy to
play around with it. Steps to repro would also be helpful.

Dan



For the record I realise that resources should be in WEB-INF by
convention, but the users where I am are expecting feature parity with
non-CF Tomcat which unpacks the whole .war.

Many thanks in advance,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


Daniel Jones
 

Hi Dan,

Thanks for getting back to me.

Sadly I can't reproduce the issue on my local Cloud Foundry instance, so I
suspect the issue has something to do with the state of the client's CF's
blob store, their version of CF, or some peculiarity of the .war.

The behaviour I'm seeing is quite odd.

There is a subdirectory called "conf" in the .war file. If I make any
changes inside this directory, they are not reflected after an app push,
even if I delete the app beforehand. If I make changes anywhere else in the
.war, the changes are take effect. If I rename "conf" to "conf-new", the
change *is* effected.

I've tried this on two completely separate Cloud Foundry instances (PCF
1.3.3) and get the same result. Both have 'seen' older versions of the war.

CF_TRACE shows that in the resource matching request all 12,000-odd files
are submitted. The response is significantly smaller, and does not include
any of the files from the "conf" directory.

Any suggestions on where to look for more clues?

On Thu, Jun 11, 2015 at 2:20 PM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Thu, Jun 11, 2015 at 7:42 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

If one pushes a .war containing files outside of WEB-INF, deletes the
app, then pushes a new version where those files outside of WEB-INF have
been changed, these changes are not reflected in the running app. Is this
expected behaviour? The buildpack is forked, from commit 71ab82a
<https://github.com/cloudfoundry/java-buildpack/commit/71ab82af4cd95465dcbc35eb89f3ac9dc9431e9a>
.
Seems odd. Every time you stage an app, you should get a copy of the
latest app bits (i.e. what was pushed up by cf). If you're not getting the
latest set of files, I don't think that would be a build pack problem. The
DEA should provide the build pack with the latest set of your files, if
it's not I don't think there's much the build pack can do about it.


I was just about to go about creating a repeatable test case and raising
a GitHub issue, but figured it'd be best to check that this isn't the way
it's *supposed* to work.
A test case would be a good idea. You could post it here, I'd be happy to
play around with it. Steps to repro would also be helpful.

Dan



For the record I realise that resources should be in WEB-INF by
convention, but the users where I am are expecting feature parity with
non-CF Tomcat which unpacks the whole .war.

Many thanks in advance,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

--
Regards,

Daniel Jones
EngineerBetter.com


Daniel Mikusa
 

On Fri, Jun 12, 2015 at 10:02 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi Dan,

Thanks for getting back to me.

Sadly I can't reproduce the issue on my local Cloud Foundry instance, so I
suspect the issue has something to do with the state of the client's CF's
blob store, their version of CF, or some peculiarity of the .war.

The behaviour I'm seeing is quite odd.

There is a subdirectory called "conf" in the .war file.
Ok. I don't think that should cause a problem.


If I make any changes inside this directory, they are not reflected after
an app push,
That's weird, but might happen. Some thoughts on how to see what's
happening. Apologizes in advance as some of this might be pretty basic.

- check that the WAR file generated includes those changes
- check that you're pointing to the right WAR file, i.e. that `-p` or
path in the manifest is pointing to the right location
- check that cf is not skipping the files for some reason (resource
matching). It shouldn't be an issue with files under 65k, which are always
pushed. In the CF_TRACE output, you should see the hashes of the files
that are sent to the server. I'd suggest hashing the file manually and
making sure it matches. Just to confirm everything is OK there.
- check the build pack that you're using. If it's been customized, make
sure it's not doing anything with that directory, like perhaps trying to
dynamically modify the config or something that could overwrite your local
changes.
- Modify the build pack so that the first thing it does is to hash the
files in conf and prints the hashes to the screen. That would let you see
the hash of the file that's given to the build pack and help to narrow down
where that change might happen.


even if I delete the app beforehand.
If you delete the app, that should delete any state associated with it.
The next push should include the most recent set of files, as CF wouldn't
have any other files for the app. Since it seems like this does not help,
it makes me thing that something else is modifying or overwriting your
files. Perhaps in the build pack or when the app starts.


If I make changes anywhere else in the .war, the changes are take effect.
If I rename "conf" to "conf-new", the change *is* effected.
I don't think that the official JBP would do anything with a "conf"
directory. It stores it's internal stuff in a hidden folder called
`.java-buildpack/`. It is suspicious that it's a "conf" directory though.
Perhaps something in a custom build pack or in the app is overwriting
this? Perhaps to auto configure it to use a bound service or something
like that?



I've tried this on two completely separate Cloud Foundry instances (PCF
1.3.3) and get the same result. Both have 'seen' older versions of the war.
That could be possible, if it's something in the app or build pack that's
overwriting that conf directory or it's contents.



CF_TRACE shows that in the resource matching request all 12,000-odd files
are submitted. The response is significantly smaller, and does not include
any of the files from the "conf" directory.
That sounds OK. I believe that returns anything that matches and is larger
than the defined threshold, which defaults to 65k.



Any suggestions on where to look for more clues?
See above. If that doesn't work, open a support ticket w/Pivotal Support
since this is happening on PCF and we can do some interactive debugging.

Dan





On Thu, Jun 11, 2015 at 2:20 PM, Daniel Mikusa <dmikusa(a)pivotal.io> wrote:

On Thu, Jun 11, 2015 at 7:42 AM, Daniel Jones <
daniel.jones(a)engineerbetter.com> wrote:

Hi all,

If one pushes a .war containing files outside of WEB-INF, deletes the
app, then pushes a new version where those files outside of WEB-INF have
been changed, these changes are not reflected in the running app. Is this
expected behaviour? The buildpack is forked, from commit 71ab82a
<https://github.com/cloudfoundry/java-buildpack/commit/71ab82af4cd95465dcbc35eb89f3ac9dc9431e9a>
.
Seems odd. Every time you stage an app, you should get a copy of the
latest app bits (i.e. what was pushed up by cf). If you're not getting the
latest set of files, I don't think that would be a build pack problem. The
DEA should provide the build pack with the latest set of your files, if
it's not I don't think there's much the build pack can do about it.


I was just about to go about creating a repeatable test case and raising
a GitHub issue, but figured it'd be best to check that this isn't the way
it's *supposed* to work.
A test case would be a good idea. You could post it here, I'd be happy
to play around with it. Steps to repro would also be helpful.

Dan



For the record I realise that resources should be in WEB-INF by
convention, but the users where I am are expecting feature parity with
non-CF Tomcat which unpacks the whole .war.

Many thanks in advance,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Regards,

Daniel Jones
EngineerBetter.com

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev