Date   

Re: Exposing Multiple Ports from an Application

Shannon Coen
 

On Wed, Jun 8, 2016 at 5:03 AM, Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi Shannon,

Thanks for the reply.

On Wed, Jun 8, 2016 at 4:36 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

If I understand you correctly, you would like your application to receive
requests on multiple ports. Is that correct? Does it matter what ports your
application clients send requests to? Could you please share the details of
your use cases for this functionality?
The usecase is that an application listening to multiple types of traffic
(http, tcp, binary, etc.) from multiple ports exposed in the container.
Therefore I guess we would need to have multiple ports open from the host
side as well.
If your application receives http traffic on more than one port, you would
not need another port opened, only another route. For non-HTTP traffic, you
would need a TCP route (with a reserved port) for each app port. The route
ports will not match your application ports. Does this sound like it would
fulfill your needs?



Currently HTTP requests may only be sent to 80 or 443. Your app may
receive requests on one port only. On DEAs, this port is randomized and can
be discovered from the $PORT env var. On Diego, this env var is also
present but is always 8080.

This isn't a limitation of Diego; Diego will open whatever container
ports a client tells it to. However, we have not yet added the business
logic to Cloud Controller to expose management of these ports to
application developers.

You discovered that two ports are already opened on the container. The
second port is opened automatically in support of the feature that enables
a developer to SSH into the container. The env var you identified, CF_INSTANCE_PORTS,
is a list of all ports opened on the container. This env var should be
considered an internal implementation detail as it contains ports used by
internal system components to which an app should not bind. The env var is
documented as there are a few use cases where an app needs to know what the
host port is.

We do plan to enable applications to listen on multiple ports. Clients
will still make HTTP requests to 80 or 443, but a developer will be able to
specify the application port when mapping a route to an application.

E.g.
myapp.cf.com -> myapp listening on 8080
myapp.cf.com/admin -> myapp listening on 6000

I'm currently drafting a proposal for this feature and will share it soon
for feedback.
This is indeed great news!


If your clients must call your application on a port besides 80 or 443, I
would recommend exploring our support for TCP routing (see the CLI help
file for create-route). Developers can reserve non-standard ports for a
route, like tcp.cf.com:6000, and CF will route requests for it to your
application port on 8080. After we deliver support for custom app ports, as
described above, you would be able to specify both the route port and the
app port.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jun 7, 2016 at 1:18 AM, Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi all,

Is it possible to expose two ports of an Application via from CF to
external traffic, via an externally routable address and multiple ports?

While going through the documentation, came across [1], which suggests
that requests are served from port 80 and 443 only. However, while trying
out the sample spring application [2] on CF (using the hosted CF in pivotal
web services), I could actually see that there are multiple port mappings
defined by invoking the rest API [3]. Additionally, in pivotal.io docs,
I see the definition of CF-INSTANCE-PORTS, which seems to support multiple
port mappings from host to container.

I'm curios to know if exposing multiple ports from the host and
container side is possible.

Thanks in advance.

[1].
https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#ports

[2]. https://github.com/cloudfoundry-samples/spring-music

[3].
{
"0": {
"state": "RUNNING",
"stats": {
"name": "spring-music",
"uris": [
"spring-music-moonlit-melioration.cfapps.io"
],
"host": "10.10.115.53",
"port": 64248,
"net_info": {
"address": "10.10.115.53",
* "ports": [*
* {*
* "container_port": 8080,*
* "host_port": 64248*
* },*
* {*
* "container_port": 2222,*
* "host_port": 64249*
* }*
* ]*
},
"uptime": 9484,
"mem_quota": 536870912,
"disk_quota": 1073741824,
"fds_quota": 16384,
"usage": {
"time": "2016-05-20T11:12:38.944471672Z",
"cpu": 0.0010850797743361504,
"mem": 460423168,
"disk": 166739968
}
}
}
}

[4].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-PORTS


--
Thanks and Regards,
Isuru


Re: Why Openstack: N/A for stemcell versions in v237 release notes?

Dr Nic Williams <drnicwilliams@...>
 

A fine and worthy shameless plug. Well played!

On Thu, Jun 9, 2016 at 4:28 AM +1000, "Amit Gupta" <agupta(a)pivotal.io> wrote:










Hi Tom,
Apologies for having had several releases with "N/A" under the OpenStack stemcell.  We had a very lengthy process of obtaining a working OpenStack environment from our provider.  While waiting for the environment, the Release Integration team picked up a track of work on adding Diego to our pipelines that we're still in the middle of, and we've also had several high urgency security issues that have pre-empted other work in our backlog.  The Diego and security tracks are wrapping up, and rebooting our OpenStack environment is one of our next two tracks of work in our roadmap [0].  At our current team strength we are able to parallelize two tracks of work, so we will be getting to this soon.
I'll use this as an opportunity for a shameless plug.  If you're interested in how Cloud Foundry is integrated, released, and deployed, I welcome you to dojo [1] with Pivotal on the Release Integration team.  This team is responsible for the pipelines which fully certify and ship the Cloud Foundry release, and are building the BOSH 2.0 manifests for Cloud Foundry for the community to use.
[0] https://www.pivotaltracker.com/n/projects/1382120[1] https://www.cloudfoundry.org/community/contribute/dojos/
Cheers,Amit

On Wed, Jun 8, 2016 at 6:05 AM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:
AWS: light-bosh-stemcell-3232.3-aws-xen-hvm-ubuntu-trusty-go_agent

vSphere: bosh-stemcell-3232.3-vsphere-esxi-ubuntu-trusty-go_agent

OpenStack: N/A

BOSH-Lite: bosh-stemcell-3147-warden-boshlite-ubuntu-trusty-go_agent



What's up with the N/A?



Thanks,

Tom


Re: Why Openstack: N/A for stemcell versions in v237 release notes?

Amit Kumar Gupta
 

Hi Tom,

Apologies for having had several releases with "N/A" under the OpenStack
stemcell. We had a very lengthy process of obtaining a working OpenStack
environment from our provider. While waiting for the environment, the
Release Integration team picked up a track of work on adding Diego to our
pipelines that we're still in the middle of, and we've also had several
high urgency security issues that have pre-empted other work in our
backlog. The Diego and security tracks are wrapping up, and rebooting our
OpenStack environment is one of our next two tracks of work in our roadmap [
0 <https://www.pivotaltracker.com/n/projects/1382120>]. At our current
team strength we are able to parallelize two tracks of work, so we will be
getting to this soon.

I'll use this as an opportunity for a shameless plug. If you're interested
in how Cloud Foundry is integrated, released, and deployed, I welcome you
to dojo [1 <https://www.cloudfoundry.org/community/contribute/dojos/>] with
Pivotal on the Release Integration team. This team is responsible for the
pipelines which fully certify and ship the Cloud Foundry release, and are
building the BOSH 2.0 manifests for Cloud Foundry for the community to use.

[0] https://www.pivotaltracker.com/n/projects/1382120
[1] https://www.cloudfoundry.org/community/contribute/dojos/

Cheers,
Amit

On Wed, Jun 8, 2016 at 6:05 AM, Tom Sherrod <tom.sherrod(a)gmail.com> wrote:

AWS: light-bosh-stemcell-3232.3-aws-xen-hvm-ubuntu-trusty-go_agent
vSphere: bosh-stemcell-3232.3-vsphere-esxi-ubuntu-trusty-go_agent
OpenStack: N/A
BOSH-Lite: bosh-stemcell-3147-warden-boshlite-ubuntu-trusty-go_agent

What's up with the N/A?

Thanks,
Tom


Why Openstack: N/A for stemcell versions in v237 release notes?

Tom Sherrod <tom.sherrod@...>
 

AWS: light-bosh-stemcell-3232.3-aws-xen-hvm-ubuntu-trusty-go_agent
vSphere: bosh-stemcell-3232.3-vsphere-esxi-ubuntu-trusty-go_agent
OpenStack: N/A
BOSH-Lite: bosh-stemcell-3147-warden-boshlite-ubuntu-trusty-go_agent

What's up with the N/A?

Thanks,
Tom


Re: Exposing Multiple Ports from an Application

Isuru Haththotuwa <isurulucky@...>
 

Hi Shannon,

Thanks for the reply.

On Wed, Jun 8, 2016 at 4:36 AM, Shannon Coen <scoen(a)pivotal.io> wrote:

If I understand you correctly, you would like your application to receive
requests on multiple ports. Is that correct? Does it matter what ports your
application clients send requests to? Could you please share the details of
your use cases for this functionality?
The usecase is that an application listening to multiple types of traffic
(http, tcp, binary, etc.) from multiple ports exposed in the container.
Therefore I guess we would need to have multiple ports open from the host
side as well.


Currently HTTP requests may only be sent to 80 or 443. Your app may
receive requests on one port only. On DEAs, this port is randomized and can
be discovered from the $PORT env var. On Diego, this env var is also
present but is always 8080.

This isn't a limitation of Diego; Diego will open whatever container ports
a client tells it to. However, we have not yet added the business logic to
Cloud Controller to expose management of these ports to application
developers.

You discovered that two ports are already opened on the container. The
second port is opened automatically in support of the feature that enables
a developer to SSH into the container. The env var you identified, CF_INSTANCE_PORTS,
is a list of all ports opened on the container. This env var should be
considered an internal implementation detail as it contains ports used by
internal system components to which an app should not bind. The env var is
documented as there are a few use cases where an app needs to know what the
host port is.

We do plan to enable applications to listen on multiple ports. Clients
will still make HTTP requests to 80 or 443, but a developer will be able to
specify the application port when mapping a route to an application.

E.g.
myapp.cf.com -> myapp listening on 8080
myapp.cf.com/admin -> myapp listening on 6000

I'm currently drafting a proposal for this feature and will share it soon
for feedback.
This is indeed great news!


If your clients must call your application on a port besides 80 or 443, I
would recommend exploring our support for TCP routing (see the CLI help
file for create-route). Developers can reserve non-standard ports for a
route, like tcp.cf.com:6000, and CF will route requests for it to your
application port on 8080. After we deliver support for custom app ports, as
described above, you would be able to specify both the route port and the
app port.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jun 7, 2016 at 1:18 AM, Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi all,

Is it possible to expose two ports of an Application via from CF to
external traffic, via an externally routable address and multiple ports?

While going through the documentation, came across [1], which suggests
that requests are served from port 80 and 443 only. However, while trying
out the sample spring application [2] on CF (using the hosted CF in pivotal
web services), I could actually see that there are multiple port mappings
defined by invoking the rest API [3]. Additionally, in pivotal.io docs,
I see the definition of CF-INSTANCE-PORTS, which seems to support multiple
port mappings from host to container.

I'm curios to know if exposing multiple ports from the host and container
side is possible.

Thanks in advance.

[1].
https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#ports

[2]. https://github.com/cloudfoundry-samples/spring-music

[3].
{
"0": {
"state": "RUNNING",
"stats": {
"name": "spring-music",
"uris": [
"spring-music-moonlit-melioration.cfapps.io"
],
"host": "10.10.115.53",
"port": 64248,
"net_info": {
"address": "10.10.115.53",
* "ports": [*
* {*
* "container_port": 8080,*
* "host_port": 64248*
* },*
* {*
* "container_port": 2222,*
* "host_port": 64249*
* }*
* ]*
},
"uptime": 9484,
"mem_quota": 536870912,
"disk_quota": 1073741824,
"fds_quota": 16384,
"usage": {
"time": "2016-05-20T11:12:38.944471672Z",
"cpu": 0.0010850797743361504,
"mem": 460423168,
"disk": 166739968
}
}
}
}

[4].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-PORTS


--
Thanks and Regards,
Isuru


Re: Exposing Multiple Ports from an Application

Shannon Coen
 

If I understand you correctly, you would like your application to receive
requests on multiple ports. Is that correct? Does it matter what ports your
application clients send requests to? Could you please share the details of
your use cases for this functionality?

Currently HTTP requests may only be sent to 80 or 443. Your app may receive
requests on one port only. On DEAs, this port is randomized and can be
discovered from the $PORT env var. On Diego, this env var is also present
but is always 8080.

This isn't a limitation of Diego; Diego will open whatever container ports
a client tells it to. However, we have not yet added the business logic to
Cloud Controller to expose management of these ports to application
developers.

You discovered that two ports are already opened on the container. The
second port is opened automatically in support of the feature that enables
a developer to SSH into the container. The env var you identified,
CF_INSTANCE_PORTS,
is a list of all ports opened on the container. This env var should be
considered an internal implementation detail as it contains ports used by
internal system components to which an app should not bind. The env var is
documented as there are a few use cases where an app needs to know what the
host port is.

We do plan to enable applications to listen on multiple ports. Clients will
still make HTTP requests to 80 or 443, but a developer will be able to
specify the application port when mapping a route to an application.

E.g.
myapp.cf.com -> myapp listening on 8080
myapp.cf.com/admin -> myapp listening on 6000

I'm currently drafting a proposal for this feature and will share it soon
for feedback.

If your clients must call your application on a port besides 80 or 443, I
would recommend exploring our support for TCP routing (see the CLI help
file for create-route). Developers can reserve non-standard ports for a
route, like tcp.cf.com:6000, and CF will route requests for it to your
application port on 8080. After we deliver support for custom app ports, as
described above, you would be able to specify both the route port and the
app port.





Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.

On Tue, Jun 7, 2016 at 1:18 AM, Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi all,

Is it possible to expose two ports of an Application via from CF to
external traffic, via an externally routable address and multiple ports?

While going through the documentation, came across [1], which suggests
that requests are served from port 80 and 443 only. However, while trying
out the sample spring application [2] on CF (using the hosted CF in pivotal
web services), I could actually see that there are multiple port mappings
defined by invoking the rest API [3]. Additionally, in pivotal.io docs, I
see the definition of CF-INSTANCE-PORTS, which seems to support multiple
port mappings from host to container.

I'm curios to know if exposing multiple ports from the host and container
side is possible.

Thanks in advance.

[1].
https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#ports

[2]. https://github.com/cloudfoundry-samples/spring-music

[3].
{
"0": {
"state": "RUNNING",
"stats": {
"name": "spring-music",
"uris": [
"spring-music-moonlit-melioration.cfapps.io"
],
"host": "10.10.115.53",
"port": 64248,
"net_info": {
"address": "10.10.115.53",
* "ports": [*
* {*
* "container_port": 8080,*
* "host_port": 64248*
* },*
* {*
* "container_port": 2222,*
* "host_port": 64249*
* }*
* ]*
},
"uptime": 9484,
"mem_quota": 536870912,
"disk_quota": 1073741824,
"fds_quota": 16384,
"usage": {
"time": "2016-05-20T11:12:38.944471672Z",
"cpu": 0.0010850797743361504,
"mem": 460423168,
"disk": 166739968
}
}
}
}

[4].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-PORTS


Re: GraphViz support for rootfs?

john mcteague <john.mcteague@...>
 

Right now, I have little to no control over the rootfs short of rebuilding
alot of the components from source (assuming you aren't using a vendor
version). I have a choice with buildpacks, if I don't want something in my
environment, as an operator I can remove it.

Rootfs bloat vs buildpack complexity is always going to be a tricky
balancing act. Buildpacks as we know them today are too simplistic in
nature. Perhaps some of the recent proposals for how buildpacks could
evolve offer some help in this space.

On 7 Jun 2016 6:10 p.m., "Mike Youngstrom" <youngm(a)gmail.com> wrote:

This is an interesting question/problem. Although I see the value in not
bloating the RootFS it can be nice from a security perspective to know that
when a CVE does come up upgrading the RootFS can fix it for all instead of
relying on application developers to update. Could a bloated RootFS where
security is managed outside the application make a better counter approach
to a Docker solution?

A more bloated RootFS could also help keep buldpack complexity down.

Thoughts?

Mike

On Tue, Jun 7, 2016 at 4:56 AM, john mcteague <john.mcteague(a)gmail.com>
wrote:

I would be inclined to suggest we should be heading in the opposite
direction, stripping libraries out of the rootfs and finding a way to allow
buildpacks to add in required dependencies, thereby reducing the size and
complexity of the rootfs and minimising the number of potential CVE's in
this area.

In addition, something I have spoken to a number of people about in the
past is the presence of compilers in the rootfs which some regulated
environments do not allow.

Whether its the ability to properly customize rootfs ourselves (as CF
operators) or finding ways for buildpacks to add in missing dependencies,
we need to limit what we add to the rootfs.

On Mon, Jun 6, 2016 at 10:02 PM, Gabriel Ramirez <gramirez(a)pivotal.io>
wrote:

We are looking for community feedback on adding GraphViz support for
rootfs. This is an issue that came up in a request last week. (
https://github.com/cloudfoundry/python-buildpack/pull/41#issuecomment-223097886
)

We appreciate your feedback on this request. Please send us your
comments on this email list or on this github issue:
https://github.com/cloudfoundry/stacks/issues/33

--
Buildpacks team


Re: GraphViz support for rootfs?

Mike Youngstrom
 

This is an interesting question/problem. Although I see the value in not
bloating the RootFS it can be nice from a security perspective to know that
when a CVE does come up upgrading the RootFS can fix it for all instead of
relying on application developers to update. Could a bloated RootFS where
security is managed outside the application make a better counter approach
to a Docker solution?

A more bloated RootFS could also help keep buldpack complexity down.

Thoughts?

Mike

On Tue, Jun 7, 2016 at 4:56 AM, john mcteague <john.mcteague(a)gmail.com>
wrote:

I would be inclined to suggest we should be heading in the opposite
direction, stripping libraries out of the rootfs and finding a way to allow
buildpacks to add in required dependencies, thereby reducing the size and
complexity of the rootfs and minimising the number of potential CVE's in
this area.

In addition, something I have spoken to a number of people about in the
past is the presence of compilers in the rootfs which some regulated
environments do not allow.

Whether its the ability to properly customize rootfs ourselves (as CF
operators) or finding ways for buildpacks to add in missing dependencies,
we need to limit what we add to the rootfs.

On Mon, Jun 6, 2016 at 10:02 PM, Gabriel Ramirez <gramirez(a)pivotal.io>
wrote:

We are looking for community feedback on adding GraphViz support for
rootfs. This is an issue that came up in a request last week. (
https://github.com/cloudfoundry/python-buildpack/pull/41#issuecomment-223097886
)

We appreciate your feedback on this request. Please send us your comments
on this email list or on this github issue:
https://github.com/cloudfoundry/stacks/issues/33

--
Buildpacks team


Re: Reg the --no-route option in cf push

Amit Kumar Gupta
 

Hi Nithiyarsi,

Check out
https://docs.cloudfoundry.org/devguide/deploy-apps/troubleshoot-app-health.html#start

*Make sure your application code uses the PORT environment variable. Your
application may be failing because it is listening on the wrong port.
Instead of hard coding the port on which your application listens, use the
PORT environment variable.*

That might fix it for you.

Amit

On Tue, Jun 7, 2016 at 9:06 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH
MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi



I have created a custom buildpack with python,nginx and gunicorn.. And am
cf pushing the Django app by specifying it in the manifest.yml file.

The app is started successfully if I give –no-route option while cf push.



But if I remove the app is not starting..



Please let me know what is the reason

I get the app crashed error and no other information to find the reason..



Regards

Nithiyarsi


Reg the --no-route option in cf push

Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM@Cisco) <ngnanase at cisco.com...>
 

Hi

I have created a custom buildpack with python,nginx and gunicorn.. And am cf pushing the Django app by specifying it in the manifest.yml file.
The app is started successfully if I give -no-route option while cf push.

But if I remove the app is not starting..

Please let me know what is the reason
I get the app crashed error and no other information to find the reason..

Regards
Nithiyarsi


Re: UX proposal App manifests improvements for Routes, open for review

Koper, Dies <diesk@...>
 

Hi cf CLI (app manifest) users,

This proposal has been on hold while the Routing team was reconsidering potential changes to the multiple app ports API.
I have confirmed with Shannon that these changes would not affect the parts of the proposal not related to multiple app ports, so I have updated the proposal to take these out.

FYI,
I'd like to start writing stories this week and prioritize them for the CLI team to start very soon.
We received a (slightly yet significant) bigger preference for proposal B (specifying routes as a whole, as opposed to broken up into host, domain, port and path).

Note that I have one action item left to confirm we can do without the "no-hostname" attribute. I'd like to start writing the stories even if I can't resolve that first.

Regards,
Dies Koper
Cloud Foundry CLI PM


From: Koper, Dies [mailto:diesk(a)fast.au.fujitsu.com]
Sent: Saturday, March 26, 2016 12:55 PM
To: cf-dev(a)lists.cloudfoundry.org
Subject: [cf-dev] UX proposal App manifests improvements for Routes, open for review

Hi,

I'd like to introduce a set of new attributes to app manifests to support the specification of HTTP routes with paths, TCP routes and application ports.
I am also exploring a new way of specifying routes (proposal B).

Please take a look and leave your feedback in the doc or here on the list.

https://docs.google.com/document/d/17c1FwsuK_YCjQhH_7CebQkjrot4sfIsbAMQ0Cwcvv0M/edit?usp=sharing

Cheers,
Dies Koper
Cloud Foundry CLI PM


Re: Reg executing sudo commands in DEA vm

Daniel Mikusa
 

On Tue, Jun 7, 2016 at 6:00 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH
MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Eric



Thanks for your reply..



We are pushing the app with python –nginx buildpack ( custom created ).
Nginx needs to be run with super-user privileges. Following is the error
related to that…



*ERR 2016/06/07 05:59:29 [warn] 34#0: the "user" directive makes sense
only if the master process runs with super-user privileges, ignored in
/home/vcap/app/nginx/conf/nginx.conf:5*


This isn't saying that Nginx *needs* to be run with super user privileges.
It's saying that the option you're setting on line #5 doesn't make sense
*unless* you're running with super user privileges. Just remove or comment
out line #5 and it'll remove this warning.

Essentially what the warning is saying is that you're already running as a
non-privileged user (i.e. not root), so you don't need to set the
configuration option to change to a non-privileged user.


Please let me know how can I run the nginx with super-user privileges in
the warden container.
When it comes to your application code, it's not possible. You can't run
anything as root in an application container. Everything runs as the
`vcap` user.

You can technically enter the container as root using wsh. It is a manual
process though and any changes you make will disappear when your app is
restarted.

Dan




*Reg Manual Logging to the VM:*



I followed both these steps separately, to run the script as sudo user.
I tried to simulate the behavior manually.



1. vcap(a)e3da8039-ff08-4b9e-9492-922b0f4b30f4:/var/vcap/data/warden/depot/19k1kejlrto$
echo "c1oudc0w" | sudo -S ./test2.sh

[sudo] password for vcap: I am running test2.sh

The same command is not executed via the cf push –no-route –c “bash
./init_db.sh”.



2. Logging into wsh: If I do wsh , I cannot access the
depot/19k1kejlrto folder, because there is where the scripts are placed

vcap(a)e3da8039-ff08-4b9e-9492-922b0f4b30f4:/var/vcap/data/warden/depot/19k1kejlrto#
sudo ./bin/wsh

root(a)19k1kejlrto:~# ls

firstboot.sh

root(a)19k1kejlrto:~# pwd

/root

root(a)19k1kejlrto:~# cd /var/vcap/data/warden/depot/19k1kejlrto

bash: cd: /var/vcap/data/warden/depot/19k1kejlrto: No such file or
directory

root(a)19k1kejlrto:~# cd /var/vcap/data/warden/depot

bash: cd: /var/vcap/data/warden/depot: No such file or directory

root(a)19k1kejlrto:~# cd /var/vcap/data/

root(a)19k1kejlrto:/var/vcap/data# ls

dea_next



sudo su lands in the same location but via wsh, it lands in a different
folder and not accessible to the depot/19k1kejlrto .. But in both the
cases, they land in as root user only..

I could not decipher the difference..



Regards

Nithiyasri



*From: *Eric Malm <emalm(a)pivotal.io>
*Date: *Monday, June 6, 2016 at 11:02 AM
*To: *"Discussions about Cloud Foundry projects and the system overall." <
cf-dev(a)lists.cloudfoundry.org>
*Cc: *Amit Gupta <agupta(a)pivotal.io>, Jayarajan Ramapurath Kozhummal <
jayark(a)cisco.com>
*Subject: *Re: [cf-dev] Re: Reg executing sudo commands in DEA vm



Hi, Nithiyasri,



The 'vcap' user in the containers on both the DEAs and the Diego Cells
isn't authorized to use sudo. What do you need to do inside the container
that requires superuser privileges?



Also, could you clarify what you mean by "running manually in the warden"?
Is this after you've used `bosh ssh` to access the DEA VM, then entered the
container from the warden depot via `wsh`? If so, you're entering the
container as root, not vcap, so sudo will then work automatically.



Thanks,

Eric, CF Runtime Diego PM



On Mon, Jun 6, 2016 at 10:39 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

+ Gentle Reminder…



*From:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at
Cisco)
*Sent:* Sunday, June 05, 2016 4:25 PM
*To:* 'Amit Gupta' <agupta(a)pivotal.io>
*Cc:* 'Discussions about Cloud Foundry projects and the system overall.' <
cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>
*Subject:* Reg executing sudo commands in DEA vm



Hi Amit



I am trying to execute a boot script in the warden container of DEA vm of
cf-231. The boot script needs to executed as sudo user which has a password
to enter..



echo “c1oudc0w” | sudo -S sh boot.sh



But this repeatedly fails with the following error:



*ERR [sudo] password for vcap: Sorry, try again.*

*2016-06-05T10:44:07.86+0000 [App/0] ERR [sudo] password for vcap:*

*2016-06-05T10:44:07.86+0000 [App/0] ERR sudo: 1 incorrect password
attempt*



When running manually in the warden, this sudo command is successful.. I
tried many options to make it work. Do you have any restrictions to run the
sudo cmd via cf push..

I googled, but no luck. Kindly do the needful. If not relevant, please
forward to the relevant mailer list, as this issue is a showstopper..



I am pushing the app into DEA, using the following:

cf push --no-route -c "bash ./init_db.sh"


*Contents of Init_db.sh*

#!/bin/bash



echo "------ Create database tables ------"

python manage.py migrate --noinput



echo "------ create default admin user ------"

#echo "from django.contrib.auth.models import User;
User.objects.create_superuser('admin', 'admin(a)myapp.local', 'Passw0rd')"
| python manage.py shell



echo “c1oudc0w” | sudo -S sh boot.sh



regards

Nithiyasri



Re: GraphViz support for rootfs?

john mcteague <john.mcteague@...>
 

I would be inclined to suggest we should be heading in the opposite
direction, stripping libraries out of the rootfs and finding a way to allow
buildpacks to add in required dependencies, thereby reducing the size and
complexity of the rootfs and minimising the number of potential CVE's in
this area.

In addition, something I have spoken to a number of people about in the
past is the presence of compilers in the rootfs which some regulated
environments do not allow.

Whether its the ability to properly customize rootfs ourselves (as CF
operators) or finding ways for buildpacks to add in missing dependencies,
we need to limit what we add to the rootfs.

On Mon, Jun 6, 2016 at 10:02 PM, Gabriel Ramirez <gramirez(a)pivotal.io>
wrote:

We are looking for community feedback on adding GraphViz support for
rootfs. This is an issue that came up in a request last week. (
https://github.com/cloudfoundry/python-buildpack/pull/41#issuecomment-223097886
)

We appreciate your feedback on this request. Please send us your comments
on this email list or on this github issue:
https://github.com/cloudfoundry/stacks/issues/33

--
Buildpacks team


Re: Reg executing sudo commands in DEA vm

Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM@Cisco) <ngnanase at cisco.com...>
 

Hi Eric

Thanks for your reply..

We are pushing the app with python -nginx buildpack ( custom created ). Nginx needs to be run with super-user privileges. Following is the error related to that...

ERR 2016/06/07 05:59:29 [warn] 34#0: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /home/vcap/app/nginx/conf/nginx.conf:5

Please let me know how can I run the nginx with super-user privileges in the warden container.

Reg Manual Logging to the VM:

I followed both these steps separately, to run the script as sudo user. I tried to simulate the behavior manually.


1. vcap(a)e3da8039-ff08-4b9e-9492-922b0f4b30f4:/var/vcap/data/warden/depot/19k1kejlrto$ echo "c1oudc0w" | sudo -S ./test2.sh

[sudo] password for vcap: I am running test2.sh

The same command is not executed via the cf push -no-route -c "bash ./init_db.sh".



2. Logging into wsh: If I do wsh , I cannot access the depot/19k1kejlrto folder, because there is where the scripts are placed
vcap(a)e3da8039-ff08-4b9e-9492-922b0f4b30f4:/var/vcap/data/warden/depot/19k1kejlrto# sudo ./bin/wsh
root(a)19k1kejlrto:~# ls
firstboot.sh
root(a)19k1kejlrto:~# pwd
/root
root(a)19k1kejlrto:~# cd /var/vcap/data/warden/depot/19k1kejlrto
bash: cd: /var/vcap/data/warden/depot/19k1kejlrto: No such file or directory
root(a)19k1kejlrto:~# cd /var/vcap/data/warden/depot
bash: cd: /var/vcap/data/warden/depot: No such file or directory
root(a)19k1kejlrto:~# cd /var/vcap/data/
root(a)19k1kejlrto:/var/vcap/data# ls
dea_next

sudo su lands in the same location but via wsh, it lands in a different folder and not accessible to the depot/19k1kejlrto .. But in both the cases, they land in as root user only..
I could not decipher the difference..

Regards
Nithiyasri

From: Eric Malm <emalm(a)pivotal.io<mailto:emalm(a)pivotal.io>>
Date: Monday, June 6, 2016 at 11:02 AM
To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Cc: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>, Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>
Subject: Re: [cf-dev] Re: Reg executing sudo commands in DEA vm

Hi, Nithiyasri,

The 'vcap' user in the containers on both the DEAs and the Diego Cells isn't authorized to use sudo. What do you need to do inside the container that requires superuser privileges?

Also, could you clarify what you mean by "running manually in the warden"? Is this after you've used `bosh ssh` to access the DEA VM, then entered the container from the warden depot via `wsh`? If so, you're entering the container as root, not vcap, so sudo will then work automatically.

Thanks,
Eric, CF Runtime Diego PM

On Mon, Jun 6, 2016 at 10:39 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
+ Gentle Reminder...

From: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)
Sent: Sunday, June 05, 2016 4:25 PM
To: 'Amit Gupta' <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Cc: 'Discussions about Cloud Foundry projects and the system overall.' <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>
Subject: Reg executing sudo commands in DEA vm

Hi Amit

I am trying to execute a boot script in the warden container of DEA vm of cf-231. The boot script needs to executed as sudo user which has a password to enter..

echo "c1oudc0w" | sudo -S sh boot.sh

But this repeatedly fails with the following error:

ERR [sudo] password for vcap: Sorry, try again.
2016-06-05T10:44:07.86+0000 [App/0] ERR [sudo] password for vcap:
2016-06-05T10:44:07.86+0000 [App/0] ERR sudo: 1 incorrect password attempt

When running manually in the warden, this sudo command is successful.. I tried many options to make it work. Do you have any restrictions to run the sudo cmd via cf push..
I googled, but no luck. Kindly do the needful. If not relevant, please forward to the relevant mailer list, as this issue is a showstopper..

I am pushing the app into DEA, using the following:
cf push --no-route -c "bash ./init_db.sh"
Contents of Init_db.sh
#!/bin/bash

echo "------ Create database tables ------"
python manage.py migrate --noinput

echo "------ create default admin user ------"
#echo "from django.contrib.auth.models import User; User.objects.create_superuser('admin', 'admin(a)myapp.local<mailto:'admin(a)myapp.local>', 'Passw0rd')" | python manage.py shell

echo "c1oudc0w" | sudo -S sh boot.sh

regards
Nithiyasri


Need a new version of docker-boshrelease

Stanley Shen <meteorping@...>
 

https://github.com/cloudfoundry-community/docker-boshrelease

There is one PR https://github.com/cloudfoundry-community/docker-boshrelease/pull/62/files to fix an issue in release v26 and it has been merged for quite a long time.

Can someone help to do a new release for it?


Exposing Multiple Ports from an Application

Isuru Haththotuwa <isurulucky@...>
 

Hi all,

Is it possible to expose two ports of an Application via from CF to
external traffic, via an externally routable address and multiple ports?

While going through the documentation, came across [1], which suggests that
requests are served from port 80 and 443 only. However, while trying out
the sample spring application [2] on CF (using the hosted CF in pivotal web
services), I could actually see that there are multiple port mappings
defined by invoking the rest API [3]. Additionally, in pivotal.io docs, I
see the definition of CF-INSTANCE-PORTS, which seems to support multiple
port mappings from host to container.

I'm curios to know if exposing multiple ports from the host and container
side is possible.

Thanks in advance.

[1].
https://docs.cloudfoundry.org/devguide/deploy-apps/prepare-to-deploy.html#ports

[2]. https://github.com/cloudfoundry-samples/spring-music

[3].
{
"0": {
"state": "RUNNING",
"stats": {
"name": "spring-music",
"uris": [
"spring-music-moonlit-melioration.cfapps.io"
],
"host": "10.10.115.53",
"port": 64248,
"net_info": {
"address": "10.10.115.53",
* "ports": [*
* {*
* "container_port": 8080,*
* "host_port": 64248*
* },*
* {*
* "container_port": 2222,*
* "host_port": 64249*
* }*
* ]*
},
"uptime": 9484,
"mem_quota": 536870912,
"disk_quota": 1073741824,
"fds_quota": 16384,
"usage": {
"time": "2016-05-20T11:12:38.944471672Z",
"cpu": 0.0010850797743361504,
"mem": 460423168,
"disk": 166739968
}
}
}
}

[4].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-PORTS


Re: Getting the Internal IPs of the Applicatiom Container Before the Application State become RUNNING

Isuru Haththotuwa <isurulucky@...>
 

Hi Jason,

Thanks for the input. So I gather currently we do not have a way to do
instance to instance communication in CF. Great to hear that there is an
ongoing effort to support an overlay network. Will go through the proposal
and share any feedback that I have.

On Mon, Jun 6, 2016 at 10:31 PM, Jason Sherron <jsherron(a)pivotal.io> wrote:

Hi Isuru,

What you describe is largely similar to what we're doing with the
container networking effort -- providing the ability to communicate between
containers internally without sending all traffic back out to the external
router. I invite you take a look at our recent blog post [1] and provide
feedback, either on this list, directly to me jsherron(a)pivotal.io or join
us on our Cloud Foundry Slack channel #container-networking.

1 -
https://www.cloudfoundry.org/vision-future-container-networking-cloud-foundry/

On Mon, Jun 6, 2016 at 1:00 AM Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi Eric,

Thanks for the input.

On Mon, Jun 6, 2016 at 6:54 AM, Eric Malm <emalm(a)pivotal.io> wrote:

Hi, Isuru,

In general, traffic destined for an app container should be sent to the
externally routable address and port for that container, namely the one
exposed via the CF_INSTANCE_ADDR environment variable. So if the other
instances of your app need to communicate with each other directly, they
should do so via that socket address. That information is also available
for all other instances via CC's /v2/apps/:guid/stats endpoint. Note that
the app should also have a security group applied to its space that allows
its instances to communicate directly with the DEAs and/or Diego Cells in
the deployment.

The app process (on Linux, anyway) runs in a separate network namespace
from the host with a separate, private IP on a virtual ethernet interface.
You can therefore find that private IP with tools such as `ifconfig`, but
nothing outside the container should need to know that information.
Generally, the application should listen on the port set in the PORT
environment variable on all interfaces.
Fully agree that for exposing/accessing a service running on an instance
should happen via the externally routable address + port. However, if the
instances of the same app are in the same network, can they communicate via
the internal IP and internal ports? If there is a requirement for internal
communication between instances, which is not required to go via the
externally routable address + port, this would be useful. WDYT?


Thanks,
Eric, CF Runtime Diego PM

On Fri, Jun 3, 2016 at 6:11 AM, Isuru Haththotuwa <isurulucky(a)gmail.com>
wrote:

Hi Victor,

Thanks for the reply.

On Fri, Jun 3, 2016 at 4:20 PM, Victor Fonseca <vfonseca(a)pivotal.io>
wrote:

The application port changes every time you boot it. There's no means
to do what you want, determining what the ip and port your app. Can you
make the cluster instead of searching by IP: PORT look for AppName and
resolve the IP: Port?
When a new instance of an application comes up, the other it will get
to know the other instances of the cluster at startup. And there is an
health check mechanism, which will detect any invalid IP:PORT tuples
(non-existing instances), and remove them from the cluster. So this
scenario is actually handled. Instances spinning up and going down
dynamically is not an issue.

Is there a method to get the container's own IP:PORT from the container
itself? As I see, the available env. variable CF-INSTANCE-ADDR [1] will
return the host Ip and host port.

[1].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-ADDR


On Fri, Jun 3, 2016 at 7:43 AM, Isuru Haththotuwa <
isurulucky(a)gmail.com> wrote:

Hi all,

I'm working on a clustering mechanism for a specific java server that
are running on CF. The requirement is, before this server is started, it
needs a list of IPs + ports of the other instances that should cluster
together. I had a look at the CF API, specially at the /v2/apps/:guid/instances
API [1] endpoint. But it only return the IPs and ports of applications
which are in the RUNNING state. Is there a way to get the IPs of an
Application regardless of the state its in?

Also, as per the docs, CF_INSTANCE_IP is the external IP of the host
in which the container is running. Similarly, is there a way to get the
internal IP (and maybe the internal port as well) of a container, from the
container itself?

[1].
https://apidocs.cloudfoundry.org/237/apps/get_the_instance_information_for_a_started_app.html

[2].
http://docs.run.pivotal.io/devguide/deploy-apps/environment-variable.html#CF-INSTANCE-IP

--
Thanks and Regards,
Isuru


--

Victor Fonseca | Sr Field Engineer | Pivotal
Mobile: +55 11 996213861

pivotal.io


--
Thanks and Regards,
Isuru

--
Thanks and Regards,
Isuru
--
Thanks and Regards,
Isuru


CAB call for June - Cancelled

Michael Maximilien
 

Hi, all,

Since are just coming off a great summit in Santa Clara where 1000s of us
got to meet and share where the platform is now and where it's headed, we
are canceling the call for this month.

Talk to you in July. All the best,

dr.max
ibm cloud labs
sillicon valley, ca

Sent from my iPhone


GraphViz support for rootfs?

Gabriel Ramirez
 

We are looking for community feedback on adding GraphViz support for rootfs. This is an issue that came up in a request last week. (https://github.com/cloudfoundry/python-buildpack/pull/41#issuecomment-223097886)

We appreciate your feedback on this request. Please send us your comments on this email list or on this github issue: https://github.com/cloudfoundry/stacks/issues/33

--
Buildpacks team


Re: Reg executing sudo commands in DEA vm

Eric Malm <emalm@...>
 

Hi, Nithiyasri,

The 'vcap' user in the containers on both the DEAs and the Diego Cells
isn't authorized to use sudo. What do you need to do inside the container
that requires superuser privileges?

Also, could you clarify what you mean by "running manually in the warden"?
Is this after you've used `bosh ssh` to access the DEA VM, then entered the
container from the warden depot via `wsh`? If so, you're entering the
container as root, not vcap, so sudo will then work automatically.

Thanks,
Eric, CF Runtime Diego PM

On Mon, Jun 6, 2016 at 10:39 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

+ Gentle Reminder…



*From:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at
Cisco)
*Sent:* Sunday, June 05, 2016 4:25 PM
*To:* 'Amit Gupta' <agupta(a)pivotal.io>
*Cc:* 'Discussions about Cloud Foundry projects and the system overall.' <
cf-dev(a)lists.cloudfoundry.org>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>
*Subject:* Reg executing sudo commands in DEA vm



Hi Amit



I am trying to execute a boot script in the warden container of DEA vm of
cf-231. The boot script needs to executed as sudo user which has a password
to enter..



echo “c1oudc0w” | sudo -S sh boot.sh



But this repeatedly fails with the following error:



*ERR [sudo] password for vcap: Sorry, try again.*

*2016-06-05T10:44:07.86+0000 [App/0] ERR [sudo] password for vcap:*

*2016-06-05T10:44:07.86+0000 [App/0] ERR sudo: 1 incorrect password
attempt*



When running manually in the warden, this sudo command is successful.. I
tried many options to make it work. Do you have any restrictions to run the
sudo cmd via cf push..

I googled, but no luck. Kindly do the needful. If not relevant, please
forward to the relevant mailer list, as this issue is a showstopper..



I am pushing the app into DEA, using the following:

cf push --no-route -c "bash ./init_db.sh"


*Contents of Init_db.sh*

#!/bin/bash



echo "------ Create database tables ------"

python manage.py migrate --noinput



echo "------ create default admin user ------"

#echo "from django.contrib.auth.models import User;
User.objects.create_superuser('admin', 'admin(a)myapp.local', 'Passw0rd')"
| python manage.py shell



echo “c1oudc0w” | sudo -S sh boot.sh



regards

Nithiyasri

4361 - 4380 of 9425