cf-dev@lists.cloudfoundry.org

Re: how to debug "BuildpackCompileFailed" issue?

Mike Dalessio

#4109

Hi Ning,

If you generate an application manifest based on the current state of the
app, you'll have more information with which to try to diagnose the issue.

The command is `cf create-app-manifest APP_NAME [-p
/path/to/<app-name>-manifest.yml ]`

Can you share the contents of that manifest file?

-m

Show quoted text

On Tue, Mar 8, 2016 at 10:29 PM, Ning Fu <nfu(a)pivotal.io> wrote:

Yes, Gemfile and Gemfile.lock files are present at the top level.
I can get "cf logs" output for other apps I pushed. So I think the reason
that I don't see anything from "cf logs --recent" is that there are no logs
produced at all.
Anyway, whatever the root cause it may be, I think the key point is not
the root cause itself, it is the utilities that we provide to developers to
investigate when such an error occurs. I think it is really a bad user
experience for developers when such an error occurs but they can only guess
the root cause and try the solutions, rather than find clues from logs. PHP
build pack has a BP_DEBUG environment variable that helps debugging. But it
seems Ruby build pack doesn't support it.

Thanks,
Ning


On Wed, Mar 9, 2016 at 12:45 AM, Jesse Alford <jalford(a)pivotal.io> wrote:

Are your Gemfile and Gemfile.lock files literally named Gemfile and
Gemfile.lock, and both checked present at the top level of the directory
you're pushing from, on the machine you're pushing from?

The Ruby Buildpack looks for (and requires) the presence of these to
identify a Ruby app. If you've got some other setup, you can put empty
files with the appropriate names in the top level dir.

This might not be your problem, but I've seen it cause the error you
report many times.

On Tue, Mar 8, 2016, 7:12 AM JT Archie <jarchie(a)pivotal.io> wrote:

Ning,

What is the output of `cf buildpacks`?

JT

On Tue, Mar 8, 2016 at 1:27 AM, Ning Fu <nfu(a)pivotal.io> wrote:

Hi,

Does anyone know how to debug "BuildpackCompileFailed" issue?
When I push a ruby app:
========================
...
Done uploadingOK
Starting app happy in org funorg / space development as funcloud...
FAILEDBuildpackCompileFailed
TIP: use 'cf logs happy --recent' for more
informationPivotals-iMac:happy-root-route-app pivotal$ cf logs happy
--recent
Connected, dumping recent logs for app happy in org funorg / space
development as funcloud...
========================
But I got nothing from "cf logs happy --recent".
It is "ruby '2.2.2'" in Gemfile, and my ruby build back is
cached-1.6.11. I've also tried "bundle package --all" before I push.

Any suggestions? Doesn't ruby build pack provide any logs?

Thanks,
Ning

attachment.html

Re: Dial tcp: i/o timeout while pushing a sample app to Cloud Foundry BOSH-Lite deployment

Rob Dimsdale

#4108

We have noticed similar things recently. Our issue was a misconfigured DNS record in /etc/resolv.conf so we suggest you check your nameservers point to valid DNS servers, and that those servers have the required DNS entries for the hostnames.

It might also be related to the CLI switching from golang 1.5 to 1.6 in cf v6.16, though we haven't confirmed this. You could try downloading the previous version of the CLI and seeing if that resolves your issue.

It's also possible that DNS is taking longer than 5 seconds to return correctly - the CLI will not handle this due to a hard-coded timeout. See https://github.com/cloudfoundry/cli/issues/763 for more information.

Thanks,
Rob && Al
CF Release Integration
Pivotal

Re: Can resources of a IDLE application be shared by others?

Montenegro, Allan Mauricio (EESO CC PREMIER) <amontenegro@...>

#4107

Hello,

I'm sorry, I'm constantly receiving emails from this source, in which I have no part on, as I am a new Hewlett Packard Enterprise Employee.

Please remove my email.

Thanks,

Montenegro Murillo, Allan
Support Specialist
ITIO Customer Care

+1-877-785-2155 Office
Heredia, Costa Rica
allan.montenegro(a)hpe.com<mailto:allan.montenegro(a)hpe.com>

[cid:image001.jpg(a)01D11DDE.2289DAA0]

Show quoted text

-----Original Message-----
From: Deepak Vij (A) [mailto:deepak.vij(a)huawei.com]
Sent: Wednesday, March 09, 2016 1:09 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Can resources of a IDLE application be shared by others?

Over allocation of computing resources is a very serious problem in the contemporary cloud computing environment and across enterprise data centers in general. Famous Quasar research paper published by Stanford researches talks about this issue in great detail. There are startling numbers such as industry-wide utilization between 6% and 12%. A recent study estimated server utilization on Amazon EC2 in the 3% to 17% range.

Techniques which Google ("Borg") internally employs is called "Resource Reclamation" whereby over allocated resources are taken advantage of for executing best-effort workloads such as background analytics, and other low priority jobs. Such best-effort workloads could be pre-empted in case of any interference to the original normal running workload or resources are needed back at a later time. In order to be able to utilize unused resources in such a manner ("Resource Reclamation") requires underlying support for capabilities such as "Preemption & Resizing", "Isolation Mechanisms", "Interference Detection" etc.

Another important aspect of all this is the fact that users never know how to precisely do resource reservations - this leads to underutilization of computing resources. Instead, what is being proposed is some kind of classification based predictive analytics techniques whereby resource management system itself determines right amount of resources to meet the user performance constraints. In this case, user only specifies the performance constraints (SLO) not the actual low level resource reservations itself. Combination of predictive analytics based proactive approach in conjunction with reactive "resource reclamation" approach is the optimal strategy. This accounts for any mis-predictions as well.

Mesos Resource Management also supports "Resource Reclamation". Although, both Google ("Borg") and Mesos do not employ predictive analytics based proactive approach in their environment.

I hope this makes sense and helps.

- Deepak Vij (Huawei, Software Lab., Santa Clara)

-----Original Message-----

From: Stanley Shen [mailto:meteorping(a)gmail.com]

Sent: Tuesday, March 08, 2016 7:10 PM

To: cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>

Subject: [cf-dev] Can resources of a IDLE application be shared by others?

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.

The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.

For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.

So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.

Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?

It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.

attachment.html

Re: Can resources of a IDLE application be shared by others?

Deepak Vij

#4106

Over allocation of computing resources is a very serious problem in the contemporary cloud computing environment and across enterprise data centers in general. Famous Quasar research paper published by Stanford researches talks about this issue in great detail. There are startling numbers such as industry-wide utilization between 6% and 12%. A recent study estimated server utilization on Amazon EC2 in the 3% to 17% range.

Techniques which Google ("Borg") internally employs is called "Resource Reclamation" whereby over allocated resources are taken advantage of for executing best-effort workloads such as background analytics, and other low priority jobs. Such best-effort workloads could be pre-empted in case of any interference to the original normal running workload or resources are needed back at a later time. In order to be able to utilize unused resources in such a manner ("Resource Reclamation") requires underlying support for capabilities such as "Preemption & Resizing", "Isolation Mechanisms", "Interference Detection" etc.

Another important aspect of all this is the fact that users never know how to precisely do resource reservations - this leads to underutilization of computing resources. Instead, what is being proposed is some kind of classification based predictive analytics techniques whereby resource management system itself determines right amount of resources to meet the user performance constraints. In this case, user only specifies the performance constraints (SLO) not the actual low level resource reservations itself. Combination of predictive analytics based proactive approach in conjunction with reactive "resource reclamation" approach is the optimal strategy. This accounts for any mis-predictions as well.

Mesos Resource Management also supports "Resource Reclamation". Although, both Google ("Borg") and Mesos do not employ predictive analytics based proactive approach in their environment.

I hope this makes sense and helps.

- Deepak Vij (Huawei, Software Lab., Santa Clara)

Show quoted text

-----Original Message-----
From: Stanley Shen [mailto:meteorping(a)gmail.com] 
Sent: Tuesday, March 08, 2016 7:10 PM
To: cf-dev(a)lists.cloudfoundry.org
Subject: [cf-dev] Can resources of a IDLE application be shared by others?

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation. 
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.

Adding previous_instances and previous_memory fields to cf_event

KRuelY <kevinyudhiswara@...>

#4105

Hi,

I am currently working on metering runtime usage, and one issue I'm facing
is that there is a possibility that usage submission comes in out of
order(due to network error / other possibilities). Before the issue, the way
metering runtime usage works is quiet simple. There is an app that will look
at cf_events and submit usages to
[cf-abacus](https://github.com/cloudfoundry-incubator/cf-abacus). 


    {
      "metadata": {
        "guid": "40afe01a-b15a-4b8d-8bd1-e36a0ba2f6f5",
        "url": "/v2/app_usage_events/40afe01a-b15a-4b8d-8bd1-e36a0ba2f6f5",
        "created_at": "2016-03-02T09:48:09Z"
      },
      "entity": {
        "state": "STARTED",
        "memory_in_mb_per_instance": 512,
        "instance_count": 1,
        "app_guid": "a2ab1b5a-94c0-4344-9a71-a1d2b11f483a",
        "app_name": "abacus-usage-collector",
        "space_guid": "d34d770d-4cd0-4bdc-8c83-8fdfa5f0b3cb",
        "space_name": "dev",
        "org_guid": "238a3e78-3fc8-4542-928a-88ee99643732",
        "buildpack_guid": "b77d0ef8-da1f-4c0a-99cc-193449324706",
        "buildpack_name": "nodejs_buildpack",
        "package_state": "STAGED",
        "parent_app_guid": null,
        "parent_app_name": null,
        "process_type": "web"
      }
    }


The way this app works is by looking at the state.
If the state is STARTED, it will submit usage to abacus with the
instance_memory = memory_in_mb_per_instance, running_instances =
instance_count, and since = created_at.
If the state is STOPPED, it will submit usage to abacus with the
instance_memory = 0, running_instances = 0, and since = created_at.

In ideal situation, where there is no out of order submission this is fine.
'Simple, but Exaggerated' Example:
Usage instance_memory = 1GB, running_instances = 1, since = 3/9 00:00 comes
in. (STARTED)
Usage instance_memory = 0GB, running_instances = 0, since = 3/10 00:00 comes
in. (STOPPED)
Then Abacus know that the app consumed 1GB * (3/10 - 3/9 = 24 hours) = 24
GB-hour.

But when the usage comes in out of order:
Usage instance_memory = 0GB, running_instances = 0, since = 3/10 00:00 comes
in. (STOPPED)
Usage instance_memory = 1GB, running_instances = 1, since = 3/9 00:00 comes
in. (STARTED)
The formula that Abacus currently have would not works.

Abacus has another formula that would take care of this out of order
submission, but it would only works if we have previous_instance_memory and
previous_running_instances.

When looking for a way to have this fields, we concluded that the cleanest
way would be to add previous_memory_in_mb_per_instance and
previous_instance_count to the cf_event. It will make App reconfigure or cf
scale makes more sense too because currently cf scale is a STOP and a START.

To sum up, the cf_event state submitted would include information:

// Starting
{
  "state": "STARTED",
  "memory_in_mb_per_instance": 512,
  "instance_count": 1,
  "previous_memory_in_mb_per_instance": 0,
  "previous_instance_count": 0
}

// Scaling up
{
  "state": "SCALE"?,
  "memory_in_mb_per_instance": 512,
  "instance_count": 2,
  "previous_memory_in_mb_per_instance": 512,
  "previous_instance_count": 1
}

// Scale down
{
  "state": "SCALE"?,
  "memory_in_mb_per_instance": 512,
  "instance_count": 1,
  "previous_memory_in_mb_per_instance": 512,
  "previous_instance_count": 2
}

// Stopping
{
  "state": "STOPPED",
  "memory_in_mb_per_instance": 0,
  "instance_count": 0,
  "previous_memory_in_mb_per_instance": 512,
  "previous_instance_count": 1
}


Any thoughts/feedbacks/guidance?








--
View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-Adding-previous-instances-and-previous-memory-fields-to-cf-event-tp4100.html
Sent from the CF Dev mailing list archive at Nabble.com.

Re: Can resources of a IDLE application be shared by others?

Daniel Mikusa

#4104

This thread might be helpful.

http://cf-dev.70369.x6.nabble.com/cf-dev-Overcommit-on-Diego-Cells-td1162.html

Dan

Show quoted text

On Tue, Mar 8, 2016 at 10:09 PM, Stanley Shen <meteorping(a)gmail.com> wrote:

Hello, all

When pushing an application to CF, we need to define its disk/memory
limitation.
The memory limitation is just the possible maximum value will be needed in
this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at
startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is
allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should
can push more applications, like 3 since only 2G is needed for each
application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing
application, it will not be released even I stopped the application.

attachment.html

Re: Can resources of a IDLE application be shared by others?

Stanley Shen <meteorping@...>

#4103

It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.

This statement is wrong, and the resources will be release after the application is stopped.

Re: Error while using uaac with CF r230

Sylvain Goulmy <sygoulmy@...>

#4102

It seems that i can give the solution for that one, I had a F5 in front of
my CF platform with compression set to deflate.

UAAC sends the HTTP header : Accept-Encoding:
gzip;q=1.0,deflate;q=0.6,identity;q=0.3\r\n on its request but the UAA send
an uncompressed response. As the F5 know that the client UAAC accept
compression, it does the job and compress the UAA response.

Unfortunately there is a bug in Ruby with the deflate compressiuon :
https://bugs.ruby-lang.org/issues/11268

Everything was finally solved by switching the F5 compression mode from
deflate to gzip.

End of the story.

Show quoted text

On Thu, Feb 25, 2016 at 6:31 PM, Sylvain Goulmy <sygoulmy(a)gmail.com> wrote:

Hi all,

I'm currently experiencing issues with the uaac :

uaac token client get admin -s mysecretftw
Zlib::DataError: incorrect header check
attempt to get token failed
I'm working with CF r230 and UAA client 3.1.7.

Thanks in advance for your support.

Sylvain

attachment.html

Re: New CF Service Broker "chaos-galago" - a chaos-monkey for your Cloud Foundry

Sam Bryant

#4101

Hi David,

The service broker actually uses an already existing API endpoint: https://apidocs.cloudfoundry.org/231/apps/terminate_the_running_app_instance_at_the_given_index.html

It will terminate the app instance, in my experience CF always tidies up and restarts the app within a minute or so.

If you have any other ideas/ opinions on how application instances should be terminated I am happy to investigate the options of enhancements. We are also open to pull requests.

Regards,
Sam

Re: New CF Service Broker "chaos-galago" - a chaos-monkey for your Cloud Foundry

David Illsley <davidillsley@...>

#4100

Really cool. How nasty is the kill? Is the process killed, then CF
cleans-up on healthcheck failure, or is the app instance removed from the
router before instance termination?

Show quoted text

On Thu, Mar 3, 2016 at 4:33 PM, Sam Bryant <srbry(a)hotmail.com> wrote:

Hi all,

Fidelity International have recently developed and open-sourced a new
Service Broker for Cloud Foundry called "chaos-galago". It has been created
with aim of providing a marketplace service to users that will cause chaos
to their bound applications. It works by randomly killing service instances
based on a provided probability and frequency.

We see a massive benefit in chaos-galago as it should allow developers to
test if their CF applications truly are cloud-resilient, do they cope with
unexpected failures? Are they deployed with enough instances so that they
meantime uptime?

More information and how to deploy it can be found on our Github repo:
https://github.com/FidelityInternational/chaos-galago

Any input, feedback and questions are encouraged, we hope that everyone
finds it the same useful tool that we do.

Regards,
Sam

attachment.html

Re: How to enable Windows2012r2 in PivotalCF account

Daniel Mikusa

#4099

Please direct Pivotal specific questions to Pivotal, not the open source
mailing list.  PWS questions can be sent to support(a)run.pivotal.io and PCF
questions can be logged here: https://support.pivotal.io/hc/en-us.

Dan

Show quoted text

On Wed, Mar 9, 2016 at 1:16 AM, Gowri L N <gowri.ln123(a)gmail.com> wrote:

i am trying to push a .NET based app to cloud foundry .
I have an account at https://run.pivotal.io/ and when I try to list the
stacks available in my org and space, I see only cflinuxfs2 but not the new
windows2012r2 stack.

Can anyone suggest me the documentation on how to do that or help me with
this ?

Link( http://engineering.pivotal.io/post/dotnet-quick-intro/) suggests
that I need access to Cloud Foundry installation with Elastic Runtime/Diego
support enabled. And when I searched for the same, I got informatoion on
how I can deploy an ops manger and enable Diego/elastic runtime(
https://docs.pivotal.io/pivotalcf/opsguide/deploying-diego.html)  .Does
this mean I need to have an AWS accountand a CF deployment on that ? Can I
not try a sample .NET app using my pivotalCF account ? i.e Is there a way
to add windows Stack to my current org ?

Thanks,

attachment.html

Re: Reg the minimal-openstack yml files

Yitao Jiang

#4098

​Of course yes, assuming you have add the director as the dns resolver in
UAA job.​

On Wed, Mar 9, 2016 at 4:38 PM, Nithiyasri Gnanasekaran -X (ngnanase - TECH
MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi Amit

Thanks your reply

We understood the must of static IP of routers in uaa jobs.

Kindly let me know if we can give the DNS entry name (hostname) of the
router in the uaa job

*        - 0.router-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh*

We are also parallel trying to get the help of a private vendor to help us
on the installation, as told by you

Regards

Nithiyasri

*From:* Amit Gupta [mailto:agupta(a)pivotal.io]
*Sent:* Tuesday, March 08, 2016 3:43 AM
*To:* Discussions about Cloud Foundry projects and the system overall. <
cf-dev(a)lists.cloudfoundry.org>
*Cc:* Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <
ngnanase(a)cisco.com>; Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com>
*Subject:* Re: Reg the minimal-openstack yml files

You need some static IPs, because some components don't use service
discovery, so clients need to be configured up front to know those
components' IPs. For example, the route-registrar jobs talk to the NATS
cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers
must be assigned static IPs in the manifest and components like
route-registrar that talk to NATS need to have those IPs provided in their
config in the manifest.

This current requirement on static IPs exists for some other components as
well. Some other components however would be fine with dynamic networking.
A private vendor may be able to work with you to manage a Cloud Foundry
installation that suits your networking requirements, but said guidance is
beyond the scope of this mailing list.  The deployments that the Foundation
tests continuously use static IPs for simplicity.

Cheers,

Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <
jayark(a)cisco.com> wrote:

Thanks Amit!

A minor correction in our case.

We are not using static IP. We are only using dynamic IP.

None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?

Given that we are using dynamic IPs, do you foresee any issues with our
current approach to generate the manifest for CF-229?

Regards

 Jayaraj

*From: *Amit Gupta <agupta(a)pivotal.io>
*Date: *Monday, March 7, 2016 at 12:24 PM
*To: *"Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at
Cisco)" <ngnanase(a)cisco.com>
*Cc: *Rohit Kumar <rokumar(a)pivotal.io>, Jayarajan Ramapurath Kozhummal <
jayark(a)cisco.com>, "Discussions about Cloud Foundry projects and the
system overall." <cf-dev(a)lists.cloudfoundry.org>
*Subject: *Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest
any time soon.

As for generating the manifest, you can look at these docs for what needs
to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is
a good way to generate the manifest.  This is not guaranteed to work for
100% of use cases, e.g. if you want to use all floating IPs instead of
static IPs.  But it is a good starting point.

Cheers,

Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase -
TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com> wrote:

Hi

We are trying to upgrade our deployment with the latest cloud-foundry,
from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. *Can we have a
similar one available for openstack environment, with which we can deploy
the basic cloud foundry* and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided
by the errors given by the generate_deployment_manifest script.  Kindly let
us know if this is the correct way to generate the manifest.

Regards

Nithiyasri

-- 

Regards,

Yitao

attachment.html

Re: Update Parallelization in Cloud Foundry

Marco Voelz

#4097

Thanks for clarifying this for me, Amit.

Warm regards
Marco

Show quoted text

On 09/03/16 07:43, "Amit Gupta" <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote:

You can probably try to start everything in parallel, and either set very long update timeouts, or allow the deployment to fail with the expectation that it will eventually correct itself.  Or you can start things in a strict order, and have stronger constraints on the possible failure scenarios, and be able to debug the root cause of a failure better.

Certain things do depend on NATS, and thus won't work until NATS is up.  The main thing I can currently think of is registering routes with gorouter, which is done both for apps and for system components (e.g. the route-registrar registers api.SYSTEM_DOMAIN on behalf of the CC).

Best,
Amit

On Tue, Mar 8, 2016 at 2:14 AM, Voelz, Marco <marco.voelz(a)sap.com<mailto:marco.voelz(a)sap.com>> wrote:
Does NATS also need to come up before any of the other components?

On 07/03/16 21:16, "Amit Gupta" <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote:

Hey Omar,

You can set the "serial" property at the global level of a deployment (you can think of it as setting a default for all jobs), and then override it at the individual job levels.  You will want the consul server jobs to be deployed first, with serial: true, and max_in_flight: 1.  The important thing here is, if you have more than one server in your consul cluster, they need to come up one at a time to ensure the cluster orchestration goes smoothly.  The same is true if your etcd cluster has more than one server in it.  If you're using the postgres job for CCDB and/or UAADB (instead of some external database), then you will want the postgres job to come up before CC and/or UAA.  Similarly, if you're using the provided blobstore job instead of an external blobstore, you'll want it up before CC comes up.

You might be able to get away with parallelizing some of the things above.  E.g. if you bring the CC and blobstore up at the same time, CC might fail to start for a while until Blobstore comes up, and then CC might successfully start up.  Monit also generally keeps retrying even after BOSH gives up.  So your deploy might fail but later on, you might see everything up and running.

Cheers,
Amit

On Mon, Mar 7, 2016 at 5:54 AM, Omar Elazhary <omazhary(a)gmail.com<mailto:omazhary(a)gmail.com>> wrote:
Hello everyone,

I know it is possible to update and redeploy components in parallel in cloud foundry by setting the "serial" property in the deployment manifest to "false". However, is such a thing recommended? Are there particular job dependencies that I need to pay attention to?

Regards,
Omar

attachment.html

Re: Reg the minimal-openstack yml files

Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM@Cisco) <ngnanase at cisco.com...>

#4096

Hi Amit

Thanks your reply

We understood the must of static IP of routers in uaa jobs.
Kindly let me know if we can give the DNS entry name (hostname) of the router in the uaa job
        - 0.router-z1.ccc-bosh-net.<%= $deployment_name %>.microbosh

We are also parallel trying to get the help of a private vendor to help us on the installation, as told by you


Regards
Nithiyasri

From: Amit Gupta [mailto:agupta(a)pivotal.io]
Sent: Tuesday, March 08, 2016 3:43 AM
To: Discussions about Cloud Foundry projects and the system overall. <cf-dev(a)lists.cloudfoundry.org>
Cc: Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com>; Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com>
Subject: Re: Reg the minimal-openstack yml files

You need some static IPs, because some components don't use service discovery, so clients need to be configured up front to know those components' IPs. For example, the route-registrar jobs talk to the NATS cluster to register routes for CC, HM9k, UAA, and Doppler. NATS servers must be assigned static IPs in the manifest and components like route-registrar that talk to NATS need to have those IPs provided in their config in the manifest.

This current requirement on static IPs exists for some other components as well. Some other components however would be fine with dynamic networking. A private vendor may be able to work with you to manage a Cloud Foundry installation that suits your networking requirements, but said guidance is beyond the scope of this mailing list.  The deployments that the Foundation tests continuously use static IPs for simplicity.

Cheers,
Amit

On Monday, March 7, 2016, Jayarajan Ramapurath Kozhummal (jayark) <jayark(a)cisco.com<mailto:jayark(a)cisco.com>> wrote:
Thanks Amit!

A minor correction in our case.
We are not using static IP. We are only using dynamic IP.
None of the Cloud Foundry components are using Floating IP.

What is Pivotal recommendation for the networking? Is it static or dynamic?
Given that we are using dynamic IPs, do you foresee any issues with our current approach to generate the manifest for CF-229?

Regards
 Jayaraj

From: Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>>
Date: Monday, March 7, 2016 at 12:24 PM
To: "Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco)" <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>>
Cc: Rohit Kumar <rokumar(a)pivotal.io<mailto:rokumar(a)pivotal.io>>, Jayarajan Ramapurath Kozhummal <jayark(a)cisco.com<mailto:jayark(a)cisco.com>>, "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>>
Subject: Re: Reg the minimal-openstack yml files

We're not likely to be able to create and maintain a new minimal manifest any time soon.

As for generating the manifest, you can look at these docs for what needs to go into the OpenStack stub as of the latest release:

http://docs.cloudfoundry.org/deploying/openstack/cf-stub.html

Combining this with updating your current template guided by the errors is a good way to generate the manifest.  This is not guaranteed to work for 100% of use cases, e.g. if you want to use all floating IPs instead of static IPs.  But it is a good starting point.

Cheers,
Amit

On Mon, Mar 7, 2016 at 4:09 AM, Nithiyasri Gnanasekaran -X (ngnanase - TECH MAHINDRA LIM at Cisco) <ngnanase(a)cisco.com<mailto:ngnanase(a)cisco.com>> wrote:
Hi

We are trying to upgrade our deployment with the latest cloud-foundry, from 205 to 230 release, as per your advice.

We could see minimal-aws.yml available in the GIT repo. Can we have a similar one available for openstack environment, with which we can deploy the basic cloud foundry and do our custom changes on top of it

Parallely we are updating our stub to match the template yml files guided by the errors given by the generate_deployment_manifest script.  Kindly let us know if this is the correct way to generate the manifest.


Regards
Nithiyasri

attachment.html

Re: Dial tcp: i/o timeout while pushing a sample app to Cloud Foundry BOSH-Lite deployment

Giovanni Napoli

#4095

EDIT: Strange thing. I tried to not use a wired internet connection, and it's works. Infact i used an hotspot with my cell phone and it pushes the app to the local Cloud without any problem.
Just saying.

Re: Update Parallelization in Cloud Foundry

Amit Kumar Gupta

#4094

You can probably try to start everything in parallel, and either set very
long update timeouts, or allow the deployment to fail with the expectation
that it will eventually correct itself.  Or you can start things in a
strict order, and have stronger constraints on the possible failure
scenarios, and be able to debug the root cause of a failure better.

Certain things do depend on NATS, and thus won't work until NATS is up.
The main thing I can currently think of is registering routes with
gorouter, which is done both for apps and for system components (e.g. the
route-registrar registers api.SYSTEM_DOMAIN on behalf of the CC).

Best,
Amit

Show quoted text

On Tue, Mar 8, 2016 at 2:14 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote:

Does NATS also need to come up before any of the other components?

On 07/03/16 21:16, "Amit Gupta" <agupta(a)pivotal.io> wrote:

Hey Omar,

You can set the "serial" property at the global level of a deployment (you
can think of it as setting a default for all jobs), and then override it at
the individual job levels.  You will want the consul server jobs to be
deployed first, with serial: true, and max_in_flight: 1.  The important
thing here is, if you have more than one server in your consul cluster,
they need to come up one at a time to ensure the cluster orchestration goes
smoothly.  The same is true if your etcd cluster has more than one server
in it.  If you're using the postgres job for CCDB and/or UAADB (instead of
some external database), then you will want the postgres job to come up
before CC and/or UAA.  Similarly, if you're using the provided blobstore
job instead of an external blobstore, you'll want it up before CC comes up.

You might be able to get away with parallelizing some of the things
above.  E.g. if you bring the CC and blobstore up at the same time, CC
might fail to start for a while until Blobstore comes up, and then CC might
successfully start up.  Monit also generally keeps retrying even after BOSH
gives up.  So your deploy might fail but later on, you might see everything
up and running.

Cheers,
Amit

On Mon, Mar 7, 2016 at 5:54 AM, Omar Elazhary <omazhary(a)gmail.com> wrote:

Hello everyone,

I know it is possible to update and redeploy components in parallel in
cloud foundry by setting the "serial" property in the deployment manifest
to "false". However, is such a thing recommended? Are there particular job
dependencies that I need to pay attention to?

Regards,
Omar

attachment.html

How to enable Windows2012r2 in PivotalCF account

Gw .

#4093

i am trying to push a .NET based app to cloud foundry .
I have an account at https://run.pivotal.io/ and when I try to list the stacks available in my org and space, I see only cflinuxfs2 but not the new windows2012r2 stack.

Can anyone suggest me the documentation on how to do that or help me with this ?

Link( http://engineering.pivotal.io/post/dotnet-quick-intro/) suggests that I need access to Cloud Foundry installation with Elastic Runtime/Diego support enabled. And when I searched for the same, I got informatoion on how I can deploy an ops manger and enable Diego/elastic runtime(https://docs.pivotal.io/pivotalcf/opsguide/deploying-diego.html)  .Does this mean I need to have an AWS accountand a CF deployment on that ? Can I not try a sample .NET app using my pivotalCF account ? i.e Is there a way to add windows Stack to my current org ?

Thanks,

Re: how to debug "BuildpackCompileFailed" issue?

Ning Fu

#4092

Yes, Gemfile and Gemfile.lock files are present at the top level.
I can get "cf logs" output for other apps I pushed. So I think the reason
that I don't see anything from "cf logs --recent" is that there are no logs
produced at all.
Anyway, whatever the root cause it may be, I think the key point is not the
root cause itself, it is the utilities that we provide to developers to
investigate when such an error occurs. I think it is really a bad user
experience for developers when such an error occurs but they can only guess
the root cause and try the solutions, rather than find clues from logs. PHP
build pack has a BP_DEBUG environment variable that helps debugging. But it
seems Ruby build pack doesn't support it.

Thanks,
Ning

Show quoted text

On Wed, Mar 9, 2016 at 12:45 AM, Jesse Alford <jalford(a)pivotal.io> wrote:

Are your Gemfile and Gemfile.lock files literally named Gemfile and
Gemfile.lock, and both checked present at the top level of the directory
you're pushing from, on the machine you're pushing from?

The Ruby Buildpack looks for (and requires) the presence of these to
identify a Ruby app. If you've got some other setup, you can put empty
files with the appropriate names in the top level dir.

This might not be your problem, but I've seen it cause the error you
report many times.

On Tue, Mar 8, 2016, 7:12 AM JT Archie <jarchie(a)pivotal.io> wrote:

Ning,

What is the output of `cf buildpacks`?

JT

On Tue, Mar 8, 2016 at 1:27 AM, Ning Fu <nfu(a)pivotal.io> wrote:

Hi,

Does anyone know how to debug "BuildpackCompileFailed" issue?
When I push a ruby app:
========================
...
Done uploadingOK
Starting app happy in org funorg / space development as funcloud...
FAILEDBuildpackCompileFailed
TIP: use 'cf logs happy --recent' for more
informationPivotals-iMac:happy-root-route-app pivotal$ cf logs happy
--recent
Connected, dumping recent logs for app happy in org funorg / space
development as funcloud...
========================
But I got nothing from "cf logs happy --recent".
It is "ruby '2.2.2'" in Gemfile, and my ruby build back is
cached-1.6.11. I've also tried "bundle package --all" before I push.

Any suggestions? Doesn't ruby build pack provide any logs?

Thanks,
Ning

attachment.html

Re: how to debug "BuildpackCompileFailed" issue?

Ning Fu

#4091

Here is the output of "cf buildpacks":

cf buildpacks
Getting buildpacks...

buildpack                position   enabled   locked   filename
staticfile_buildpack     1          true      false
 staticfile_buildpack-cached-v1.2.3.zip
java_buildpack_offline   2          true      false
 java-buildpack-offline-v3.6.zip
ruby_buildpack           3          true      false
 ruby_buildpack-cached-v1.6.11.zip
nodejs_buildpack         4          true      false
 nodejs_buildpack-cached-v1.5.4.zip
go_buildpack             5          true      false
 go_buildpack-cached-v1.7.1.zip
python_buildpack         6          true      false
 python_buildpack-cached-v1.5.3.zip
php_buildpack            7          true      false
 php_buildpack-cached-v4.3.1.zip
binary_buildpack         8          true      false
 binary_buildpack-cached-v1.0.1.zip
java_buildpack           9          true      false
 java-buildpack-v3.3.1.zip

Show quoted text

On Wed, Mar 9, 2016 at 12:45 AM, Jesse Alford <jalford(a)pivotal.io> wrote:

Are your Gemfile and Gemfile.lock files literally named Gemfile and
Gemfile.lock, and both checked present at the top level of the directory
you're pushing from, on the machine you're pushing from?

The Ruby Buildpack looks for (and requires) the presence of these to
identify a Ruby app. If you've got some other setup, you can put empty
files with the appropriate names in the top level dir.

This might not be your problem, but I've seen it cause the error you
report many times.

On Tue, Mar 8, 2016, 7:12 AM JT Archie <jarchie(a)pivotal.io> wrote:

Ning,

What is the output of `cf buildpacks`?

JT

On Tue, Mar 8, 2016 at 1:27 AM, Ning Fu <nfu(a)pivotal.io> wrote:

Hi,

Does anyone know how to debug "BuildpackCompileFailed" issue?
When I push a ruby app:
========================
...
Done uploadingOK
Starting app happy in org funorg / space development as funcloud...
FAILEDBuildpackCompileFailed
TIP: use 'cf logs happy --recent' for more
informationPivotals-iMac:happy-root-route-app pivotal$ cf logs happy
--recent
Connected, dumping recent logs for app happy in org funorg / space
development as funcloud...
========================
But I got nothing from "cf logs happy --recent".
It is "ruby '2.2.2'" in Gemfile, and my ruby build back is
cached-1.6.11. I've also tried "bundle package --all" before I push.

Any suggestions? Doesn't ruby build pack provide any logs?

Thanks,
Ning

attachment.html

Can resources of a IDLE application be shared by others?

Stanley Shen <meteorping@...>

#4090

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation. 
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.