I understand what you're trying to avoid, I just think that is actually the
normal case for the port healthchecks. Nothing on the Linux or Docker side
ever touches the app so it's entirely possible it will be added to the
router without it actually working and that is what I expect the platform
to
do. Hopefully the more generic HTTP check can be added quickly to all the
right places so that we'll at least have more sensible options.

Now we just have to decide if we hang onto Iron Foundry that just uses a
port check until then, or try to explain to my users that most of their
apps
won't work unless they turn off the healthcheck. I'm expecting most of
them
won't RTFM and we'll get constant complaints about how our .NET support is
broken because their apps won't start up.

Aaron



--
View this message in context:
http://cf-dev.70369.x6.nabble.com/Issue-with-crashing-Windows-apps-on-Diego-tp3586p3690.html
Sent from the CF Dev mailing list archive at Nabble.com.

Sorry, for the typo I meant 6868

Thanks,
Kinjal

On Tue, Feb 2, 2016 at 11:04 PM, Kinjal Doshi <kindoshi(a)gmail.com> wrote:

Hi Amit,

This does not seem to be a port issue on 6968. I tried the same
deployment by modifying the security groups (both bosh and cf ) to allow
All Protocol All Ports. even wit this change the deployment fails while
compiling packages.

Would be great i you could provide some pointers to have this corrected.
One thing I noticed is that the config ha_proxy is set to null in the
generate deployment manifest.

Thanks,
Kinjal

On Tue, Feb 2, 2016 at 12:35 AM, Kinjal Doshi <kindoshi(a)gmail.com> wrote:

Hi Amit,

I checked the ports on all the security groups and found that 6868 is
enabled on the inbound for 0.0.0.0/0 in all the groups.

Am sending you the the bosh logs on your personal email address.

Would be great if you could please take a look.

Thanks,
Kinjal

On Sat, Jan 30, 2016 at 4:00 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hey Kinjal,

Happy to help!

Looks like your director is failing to connect to your compilation
VMs. In your manifest you have a network called "cf1" with an associated
subnet ID and security groups. I believe specifically trying to reach
those VMs on port 6868. Can you look at the security group rules,
including the security groups applied to the micro bosh VM, and see why
there might be problems communicating?

Best,
Amit

On Fri, Jan 29, 2016 at 12:49 PM, Kinjal Doshi <kindoshi(a)gmail.com>
wrote:

Hi Amit,

Really appreciate all the help I receive on this forum. Hats off to
you all.

Here is the deployment log output:
https://gist.github.com/kinjaldoshi/0925fdf6022b079ca2b5

Thanks,
Kinjal

On Sat, Jan 30, 2016 at 2:03 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Since you started with the "minimal-aws" flow which happens to
document using micro bosh, you should be fine to continue with micro bosh,
instead of the newer bosh-init workflow. You may run into some
discrepancies in the downstream documentation depending on whether it
assumed a bosh-init workflow vs a micro bosh workflow, but we can guide you
through those should you hit any problems.

On Fri, Jan 29, 2016 at 12:24 PM, Kinjal Doshi <kindoshi(a)gmail.com>
wrote:

Hi Amit,

Thanks a lot for the quick response.

I am currently sanitizing the output, will send it soon. In the mean
while, I wanted to confirm if I have created microbosh using the correct
process. I have followed the instructions at:
https://bosh.io/docs/deploy-microbosh-to-aws.html. However, I see
that there are other instructions too to create microbosh as follows:
https://docs.cloudfoundry.org/deploying/aws/setup_aws.html
and
http://bosh.io/docs/init-aws.html

I am guessing I have used the wrong procedure, is taht correct?

Thanks in advance,
Kinjal


On Sat, Jan 30, 2016 at 1:27 AM, Amit Gupta <agupta(a)pivotal.io>
wrote:

Hi Kinjal,

The task logs have sensitive credentials in them. "bosh tasks 255
--debug" will give that output, and will probably also include the full
manifest in the output. You may wish to sanitize the output before sharing
it or send me the output privately (agupta(a)pivotal.io) if you're
concerned about leaking some info.

Best,
Amit

On Fri, Jan 29, 2016 at 11:44 AM, Kinjal Doshi <kindoshi(a)gmail.com>
wrote:

Hi

I have resolved the errors in generating deployment manifest. on
executing bosh deploy, the below error is encountered while compiling
packages:

Started compiling packages
Started compiling packages >
rtr/2d7de4f6fc25938c21c5be87174f95583feb14b5
Started compiling packages >
syslog_drain_binder/3c9c0b02c11c8dba10d059fe07e6d2ee641ec053
Started compiling packages >
routing-api/b4a3e7034c4a925aa42d45419b46ad6b128d92b1
Started compiling packages >
collector/158398837665181c70bd786b46e6f4d772523017
Failed compiling packages >
routing-api/b4a3e7034c4a925aa42d45419b46ad6b128d92b1: Timed out pinging to
dc15da09-8086-4231-a5b4-15efafa27eaf after 600 seconds (00:11:03)
Failed compiling packages >
syslog_drain_binder/3c9c0b02c11c8dba10d059fe07e6d2ee641ec053: Timed out
pinging to d150aff4-095c-4d48-8c6d-f182fc3738c7 after 600 seconds (00:11:03)
Failed compiling packages >
collector/158398837665181c70bd786b46e6f4d772523017: Timed out pinging to
824b2de9-bb39-4b24-8491-4e26f79adb50 after 600 seconds (00:11:03)
Failed compiling packages >
rtr/2d7de4f6fc25938c21c5be87174f95583feb14b5: Timed out pinging to
4d636c66-690a-43e7-8481-71258732d066 after 600 seconds (00:11:35)

Error 450002: Timed out pinging to
dc15da09-8086-4231-a5b4-15efafa27eaf after 600 seconds

Task 255 error

Would be great if some pointers can be provided to proceed
further. Please let me know if the logs for this bosh task are required.

Thanks in advance,
Kinjal


On Fri, Jan 29, 2016 at 10:45 PM, Kinjal Doshi <kindoshi(a)gmail.com

wrote:

Hi Amit,

Please ignore the unresolved nodes error in the above email. I
have been able to correct it, running into some more problems, checking it
right now.

Please do let me know about my question on the dbs, though.

Thanks in advance,
Kinjal

On Fri, Jan 29, 2016 at 1:29 PM, Kinjal Doshi <kindoshi(a)gmail.com

wrote:

Hi Amit,

Thanks a lot for your response on this.

I was trying to use the manifest generation scripts to redeploy
cf but I ran into errors during spiff merge as below:

ubuntu(a)ip-172-31-45-52:~/cf-deployment/cf-release$
scripts/generate_deployment_manifest aws ../cf-stub.yml > cf-deployment.yml
2016/01/29 07:49:05 error generating manifest: unresolved nodes:
(( static_ips(1) )) in
/home/ubuntu/cf-deployment/cf-release/templates/cf-infrastructure-aws.yml
jobs.[0].networks.[0].static_ips
(( static_ips(5, 6, 15, 16, 17, 18, 19, 20) )) in
/home/ubuntu/cf-deployment/cf-release/templates/cf-infrastructure-aws.yml
jobs.[1].networks.[0].static_ips
(( static_ips(27, 28, 29) )) in
/home/ubuntu/cf-deployment/cf-release/templates/cf-infrastructure-aws.yml
jobs.[5].networks.[0].static_ips
(( static_ips(10, 25) )) in
/home/ubuntu/cf-deployment/cf-release/templates/cf-infrastructure-aws.yml
jobs.[6].networks.[0].static_ips


The public gist pointing to the cf-stub created for this attempt
is at: https://gist.github.com/kinjaldoshi/b0dc004876d2a4615c65

I am not very sure but I think this has something to do with the
way I configured the subnets. Could you please guide me on the
corrections required here. I know how (( static_ips(27, 28, 29) )) works,
but not sure why there is a problem in resolving to the required values.

Another question, I have is on the editing instructions at:
http://docs.cloudfoundry.org/deploying/aws/cf-stub.html#editing

For the ccdb and uaadb, as per comments, is it required for me
to create a service and host these DBs as mentioned in the 'Editing
Instructions' column? In that case where can i find the DDL to create the
db and tables?


Thanks a lot in advance,
Kinjal


On Fri, Jan 29, 2016 at 10:31 AM, Amit Gupta <agupta(a)pivotal.io>
wrote:

Hi Kinjal,

The minimal-aws manifest would be quite difficult to augment to
get it to work with diego. You would need to add static IP to your private
network, add a resource pool or increase the size of an existing one, add
the consul job, colocate the consul agent with some of the CF jobs, and add
a few configuration properties that aren't in the minimal one (e.g.
loggregator.tls.ca). It's probably simpler to use the
manifest generations scripts to redeploy cf (before deploying diego).

Use:

*
http://docs.cloudfoundry.org/deploying/common/create_a_manifest.html
* http://docs.cloudfoundry.org/deploying/common/deploy.html

Let us know if you run into some difficulties. These documents
ask you to define stubs, which require you to input data from your AWS IaaS
setup, and may not exactly play nicely with the AWS setup described in the
minimal-aws doc, I'm not sure.

Best,
Amit



On Wed, Jan 27, 2016 at 3:17 AM, Kinjal Doshi <
kindoshi(a)gmail.com> wrote:

Hi Eric,

Thanks a lot for the detailed response to my query.

I used the minimal-aws.yml configuration (
https://github.com/cloudfoundry/cf-release/tree/v226/example_manifests
) to create my deployment manifest which does not have the
consul VMs set up. I am guessing that the first step would be to change
this.

In this case should I use the script generators to generate
the CF deployment manifest and re-deploy cloud foundry, or are there any
other techniques/shorter path for doing this?

Thanks in advance,
Kinjal



On Mon, Jan 25, 2016 at 6:57 AM, Eric Malm <emalm(a)pivotal.io>
wrote:

Hi, Kinjal,

The stub I included in-line in my previous email may not have
come through so well for all mail clients, so I've also included it in a
public gist at
https://gist.github.com/ematpl/149ac1bac691caae0722.

Thanks,
Eric

On Fri, Jan 22, 2016 at 6:32 PM, Eric Malm <emalm(a)pivotal.io>
wrote:

Hi, Kinjal,

Thanks for asking: this is an area in which the Diego team
is looking forward to improving documentation and tooling in the near term.
For the time being, here are some more manual instructions:

Assuming you have AWS infrastructure already provisioned for
your CF deployment (VPC, subnets, NAT box, ELBs, etc.), you should need
only to add one or more additional subnets for the VMs in the Diego
deployment, and optionally an ELB for the SSH proxy routing tier (you can
also use the HAproxy in the CF deployment to do the same load-balancing,
but you'll need to give it an Elastic IP). If you're brave, and can
coordinate the reserved sections in the CF and Diego deployment manifests'
networking configs correctly, you could even share the same subnet(s)
between the two deployments.

Once you have those subnets provisioned, you'll need to
translate their properties into the iaas-settings.yml stub that you supply
to the generate-deployment-manifest script in diego-release. Since you're
deploying CF v226, we recommend you use Diego final version v0.1442.0 and
the associated manifest-generation script in that version of the release.
The other stubs should be independent of that iaas-settings one, and should
be pretty much the same as the ones for the BOSH-Lite deployment. You'll
likely want to provide different secrets and credentials in the
property-overrides stub, though, and perhaps different instance counts
depending on the availability needs of your deployment. I've included at
the end of this email a representative iaas-settings.yml file from one of
the Diego team's environments, with any specific identifiers for AWS
entities replaced by PLACEHOLDER values.

As a side note, if you don't already have the consul VMs
deployed in your CF deployment, you'll need to enable them so that the
Diego components can use it to communicate. We recommend you operate an odd
number of consul VMs: 1 if don't need high availability, and 3 or 5 if you
do (as in a production environment). You can enable them by changing the
instance count on the consul_z1 and consul_z2 jobs in the CF manifest.

After you've customized those stubs and adjusted your CF
manifest if necessary, you can generate the Diego manifest by running
something like the following from your diego-release directory:

$ ./scripts/generate-deployment-manifest \
PATH/TO/MY/CUSTOMIZED-PROPERTY-OVERRIDES.YML \
PATH/TO/MY/CUSTOMIZED-INSTANCE-COUNT-OVERRIDES.YML \

manifest-generation/bosh-lite-stubs/persistent-disk-overrides.yml \
PATH/TO/MY/CUSTOMIZED-IAAS-SETTINGS.YML \
manifest-generation/bosh-lite-stubs/additional-jobs.yml \
manifest-generation/bosh-lite-stubs/release-versions.yml \
PATH/TO/MY/MANIFEST/DIRECTORY \
> PATH/TO/MY/MANIFEST/DIRECTORY/diego.yml

'PATH/TO/MY/MANIFEST/DIRECTORY' should contain your CF
manifest in a file named 'cf.yml'. Also, please note that if you move to CF
v227 or later, which recommend Diego v0.1445.0 or later, the
manifest-generation script has changed to take its stub arguments via
flags, instead of as these positional arguments, and some of the stubs have
changed slightly.

We also realize this is currently an obscure and potentially
error-prone process, and the Diego team does have a couple stories queued
up to do soon to provide more information about how to set up Diego on AWS:

- We plan in
https://www.pivotaltracker.com/story/show/100909610 to
parametrize, document, and publish the tools and additional templates we
use to provision the AWS environments we use for CI and for our developers'
experiments and investigations, all the way from an empty account to a VPC
with BOSH, CF, and Diego.
- We plan in
https://www.pivotaltracker.com/story/show/100909610 to
provide more manual instructions to set up a Diego environment compatible
with the 'minimal-aws' CF deployment manifest and infrastructure settings,
including provisioning any additional infrastructure such as subnets and
translating their information into the stubs for the diego-release
manifest-generation script.

We'll also be eager to adopt and to integrate with the
tooling the CF Infrastructure and CF Release Integration teams will produce
at some point to automate environment bootstrapping and CF manifest
generation as much as possible.

Please let me and the rest of the team know here if you need
further assistance or clarification.

Thanks again,
Eric, CF Runtime Diego PM

*****

Example iaas-settings.yml file, with PLACEHOLDER entries for
your environment's info:

iaas_settings:
compilation_cloud_properties:
availability_zone: us-east-1a
instance_type: c3.large
resource_pool_cloud_properties:
- cloud_properties:
availability_zone: us-east-1a
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z1
- cloud_properties:
availability_zone: us-east-1b
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z2
- cloud_properties:
availability_zone: us-east-1c
elbs:
- PLACEHOLDER-SSHProxyELB-ID
instance_type: m3.medium
name: access_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: brain_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: brain_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: brain_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: cc_bridge_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: cc_bridge_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: cc_bridge_z3
- cloud_properties:
availability_zone: us-east-1a
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z1
- cloud_properties:
availability_zone: us-east-1b
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z2
- cloud_properties:
availability_zone: us-east-1c
ephemeral_disk:
iops: 1200
size: 50000
type: io1
instance_type: m3.large
name: cell_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.large
name: colocated_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.large
name: colocated_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.large
name: colocated_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.large
name: database_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.large
name: database_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.large
name: database_z3
- cloud_properties:
availability_zone: us-east-1a
instance_type: m3.medium
name: route_emitter_z1
- cloud_properties:
availability_zone: us-east-1b
instance_type: m3.medium
name: route_emitter_z2
- cloud_properties:
availability_zone: us-east-1c
instance_type: m3.medium
name: route_emitter_z3
stemcell:
name: bosh-aws-xen-hvm-ubuntu-trusty-go_agent
version: latest
subnet_configs:
- name: diego1
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-A
dns:
- 10.10.0.2
gateway: 10.10.5.1
range: 10.10.5.0/24
reserved:
- 10.10.5.2 - 10.10.5.9
static:
- 10.10.5.10 - 10.10.5.63
- name: diego2
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-B
dns:
- 10.10.0.2
gateway: 10.10.6.1
range: 10.10.6.0/24
reserved:
- 10.10.6.2 - 10.10.6.9
static:
- 10.10.6.10 - 10.10.6.63
- name: diego3
subnets:
- cloud_properties:
security_groups:
- PLACEHOLDER-InternalSecurityGroup-ID
subnet: PLACEHOLDER-subnet-id-C
dns:
- 10.10.0.2
gateway: 10.10.7.1
range: 10.10.7.0/24
reserved:
- 10.10.7.2 - 10.10.7.9
static:
- 10.10.7.10 - 10.10.7.63


On Fri, Jan 22, 2016 at 4:28 AM, Kinjal Doshi <
kindoshi(a)gmail.com> wrote:

Hi,

After deploying CF version 226 on AWS using microbosh, I am
trying to understand how to deploy Diego now to work with this version of
CF but have not been able to figure out much yet. I was able to find steps
for deploying Diego on BOSH-Lite at
https://github.com/cloudfoundry-incubator/diego-release#deploying-diego-to-bosh-lite
but not for BOSH.

Would appreciate some pointers in this direction .

Thanks in advance,
Kinjal

Thanks a lot Dieu, that answers my question.

I have another question about creating a deployment manifest using the
following as guidelines: http://do
<http://docs.cloudfoundry.org/deploying/aws/cf-stub.html#editing>
cs.cloudfoundry.org/deploying/aws/cf-stub.html#editing
<http://docs.cloudfoundry.org/deploying/aws/cf-stub.html#editing>

The stud mentions that it requires 2 networks cf1 and cf2. I have created
a VPC with CIDR as 10.10.0.0/16 and am creating two public subnets cf1
and cf2 with CIDR as 10.10.160/20 and 10.10.80.0/20. Is that the correct
approach? Or are these required to be private subnets.

In case of the minimal-aws deployment one private and one public subnet
was required but here since they are used for different zones, I am not
sure if both should be private or public or 1 each. Would be great if you
could please help me understand.

Thanks,
Kinjal





On Wed, Feb 3, 2016 at 1:05 AM, Dieu Cao <dcao(a)pivotal.io> wrote:

Hi Kinjal,

We generally recommend naming it with a specific identifiable folder name
that is not the bucket root.

This might look like:
...
buildpack_directory_key: buildpacks
....
droplet_directory_key: droplets
...

or
...
buildpack_directory_key: dev-cc-buildpacks
...
droplet_directory_key: dev-cc-droplets
...

Hope that helps.
Similarly for app_package_directory_key and resource_directory_key

-Dieu
CF CAPI PM

On Tue, Feb 2, 2016 at 7:29 AM, Kinjal Doshi <kindoshi(a)gmail.com> wrote:

Hi,

I have a question regarding the configuration of cloud controller.

It is described at
http://docs.cloudfoundry.org/deploying/aws/cf-stub.html#editing that
DROPLET_DIRECTORY_KEY should be replaced with the directory/bucket used to
store the droplets and BUILDPACK_DIRECTORY_KEY should be replaced with the
directory/bucket used to store buildpacks

Please help me understand if there should be specific values for these
two parameters or it could be just any random directory name which is
created later?

Thanks,
Kinjal

Based on this discussion, where are we on the priority of switching the
current "port" check for the Windows lifecycle back to actually be a port
check? I get the impression that the changes to support a new HTTP check
in
the CC, CLI, BBS, etc. will probably take a while so until then I'm hoping
we can make the other change a bit quicker.

Aaron



--
View this message in context:
http://cf-dev.70369.x6.nabble.com/Issue-with-crashing-Windows-apps-on-Diego-tp3586p3686.html
Sent from the CF Dev mailing list archive at Nabble.com.

Hi Gwenn

Thanks for your feedback, and taking the time to look through the
changelog!

cf-release/consul-release have been on 0.5.2 (post BoltDB migration) since
Jun 2015:
https://github.com/cloudfoundry/cf-release/commit/1683884704c91ec1609efe3c8390c058ccc6c537.
So there should be no concerns around any migrations from LMDB.

Best,
Amit

On Wed, Feb 3, 2016 at 6:04 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:

My biggest concern is the Data migration from LMDB to BoltDB, I know
there is a CLI to do that but I am concern about the migration of the DATA
itself.


*" Previously, Consul would automatically detect data directories using
the old LMDB format, and convert them to the newer BoltDB format. This
automatic upgrade has been removed for Consul 0.6, and instead a safeguard
has been put in place which will prevent Consul from booting if the old
directory format is detected. "*
*AND*
* Consul should be provisioned with physical memory approximately 2X the
data set size to allow for bursty allocations and subsequent garbage
collection.*



On Thu, Feb 4, 2016 at 3:28 AM, Amit Gupta <agupta(a)pivotal.io> wrote:

Hi cf-dev, cf-diego

The CF Infrastructure team which maintains consul-release (
https://github.com/cloudfoundry-incubator/consul-release) is
considering bumping from consul 0.5.2 to 0.6.0. There are many changes
between those consecutive releases, which you can read about here:

* 0.6.0:
https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#060-december-3-2015
* 0.6.1:
https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#061-january-6-2016
* 0.6.2:
https://github.com/hashicorp/consul/blob/master/CHANGELOG.md#062-january-13-2016

We would like to explore this bump to get the change in 0.6.0 where
consul moved to a pure Go implementation, allowing us to avoid having to
set GOMAXPROCS when starting the consul server, and then some of the
security fixes in 0.6.2 for good measure.

This change would also affect clients of the consul server in
consul-release, hopefully in mainly positive ways. If you're a consumer of
consul-release as a client, I would appreciate your feedback on this
potential bump to consul in consul-release.

Thanks,
Amit, CF Infrastructure team PM