Date
1 - 4 of 4
Cloud foundry deployment on vSphere : help on stub file parameters
ML D
Hi all
I installed bosh 261.4.0 and now I would like to install cloundfoundry :-)
I read https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html, but I need more information to setup the Deployment Manifest Stub.
Perhaps this page is not the right one to understand all the parameters of this Stub.
About networks:
The sample shows 2 networks deployment. But what are the rules to decide to deploy CF on 1, 2 or 3 networks ? Is it according the number of AZ ? Or for networks segmentation ? If segmentation, between which networks ?
What are the reserved adresses ?
Static adresses belongs from a subnet without DHCP ? But for which purpose using static adresses ?
About system domain : it is said You must have <..> configured their DNS records so that these domains resolve to your load balancer.
So it means a LB is a requirement. So ha-proxy could be the right stuff. But how to deploy it with bosh ? Command line ? .yml file ? But what to write in the file ? It would be great if ha-proxy was installed directly during cf install.
What is a system domain organization ?
And will I get a gui after deploying cf to manage users, orgs and space ?
Thank you very much for any help
Best regards
MLD
I installed bosh 261.4.0 and now I would like to install cloundfoundry :-)
I read https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html, but I need more information to setup the Deployment Manifest Stub.
Perhaps this page is not the right one to understand all the parameters of this Stub.
About networks:
The sample shows 2 networks deployment. But what are the rules to decide to deploy CF on 1, 2 or 3 networks ? Is it according the number of AZ ? Or for networks segmentation ? If segmentation, between which networks ?
What are the reserved adresses ?
Static adresses belongs from a subnet without DHCP ? But for which purpose using static adresses ?
About system domain : it is said You must have <..> configured their DNS records so that these domains resolve to your load balancer.
So it means a LB is a requirement. So ha-proxy could be the right stuff. But how to deploy it with bosh ? Command line ? .yml file ? But what to write in the file ? It would be great if ha-proxy was installed directly during cf install.
What is a system domain organization ?
And will I get a gui after deploying cf to manage users, orgs and space ?
Thank you very much for any help
Best regards
MLD
David Sabeti
Hi Marc-Laurant,
Look here <http://bosh.io/docs/networks.html> for more general information
about BOSH networks. This <http://bosh.io/docs/vsphere-cpi.html#networks> has
some more details about vSphere specifically. I'll try to address your
questions one at a time:
1. The manifest generation scripts provided by cf-release assume 2 networks
-- typically to provide redundancy. Deploying to 1 AZ is fine for
development or test environments, and deploying to 3 is possible but not
really necessary. To deploy 1 network, you'll need to put something strange
in your stub; to deploy to 3 networks, you'll need to add a new network to
the deployment manifest manually after running the normal manifest
generation scripts.
2. vSphere itself has a notion of a network that you can create through the
vSphere GUI -- that's what the BOSH networks correspond to. As the
deployer, you choose which vSphere networks you want your CF deployment to
occupy, and set the network name accordingly.
3. Reserved addresses are IP addresses that are "reserved" for other VMs
besides BOSH -- placing an IP range or CIDR block in that section of the
manifest tells BOSH not to assign any of those IP addresses to the VMs that
it manages.
4. The Static IPs section is important because a few jobs need to be
assigned static IPs so that all other jobs know how to address them. For
example, most jobs in the deployment need to connect to the Consul cluster,
so the Consul servers must live at specific IP addresses in order to be
found by their clients
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf.yml#L1321>.
In your stub, you should make sure to have about 20 static IPs in each
network.
5. The manifest generation scripts for vSphere will ensure that BOSH
deploys an HAProxy
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-vsphere.yml#L188-L189>.
The system domain should resolve to the (public) IP address of the HAProxy.
6. The vSphere doc
<https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html>
you pointed to gives directions for that field -- you can generate any name
as the value -- but it doesn't explain the purpose of the field.
Technically, domains in CF belong to organizations. That field simply
creates an organization, accessible only by the admin, that holds the
system_domain.
7. There is not GUI for CF by default. To start using your new deployment,
use the CF CLI <https://github.com/cloudfoundry/cli>.
Hope that helps!
David Sabeti
CF Release Integration
toggle quoted message
Show quoted text
Look here <http://bosh.io/docs/networks.html> for more general information
about BOSH networks. This <http://bosh.io/docs/vsphere-cpi.html#networks> has
some more details about vSphere specifically. I'll try to address your
questions one at a time:
1. The manifest generation scripts provided by cf-release assume 2 networks
-- typically to provide redundancy. Deploying to 1 AZ is fine for
development or test environments, and deploying to 3 is possible but not
really necessary. To deploy 1 network, you'll need to put something strange
in your stub; to deploy to 3 networks, you'll need to add a new network to
the deployment manifest manually after running the normal manifest
generation scripts.
2. vSphere itself has a notion of a network that you can create through the
vSphere GUI -- that's what the BOSH networks correspond to. As the
deployer, you choose which vSphere networks you want your CF deployment to
occupy, and set the network name accordingly.
3. Reserved addresses are IP addresses that are "reserved" for other VMs
besides BOSH -- placing an IP range or CIDR block in that section of the
manifest tells BOSH not to assign any of those IP addresses to the VMs that
it manages.
4. The Static IPs section is important because a few jobs need to be
assigned static IPs so that all other jobs know how to address them. For
example, most jobs in the deployment need to connect to the Consul cluster,
so the Consul servers must live at specific IP addresses in order to be
found by their clients
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf.yml#L1321>.
In your stub, you should make sure to have about 20 static IPs in each
network.
5. The manifest generation scripts for vSphere will ensure that BOSH
deploys an HAProxy
<https://github.com/cloudfoundry/cf-release/blob/master/templates/cf-infrastructure-vsphere.yml#L188-L189>.
The system domain should resolve to the (public) IP address of the HAProxy.
6. The vSphere doc
<https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html>
you pointed to gives directions for that field -- you can generate any name
as the value -- but it doesn't explain the purpose of the field.
Technically, domains in CF belong to organizations. That field simply
creates an organization, accessible only by the admin, that holds the
system_domain.
7. There is not GUI for CF by default. To start using your new deployment,
use the CF CLI <https://github.com/cloudfoundry/cli>.
Hope that helps!
David Sabeti
CF Release Integration
On Fri, Mar 24, 2017 at 10:12 AM Marc-Laurent Delaruelle <
marc-laurent.delaruelle(a)renault.com> wrote:
Hi all
I installed bosh 261.4.0 and now I would like to install cloundfoundry :-)
I read
https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html,
but I need more information to setup the Deployment Manifest Stub.
Perhaps this page is not the right one to understand all the parameters of
this Stub.
About networks:
The sample shows 2 networks deployment. But what are the rules to decide to
deploy CF on 1, 2 or 3 networks ? Is it according the number of AZ ? Or for
networks segmentation ? If segmentation, between which networks ?
What are the reserved adresses ?
Static adresses belongs from a subnet without DHCP ? But for which purpose
using static adresses ?
About system domain : it is said You must have <..> configured their DNS
records so that these domains resolve to your load balancer.
So it means a LB is a requirement. So ha-proxy could be the right stuff.
But how to deploy it with bosh ? Command line ? .yml file ? But what to
write in the file ? It would be great if ha-proxy was installed directly
during cf install.
What is a system domain organization ?
And will I get a gui after deploying cf to manage users, orgs and space ?
Thank you very much for any help
Best regards
MLD
marc-laurent.delaruelle(a)renault.com> wrote:
Hi all
I installed bosh 261.4.0 and now I would like to install cloundfoundry :-)
I read
https://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html,
but I need more information to setup the Deployment Manifest Stub.
Perhaps this page is not the right one to understand all the parameters of
this Stub.
About networks:
The sample shows 2 networks deployment. But what are the rules to decide to
deploy CF on 1, 2 or 3 networks ? Is it according the number of AZ ? Or for
networks segmentation ? If segmentation, between which networks ?
What are the reserved adresses ?
Static adresses belongs from a subnet without DHCP ? But for which purpose
using static adresses ?
About system domain : it is said You must have <..> configured their DNS
records so that these domains resolve to your load balancer.
So it means a LB is a requirement. So ha-proxy could be the right stuff.
But how to deploy it with bosh ? Command line ? .yml file ? But what to
write in the file ? It would be great if ha-proxy was installed directly
during cf install.
What is a system domain organization ?
And will I get a gui after deploying cf to manage users, orgs and space ?
Thank you very much for any help
Best regards
MLD
ML D
Hi,
Thank you very much.
1. What do you mean : To deploy 1 network, you'll need to put something strange in your stub ? What is the strange thing to add ?
2. I also go forward in the stub file building with : URL http://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html
Adding all the certificate is painfull. So I decided to write a code to automate the inserts of the cert/keys in the stub. I share it at the end of this mail. The script read a data file with the key=filepath to insert them with the right indent. As I'm a newbie I don't dare to put it in Github... :-)
About the URL
- In http://docs.cloudfoundry.org/deploying/common/consul-security.html, it is said to use scripts/generate-consul-certs. But theses scripts are not presents in the cf-release. I found them here
https://github.com/cloudfoundry-incubator/consul-release/archive/45e03acbbb993ac0cb93c1996406a31c5682ce39.zip
I don't know if I'm rigth to use them or if I miss something.
- For jwt, it is said to create the key using this command : openssl rsa -in jwt-key.pem -pubout > key.pub, But the private jw-key.pem do not exists. So I did
openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
openssl rsa -in key.pem -pubout > key.pub
Right ?
- For hm9000: doc says to setup theses,
ca_cert: HM9000_CA_CERT
server_cert: HM9000_SERVER_CERT
server_key: HM9000_SERVER_KEY
agent_cert: HM9000_AGENT_CERT
agent_key: HM9000_AGENT_KEY
But the stub sample file have something different :
hm9000:
server_key: HM9000_SERVER_KEY
server_cert: HM9000_SERVER_CERT
client_key: HM9000_CLIENT_KEY
client_cert: HM9000_CLIENT_CERT
ca_cert: HM9000_CA_CERT
Do I need to setup all CERT/KEY of the sample stub (including HM9000_CLIENT_*) as stated in the sample stub file or setup only those documented and remove the HM9000_CLIENT_* entries ?
- etcd : in the sample stub file, there is
But nothing is said about thes CERT/KEY in the url. cf-release have a script to generate them. But do I need to use it and put the certificates in the stub ??
etcd:
require_ssl: true
ca_cert: ETCD_CA_CERT
client_cert: ETCD_CLIENT_CERT
client_key: ETCD_CLIENT_KEY
peer_ca_cert: ETCD_PEER_CA_CERT
peer_cert: ETCD_PEER_CERT
peer_key: ETC_PEER_KEY
server_cert: ETCD_SERVER_CERT
server_key: ETCD_SERVER_KEY
- Same for
statsd_injector: (script to create certs exists)
cert: LOGGREGATOR_STATSDINJECTOR_CERT
key: LOGGREGATOR_STATSDINJECTOR_KEY
uaa:
admin: (script to create certs exists)
client_secret: ADMIN_SECRET
ca_cert: UAA_CA_CERT
scim
sslCertificate: UAA_SERVER_CERT
sslPrivateKey: UAA_SERVER_KEY
___________________________
SCRIPT :
# Make a backup
DATE=`date +%Y-%m-%d-%H%M%S`
stubfile=~/automation/CfManifestStub.yml
cp $stubfile $stubfile.$DATE.yml
stubfile=$stubfile.$DATE.yml
echo $stubfile
#cp ~/CfManifestStub.yml ~/CfManifestStub.yml.bak
# Generate certificates
cd ~/cf-release
#./cf-release-master/scripts/generate-cf-diego-certs
#./consul-release/consul-release-45e03acbbb993ac0cb93c1996406a31c5682ce39/scripts/generate-certs
#./cf-release-master/scripts/generate-loggregator-certs cf-diego-certs/cf-diego-ca.crt cf-diego-certs/cf-diego-ca.key
#mkdir ~/cf-release/http;cd ~/cf-release/http
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
# leave everything as default. You get 2 files cert.pem and key.pem
#mkdir ~/cf-release/jwt;cd ~/cf-release/jwt
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
#openssl rsa -in key.pem -pubout > key.pub
#./cf-release-master/scripts/generate-hm9000-certs
#mkdir jobs;cd jobs
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
# enter for hostname *.cff.pocx86.tstwinx.net
cd ~/automation
while read line ; do
IFS='='; entry=($line); unset IFS
echo "Adding data for ${entry[0]}"
#get the number of leading spaces of the lines with the tag to replace by cert/key then add 2 spaces for json
blanks=$(grep "${entry[0]}" $stubfile | awk -F'[^ ]' '{print length($1)}')
blanks=$(($blanks+2))
#add the number of blanks to the file of the cert/key
temp=${entry[1]}
echo "temp=$temp"
awk '{printf "%"'$blanks'"s%s\n", "", $0}' "${entry[1]}" > /tmp/entry.tmp
cat /tmp/entry.tmp
#insert the file in the stub file under the tag to replace
sed -i '/'${entry[0]}'/r /tmp/entry.tmp' $stubfile
# remove the tag and add the |
sed -i 's/'${entry[0]}'/|/' $stubfile
done < tag-files.txt
# Adding the 2 last keys
# First removing the comments
head --lines=-6 $stubfile > /tmp/file.tmp
mv /tmp/file.tmp $stubfile
cat $stubfile
# Adding the keys
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/key.pem > /tmp/entry1.tmp
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/cert.pem > /tmp/entry2.tmp
cat $stubfile /tmp/entry1.tmp /tmp/entry2.tmp > /tmp/file.tmp
rm /tmp/entry1.tmp;rm /tmp/entry2.tmp
mv /tmp/file.tmp $stubfile
echo $stubfile updated with certificates/keys
[bosh(a)jumpboxcf automation]$ cat tag-files.txt
CC_MUTUAL_TLS_CA_CERT=../cf-release/cf-diego-certs/cf-diego-ca.crt
CC_MUTUAL_TLS_PUBLIC_CERT=../cf-release/cf-diego-certs/cloud-controller.crt
CC_MUTUAL_TLS_PRIVATE_KEY=../cf-release/cf-diego-certs/cloud-controller.key
CONSUL_CA_CERT=../cf-release/consul-certs/server-ca.crt
CONSUL_SERVER_CERT=../cf-release/consul-certs/server.crt
CONSUL_SERVER_KEY=../cf-release/consul-certs/server.key
CONSUL_AGENT_CERT=../cf-release/consul-certs/agent.crt
CONSUL_AGENT_KEY=../cf-release/consul-certs/agent.key
LOGGREGATOR_CA_CERT=../cf-release/loggregator-certs/loggregator-ca.crt
LOGGREGATOR_DOPPLER_CERT=../cf-release/loggregator-certs/doppler.crt
LOGGREGATOR_DOPPLER_KEY=../cf-release/loggregator-certs/doppler.key
LOGGREGATOR_TRAFFICCONTROLLER_CERT=../cf-release/loggregator-certs/trafficcontroller.crt
LOGGREGATOR_TRAFFICCONTROLLER_KEY=../cf-release/loggregator-certs/trafficcontroller.key
LOGGREGATOR_METRON_CERT=../cf-release/loggregator-certs/metron.crt
LOGGREGATOR_METRON_KEY=../cf-release/loggregator-certs/metron.key
LOGGREGATOR_SYSLOGDRAINBINDER_CERT=../cf-release/loggregator-certs/syslogdrainbinder.crt
LOGGREGATOR_SYSLOGDRAINBINDER_KEY=../cf-release/loggregator-certs/syslogdrainbinder.key
SERVICE_PROVIDER_PRIVATE_KEY=../cf-release/http/key.pem
JWT_VERIFICATION_KEY=../cf-release/jwt/key.pem
JWT_SIGNING_KEY=../cf-release/jwt/key.pub
HM9000_CA_CERT=../cf-release/hm9000-certs/hm9000_ca.crt
HM9000_SERVER_CERT=../cf-release/hm9000-certs/hm9000_server.crt
HM9000_SERVER_KEY=../cf-release/hm9000-certs/hm9000_server.key
HM9000_AGENT_CERT=../cf-release/hm9000-certs/hm9000_client.crt
HM9000_AGENT_KEY=../cf-release/hm9000-certs/hm9000_client.key
[bosh(a)jumpboxcf automation]$
Thank you very much.
1. What do you mean : To deploy 1 network, you'll need to put something strange in your stub ? What is the strange thing to add ?
2. I also go forward in the stub file building with : URL http://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html
Adding all the certificate is painfull. So I decided to write a code to automate the inserts of the cert/keys in the stub. I share it at the end of this mail. The script read a data file with the key=filepath to insert them with the right indent. As I'm a newbie I don't dare to put it in Github... :-)
About the URL
- In http://docs.cloudfoundry.org/deploying/common/consul-security.html, it is said to use scripts/generate-consul-certs. But theses scripts are not presents in the cf-release. I found them here
https://github.com/cloudfoundry-incubator/consul-release/archive/45e03acbbb993ac0cb93c1996406a31c5682ce39.zip
I don't know if I'm rigth to use them or if I miss something.
- For jwt, it is said to create the key using this command : openssl rsa -in jwt-key.pem -pubout > key.pub, But the private jw-key.pem do not exists. So I did
openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
openssl rsa -in key.pem -pubout > key.pub
Right ?
- For hm9000: doc says to setup theses,
ca_cert: HM9000_CA_CERT
server_cert: HM9000_SERVER_CERT
server_key: HM9000_SERVER_KEY
agent_cert: HM9000_AGENT_CERT
agent_key: HM9000_AGENT_KEY
But the stub sample file have something different :
hm9000:
server_key: HM9000_SERVER_KEY
server_cert: HM9000_SERVER_CERT
client_key: HM9000_CLIENT_KEY
client_cert: HM9000_CLIENT_CERT
ca_cert: HM9000_CA_CERT
Do I need to setup all CERT/KEY of the sample stub (including HM9000_CLIENT_*) as stated in the sample stub file or setup only those documented and remove the HM9000_CLIENT_* entries ?
- etcd : in the sample stub file, there is
But nothing is said about thes CERT/KEY in the url. cf-release have a script to generate them. But do I need to use it and put the certificates in the stub ??
etcd:
require_ssl: true
ca_cert: ETCD_CA_CERT
client_cert: ETCD_CLIENT_CERT
client_key: ETCD_CLIENT_KEY
peer_ca_cert: ETCD_PEER_CA_CERT
peer_cert: ETCD_PEER_CERT
peer_key: ETC_PEER_KEY
server_cert: ETCD_SERVER_CERT
server_key: ETCD_SERVER_KEY
- Same for
statsd_injector: (script to create certs exists)
cert: LOGGREGATOR_STATSDINJECTOR_CERT
key: LOGGREGATOR_STATSDINJECTOR_KEY
uaa:
admin: (script to create certs exists)
client_secret: ADMIN_SECRET
ca_cert: UAA_CA_CERT
scim
sslCertificate: UAA_SERVER_CERT
sslPrivateKey: UAA_SERVER_KEY
___________________________
SCRIPT :
# Make a backup
DATE=`date +%Y-%m-%d-%H%M%S`
stubfile=~/automation/CfManifestStub.yml
cp $stubfile $stubfile.$DATE.yml
stubfile=$stubfile.$DATE.yml
echo $stubfile
#cp ~/CfManifestStub.yml ~/CfManifestStub.yml.bak
# Generate certificates
cd ~/cf-release
#./cf-release-master/scripts/generate-cf-diego-certs
#./consul-release/consul-release-45e03acbbb993ac0cb93c1996406a31c5682ce39/scripts/generate-certs
#./cf-release-master/scripts/generate-loggregator-certs cf-diego-certs/cf-diego-ca.crt cf-diego-certs/cf-diego-ca.key
#mkdir ~/cf-release/http;cd ~/cf-release/http
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
# leave everything as default. You get 2 files cert.pem and key.pem
#mkdir ~/cf-release/jwt;cd ~/cf-release/jwt
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
#openssl rsa -in key.pem -pubout > key.pub
#./cf-release-master/scripts/generate-hm9000-certs
#mkdir jobs;cd jobs
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out cert.pem
# enter for hostname *.cff.pocx86.tstwinx.net
cd ~/automation
while read line ; do
IFS='='; entry=($line); unset IFS
echo "Adding data for ${entry[0]}"
#get the number of leading spaces of the lines with the tag to replace by cert/key then add 2 spaces for json
blanks=$(grep "${entry[0]}" $stubfile | awk -F'[^ ]' '{print length($1)}')
blanks=$(($blanks+2))
#add the number of blanks to the file of the cert/key
temp=${entry[1]}
echo "temp=$temp"
awk '{printf "%"'$blanks'"s%s\n", "", $0}' "${entry[1]}" > /tmp/entry.tmp
cat /tmp/entry.tmp
#insert the file in the stub file under the tag to replace
sed -i '/'${entry[0]}'/r /tmp/entry.tmp' $stubfile
# remove the tag and add the |
sed -i 's/'${entry[0]}'/|/' $stubfile
done < tag-files.txt
# Adding the 2 last keys
# First removing the comments
head --lines=-6 $stubfile > /tmp/file.tmp
mv /tmp/file.tmp $stubfile
cat $stubfile
# Adding the keys
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/key.pem > /tmp/entry1.tmp
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/cert.pem > /tmp/entry2.tmp
cat $stubfile /tmp/entry1.tmp /tmp/entry2.tmp > /tmp/file.tmp
rm /tmp/entry1.tmp;rm /tmp/entry2.tmp
mv /tmp/file.tmp $stubfile
echo $stubfile updated with certificates/keys
[bosh(a)jumpboxcf automation]$ cat tag-files.txt
CC_MUTUAL_TLS_CA_CERT=../cf-release/cf-diego-certs/cf-diego-ca.crt
CC_MUTUAL_TLS_PUBLIC_CERT=../cf-release/cf-diego-certs/cloud-controller.crt
CC_MUTUAL_TLS_PRIVATE_KEY=../cf-release/cf-diego-certs/cloud-controller.key
CONSUL_CA_CERT=../cf-release/consul-certs/server-ca.crt
CONSUL_SERVER_CERT=../cf-release/consul-certs/server.crt
CONSUL_SERVER_KEY=../cf-release/consul-certs/server.key
CONSUL_AGENT_CERT=../cf-release/consul-certs/agent.crt
CONSUL_AGENT_KEY=../cf-release/consul-certs/agent.key
LOGGREGATOR_CA_CERT=../cf-release/loggregator-certs/loggregator-ca.crt
LOGGREGATOR_DOPPLER_CERT=../cf-release/loggregator-certs/doppler.crt
LOGGREGATOR_DOPPLER_KEY=../cf-release/loggregator-certs/doppler.key
LOGGREGATOR_TRAFFICCONTROLLER_CERT=../cf-release/loggregator-certs/trafficcontroller.crt
LOGGREGATOR_TRAFFICCONTROLLER_KEY=../cf-release/loggregator-certs/trafficcontroller.key
LOGGREGATOR_METRON_CERT=../cf-release/loggregator-certs/metron.crt
LOGGREGATOR_METRON_KEY=../cf-release/loggregator-certs/metron.key
LOGGREGATOR_SYSLOGDRAINBINDER_CERT=../cf-release/loggregator-certs/syslogdrainbinder.crt
LOGGREGATOR_SYSLOGDRAINBINDER_KEY=../cf-release/loggregator-certs/syslogdrainbinder.key
SERVICE_PROVIDER_PRIVATE_KEY=../cf-release/http/key.pem
JWT_VERIFICATION_KEY=../cf-release/jwt/key.pem
JWT_SIGNING_KEY=../cf-release/jwt/key.pub
HM9000_CA_CERT=../cf-release/hm9000-certs/hm9000_ca.crt
HM9000_SERVER_CERT=../cf-release/hm9000-certs/hm9000_server.crt
HM9000_SERVER_KEY=../cf-release/hm9000-certs/hm9000_server.key
HM9000_AGENT_CERT=../cf-release/hm9000-certs/hm9000_client.crt
HM9000_AGENT_KEY=../cf-release/hm9000-certs/hm9000_client.key
[bosh(a)jumpboxcf automation]$
David Sabeti
1. Oh, I was hoping you wouldn't ask! The manifest generation scripts
expect that you have two networks: `cf1` and `cf2`. In order to deploy to a
single network, you can "trick" the scripts into thinking that your single
network is actually two networks:
```
networks:
- name: cf1
type: manual
subnets:
- range: ...
....
- name: cf2
type: manual
subnets: (( networks.cf1.subnets )) # Alias cf1 to cf2 to satisfy
manifest generation
```
For this to deploy successfully, you'll have to reduce the instance count
of all *_z2 jobs to 0. *I should mention that this is very atypical and I
would suggest that you deploy with two networks instead.*
2. Yeah, writing scripts is always nicer than manual work. If you need more
help, you could also look at the cf-release scripts
<https://github.com/cloudfoundry/cf-release/tree/master/scripts> for
generating certs. Also, cf-deployment (what we will eventually use to
replace the manifest generation scripts) declares
<https://github.com/cloudfoundry/cf-deployment/blob/master/cf-deployment.yml#L1220-L1450>
how to generate those certs as well -- I should emphasize that
cf-deployment is still being tested and not ready for general consumption,
but you look there for help with the certs.
For the consul-certs script, make sure you have an up-to-date cf-release
and have run `scripts/update` to sync all the submodules. Once that's done,
you should find the script in the scripts directory
<https://github.com/cloudfoundry/cf-release/blob/master/scripts/generate-consul-certs>
with everything else.
For the jwt signing key, I'm pretty sure you don't need to generate an x509
certificate. Instead, all you want is a simple RSA private key:
```
openssl genrsa -out privatekey 1024
```
For HM9000 configuration, follow the same stub. You should generate the
following values:
- HM9000_SERVER_KEY
- HM9000_SERVER_CERT
- HM9000_CLIENT_KEY
- HM9000_CLIENT_CERT
- HM9000_CA_CERT
Again, you can use the scripts
<https://github.com/cloudfoundry/cf-release/blob/master/scripts/generate-hm9000-certs>
in cf-release.
For the remaining certs (etcd, statsd injector, and uaa), you should
generate those certs as well, again, using the scripts in cf-release. This
doc
<https://github.com/cloudfoundry/cf-release/blob/develop/on-tls-certificates.md>
tries to explain how to generate all of the necessary certificates for CF.
Unfortunately, the docs get stale because they lag behind when we add new
required certificates.
Hope that helps!
David
On Wed, Mar 29, 2017 at 8:37 AM Marc-Laurent Delaruelle <
marc-laurent.delaruelle(a)renault.com> wrote:
expect that you have two networks: `cf1` and `cf2`. In order to deploy to a
single network, you can "trick" the scripts into thinking that your single
network is actually two networks:
```
networks:
- name: cf1
type: manual
subnets:
- range: ...
....
- name: cf2
type: manual
subnets: (( networks.cf1.subnets )) # Alias cf1 to cf2 to satisfy
manifest generation
```
For this to deploy successfully, you'll have to reduce the instance count
of all *_z2 jobs to 0. *I should mention that this is very atypical and I
would suggest that you deploy with two networks instead.*
2. Yeah, writing scripts is always nicer than manual work. If you need more
help, you could also look at the cf-release scripts
<https://github.com/cloudfoundry/cf-release/tree/master/scripts> for
generating certs. Also, cf-deployment (what we will eventually use to
replace the manifest generation scripts) declares
<https://github.com/cloudfoundry/cf-deployment/blob/master/cf-deployment.yml#L1220-L1450>
how to generate those certs as well -- I should emphasize that
cf-deployment is still being tested and not ready for general consumption,
but you look there for help with the certs.
For the consul-certs script, make sure you have an up-to-date cf-release
and have run `scripts/update` to sync all the submodules. Once that's done,
you should find the script in the scripts directory
<https://github.com/cloudfoundry/cf-release/blob/master/scripts/generate-consul-certs>
with everything else.
For the jwt signing key, I'm pretty sure you don't need to generate an x509
certificate. Instead, all you want is a simple RSA private key:
```
openssl genrsa -out privatekey 1024
```
For HM9000 configuration, follow the same stub. You should generate the
following values:
- HM9000_SERVER_KEY
- HM9000_SERVER_CERT
- HM9000_CLIENT_KEY
- HM9000_CLIENT_CERT
- HM9000_CA_CERT
Again, you can use the scripts
<https://github.com/cloudfoundry/cf-release/blob/master/scripts/generate-hm9000-certs>
in cf-release.
For the remaining certs (etcd, statsd injector, and uaa), you should
generate those certs as well, again, using the scripts in cf-release. This
doc
<https://github.com/cloudfoundry/cf-release/blob/develop/on-tls-certificates.md>
tries to explain how to generate all of the necessary certificates for CF.
Unfortunately, the docs get stale because they lag behind when we add new
required certificates.
Hope that helps!
David
On Wed, Mar 29, 2017 at 8:37 AM Marc-Laurent Delaruelle <
marc-laurent.delaruelle(a)renault.com> wrote:
Hi,
Thank you very much.
1. What do you mean : To deploy 1 network, you'll need to put something
strange in your stub ? What is the strange thing to add ?
2. I also go forward in the stub file building with : URL
http://docs.cloudfoundry.org/deploying/common/vsphere-vcloud-cf-stub.html
Adding all the certificate is painfull. So I decided to write a code to
automate the inserts of the cert/keys in the stub. I share it at the end of
this mail. The script read a data file with the key=filepath to insert them
with the right indent. As I'm a newbie I don't dare to put it in Github...
:-)
About the URL
- In http://docs.cloudfoundry.org/deploying/common/consul-security.html,
it is said to use scripts/generate-consul-certs. But theses scripts are not
presents in the cf-release. I found them here
https://github.com/cloudfoundry-incubator/consul-release/archive/45e03acbbb993ac0cb93c1996406a31c5682ce39.zip
I don't know if I'm rigth to use them or if I miss something.
- For jwt, it is said to create the key using this command : openssl rsa
-in jwt-key.pem -pubout > key.pub, But the private jw-key.pem do not
exists. So I did
openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out
cert.pem
openssl rsa -in key.pem -pubout > key.pub
Right ?
- For hm9000: doc says to setup theses,
ca_cert: HM9000_CA_CERT
server_cert: HM9000_SERVER_CERT
server_key: HM9000_SERVER_KEY
agent_cert: HM9000_AGENT_CERT
agent_key: HM9000_AGENT_KEY
But the stub sample file have something different :
hm9000:
server_key: HM9000_SERVER_KEY
server_cert: HM9000_SERVER_CERT
client_key: HM9000_CLIENT_KEY
client_cert: HM9000_CLIENT_CERT
ca_cert: HM9000_CA_CERT
Do I need to setup all CERT/KEY of the sample stub (including
HM9000_CLIENT_*) as stated in the sample stub file or setup only those
documented and remove the HM9000_CLIENT_* entries ?
- etcd : in the sample stub file, there is
But nothing is said about thes CERT/KEY in the url. cf-release have a
script to generate them. But do I need to use it and put the certificates
in the stub ??
etcd:
require_ssl: true
ca_cert: ETCD_CA_CERT
client_cert: ETCD_CLIENT_CERT
client_key: ETCD_CLIENT_KEY
peer_ca_cert: ETCD_PEER_CA_CERT
peer_cert: ETCD_PEER_CERT
peer_key: ETC_PEER_KEY
server_cert: ETCD_SERVER_CERT
server_key: ETCD_SERVER_KEY
- Same for
statsd_injector: (script to create certs exists)
cert: LOGGREGATOR_STATSDINJECTOR_CERT
key: LOGGREGATOR_STATSDINJECTOR_KEY
uaa:
admin: (script to create certs exists)
client_secret: ADMIN_SECRET
ca_cert: UAA_CA_CERT
scim
sslCertificate: UAA_SERVER_CERT
sslPrivateKey: UAA_SERVER_KEY
___________________________
SCRIPT :
# Make a backup
DATE=`date +%Y-%m-%d-%H%M%S`
stubfile=~/automation/CfManifestStub.yml
cp $stubfile $stubfile.$DATE.yml
stubfile=$stubfile.$DATE.yml
echo $stubfile
#cp ~/CfManifestStub.yml ~/CfManifestStub.yml.bak
# Generate certificates
cd ~/cf-release
#./cf-release-master/scripts/generate-cf-diego-certs
#./consul-release/consul-release-45e03acbbb993ac0cb93c1996406a31c5682ce39/scripts/generate-certs
#./cf-release-master/scripts/generate-loggregator-certs
cf-diego-certs/cf-diego-ca.crt cf-diego-certs/cf-diego-ca.key
#mkdir ~/cf-release/http;cd ~/cf-release/http
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out
cert.pem
# leave everything as default. You get 2 files cert.pem and key.pem
#mkdir ~/cf-release/jwt;cd ~/cf-release/jwt
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out
cert.pem
#openssl rsa -in key.pem -pubout > key.pub
#./cf-release-master/scripts/generate-hm9000-certs
#mkdir jobs;cd jobs
#openssl req -x509 -nodes -newkey rsa:2048 -days 365 -keyout key.pem -out
cert.pem
# enter for hostname *.cff.pocx86.tstwinx.net
cd ~/automation
while read line ; do
IFS='='; entry=($line); unset IFS
echo "Adding data for ${entry[0]}"
#get the number of leading spaces of the lines with the tag to replace by
cert/key then add 2 spaces for json
blanks=$(grep "${entry[0]}" $stubfile | awk -F'[^ ]' '{print length($1)}')
blanks=$(($blanks+2))
#add the number of blanks to the file of the cert/key
temp=${entry[1]}
echo "temp=$temp"
awk '{printf "%"'$blanks'"s%s\n", "", $0}' "${entry[1]}" > /tmp/entry.tmp
cat /tmp/entry.tmp
#insert the file in the stub file under the tag to replace
sed -i '/'${entry[0]}'/r /tmp/entry.tmp' $stubfile
# remove the tag and add the |
sed -i 's/'${entry[0]}'/|/' $stubfile
done < tag-files.txt
# Adding the 2 last keys
# First removing the comments
head --lines=-6 $stubfile > /tmp/file.tmp
mv /tmp/file.tmp $stubfile
cat $stubfile
# Adding the keys
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/key.pem >
/tmp/entry1.tmp
awk '{printf "%"'8'"s%s\n", "", $0}' ../cf-release/jobs/cert.pem >
/tmp/entry2.tmp
cat $stubfile /tmp/entry1.tmp /tmp/entry2.tmp > /tmp/file.tmp
rm /tmp/entry1.tmp;rm /tmp/entry2.tmp
mv /tmp/file.tmp $stubfile
echo $stubfile updated with certificates/keys
[bosh(a)jumpboxcf automation]$ cat tag-files.txt
CC_MUTUAL_TLS_CA_CERT=../cf-release/cf-diego-certs/cf-diego-ca.crt
CC_MUTUAL_TLS_PUBLIC_CERT=../cf-release/cf-diego-certs/cloud-controller.crt
CC_MUTUAL_TLS_PRIVATE_KEY=../cf-release/cf-diego-certs/cloud-controller.key
CONSUL_CA_CERT=../cf-release/consul-certs/server-ca.crt
CONSUL_SERVER_CERT=../cf-release/consul-certs/server.crt
CONSUL_SERVER_KEY=../cf-release/consul-certs/server.key
CONSUL_AGENT_CERT=../cf-release/consul-certs/agent.crt
CONSUL_AGENT_KEY=../cf-release/consul-certs/agent.key
LOGGREGATOR_CA_CERT=../cf-release/loggregator-certs/loggregator-ca.crt
LOGGREGATOR_DOPPLER_CERT=../cf-release/loggregator-certs/doppler.crt
LOGGREGATOR_DOPPLER_KEY=../cf-release/loggregator-certs/doppler.key
LOGGREGATOR_TRAFFICCONTROLLER_CERT=../cf-release/loggregator-certs/trafficcontroller.crt
LOGGREGATOR_TRAFFICCONTROLLER_KEY=../cf-release/loggregator-certs/trafficcontroller.key
LOGGREGATOR_METRON_CERT=../cf-release/loggregator-certs/metron.crt
LOGGREGATOR_METRON_KEY=../cf-release/loggregator-certs/metron.key
LOGGREGATOR_SYSLOGDRAINBINDER_CERT=../cf-release/loggregator-certs/syslogdrainbinder.crt
LOGGREGATOR_SYSLOGDRAINBINDER_KEY=../cf-release/loggregator-certs/syslogdrainbinder.key
SERVICE_PROVIDER_PRIVATE_KEY=../cf-release/http/key.pem
JWT_VERIFICATION_KEY=../cf-release/jwt/key.pem
JWT_SIGNING_KEY=../cf-release/jwt/key.pub
HM9000_CA_CERT=../cf-release/hm9000-certs/hm9000_ca.crt
HM9000_SERVER_CERT=../cf-release/hm9000-certs/hm9000_server.crt
HM9000_SERVER_KEY=../cf-release/hm9000-certs/hm9000_server.key
HM9000_AGENT_CERT=../cf-release/hm9000-certs/hm9000_client.crt
HM9000_AGENT_KEY=../cf-release/hm9000-certs/hm9000_client.key
[bosh(a)jumpboxcf automation]$