Date
1 - 5 of 5
database migration
Jeff Weber <jweber@...>
I've got a monolithic CF deployment that has become quite a burden to
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as gotchas
or should this be a straight forward set of tasks?
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as gotchas
or should this be a straight forward set of tasks?
James Bayer
that looks pretty good. i'd probably change the following:
8) Stop jobs on old database cluster
9) Validate CF is working and connected to new database
10) Validate old database files are not changed since last backup and
back-up to new set of backup files just in case
11) Remove old database instance from the manifest
toggle quoted message
Show quoted text
8) Stop jobs on old database cluster
9) Validate CF is working and connected to new database
10) Validate old database files are not changed since last backup and
back-up to new set of backup files just in case
11) Remove old database instance from the manifest
On Mon, Feb 15, 2016 at 10:46 AM, Jeff Weber <jweber(a)cofront.net> wrote:
I've got a monolithic CF deployment that has become quite a burden to
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as gotchas
or should this be a straight forward set of tasks?
--
Thank you,
James Bayer
Thank you,
James Bayer
Amit Kumar Gupta
Hey Jeff,
Can you describe the burden you're feeling from the monolithic deployment?
We are actively working on splitting up *cf-release* into multiple smaller,
reusable, composable *releases*, but the *manifest deployment* will stay as
one unit to make it easier to ensure that your pieces are working together,
configurations make sense together, and knowing the state of your full
cluster is easier (if you're upgrading e.g. how do you know you've upgraded
all your smaller deployments if you separate them?)
Cheers,
Amit
toggle quoted message
Show quoted text
Can you describe the burden you're feeling from the monolithic deployment?
We are actively working on splitting up *cf-release* into multiple smaller,
reusable, composable *releases*, but the *manifest deployment* will stay as
one unit to make it easier to ensure that your pieces are working together,
configurations make sense together, and knowing the state of your full
cluster is easier (if you're upgrading e.g. how do you know you've upgraded
all your smaller deployments if you separate them?)
Cheers,
Amit
On Mon, Feb 15, 2016 at 10:46 AM, Jeff Weber <jweber(a)cofront.net> wrote:
I've got a monolithic CF deployment that has become quite a burden to
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as gotchas
or should this be a straight forward set of tasks?
Jeff Weber <jweber@...>
I believe in the long run the current work being done in BOSH around AZ and
links will get rid of this problem for us, but I have to figure out a way
to upgrade in the interim. My main goal currently is to remove the
deployment of our core CF database and another instance of cf-mysql-release
service from the cloud foundry manifest to make working with the
combination easier to reason about what changes impact what things. This
will leave me in a situation where I can try to fix up the rest of my
deployment without worrying about breaking the databases.
The CF manifest problem its self is a mix between not having a good grasp
on how to use spiff to compose a custom manifest and our OpenStack
deployment requirements.
The out of the box cf-jobs.yml only includes job templates for two zones
and to deal with my environment today I need a minimum of five possible
placement zones which means creating three extra sets of jobs and then
updating them every time a new release comes out. This has led to having
custom cf-jobs.yml when new properties or release requirements are added
sometimes they're missed during the upgrade process. Maybe there is a
simpler way to keep the baseline cf-jobs.yml and also add new jobs and
reference a job from cf-jobs.yml to inherit the properties I need instead
of breaking things into pieces so its easier to keep updated with what's
changed piece by piece?
toggle quoted message
Show quoted text
links will get rid of this problem for us, but I have to figure out a way
to upgrade in the interim. My main goal currently is to remove the
deployment of our core CF database and another instance of cf-mysql-release
service from the cloud foundry manifest to make working with the
combination easier to reason about what changes impact what things. This
will leave me in a situation where I can try to fix up the rest of my
deployment without worrying about breaking the databases.
The CF manifest problem its self is a mix between not having a good grasp
on how to use spiff to compose a custom manifest and our OpenStack
deployment requirements.
The out of the box cf-jobs.yml only includes job templates for two zones
and to deal with my environment today I need a minimum of five possible
placement zones which means creating three extra sets of jobs and then
updating them every time a new release comes out. This has led to having
custom cf-jobs.yml when new properties or release requirements are added
sometimes they're missed during the upgrade process. Maybe there is a
simpler way to keep the baseline cf-jobs.yml and also add new jobs and
reference a job from cf-jobs.yml to inherit the properties I need instead
of breaking things into pieces so its easier to keep updated with what's
changed piece by piece?
On Tue, Feb 16, 2016 at 2:12 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Hey Jeff,
Can you describe the burden you're feeling from the monolithic deployment?
We are actively working on splitting up *cf-release* into multiple
smaller, reusable, composable *releases*, but the *manifest deployment*
will stay as one unit to make it easier to ensure that your pieces are
working together, configurations make sense together, and knowing the state
of your full cluster is easier (if you're upgrading e.g. how do you know
you've upgraded all your smaller deployments if you separate them?)
Cheers,
Amit
On Mon, Feb 15, 2016 at 10:46 AM, Jeff Weber <jweber(a)cofront.net> wrote:I've got a monolithic CF deployment that has become quite a burden to
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as
gotchas or should this be a straight forward set of tasks?
Jeff Weber <jweber@...>
Thanks for the feedback.
I've updated my plans with those steps which should help make it safer to
verify the migration did work.
toggle quoted message
Show quoted text
I've updated my plans with those steps which should help make it safer to
verify the migration did work.
On Tue, Feb 16, 2016 at 1:42 PM, James Bayer <jbayer(a)pivotal.io> wrote:
that looks pretty good. i'd probably change the following:
8) Stop jobs on old database cluster
9) Validate CF is working and connected to new database
10) Validate old database files are not changed since last backup and
back-up to new set of backup files just in case
11) Remove old database instance from the manifest
On Mon, Feb 15, 2016 at 10:46 AM, Jeff Weber <jweber(a)cofront.net> wrote:I've got a monolithic CF deployment that has become quite a burden to
upgraded because of it's all releases in one manifest nature. I'm currently
working on a plan to cleanup and disaggregate and the first step is to
migrate my core database into its own separate deployment.
After putting my plan together I wanted to confirm it's as simple as it
seems:
1) Deploy new database cluster
2) Stop CF services
3) Perform backup of databases on old cluster
4) Restore backups to new cluster
5) Reconfigure manifest to point to new database cluster
6) Redeploy / Start CF services with new manifest
7) Validate
8) Remove old database instance from the manifest
Is there anything outside of these basic steps to watch out for as
gotchas or should this be a straight forward set of tasks?
--
Thank you,
James Bayer