1. Messages
  2. CF app that helps with self-healing

Re: CF app that helps with self-healing

Siva <mailsiva@...>
 

Hi Daniel,
Thanks for your response.
I am aware of all the options you are suggesting. But what we are looking for is a process to restart an app instance without human intervention from an alert policy in our monitoring system. This monitoring system is outside of CF and does not have access to CF CLI. But it can access REST endpoints.
For eg - The monitoring system will detect a high CPU utilization on one of the app instance. It will raise an alert which will trigger a policy that will call a REST endpoint of this self healing app. Based on the parameters passed in the request, the self-healing app will restart the requested app instance.
This is required when the app does not know that it is in a bad state but some metrics we are tracking are indicating that the app instance need to be restarted.
Hope that makes sense.

Thanks
Siva


On Fri, Jan 24, 2020 at 9:55 AM Daniel Mikusa <dmikusa@...> wrote:
Not sure I totally get what you are asking, but `cf restart-app-instance` will restart an instance, so if you have an alert trigger a script, you could script the restart.

Or you could just have the app itself know when it gets into a bad state, presumably it would if it's emitting the metrics to indicate this, and exit. When it exits the platform will just restart the app.

Dan


On Fri, Jan 24, 2020 at 12:30 PM Siva <mailsiva@...> wrote:
Dear CF community,
We are trying to find a way to selectively restart some instances of apps or to restart a specific app on an as needed basis based on some alerts that we receive from our monitoring solution. One option we are considering is to have a self-healing app deployed in CF which will have some REST endpoints exposed which we can call from our alert policies that will perform those actions for us. This self-healing app will essentially have the capabilities of CF CLI for stopping and starting services and instances. This app will also be protected by UAA.
Before we go off and start developing this app, I wanted to check if anyone in the CF community has thought about this approach before and have a solution in place or any ideas to consider.

Thanks,
Siva Balan



--
http://www.twitter.com/sivabalans
Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.