Re: scheduler


Onsi Fakhouri <ofakhouri@...>
 

Diego is very much usable at this point and we're encouraging beta testers to start putting workloads on it. Check out github.com/cloudfoundry-incubator/diego for all things Diego.

Diego supports one off tasks. It's up to the consumer (e.g. Cloud Controller) to submit the tasks when they want them run. We'd like to bubble this functionality up to the CC but it's not a very high priority at the moment.

Onsi

Sent from my iPad

On May 26, 2015, at 8:21 AM, Corentin Dupont <corentin.dupont(a)create-net.org> wrote:

Another question, what is the status of Diego? Is there an expected date for its release?
Is it useable already?
If I understand correctly, Diego doesn't supports cron-like jobs, but will facilitate them?

On Tue, May 26, 2015 at 5:08 PM, James Bayer <jbayer(a)pivotal.io> wrote:
those are exciting use cases, thank you for sharing the background!


On Tue, May 26, 2015 at 2:37 AM, Corentin Dupont <cdupont(a)create-net.org> wrote:
Hi James, thanks for the answer!
We are interested to implement a job scheduler for CF. Do you think this could be interesting to have?

We are working in a project called DC4Cities (http://www.dc4cities.eu) were the objective is to make data centres use more renewable energy.
We want to use PaaS frameworks such as CloudFoundry to achieve this goal.
The idea is to schedule some PaaS tasks at the moment there is more renewable energies (when the sun is shining).

That's why I had the idea to implement a job scheduler for batch jobs in CF. For example one could state "I need to have this task to run for 2 hours per day" and the scheduler could choose when to run it.

Another possibility is to have application-oriented SLA implemented at CF level. For example if some KPIs of the application are getting too low, CF would spark a new container. If the SLA is defined with some flexibility, it could also be used to schedule renewable energies. For example in our trial scenarios we have an application that convert images. Its SLA says that it needs to convert 1000 images per day, but you are free to produce them when you want i.e. when renewable energies are available...


On Mon, May 25, 2015 at 7:29 PM, James Bayer <jbayer(a)pivotal.io> wrote:
there is ongoing work to support process types using buildpacks, so that the same application codebase could be used for multiple different types of processes (web, worker, etc).

once process types and diego tasks are fully available, we expect to implement a user-facing api for running batch jobs as application processes.

what people do today is run a long-running process application which uses something like quartz scheduler [1] or ruby clock with a worker system like resque [2]

[1] http://quartz-scheduler.org/
[2] https://github.com/resque/resque-scheduler

On Mon, May 25, 2015 at 6:19 AM, Corentin Dupont <cdupont(a)create-net.org> wrote:
To complete my request, I'm thinking of something like this in the manifest.yml:

applications:
- name: virusscan
memory: 512M
instances: 1
schedule:
- startFrom : a date
endBefore : a date
walltime : a duration
precedence : other application name
moldable : true/false

What do you think?

On Mon, May 25, 2015 at 11:25 AM, Corentin Dupont <cdupont(a)create-net.org> wrote:


---------- Forwarded message ----------
From: Corentin Dupont <corentin.dupont(a)create-net.org>
Date: Mon, May 25, 2015 at 11:21 AM
Subject: scheduler
To: cf-dev(a)lists.cloudfoundry.org


Hi guys,
just to know, is there a project to add a job scheduler in Cloud Foundry?
I'm thinking of something like the Heroku scheduler (https://devcenter.heroku.com/articles/scheduler).
That would be very neat to have regular tasks triggered...
Thanks,
Corentin


--
Corentin Dupont
Researcher @ Create-Net
www.corentindupont.info

_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev


--
Thank you,

James Bayer


--
Thank you,

James Bayer


--
Corentin Dupont
Researcher @ Create-Net
www.corentindupont.info
_______________________________________________
cf-dev mailing list
cf-dev(a)lists.cloudfoundry.org
https://lists.cloudfoundry.org/mailman/listinfo/cf-dev

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.