Date
1 - 8 of 8
Pluggable Resource Scheduling
Deepak Vij
Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos.
Look forward to comments and feedback from the community. Thanks. Regards, Deepak Vij (Huawei Software Lab., Santa Clara) |
|
Gwenn Etourneau
Hi,
Interesting proposition, wondering if it make sense to hook into Diego or CF. Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ? Thanks On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com> wrote: Hi folks, I would like to start a discussion thread and get community |
|
Deepak Vij
I did not mean to replace the whole “Diego” environment itself. What I was thinking was more in terms of plug-ability within Diego itself. This is so that “Auctioneer” component can be turned into a “Mesos Framework” as one of the scheduling options. By doing that, “Auctioneer” can start accepting “Mesos Offers” instead of native “Auctioning based Diego Resource Scheduling”. Rest of the runtime environment including Garden, Rep etc., they all stay the same. Nothing else changes. I hope this makes sense.
- Deepak From: Gwenn Etourneau [mailto:getourneau(a)pivotal.io] Sent: Wednesday, November 11, 2015 5:10 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Pluggable Resource Scheduling Hi, Interesting proposition, wondering if it make sense to hook into Diego or CF. Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ? Thanks On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote: Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos. Look forward to comments and feedback from the community. Thanks. Regards, Deepak Vij (Huawei Software Lab., Santa Clara) |
|
Zhang Lei <harryzhang@...>
You can add different scheduling strategy into Diego by implementing a scheduler plugin.
But not Mesos, that would be a huge task and another story. The reason Kubernetes can integrate Mesos as scheduler (can work, not perfect) is due to Mesosphere is doing that part, I'm afraid ... 在 2015-11-13 03:57:52,"Deepak Vij (A)" <deepak.vij(a)huawei.com> 写道: I did not mean to replace the whole “Diego” environment itself. What I was thinking was more in terms of plug-ability within Diego itself. This is so that “Auctioneer” component can be turned into a “Mesos Framework” as one of the scheduling options. By doing that, “Auctioneer” can start accepting “Mesos Offers” instead of native “Auctioning based Diego Resource Scheduling”. Rest of the runtime environment including Garden, Rep etc., they all stay the same. Nothing else changes. I hope this makes sense. - Deepak From: Gwenn Etourneau [mailto:getourneau(a)pivotal.io] Sent: Wednesday, November 11, 2015 5:10 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Pluggable Resource Scheduling Hi, Interesting proposition, wondering if it make sense to hook into Diego or CF. Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ? Thanks On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com> wrote: Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos. Look forward to comments and feedback from the community. Thanks. Regards, Deepak Vij (Huawei Software Lab., Santa Clara) |
|
Deepak Vij
If Mesosphere can do it for Kubernetes, CF/Diego is doable too. I am not worried about that, it is a solved problem.
The only concern I have is regarding deployment of Garden Container environment in Mesos/Slave/Executor. Although, this is not CF/Diego issues. Because Mesos is not Garden Container environment aware, its underlying DRF scheduling algorithm may not have visibility to the resources being consumed within the Garden Container. Unless, we wrap Garden Container within the Docker container as Mesos supports Docker container environment. Although, this may not be right approach as it opens up another can of worms – Gardner Container nested within Docker Container. For Kubernetes environment this is not an issue as it uses Docker container to begin with. - Deepak From: resouer(a)163.com [mailto:resouer(a)163.com] On Behalf Of Zhang Lei Sent: Thursday, November 12, 2015 8:24 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Pluggable Resource Scheduling You can add different scheduling strategy into Diego by implementing a scheduler plugin. But not Mesos, that would be a huge task and another story. The reason Kubernetes can integrate Mesos as scheduler (can work, not perfect) is due to Mesosphere is doing that part, I'm afraid ... 在 2015-11-13 03:57:52,"Deepak Vij (A)" <deepak.vij(a)huawei.com> 写道: I did not mean to replace the whole “Diego” environment itself. What I was thinking was more in terms of plug-ability within Diego itself. This is so that “Auctioneer” component can be turned into a “Mesos Framework” as one of the scheduling options. By doing that, “Auctioneer” can start accepting “Mesos Offers” instead of native “Auctioning based Diego Resource Scheduling”. Rest of the runtime environment including Garden, Rep etc., they all stay the same. Nothing else changes. I hope this makes sense. - Deepak From: Gwenn Etourneau [mailto:getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>] Sent: Wednesday, November 11, 2015 5:10 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Pluggable Resource Scheduling Hi, Interesting proposition, wondering if it make sense to hook into Diego or CF. Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ? Thanks On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote: Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos. Look forward to comments and feedback from the community. Thanks. Regards, Deepak Vij (Huawei Software Lab., Santa Clara) |
|
Idit Levine
We can add to mesosphere a native support for Garden containers. That is very easy to do ....
toggle quoted message
Show quoted text
Sent from my iPhone On Nov 13, 2015, at 2:51 PM, Deepak Vij (A) <deepak.vij(a)huawei.com> wrote: |
|
Deepak Vij
Hi Idit, good to hear from you. In that case, we have covered all bases and good to go on this. We will touch base as we discussed earlier. Thanks.
- Deepak From: Gmail [mailto:idit.levine(a)gmail.com] Sent: Friday, November 13, 2015 1:43 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Re: Re: Pluggable Resource Scheduling We can add to mesosphere a native support for Garden containers. That is very easy to do .... Sent from my iPhone On Nov 13, 2015, at 2:51 PM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote: If Mesosphere can do it for Kubernetes, CF/Diego is doable too. I am not worried about that, it is a solved problem. The only concern I have is regarding deployment of Garden Container environment in Mesos/Slave/Executor. Although, this is not CF/Diego issues. Because Mesos is not Garden Container environment aware, its underlying DRF scheduling algorithm may not have visibility to the resources being consumed within the Garden Container. Unless, we wrap Garden Container within the Docker container as Mesos supports Docker container environment. Although, this may not be right approach as it opens up another can of worms – Gardner Container nested within Docker Container. For Kubernetes environment this is not an issue as it uses Docker container to begin with. - Deepak From: resouer(a)163.com<mailto:resouer(a)163.com> [mailto:resouer(a)163.com] On Behalf Of Zhang Lei Sent: Thursday, November 12, 2015 8:24 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Pluggable Resource Scheduling You can add different scheduling strategy into Diego by implementing a scheduler plugin. But not Mesos, that would be a huge task and another story. The reason Kubernetes can integrate Mesos as scheduler (can work, not perfect) is due to Mesosphere is doing that part, I'm afraid ... 在 2015-11-13 03:57:52,"Deepak Vij (A)" <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> 写道: I did not mean to replace the whole “Diego” environment itself. What I was thinking was more in terms of plug-ability within Diego itself. This is so that “Auctioneer” component can be turned into a “Mesos Framework” as one of the scheduling options. By doing that, “Auctioneer” can start accepting “Mesos Offers” instead of native “Auctioning based Diego Resource Scheduling”. Rest of the runtime environment including Garden, Rep etc., they all stay the same. Nothing else changes. I hope this makes sense. - Deepak From: Gwenn Etourneau [mailto:getourneau(a)pivotal.io<mailto:getourneau(a)pivotal.io>] Sent: Wednesday, November 11, 2015 5:10 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Pluggable Resource Scheduling Hi, Interesting proposition, wondering if it make sense to hook into Diego or CF. Diego is connected to CF by the CC-Bridge (big picture) why not create a CC-Bridge for other scheduling system ? Thanks On Thu, Nov 12, 2015 at 5:13 AM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote: Hi folks, I would like to start a discussion thread and get community thoughts regarding availability of Pluggable Resource Scheduling within CF/Diego. Just like Kubernetes does, wouldn’t it be nice to have an option of choosing Diego native scheduling or other uber/global resource management environments, specifically Mesos. Look forward to comments and feedback from the community. Thanks. Regards, Deepak Vij (Huawei Software Lab., Santa Clara) |
|
taichi nakashima
+1 having this pluggable scheduling feature !
toggle quoted message
Show quoted text
-- tcnksm 2015年11月14日(土) 6:55 Deepak Vij (A) <deepak.vij(a)huawei.com>: Hi Idit, good to hear from you. In that case, we have covered all bases |
|