Can resources of a IDLE application be shared by others?


Stanley Shen <meteorping@...>
 

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.


Stanley Shen <meteorping@...>
 

It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.

This statement is wrong, and the resources will be release after the application is stopped.


Daniel Mikusa
 

On Tue, Mar 8, 2016 at 10:09 PM, Stanley Shen <meteorping(a)gmail.com> wrote:

Hello, all

When pushing an application to CF, we need to define its disk/memory
limitation.
The memory limitation is just the possible maximum value will be needed in
this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at
startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is
allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should
can push more applications, like 3 since only 2G is needed for each
application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing
application, it will not be released even I stopped the application.


Deepak Vij
 

Over allocation of computing resources is a very serious problem in the contemporary cloud computing environment and across enterprise data centers in general. Famous Quasar research paper published by Stanford researches talks about this issue in great detail. There are startling numbers such as industry-wide utilization between 6% and 12%. A recent study estimated server utilization on Amazon EC2 in the 3% to 17% range.

Techniques which Google ("Borg") internally employs is called "Resource Reclamation" whereby over allocated resources are taken advantage of for executing best-effort workloads such as background analytics, and other low priority jobs. Such best-effort workloads could be pre-empted in case of any interference to the original normal running workload or resources are needed back at a later time. In order to be able to utilize unused resources in such a manner ("Resource Reclamation") requires underlying support for capabilities such as "Preemption & Resizing", "Isolation Mechanisms", "Interference Detection" etc.

Another important aspect of all this is the fact that users never know how to precisely do resource reservations - this leads to underutilization of computing resources. Instead, what is being proposed is some kind of classification based predictive analytics techniques whereby resource management system itself determines right amount of resources to meet the user performance constraints. In this case, user only specifies the performance constraints (SLO) not the actual low level resource reservations itself. Combination of predictive analytics based proactive approach in conjunction with reactive "resource reclamation" approach is the optimal strategy. This accounts for any mis-predictions as well.

Mesos Resource Management also supports "Resource Reclamation". Although, both Google ("Borg") and Mesos do not employ predictive analytics based proactive approach in their environment.

I hope this makes sense and helps.

- Deepak Vij (Huawei, Software Lab., Santa Clara)

-----Original Message-----
From: Stanley Shen [mailto:meteorping(a)gmail.com]
Sent: Tuesday, March 08, 2016 7:10 PM
To: cf-dev(a)lists.cloudfoundry.org
Subject: [cf-dev] Can resources of a IDLE application be shared by others?

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.


Montenegro, Allan Mauricio (EESO CC PREMIER) <amontenegro@...>
 

Hello,



I'm sorry, I'm constantly receiving emails from this source, in which I have no part on, as I am a new Hewlett Packard Enterprise Employee.



Please remove my email.



Thanks,

Montenegro Murillo, Allan
Support Specialist
ITIO Customer Care


+1-877-785-2155 Office
Heredia, Costa Rica
allan.montenegro(a)hpe.com<mailto:allan.montenegro(a)hpe.com>


[cid:image001.jpg(a)01D11DDE.2289DAA0]

-----Original Message-----
From: Deepak Vij (A) [mailto:deepak.vij(a)huawei.com]
Sent: Wednesday, March 09, 2016 1:09 PM
To: Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Can resources of a IDLE application be shared by others?



Over allocation of computing resources is a very serious problem in the contemporary cloud computing environment and across enterprise data centers in general. Famous Quasar research paper published by Stanford researches talks about this issue in great detail. There are startling numbers such as industry-wide utilization between 6% and 12%. A recent study estimated server utilization on Amazon EC2 in the 3% to 17% range.



Techniques which Google ("Borg") internally employs is called "Resource Reclamation" whereby over allocated resources are taken advantage of for executing best-effort workloads such as background analytics, and other low priority jobs. Such best-effort workloads could be pre-empted in case of any interference to the original normal running workload or resources are needed back at a later time. In order to be able to utilize unused resources in such a manner ("Resource Reclamation") requires underlying support for capabilities such as "Preemption & Resizing", "Isolation Mechanisms", "Interference Detection" etc.



Another important aspect of all this is the fact that users never know how to precisely do resource reservations - this leads to underutilization of computing resources. Instead, what is being proposed is some kind of classification based predictive analytics techniques whereby resource management system itself determines right amount of resources to meet the user performance constraints. In this case, user only specifies the performance constraints (SLO) not the actual low level resource reservations itself. Combination of predictive analytics based proactive approach in conjunction with reactive "resource reclamation" approach is the optimal strategy. This accounts for any mis-predictions as well.



Mesos Resource Management also supports "Resource Reclamation". Although, both Google ("Borg") and Mesos do not employ predictive analytics based proactive approach in their environment.



I hope this makes sense and helps.



- Deepak Vij (Huawei, Software Lab., Santa Clara)



-----Original Message-----

From: Stanley Shen [mailto:meteorping(a)gmail.com]

Sent: Tuesday, March 08, 2016 7:10 PM

To: cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>

Subject: [cf-dev] Can resources of a IDLE application be shared by others?



Hello, all



When pushing an application to CF, we need to define its disk/memory limitation.

The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.

For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.

So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.



Take m3.large VM for example, it has 7.5G.

Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.



Can the resources of a IDLE application be shared by other applications?

It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.


Benjamin Gandon
 

I suggest you manually “cf scale -m 2G“ after your app has booted.
Type “cf scale --help” for more info.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com> a écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.


Stanley Shen <meteorping@...>
 

Yes, it's one way but it's not flexible, and scale app need to restart the app as well.
As I said I may have some heavy operations which will definitely need more than 2G.

In my opinion the ideal way is that we just set a maximum value for each process, but during the running of the process, we don't pre-allocate the memory as we specify as the maximum in deployment.

I suggest you manually “cf scale -m 2G“ after your app has booted.
Type “cf scale --help” for more info.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com&gt; a écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this
application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some
some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is
allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more
applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it
will not be released even I stopped the application.


Amit Kumar Gupta
 

Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used to
determine if the container needs to be killed when it exceeds it. However,
since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that on
a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust the
containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not sure
whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com> wrote:

Yes, it's one way but it's not flexible, and scale app need to restart the
app as well.
As I said I may have some heavy operations which will definitely need more
than 2G.

In my opinion the ideal way is that we just set a maximum value for each
process, but during the running of the process, we don't pre-allocate the
memory as we specify as the maximum in deployment.





I suggest you manually “cf scale -m 2G“ after your app has booted.
Type “cf scale --help” for more info.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com&gt; a
écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory
limitation.
The memory limitation is just the possible maximum value will be
needed in this
application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at
startup some
some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G
memory is
allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we
should can push more
applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other
applications?
It seems right now all the resources are pre-allocated when pushing
application, it
will not be released even I stopped the application.


Stanley Shen <meteorping@...>
 

Thanks for information.

I said "pre-allocated" because after I pushed an APP with 5G memory specified, if I go to Cell VM, and I notice the available memory is 5G less than totally memory via "curl -s http://localhost:1800/state" on Cell VM.

I think overcommit factor is not very suitable in my case, but "resource reclamation and predictive analytics" are quite helpful, and it's a quite useful/flexible mechanism.

Do we have any plan on such cool ideas?

Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used to
determine if the container needs to be killed when it exceeds it. However,
since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that on
a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust the
containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not sure
whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com&gt; wrote:


Amit Kumar Gupta
 

Hey Stanley,

Yup, that /state endpoint is for the Cell's bookkeeping (it's essentially
implementing the overcommit factor of 1, aka no overcommit), but it doesn't
mean physical memory has been allocated. But yes, for your purposes, it
doesn't matter if actual memory has been allocated or if for bookkeeping
purposes that memory is "reserved" by the Cell, since either way you can't
add more work to the Cell when it's reported available memory is less than
the max limit you want to put on your new container.

There are no near-term plans I know of for those features Deepak mentioned
(resource reclamation/pre-emption, and predictive analytics), I'll let the
Diego PM know you're interested. I've also mentioned it to Pivotal's data
science team as a possible optimization to explore.

Best,
Amit

On Sun, Mar 13, 2016 at 7:06 PM, Stanley Shen <meteorping(a)gmail.com> wrote:

Thanks for information.

I said "pre-allocated" because after I pushed an APP with 5G memory
specified, if I go to Cell VM, and I notice the available memory is 5G less
than totally memory via "curl -s http://localhost:1800/state" on Cell VM.

I think overcommit factor is not very suitable in my case, but "resource
reclamation and predictive analytics" are quite helpful, and it's a quite
useful/flexible mechanism.

Do we have any plan on such cool ideas?






Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used
to
determine if the container needs to be killed when it exceeds it.
However,
since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that
on
a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust
the
containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not
sure
whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com&gt;
wrote: