Re: Can resources of a IDLE application be shared by others?


Amit Kumar Gupta
 

Hey Stanley,

Yup, that /state endpoint is for the Cell's bookkeeping (it's essentially
implementing the overcommit factor of 1, aka no overcommit), but it doesn't
mean physical memory has been allocated. But yes, for your purposes, it
doesn't matter if actual memory has been allocated or if for bookkeeping
purposes that memory is "reserved" by the Cell, since either way you can't
add more work to the Cell when it's reported available memory is less than
the max limit you want to put on your new container.

There are no near-term plans I know of for those features Deepak mentioned
(resource reclamation/pre-emption, and predictive analytics), I'll let the
Diego PM know you're interested. I've also mentioned it to Pivotal's data
science team as a possible optimization to explore.

Best,
Amit

On Sun, Mar 13, 2016 at 7:06 PM, Stanley Shen <meteorping(a)gmail.com> wrote:

Thanks for information.

I said "pre-allocated" because after I pushed an APP with 5G memory
specified, if I go to Cell VM, and I notice the available memory is 5G less
than totally memory via "curl -s http://localhost:1800/state" on Cell VM.

I think overcommit factor is not very suitable in my case, but "resource
reclamation and predictive analytics" are quite helpful, and it's a quite
useful/flexible mechanism.

Do we have any plan on such cool ideas?






Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used
to
determine if the container needs to be killed when it exceeds it.
However,
since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that
on
a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust
the
containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not
sure
whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com&gt;
wrote:

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.