Hello, all

When pushing an application to CF, we need to define its disk/memory
limitation.
The memory limitation is just the possible maximum value will be needed in
this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at
startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is
allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should
can push more applications, like 3 since only 2G is needed for each
application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing
application, it will not be released even I stopped the application.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com> a écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this application, but in most time, we don't need so much memory.
For example, I have one application which needs at most 5G memory at startup some some specific operation, but in most time it just needs 2G.
So right now I need to specify 5G in deployment manifest, and 5G memory is allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it will not be released even I stopped the application.

I suggest you manually “cf scale -m 2G“ after your app has booted.
Type “cf scale --help” for more info.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com&gt; a écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory limitation.
The memory limitation is just the possible maximum value will be needed in this

application, but in most time, we don't need so much memory.

For example, I have one application which needs at most 5G memory at startup some

some specific operation, but in most time it just needs 2G.

So right now I need to specify 5G in deployment manifest, and 5G memory is

allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we should can push more

applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other applications?
It seems right now all the resources are pre-allocated when pushing application, it

will not be released even I stopped the application.

Yes, it's one way but it's not flexible, and scale app need to restart the
app as well.
As I said I may have some heavy operations which will definitely need more
than 2G.

In my opinion the ideal way is that we just set a maximum value for each
process, but during the running of the process, we don't pre-allocate the
memory as we specify as the maximum in deployment.

I suggest you manually “cf scale -m 2G“ after your app has booted.
Type “cf scale --help” for more info.

Le 9 mars 2016 à 04:09, Stanley Shen <meteorping(a)gmail.com&gt; a

écrit :

Hello, all

When pushing an application to CF, we need to define its disk/memory

limitation.

The memory limitation is just the possible maximum value will be

needed in this

application, but in most time, we don't need so much memory.

For example, I have one application which needs at most 5G memory at

startup some

some specific operation, but in most time it just needs 2G.

So right now I need to specify 5G in deployment manifest, and 5G

memory is

allocated.

Take m3.large VM for example, it has 7.5G.
Right now we can only push one application on it, but ideally we

should can push more

applications, like 3 since only 2G is needed for each application.

Can the resources of a IDLE application be shared by other

applications?

It seems right now all the resources are pre-allocated when pushing

application, it

will not be released even I stopped the application.

Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used to
determine if the container needs to be killed when it exceeds it. However,
since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that on
a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust the
containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not sure
whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com&gt; wrote:

Thanks for information.

I said "pre-allocated" because after I pushed an APP with 5G memory
specified, if I go to Cell VM, and I notice the available memory is 5G less
than totally memory via "curl -s http://localhost:1800/state" on Cell VM.

I think overcommit factor is not very suitable in my case, but "resource
reclamation and predictive analytics" are quite helpful, and it's a quite
useful/flexible mechanism.

Do we have any plan on such cool ideas?

Hi Stanley,

No physical memory is actually pre-allocated, it's simply a maximum used

to

determine if the container needs to be killed when it exceeds it.

However,

since your VM has some fixed amount of physical memory (e.g. 7.5G), the
operator will want to be able to make some guarantees that the VM doesn't
run a bunch of apps that consume the entire physical memory even if the
apps don't individually exceed their maximum memory limit. This is
especially important in a multi-tenant scenario.

One mechanism to deal with this is an "over-commit factor". This is what
Dan Mikusa's link was about in case you didn't read it yet. If you want
absolute guarantees that the VM will only have work scheduled on it such
that applications cannot consume more memory than what's "guaranteed" to
them by whatever their max memory limits are set to, you'll want an
overcommit factor on memory of 1. An overcommit factor of 2 means that

on

a 7.5G VM, you could allocate containers whose sum total of their max
memory limits was up to 15G, and you'd be fine as long as you can trust

the

containers to not consume, in total, more than 7.5G of real memory.

The DEA architecture supports setting the overcommit factors, I'm not

sure

whether Diego supports this (yet).

The two concepts Deepak brings up, resource reclamation and predictive
analytics, are both pretty cool ideas. But these are not currently
supported in Cloud Foundry.

Best,
Amit

On Thu, Mar 10, 2016 at 7:54 AM, Stanley Shen <meteorping(a)gmail.com&gt;

wrote: