|
Re: FYI: CAB call for July 2015 - moved to 7/15 @ 8a PDT
Thanks again to Phil Whelan for crafting the greatly detailed cab minutes.
http://www.activestate.com/blog/2015/07/cloud-foundry-advisory-board-meeting-2015-july The MEGA section says the following: *Updates from this MEGA team for the past week include having extracted etcd-release and now they are working on extracting consul-release, setting up pipelines for them, testing them and publishing them to bosh.io <http://bosh.io/>.* *Longer-term they want to generalize the patterns that have been introduced for Diego around manifest generation.* *Cornelia Davis from Pivotal asked Amit to confirm whether manifest generation was moving away from Spiff, to which Amit replied "yes"."* Looking at diego repo, I still see the spiff installation instructions, and spiff used for manifest generation [2]. Where can I learn about the manifest generation pattern for diego, to better understand this CAB discussion ? Thanks in advance, Guillaume. [2] https://github.com/cloudfoundry-incubator/diego-release/blob/develop/scripts/generate-deployment-manifest#L84-L103 On Tue, Jul 14, 2015 at 4:56 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote: FYI...
|
|
|
|
Re: Utilities PMC - 2015-07-07 notes
Mike Dalessio
Sorry all,
toggle quoted messageShow quoted text
The correct URL for LicenseFinder tracker is: https://www.pivotaltracker.com/n/projects/234851 However, the Toolsmiths tracker is private as there's quite a bit of work in there that's specific to Pivotal, so I apologize for attempting to link to it.
On Thu, Jul 9, 2015 at 1:05 AM, Guillaume Berche <bercheg(a)gmail.com> wrote:
Thanks Mike for the detailed notes.
|
|
|
|
Re: Utilities PMC - 2015-07-07 notes
Mike, not sure you had seen my question to the utilities pointers below ?
toggle quoted messageShow quoted text
On Thu, Jul 9, 2015 at 7:05 AM, Guillaume Berche wrote:
Thanks Mike for the detailed notes.
|
|
|
|
App autosleep support
Hi,
I wonder if there are plans to implement an auto-sleep behavior in cloudfoundry, in which inactive apps would be automatically stopped after a max inactivity threshold, and automatically restart upon arrival of traffic on their routes. Similar to google app engine default behavior [1] I did not find mentions of this yet in mailing lists and trackers. We feel at Orange that such feature can improve the density for some of our non-prod use-cases (with environmental and financial benefits). I'd like to know if someone in the community already worked on such feature or would be interested in collaborating on an opensource implementation. I drafted some specs for a java-based implementation we're planning to work on [2]. I'd love to hear feedbacks and suggestions on this. Thanks in advance, Guillaume. [1] https://cloud.google.com/appengine/docs/java/modules/ [2] https://docs.google.com/document/d/1tMhIBX3tw7kPEOMCzKhUgmtmr26GVxyXwUTwMO71THI/edit#
|
|
|
|
Re: Soliciting feedback on a UX change for route services
Thanks Shannon for your feedback.
toggle quoted messageShow quoted text
I understand there is a small window into which the pre-determined app might not exist anymore (e.g. during blue/green deployment traffic shift). The default behavior you're suggesting (picking a different app instance) seems sensible to me, even though it will lead to seldom false associations. We can imagine to refine this behavior in a second step, when use-cases of router service being sensitive to false associations become more frequent: allow the gorouter to comply to hints provided by the route service to tune the behavior in case the pre-determined app might not exist anymore. The router service could for instance augment the router-service HTTP header with hints fields: - missing-app-policy with one of the following values: - reassign-app: the router transparently route the request to another app(default) - reject: reject the request (e.g. 502 status code with a json body providing the currently available app ids). In this case the route service may reemit the request to the gorouter, specifying the second param below - route-to-app override the predetermined app to which to route the traffic I'm currently planning to implement a route-service that would leverage the app_id in the request in an "autosleep", see [1]. The "reassign-app" default policy seems fine as a first step. The reject policy would be a nice refinement to close this corner case. [1] https://docs.google.com/document/d/1tMhIBX3tw7kPEOMCzKhUgmtmr26GVxyXwUTwMO71THI/edit# Guillaume.
On Sat, Jul 18, 2015 at 12:08 AM, Shannon Coen <scoen(a)pivotal.io> wrote:
Guillaume,
|
|
|
|
Re: CF-Abacus: incubation and inception meeting coming soon
Alexander Lomov <alexander.lomov@...>
Hello, Maxim.
I am Alexander Lomov. I work in R&D department at Altoros. I would like to participate in your Hangout meeting dedicated to CF Abacus. Could you please tell when and how I can participate. Thank you, Alex L. ------------------------ Alex Lomov *Altoros* — Cloud Foundry deployment, training and integration *Twitter:* @code1n <https://twitter.com/code1n> *GitHub:* @allomov <https://gist.github.com/allomov>
|
|
|
|
Questions about /v2/app_usage_events
王小锋 <zzuwxf at gmail.com...>
Hi, there
I noticed there are three state about the app event, started, stopped and buildpack_set. what does "buildpack_set" mean? If the app is in buildpack_set status, does it occupy cup/memory/disk? If the app is deleted, will the event reflected in app_usage_events? if no, should I use /v2/app_usage_events together with /v2/events?q=type:audit.app.delete-request to determine the lifetime of an app? Many thanks.
|
|
|
|
Re: Removing FUSE support from CF
Jack Cai
+1 for space-level configuration.
toggle quoted messageShow quoted text
Jack
On Wed, Jul 29, 2015 at 2:04 PM, Matt Cowger <matt(a)cowger.us> wrote:
We're wary of adding too many knobs to the platform and exposing themall the way down to app developers increases the cognitive load for folks
|
|
|
|
Re: Node.js Apps with small memory limits; Inaccurate Memory Availability in Containers
Christopher Piraino <cpiraino@...>
Sai,
toggle quoted messageShow quoted text
Running free/top from inside the container is going to report the VM's memory/cpu statistics in which the container is running. The correct stats are located in the appropriate cgroups filesystem, which is where Cloud Foundry pulls stats from when using the CLI. From inside the container it is actually very hard to figure out how much memory the cgroup is using (see this article for more information <http://fabiokung.com/2014/03/13/memory-inside-linux-containers/>). Best, Chris
On Wed, Jul 29, 2015 at 7:55 AM, Mike Dalessio <mdalessio(a)pivotal.io> wrote:
Hi Sai,
|
|
|
|
Re: Notifications on ORG, SPACE and USER modifications
Piotr Przybylski <piotrp@...>
We have several use cases
- in time event processing for usage, with notification instead of polling we'll be able to keep up better, getting closer to real time for metering, and thus rating and billing - timely display of the new system objects like spaces or organizations, being notified about new objects, avoids retrieving all objects and comparing which is already known - detecting and signaling entity field or property changes, for example name changes, which can require polling and comparison These in turn would allow better support uses like - console or UI reactive to entity changes - activity dashboards - analytics Piotr |------------> | From: | |------------> >--------------------------------------------------------------------------------------------------------------------------------------------------| |Mike Youngstrom <youngm(a)gmail.com> | >--------------------------------------------------------------------------------------------------------------------------------------------------| |------------> | To: | |------------> >--------------------------------------------------------------------------------------------------------------------------------------------------| |"Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org> | >--------------------------------------------------------------------------------------------------------------------------------------------------| |------------> | Date: | |------------> >--------------------------------------------------------------------------------------------------------------------------------------------------| |07/29/2015 08:54 AM | >--------------------------------------------------------------------------------------------------------------------------------------------------| |------------> | Subject: | |------------> >--------------------------------------------------------------------------------------------------------------------------------------------------| |Re: [cf-dev] Notifications on ORG, SPACE and USER modifications | >--------------------------------------------------------------------------------------------------------------------------------------------------| |------------> | Sent by: | |------------> >--------------------------------------------------------------------------------------------------------------------------------------------------| |cf-dev-bounces(a)lists.cloudfoundry.org | >--------------------------------------------------------------------------------------------------------------------------------------------------| For us the main use case is security auditing to keep a long term record of who has done anything. In the case of our Security team rather than use CF events directly they preferred to have events forwarded to Security Analytics. Today we pull events then forward the details to Security Analytics via a syslog endpoint. Mike [0] http://www.emc.com/security/security-analytics/security-analytics.htm On Tue, Jul 28, 2015 at 11:26 PM, Dieu Cao <dcao(a)pivotal.io> wrote: Hi all interested in notifications on modification of resources, It would be helpful for me in framing the "what" and the "why" of this feature if you could also describe your specific use cases and pain points on why you would want notifications on modifications and also which resources you particularly care about. Is it for real time updates on a dashboard? For consumption for billing purposes? For triggering provisioning/deprovisioning of resources? -Dieu On Tue, Jul 28, 2015 at 11:05 AM, Jean-Sebastien Delfino < jsdelfino(a)gmail.com> wrote: I’m going to need something like this too for the CF Abacus service metering project, as I’d like to track the lifecycle of orgs, users, etc to match their history with the usage data reported for them. Here’s a straw man description of what I had in mind: - For Abacus, I’d need a Lossless API. Usage metering eventually translates to billing and money, you don’t want to lose that :) - An extension or variant of the current CF /v2/events API supporting users, orgs, app usage etc, as even with a notification API I’ll still need to do GETs sometimes. - 304 responses with etags on these GETs (as suggested earlier in the thread [1]) would be good. - A Webhook style notification API where I could register interest in a selection of events with a callback URL, and get these events POSTed back to me at that URL, similar to what Github and many others do with Webhooks. - On top of Webhooks, it’d be nice to have a form of streaming (either down to the client like the Firehose does or in the other direction up to the Webhook callback URL), but I'm not sure if we’ll need that in the project right away. - We’d obviously need some form of security, maybe use my user token to register for events on entities that I have access to? - I’m also curious about the group’s thoughts on queueing and back-pressure when events get generated faster that they can be consumed for example. There was a mention of some message queuing earlier [2]. That would make sense to me (although IMO it’d be good if the underlying MQ didn’t shine through the API). What did you have in mind for this? I guess there are quite a few things to figure out here! I’ll be happy to collaborate with the community on these discussions. Thoughts? [1] http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p842.html [2] http://cf-dev.70369.x6.nabble.com/cf-dev-Notifications-on-ORG-SPACE-and-USER-modifications-tp827p834.html - Jean-Sebastien On Fri, Jul 24, 2015 at 9:59 PM, Matt Cowger <matt(a)cowger.us> wrote: I think ETags is reasonable thought as well. On Thu, Jul 23, 2015 at 4:39 PM, Benjamin Black <bblack(a)pivotal.io> wrote: ETags and a 304 response are specifically intended for that purpose. I'd recommend that over relying on Last-Modified. b On Thu, Jul 23, 2015 at 12:34 AM, Koper, Dies < diesk(a)fast.au.fujitsu.com> wrote: Or setting the Last-Modified HTTP response header accordingly, and allow clients to use HTTP caching mechanisms (Last-Modified, etc.) to get quick empty responses with the current APIs if no changes have been made? (Or maybe this is already working so – haven’t checked). Regards, Dies Koper From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto: cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Matt Cowger Sent: Thursday, July 23, 2015 4:45 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications I've wanted something similar as well. On a related note, having a CC API 'serial' number (for each object in CC - apps, spaces, etc) that increments on every change relevant to that object would be of value for detecting if something has changed. On Thu, Jul 23, 2015 at 3:27 PM, Dieu Cao <dcao(a)pivotal.io> wrote: There are a few different approaches to this and different concerns that are possible. The requests I've seen have been around wanting to be able to subscribe to and filter the various events that cc currently generates so that other behavior could be triggered. We currently have events, app usage events, and service usage events. Is it acceptable for the notifications to be lossy? Depends on the use case but If so, then the firehose may be an acceptable approach. The CAPI team is currently focusing on other work in the near term, such as the v3 API and private brokers, but would be happy to collaborate on a proposal. On Wed, Jul 22, 2015 at 2:05 PM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: My take: CC should have callbacks on for each model create, update and delete methods. Those callbacks will send a message to an MQ, which you can subscribe to consume those messages. This can be expanded to pretty much every event we need to track. What do you think? JP 2015-07-22 17:30 GMT-03:00 Matthias X Hub <matthias.hub(a)de.ibm.com >: Hi, we (=IBM) are also having the need and are currently investigating how to solve this. We plan to work on a proposal to discuss this further with the cf community. I'll keep you updated on that. Regards, Matthias From: Mike Youngstrom <youngm(a)gmail.com> To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org> Date: 22.07.2015 20:57 Subject: Re: [cf-dev] Notifications on ORG, SPACE and USER modifications Sent by: cf-dev-bounces(a)lists.cloudfoundry.org We have the same need. Today we are polling the CC. It would be nice for us also if we could get CC event notifications via something like the firehose. Mike On Wed, Jul 22, 2015 at 10:23 AM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: I mean, I know you can list those events thru the API, but I want something that will react on an event instead of having to be constantly polling for them. 2015-07-22 13:18 GMT-03:00 Juan Pablo Genovese < juanpgenovese(a)gmail.com>: Sree, thanks! Any pointers on how can I hook up to these audit events? Thank you! 2015-07-22 13:12 GMT-03:00 Sree Tummidi <stummidi(a)pivotal.io>: I believe there are audit events generated for all these actions which can be captured and forwarded to an SIEM solution like splunk Thanks, Sree Sent from my iPhone On Jul 22, 2015, at 8:54 AM, Juan Pablo Genovese < juanpgenovese(a)gmail.com> wrote: Guys, I need to somehow hook up into the Cloud Controller (CC) to capture ORG, SPACE and USER deletion, insertion and update. So far, I considered some approaches, such as forking the CC (the least favorite) and modifying the code with some hooks, tapping into Nginx to capture the requests, and using triggers in the database to capture each event and send the necessary info to a service. What do you think? Any other idea you might have? Thanks! -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Mis mejores deseos, Best wishes, Meilleurs vœux, Juan Pablo ------------------------------------------------------ http://www.jpgenovese.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- -- Matt _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- -- Matt _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- Jean-Sebastien Sent from my DynaTAC 8000x _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
|
|
Re: CF-Abacus: incubation and inception meeting coming soon
Stormy
I don't see anything on Twitter about this. Any objections to tweeting this
news? Is there a potential blog post for the CF blog here? Best, Stormy On Tue, Jul 28, 2015 at 11:15 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote: Hi, all,
|
|
|
|
Re: CF-Abacus: incubation and inception meeting coming soon
Michael Maximilien
Quick update on inception meeting.
toggle quoted messageShow quoted text
To accommodate our friends and colleagues from Europe who would like to attend, let's plan to move the meeting to 10a to 12:30p with the option of lunch after at nearby location in SF. Unless I hear any objections I will send the invites to those interested parties who have already contacted me and confirm details here. If you want to attend (local or remote) please remember to reply to me with email so I can add you to invite list. Best, dr.max ibm cloud labs silicon valley, ca Sent from my iPhone
On Jul 28, 2015, at 10:15 PM, Michael Maximilien <maxim(a)us.ibm.com> wrote:
|
|
|
|
Re: Removing FUSE support from CF
Matt Cowger
We're wary of adding too many knobs to the platform and exposing them allthe way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise? Agreed. I'd argue this this probably not be a 'per-app' thing, as I too amy way of the knobs that developers like to turn. I think a per space level is just the right level. On Wed, Jul 29, 2015 at 10:20 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote: That's still very much open for discussion. Obviously, someone with -- -- Matt
|
|
|
|
Re: Removing FUSE support from CF (Onsi Fakhouri)
Onsi Fakhouri <ofakhouri@...>
Hi Pramod,
Still TBD - we're driving hard for a Diego GA first and will then evaluate the various priorities and scope them out. Onsi On Wed, Jul 29, 2015 at 9:59 AM, Pramod Mandagere <nagapramod(a)gmail.com> wrote: If not FUSE, what is the timeline/status for support of mounting
|
|
|
|
Re: Removing FUSE support from CF
Onsi Fakhouri <ofakhouri@...>
That's still very much open for discussion. Obviously, someone with
toggle quoted messageShow quoted text
administrative privileges should be in charge of this particular piece of configuration. Also making this a runtime config (e.g. feature flag) as opposed to a deploy-time config (e.g. part of the CC config written out by BOSH) would make the different behaviors more testable. Thoughts? Preferences? We're wary of adding too many knobs to the platform and exposing them all the way down to app developers increases the cognitive load for folks using the platform. Enabling/disabling it on a per-installation level, and - maybe - a per-space level, might be a decent compromise? Onsi
On Wed, Jul 29, 2015 at 9:54 AM, Matt Cowger <matt(a)cowger.us> wrote:
Once - configurable on a per-app, per space, or per deployment basis?
|
|
|
|
Re: Droplets and Stacks
Onsi Fakhouri <ofakhouri@...>
Hey Colin,
Good stuff. I like to draw a circle around the rootfs, the buildpacks, the generated droplet, the Task/LRP recipes, and the lifecycle binaries that run inside containers to stage and launch droplets. You could label that circle an application lifecycle. Diego currently supports three application lifecycles and is loosely coupled to those lifecycles: 1. The Linux-Buildpack App lifecycle: includes the cflinuxfs2 rootfs, the various buildpacks (including a known interface for building custom buildpacks), the droplets (compiled artifacts guaranteed to run with cflinuxfs2), two binaries: the builder (performs staging) and the launcher (runs applications), and code that can convert CC requests for staging and running instances to Diego Task/LRP recipes. 2. The Windows App lifecycle: includes the notion of a correctly configured windows environment, a windows-compatible droplet, a builder, a launcher, and code that can generate Tasks/LRPs. In this context we do not yet have/need the notion of a buildpack though we are free to add one later. The builder simply prepares the droplet from source and the launcher knows how to invoke it. 3. The Docker App lifecycle: has no rootfs as the docker image provides the entire rootfs, includes a builder to extract docker-metadata and send it back to CC for safe-keeping, and a launcher to launch the requested process *and* present it with a standard CF environment. Again, there's also code that knows how to translate CC requests for a docker-based application into Tasks and LRPs. The cool thing is that Diego doesn't care about any of these details and you are free to construct your own lifecycles and have your own contracts within each lifecycle. You are spot on in noting that there is an implicit contract between the buildpacks and the rootfs. I'd go further and say that that implicit contract covers everything in the lifecycle circle (e.g. the builder has a contract with the buildpacks, it expects `detect`, `compile` and `release` to work a certain way, the recipes have a contract with the builder/launcher, they expect particular command line arguments, etc...) This is one reason why we've recently transitioned the ownership of the rootfs from the runtime team to the buildpack team - as the buildpack team is best suited to define and maintain the contract between the buildpacks and the rootfs. Would love to explore ways to make all these contracts more explicit. One last point. I didn't use the word "stack" in this e-mail until just now. I agree that it's an overloaded concept that is easily and often misunderstood ;) Onsi On Wed, Jul 29, 2015 at 9:51 AM, Colin Humphreys <colin(a)cloudcredo.com> wrote: Hi All,
|
|
|
|
Re: Removing FUSE support from CF (Onsi Fakhouri)
Pramod Mandagere
If not FUSE, what is the timeline/status for support of mounting persistent
volumes in Diego natively(remember seeing a talk in cf summit on floating/fixed volumes)? Pramod On Wed, Jul 29, 2015 at 9:53 AM <cf-dev-request(a)lists.cloudfoundry.org> wrote: Send cf-dev mailing list submissions to
|
|
|
|
Re: Removing FUSE support from CF
Matt Cowger
Once - configurable on a per-app, per space, or per deployment basis?
toggle quoted messageShow quoted text
On Wed, Jul 29, 2015 at 9:50 AM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote:
Hey all, --
-- Matt
|
|
|
|
Re: Notifications on ORG, SPACE and USER modifications
Matt Cowger
For me the primary use case to developing management tools on top of the CC
toggle quoted messageShow quoted text
API, and being able to intelligently cache information about the state of those resources. Without etags or serial numbers or similar, that means every refresh needs to query a TON of data from CC.
On Tue, Jul 28, 2015 at 10:26 PM, Dieu Cao <dcao(a)pivotal.io> wrote:
Hi all interested in notifications on modification of resources, --
-- Matt
|
|
|
|
Droplets and Stacks
Colin Humphreys <colin@...>
Hi All,
I wrote a couple of articles about droplets and stacks. http://www.cloudcredo.com/a-droplet-of-value/ http://www.cloudcredo.com/stacks-of-problems/ The droplet post is fairly self-explanatory, and enabling the choice of shipping droplets or source is well on the way in Cloud Foundry development. I feel our story around stacks is far less complete. It seems to be an overloaded concept inherited from Heroku and the contract with the stack seems to cause issues for both app and buildpack developers. I'd like to open the discussion on what the future should be for stacks, or if you think they're perfect as they are. Cheers, Colin CloudCredo Cheerleader
|
|
|
