|
Re: Stemcells, Garden, and rootfs: how are these related?
toggle quoted message
Show quoted text
On Thu, May 19, 2016 at 11:47 AM, Duncan Winn <dwinn(a)pivotal.io> wrote: sorry JK - it's still early am (ignore what I just said) - the stack is
the specific rootfs - the container image is a combination of a droplet
plus a stack. The container image will then run in your garden process.
For a windows stack you would need a windows vm/cell to run it.
On Thu, May 19, 2016 at 8:43 AM Duncan Winn <dwinn(a)pivotal.io> wrote:
Hey JK, a stack is a combination of the rootfs and droplet. This is
conceptually a "container image" that will then run in your garden
container. For a windows stack you would need a windows vm/cell to run it.
On Thu, May 19, 2016 at 8:26 AM J K <falconwing(a)gmail.com> wrote:
Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?
--
Duncan Winn
Cloud Foundry PCF Services
--
Duncan Winn
Cloud Foundry PCF Services
|
|
|
Re: Stemcells, Garden, and rootfs: how are these related?
sorry JK - it's still early am (ignore what I just said) - the stack is the
specific rootfs - the container image is a combination of a droplet plus a
stack. The container image will then run in your garden process. For a
windows stack you would need a windows vm/cell to run it.
toggle quoted message
Show quoted text
On Thu, May 19, 2016 at 8:43 AM Duncan Winn <dwinn(a)pivotal.io> wrote: Hey JK, a stack is a combination of the rootfs and droplet. This is
conceptually a "container image" that will then run in your garden
container. For a windows stack you would need a windows vm/cell to run it.
On Thu, May 19, 2016 at 8:26 AM J K <falconwing(a)gmail.com> wrote:
Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?
--
Duncan Winn
Cloud Foundry PCF Services
--
Duncan Winn
Cloud Foundry PCF Services
|
|
|
Re: Stemcells, Garden, and rootfs: how are these related?
Hey JK, a stack is a combination of the rootfs and droplet. This is
conceptually a "container image" that will then run in your garden
container. For a windows stack you would need a windows vm/cell to run it.
toggle quoted message
Show quoted text
On Thu, May 19, 2016 at 8:26 AM J K <falconwing(a)gmail.com> wrote: Thank you, Amit. Is it correct to say that a stack (in the `cf stacks`
sense) is a combination of rootfs + stemcell? If you have a "base VM" of,
say, Linux -- and if that's the one bosh is deploying to all the cells, how
does that become a `windows2012R2` stack in CF?
--
Duncan Winn
Cloud Foundry PCF Services
|
|
|
Re: Stemcells, Garden, and rootfs: how are these related?
Thank you, Amit. Is it correct to say that a stack (in the `cf stacks` sense) is a combination of rootfs + stemcell? If you have a "base VM" of, say, Linux -- and if that's the one bosh is deploying to all the cells, how does that become a `windows2012R2` stack in CF?
|
|
|
Re: Stemcells, Garden, and rootfs: how are these related?
Hi J K, To add to what Nick said: You can think of Garden as a generic API for something that creates containers and runs processes in them, and can be backed by various different backend implementations, e.g. garden-linux or garden-windows. It leaves the responsibility of what it means to actually create a container to the backend, but provides an interface for interacting with containers and container processes as logical resources. At a high level, a container consists of a root file system image, a set of resource constraints, and some commands to run inside the container. Cloud Foundry is able to take your raw application source code and build it inside a container. The root file system used for the container that builds your code is the "rootfs". So for the build process, the standard "cflinuxfs2" rootfs is used to create the container, then your code is injected and compiled, and the resulting asset is extracted. Next, when Cloud Foundry runs your application, it again creates a container starting from the rootfs image, injects your built code, and runs it. The "create a container and run xyz" responsibility belongs to Garden, so the relationship between garden and the rootfs is that Cloud Foundry finds a Garden process somewhere and instructs it to use a particular rootfs to use to create a container out of it. A single application developer wants to push their code to Cloud Foundry and have it run the code in some container. But an instance of the Cloud Foundry platform can support many (e.g. hundreds of thousands) application developers all doing the same thing. An operator managing an instance of Cloud Foundry needs to take Cloud Foundry software itself and have it be deployed and run by something. The toolchain for this is called BOSH. BOSH typically operates on VMs in a virtualized infrastructure like AWS of vSphere. BOSH can run Cloud Foundry, or any number of different distributed software services, on top of virtualized infrastructure. Analogous to the rootfs which is a common base layer image for all applications running on Cloud Foundry, a stemcell is a common base VM image for all services deployed by BOSH (e.g. each of the components comprising Cloud Foundry itself). The documentation Nick has linked you to can give you more detailed information about these concepts, as well as the reasoning behind them. Best, Amit On Wed, May 18, 2016 at 4:53 PM, Nicholas Calugar <ncalugar(a)pivotal.io> wrote: Hi JK,
I don't work on any of these parts of Cloud Foundry but I can give a very
basic overview:
[1] rootfs: the filesystem used for applications deployed on Cloud
Foundry. The current default is cflinuxfs2, based on Ubuntu 14.04
[2] garden: "A rich golang client and server for container creation and
management with pluggable backends for linux, windows and The Open
Container Initiative Spec". Garden is running on the Diego cells where
Cloud Foundry applications are run.
[3] stemcells: The base image for VMs deployed by bosh.
Highly recommend staring with the Cloud Foundry documentation:
http://docs.cloudfoundry.org/. You can also "kick the tires" by trying
one of the hosted versions of Cloud Foundry:
https://www.cloudfoundry.org/learn/certified-providers/
Nick
[1] https://github.com/cloudfoundry/stacks
[2] https://github.com/cloudfoundry-incubator/garden
[3] http://bosh.cloudfoundry.org/docs/stemcell.html
On Wed, May 18, 2016 at 1:00 PM, J K <falconwing(a)gmail.com> wrote:
hi,
I'm learning about Cloud Foundry's ideas of how to organize applications
and how it works under the covers. At the moment I'm having a little
trouble understanding the relationship between a few ideas in CF: "rootfs",
"Garden", and "stem cells".
Can someone explain:
(1) their understanding of what these ideas are; and
(2) how they are related?
Thank you!
|
|
|
Re: Buildpacks Checksum Site for Release Validation
How the pws / cloud-ops team verify the buildpack checksum before deploying
it ?
toggle quoted message
Show quoted text
On Thu, May 19, 2016 at 3:57 AM, Danny Rosen <drosen(a)pivotal.io> wrote: In the future we may consider implementing a json feed. If we did go
forward with this work, would you be interested in creating example CI / CD
implementations as references?
On Tue, May 17, 2016 at 10:30 PM, Gwenn Etourneau <getourneau(a)pivotal.io>
wrote:
Hi,
Any json feed /api ?
Can be nice and more easy to integrate with any CI/CD tool.
Thanks
On Wed, May 18, 2016 at 11:15 AM, taichi nakashima <nsd22843(a)gmail.com>
wrote:
Great,
I hope cloudfoundry/cli will provide the same thing.
cf.
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/K3BEBY4A2WSUKS7YS5IF2UDQHHSU35A7/
Taichi Nakashima
2016年5月18日(水) 6:20 David Jahn <djahn(a)pivotal.io>:
Dear Cloud Foundry Users,
To help operators and users of Cloud Foundry establish a "chain of
custody" for buildpacks, we have launched the following checksum site:
https://buildpackverify.cloudfoundry.org
This site provides a checksum for all cached buildpack release zip
files (except for the java-buildpack). Whenever the buildpacks team
generates a new buildpack release, we will immediately compute the SHA256
checksum of that file and upload it to this website.
The site is hosted on a different repository from the main buildpack
github repositories. It allows operators to validate that the zip file we
produced is the same artifact that has been downloaded and installed.
Additionally, if an operator wishes to further investigate the
components of a buildpack, the "manifest.yml" in each buildpack root
directory (for
example,
https://github.com/cloudfoundry/go-buildpack/blob/master/manifest.yml)
provides a catalog of every third party component in the buildpack, a URL
of that component's location, and an MD5 checksum of that component.
We hope that this will assist people in auditing the source of their
buildpack code!
Cheers,
Buildpacks Team
|
|
|
Re: Proposal: Reducing State in Service Brokers - Service Broker API Enhancement
Oh interesting! I was thinking about creating a broker that can do same thing, Are you looking to open source this broker?
toggle quoted message
Show quoted text
On May 17, 2016 at 12:02:48 PM, Alex Ley (aley(a)pivotal.io) wrote:
hat builds lots of service brokers. We are currently working on broker that backs onto BOSH and want to move towards making it stateless. We have written a proposal to enhance the CF Service Broker API to allow us to achieve this. We believe this will he
|
|
|
Re: Stemcells, Garden, and rootfs: how are these related?
toggle quoted message
Show quoted text
On Wed, May 18, 2016 at 1:00 PM, J K <falconwing(a)gmail.com> wrote: hi,
I'm learning about Cloud Foundry's ideas of how to organize applications
and how it works under the covers. At the moment I'm having a little
trouble understanding the relationship between a few ideas in CF: "rootfs",
"Garden", and "stem cells".
Can someone explain:
(1) their understanding of what these ideas are; and
(2) how they are related?
Thank you!
|
|
|
Re: User defined variable "key" validation doesn't happen at cf set-env phase
No, we are trying to focus on development of the V3 API, we will only fix
major issues with the V2 API.
Thanks,
Nick
toggle quoted message
Show quoted text
On Sun, May 15, 2016 at 8:44 PM, Padmashree B <padmashree.b(a)sap.com> wrote: Hi Nick,
Thanks for the update!
Are there any plans of bringing it to V2 APIs as well?
Kind Regards,
Padma
|
|
|
Re: aligning cf push health-check default value
Hi Dies, The CC doesn't see these as steps of a push, they are all distinct API requests. We cannot make decisions based on previous requests or lack thereof, it has to be done in the CLI. Based on a quick test, unknown fields are ignored in POST bodies in V2, so you can send "health_check_type":"none" on older CC versions. Thanks, Nick On Sun, May 15, 2016 at 5:23 PM, Koper, Dies <diesk(a)fast.au.fujitsu.com> wrote: Hi Nick, Eric,
I have confirmed that with the current CC release we could change the
default behaviour on the CLI side without making additional API calls:
In the body of POST /v2/apps?async=true we simply add
"health_check_type":"none" if the user specified “--no-route” and that
seems to be accepted regardless whether the app defaults to DEA or Diego.
Can you confirm CC does not return an error on older CC versions (before
the health_check_type attribute was introduced) if we add this attribute to
the API request?
Also, can you confirm you definitely don’t want CC to change its default
behaviour to align with what the user (and CLI) sees compared with Diego?
The current API sequence the CLI uses is:
1. POST /v2/apps?async=true – create app
2. POST /v2/routes?async=true&inline-relations-depth=1 – create
route(s)
3. PUT /v2/apps/guid/routes/guid – map route(s)
4. PUT /v2/resource_match & PUT /v2/apps/guid/bits?async=true –
upload app
5. GET /v2/jobs/guid – check its status
Although at step 1 CC would not know whether the app will get any routes
(i.e. whether the user specified “--no-route”), at step 4 CC would, so it
could toggle the health-check-type accordingly. Not sure how CC does this
for apps deployed to DEA – is it the DEA backend that makes this decision
at the timing of 4?
Regards,
Dies Koper
Cloud Foundry CLI PM
*From:* Nicholas Calugar [mailto:ncalugar(a)pivotal.io]
*Sent:* Thursday, May 12, 2016 4:34 AM
*To:* Discussions about Cloud Foundry projects and the system overall.
*Subject:* [cf-dev] Re: Re: Re: aligning cf push health-check default
value
Hi Dies,
I spoke with Eric and he explained that this could be the desired UX for a
majority of the apps pushed with --no-route. There are more advanced
deployment strategies that might set --no-route and bind to routes later,
but I think we can expect these users to be explicit with their health
check as well. I think my discomfort with this arose when you mentioned to
me that we might want to do this in the Cloud Controller. As long as this
continues to be explicit from the API perspective, I'm fine with changing
the UX of the CLI per your above proposal.
Thanks,
Nick
On Wed, May 4, 2016 at 1:13 PM, Shannon Coen <scoen(a)pivotal.io> wrote:
Hi Dies,
IMO the healthcheck of the app should be determined independently of
whether a developer wants their app routable.
My understanding of the implications for you proposal are that a developer
could not have a port-based healthcheck without mapping a route. This seems
unnecessarily restrictive. Soon developers will be able to specify http
healthchecks. Would these be prevented also?
Best,
Shannon Coen
Product Manager, Cloud Foundry
Pivotal, Inc.
On Wed, May 4, 2016 at 12:26 PM, Nicholas Calugar <ncalugar(a)pivotal.io>
wrote:
Hi Dies,
I considered this a bit more after we chatted yesterday and I don't think
we should try to create parity between DEAs and Diego in this case. My
personal opinion is that behavior should be explicit and these two flags
provide a more correct experience with the Diego backend.
Thanks,
Nick
On Mon, Apr 25, 2016 at 6:09 PM, Koper, Dies <diesk(a)fast.au.fujitsu.com>
wrote:
Hi CLI users,
With apps deployed to DEAs, a health check is performed at application
start-up targeting the app’s port, unless you specified `--no-route`, in
which case the process is monitored.
With Diego, the health check is performed continuously and the type of
check was exposed through an option to the `cf push` command.
This option defaults to `port`, which isn't always appropriate for apps
pushed without a route, such as worker apps.
We propose fixing the `--health-check-type` option’s default value to
align with the behaviour seen for DEAs, i.e. to use “none” if option
`--no-route` is used:
--health-check-type, -u Application health check type (Default:
'none' if '--no-route' is set, otherwise 'port')`
Would anyone object to such change?
Cheers,
Dies Koper
Cloud Foundry CLI PM
|
|
|
Re: [abacus] Separate time-based from discrete usage metrics
Hristo Iliev
Hi,
When we talk for a DB storing stateful metrics, do we really mean a single
DB storing all the Abacus pipeline data, or an input & output DBs for each
of the Abacus micro-services?
+1 for storing the data in the historical/log-like databases. This gives us
the possibility to extend the implementation of the failed events
management to the stateful measures.
We already started a spike on dataflow module. It would be straightforward
to detect stateful metrics in POST requests. This can be done by extending
the account plugin API and the metering config & schema.
We have two challenges:
* GET requests do not have access to the stateful flag, so we need a way to
detect stateful data using the document id. The idea we have is to use a
new ID schema (or just a prefix?), as you proposed on the last IPM.
* We think Replay function might miss some of the data, exactly due to the
problem we try to solve.
Regards,
Hristo Iliev
2016-05-18 19:34 GMT+03:00 Jean-Sebastien Delfino <jsdelfino(a)gmail.com>:
toggle quoted message
Show quoted text
Hi,
To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage metrics (the rest basically)
2. Store the time-based metrics in a separate DB(s) Your proposal looks good to me. Piotr, Kevin and Raj and I had several
design discussions on this topic in the last few days and we've come up
with a few more ideas on top of what you're describing here:
- The distinction between time-based and discrete resource usage metering
could also be understood as usage metering of a stateful vs stateless
resource or metric. In the stateful case, we simply store the state of the
resource instance in a separate DB like you're proposing (e.g. we store the
fact that an app or a container is currently running, or has stopped), and
update that state in place when it changes. Then to compute and report
usage later on we just need to the current resource instance state from
that DB.
- We could continue to store the metrics in the current historical
databases as well (on top of that new DB) to preserve the resource instance
history as many users typically want to know their past usage.
- Some of us were not sure if the time-based / discrete distinction should
be at the resource type level or at the metric level... IMO your proposal
to do that at the metric level is cleaner so I'm happy with it :)
- The dataflow module will probably need a few minor code changes to
detect the case where some of the output docs need to go to a separate DB
(IIRC you or someone else also mentioned that on one of our scrums or on
slack...)
- Like you said, we may still need to maintain 2 DBs to purge old entries.
If that's easier, we could also adjust the usage accumulator service and
the dataflow module a bit to delete entries for inactive resource instances
right away (e.g. when an app or container stops.)
Thoughts?
P.S. I'll add these comments to issue #88 as well to make it be easier to
follow up there.
- Jean-Sebastien
On Thu, May 12, 2016 at 5:54 AM, Hristo Iliev <hsiliev(a)gmail.com> wrote:
Hi,
We're trying to fix Abacus issue 88: Missing aggregated usage for the
running application [1].
Background
=========
See the jsdelfino comment in the GitHub issue [2]. TL;DR: Resource
providers have to send a 'ping' doc per month for time-based metrics.
Proposed solution
==============
We decided to implement a solution in Abacus that frees the usage
providers from sending the 'ping' submission.
To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage
metrics (the rest basically)
2. Store the time-based metrics in a separate DB(s)
We already drafted a proposal for adding measurement type in the usage
plans with PR #320 [3].
We're about to spike on storing the time-based metrics in their own
Database, but we wanted to get the community opinion on the topic.
Motivation
========
The discrete usage submitted to Abacus is:
* stored in partitioned databases, due to their size/number
* like an event log, storing the history of the usage/resources
In contrast the current time-based metrics are:
* limited number (usually around 2 million on a loaded CF system)
* storing just the app resources usage state (GB/h consumed so far, GB/h
consuming currently)
Therefore it looks like a good idea to separate the two usage metrics
types and store the time-based metrics in a separate database. This will
allow us not only to solve the issue, but also to store and query the data
more effectively.
We may still need to maintain 2 databases and swap new/old (irrelevant)
metrics to reduce the DB size on the month boundaries.
Regards,
Hristo & Adriana
[1] https://github.com/cloudfoundry-incubator/cf-abacus/issues/88
[2]
https://github.com/cloudfoundry-incubator/cf-abacus/issues/88#issuecomment-148498164
[3] https://github.com/cloudfoundry-incubator/cf-abacus/pull/320
|
|
|
Amulya Sharma <amulya.sharma@...>
THUMBS UP Layne Peng
toggle quoted message
Show quoted text
On Tue, May 17, 2016 at 11:51 PM Layne Peng <layne.peng(a)emc.com> wrote: but we all are saying same thing .. believe me after running multiple bosh
services managing marketplace in a large org is kind of a messy jobs ..
I cannot agree more!
|
|
|
Stemcells, Garden, and rootfs: how are these related?
hi,
I'm learning about Cloud Foundry's ideas of how to organize applications and how it works under the covers. At the moment I'm having a little trouble understanding the relationship between a few ideas in CF: "rootfs", "Garden", and "stem cells".
Can someone explain:
(1) their understanding of what these ideas are; and
(2) how they are related?
Thank you!
|
|
|
Re: Buildpacks Checksum Site for Release Validation
In the future we may consider implementing a json feed. If we did go forward with this work, would you be interested in creating example CI / CD implementations as references? On Tue, May 17, 2016 at 10:30 PM, Gwenn Etourneau <getourneau(a)pivotal.io> wrote: Hi,
Any json feed /api ?
Can be nice and more easy to integrate with any CI/CD tool.
Thanks
On Wed, May 18, 2016 at 11:15 AM, taichi nakashima <nsd22843(a)gmail.com>
wrote:
Great,
I hope cloudfoundry/cli will provide the same thing.
cf.
https://lists.cloudfoundry.org/archives/list/cf-dev(a)lists.cloudfoundry.org/thread/K3BEBY4A2WSUKS7YS5IF2UDQHHSU35A7/
Taichi Nakashima
2016年5月18日(水) 6:20 David Jahn <djahn(a)pivotal.io>:
Dear Cloud Foundry Users,
To help operators and users of Cloud Foundry establish a "chain of
custody" for buildpacks, we have launched the following checksum site:
https://buildpackverify.cloudfoundry.org
This site provides a checksum for all cached buildpack release zip files
(except for the java-buildpack). Whenever the buildpacks team generates a
new buildpack release, we will immediately compute the SHA256 checksum of
that file and upload it to this website.
The site is hosted on a different repository from the main buildpack
github repositories. It allows operators to validate that the zip file we
produced is the same artifact that has been downloaded and installed.
Additionally, if an operator wishes to further investigate the
components of a buildpack, the "manifest.yml" in each buildpack root
directory (for
example,
https://github.com/cloudfoundry/go-buildpack/blob/master/manifest.yml)
provides a catalog of every third party component in the buildpack, a URL
of that component's location, and an MD5 checksum of that component.
We hope that this will assist people in auditing the source of their
buildpack code!
Cheers,
Buildpacks Team
|
|
|
Re: Miniconda has been added to the Python Buildpack
Awesome! On Wed, May 18, 2016 at 11:10 AM, Mike Dalessio <mdalessio(a)pivotal.io> wrote: Fantastic! Great work, Buildpacks team. I'm especially excited that this
will allow data scientists to more easily run their applications on Cloud
Foundry.
On Wed, May 18, 2016 at 1:56 PM, Danny Rosen <drosen(a)pivotal.io> wrote:
Over the last few weeks the Buildpacks team has experimented with
different methods of getting the Python buildpack working with various data
science dependencies (ex: sci-py, num-py, sklearn). We believe we have
found a solution that does not create significant challenges and fulfills
the underlying goal of enabling these dependencies in a native CF
application.
We have included the addition of Miniconda into the newest version [1
<https://github.com/cloudfoundry/python-buildpack/releases/tag/v1.5.6>]
of the python-buildpack and urge you to give it a try. To do so, include an
environment.yml [2] in your app and cf push *or* try out this sample app
[3 <https://github.com/ihuston/pydata_package_test>].
We're very excited about this addition and note that this is our first
step towards providing the functionality. As always, we're interested in
obtaining feedback and pull requests for improvement.
[1] -
https://github.com/cloudfoundry/python-buildpack/releases/tag/v1.5.6
[2] -
http://conda.pydata.org/docs/using/envs.html#use-environment-from-file
[3] - https://github.com/ihuston/pydata_package_test/
|
|
|
Re: Miniconda has been added to the Python Buildpack
Fantastic! Great work, Buildpacks team. I'm especially excited that this
will allow data scientists to more easily run their applications on Cloud
Foundry.
toggle quoted message
Show quoted text
|
|
|
Miniconda has been added to the Python Buildpack
|
|
|
Re: Regarding UAA service
Siva Balan <mailsiva@...>
Hi Dax, Can I request to log your issue at https://forum.predix.io ? There are a lot more Predix focussed developers on that forum than this mailing list and you are more likely to get a faster answer there. Thanks Siva
On Wed, May 18, 2016 at 8:25 AM, Sree Tummidi <stummidi(a)pivotal.io> wrote: Hi Dax,
This is happening because your SAML has not been set up properly.
The email, first name and last name need to be mapped to attributes from
the incoming SAML assertion.
Please reach out to the Predix team so that they can set the correct
attribute mappings.
Thanks,
Sree
Sent from my iPhone
On May 17, 2016, at 7:22 PM, Dax Joshi <dax.joshi(a)tcs.com> wrote:
Hi,
Any update on this ?
Please let me know. I need to solve this issue as soon as possible.
Thanks & Regards,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- 9586581656
Mailto: dax.joshi(a)tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
From: Dax Joshi/AHD/TCS
To: Sree Tummidi <stummidi(a)pivotal.io>
Cc: Jonathan Lo <jlo(a)us.ibm.com>, cf-dev(a)lists.cloudfoundry.org
Date: 05/14/2016 10:55 AM
Subject: Re: Regarding UAA service
------------------------------
Hi Sree and Jonathan,
Thank you very much for your consideration and reply.
I am working on predix. I have bind my UAA service with SAML.
I use the GE's common login page to login so that using SSO anyone from
the same business can use my application.
*https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/oauth/authorize?client_id=<client-id>&response_type=code*
<https://489afafd-c6b4-4d81-ae52-e51116af4597.predix-uaa.run.asv-pr.ice.predix.io/oauth/authorize?client_id=ppduaa&response_type=code>
Which redirect me to GE's common login page. After successful login SAML
is sending user information to
*https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/saml/SSO/alias/<uaa-url>.cloudfoundry-saml-login*
<https://489afafd-c6b4-4d81-ae52-e51116af4597.predix-uaa.run.asv-pr.ice.predix.io/saml/SSO/alias/489afafd-c6b4-4d81-ae52-e51116af4597.cloudfoundry-saml-login>
While this redirection I have seen in browser network, in encoded format
as form data SAML is passing user's correct and full information to UAA.
including first name, last name, email, groups, roles etc..
After that UAA service redirects the browser to my landing page which I
have set as *redirect_uri *with one cookie named *TS0164a009* and one *code
in request param.*
In our application we use that code with */oauth/token* service and get
the *auth_token. *After this we use the *auth_token *with */check_token*
or /*userinfo* service of uaa to get the user information.
In which we get following json
{
"user_id": "d9cf7779-744a-407d-a846-36e0570d70d9",
"user_name": "sso",
"email": "sso(a)unknown.org",
"client_id": "client_id",
"exp": 1462921362,
"scope": [
"scim.me",
"openid"
],
"jti": "684643f2-a15a-4fca-b9ca-2f9ba2c22f82",
"aud": [
"scim",
"openid",
"ppduaa"
],
"sub": "d9cf7779-744a-407d-a846-36e0570d70d9",
"iss": "
*https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/oauth/token"*
<https://489afafd-c6b4-4d81-ae52-e51116af4597.predix-uaa.run.asv-pr.ice.predix.io/oauth/token%27>
,
"iat": 1462878162,
"cid": "client-id",
"grant_type": "authorization_code",
"azp": "client_id",
"auth_time": 1462878076,
"zid": "489afafd-c6b4-4d81-ae52-e51116af4597",
"rev_sig": "d8ddc2e6",
"origin": "gefssstg"
}
Here I have replaced *uaa-url, sso *and *client_id* with its actual
values.
You can notice that I am not even receiving correct mail id. it gives @
unknown.org
In this json nothing except sso seems to be useful to me.
Please guide me how to get the exact user details from UAA that SAML is
passing to it.
Let me know if anything else you need.
Regards,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- 9586581656
Mailto: dax.joshi(a)tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
From: Sree Tummidi <stummidi(a)pivotal.io>
To: Jonathan Lo <jlo(a)us.ibm.com>
Cc: Dax Joshi <dax.joshi(a)tcs.com>
Date: 05/14/2016 04:01 AM
Subject: Re: Regarding UAA service
------------------------------
Hi Dax,
Nice to meet you virtually. In the future you can use the cf-dev@
*lists.cloudfoundry.org* <http://lists.cloudfoundry.org/> for any UAA &
CloudFoundry related questions.
Can you elaborate on what you are trying to achieve with UAA & SAML
Integration and the use-case you have in mind
Thanks,
Sree Tummidi
Sr. Product Manager
Identity - Pivotal Cloud Foundry
On Fri, May 13, 2016 at 10:50 AM, Jonathan Lo <*jlo(a)us.ibm.com*
<jlo(a)us.ibm.com>> wrote:
Hi Dax,
Could you provide a bit more detail so that I can better direct your
query? As far as I know, you would be able to decode your access token in
order to obtain a user id, with which you could then get more user
information.
I've CCed Sree, our UAA PM, on the email.
Regards,
Jonathan
Sent from my iPhone
------------------------------
On May 13, 2016, 4:30:16 AM, *dax.joshi(a)tcs.com* <dax.joshi(a)tcs.com>
wrote:
From: *dax.joshi(a)tcs.com* <dax.joshi(a)tcs.com>
To: *jlo(a)us.ibm.com* <jlo(a)us.ibm.com>
Cc:
Date: May 13, 2016 4:30:16 AM
Subject: Regarding UAA service
Hi Jonathan,
This is Dax Joshi From TCS.
I found you email from *https://github.com/GESoftware-CF/uaa*
<https://github.com/GESoftware-CF/uaa>.
I have a query regarding getting loggedin
user details in case of UAA service and SAML Integration.
Please let me know if we can talk over
phone at your convenient time.
Please include other persons in this
loop if they can help me.
Thanks,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- *9586581656* <9586581656>
Mailto: *dax.joshi(a)tcs.com* <dax.joshi(a)tcs.com>
Website: *http://www.tcs.com* <http://www.tcs.com/>
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
|
|
|
Re: [abacus] Separate time-based from discrete usage metrics
Hi, To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage metrics (the rest basically) 2. Store the time-based metrics in a separate DB(s) Your proposal looks good to me. Piotr, Kevin and Raj and I had several design discussions on this topic in the last few days and we've come up with a few more ideas on top of what you're describing here: - The distinction between time-based and discrete resource usage metering could also be understood as usage metering of a stateful vs stateless resource or metric. In the stateful case, we simply store the state of the resource instance in a separate DB like you're proposing (e.g. we store the fact that an app or a container is currently running, or has stopped), and update that state in place when it changes. Then to compute and report usage later on we just need to the current resource instance state from that DB. - We could continue to store the metrics in the current historical databases as well (on top of that new DB) to preserve the resource instance history as many users typically want to know their past usage. - Some of us were not sure if the time-based / discrete distinction should be at the resource type level or at the metric level... IMO your proposal to do that at the metric level is cleaner so I'm happy with it :) - The dataflow module will probably need a few minor code changes to detect the case where some of the output docs need to go to a separate DB (IIRC you or someone else also mentioned that on one of our scrums or on slack...) - Like you said, we may still need to maintain 2 DBs to purge old entries. If that's easier, we could also adjust the usage accumulator service and the dataflow module a bit to delete entries for inactive resource instances right away (e.g. when an app or container stops.) Thoughts? P.S. I'll add these comments to issue #88 as well to make it be easier to follow up there. - Jean-Sebastien On Thu, May 12, 2016 at 5:54 AM, Hristo Iliev <hsiliev(a)gmail.com> wrote: Hi,
We're trying to fix Abacus issue 88: Missing aggregated usage for the
running application [1].
Background
=========
See the jsdelfino comment in the GitHub issue [2]. TL;DR: Resource
providers have to send a 'ping' doc per month for time-based metrics.
Proposed solution
==============
We decided to implement a solution in Abacus that frees the usage
providers from sending the 'ping' submission.
To fix the issue we decided to:
1. Distinguish between time-based (linux-container) and discrete usage
metrics (the rest basically)
2. Store the time-based metrics in a separate DB(s)
We already drafted a proposal for adding measurement type in the usage
plans with PR #320 [3].
We're about to spike on storing the time-based metrics in their own
Database, but we wanted to get the community opinion on the topic.
Motivation
========
The discrete usage submitted to Abacus is:
* stored in partitioned databases, due to their size/number
* like an event log, storing the history of the usage/resources
In contrast the current time-based metrics are:
* limited number (usually around 2 million on a loaded CF system)
* storing just the app resources usage state (GB/h consumed so far, GB/h
consuming currently)
Therefore it looks like a good idea to separate the two usage metrics
types and store the time-based metrics in a separate database. This will
allow us not only to solve the issue, but also to store and query the data
more effectively.
We may still need to maintain 2 databases and swap new/old (irrelevant)
metrics to reduce the DB size on the month boundaries.
Regards,
Hristo & Adriana
[1] https://github.com/cloudfoundry-incubator/cf-abacus/issues/88
[2]
https://github.com/cloudfoundry-incubator/cf-abacus/issues/88#issuecomment-148498164
[3] https://github.com/cloudfoundry-incubator/cf-abacus/pull/320
|
|
|
Re: Regarding UAA service
Hi Dax,
This is happening because your SAML has not been set up properly.
The email, first name and last name need to be mapped to attributes from the incoming SAML assertion.
Please reach out to the Predix team so that they can set the correct attribute mappings.
Thanks,
Sree
Sent from my iPhone
toggle quoted message
Show quoted text
On May 17, 2016, at 7:22 PM, Dax Joshi <dax.joshi(a)tcs.com> wrote:
Hi,
Any update on this ?
Please let me know. I need to solve this issue as soon as possible.
Thanks & Regards,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- 9586581656
Mailto: dax.joshi(a)tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
From: Dax Joshi/AHD/TCS
To: Sree Tummidi <stummidi(a)pivotal.io>
Cc: Jonathan Lo <jlo(a)us.ibm.com>, cf-dev(a)lists.cloudfoundry.org
Date: 05/14/2016 10:55 AM
Subject: Re: Regarding UAA service
Hi Sree and Jonathan,
Thank you very much for your consideration and reply.
I am working on predix. I have bind my UAA service with SAML.
I use the GE's common login page to login so that using SSO anyone from the same business can use my application.
https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/oauth/authorize?client_id=<client-id>&response_type=code
Which redirect me to GE's common login page. After successful login SAML is sending user information to
https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/saml/SSO/alias/<uaa-url>.cloudfoundry-saml-login
While this redirection I have seen in browser network, in encoded format as form data SAML is passing user's correct and full information to UAA. including first name, last name, email, groups, roles etc..
After that UAA service redirects the browser to my landing page which I have set as redirect_uri with one cookie named TS0164a009 and one code in request param.
In our application we use that code with /oauth/token service and get the auth_token. After this we use the auth_token with /check_token or /userinfo service of uaa to get the user information.
In which we get following json
{
"user_id": "d9cf7779-744a-407d-a846-36e0570d70d9",
"user_name": "sso",
"email": "sso(a)unknown.org",
"client_id": "client_id",
"exp": 1462921362,
"scope": [
"scim.me",
"openid"
],
"jti": "684643f2-a15a-4fca-b9ca-2f9ba2c22f82",
"aud": [
"scim",
"openid",
"ppduaa"
],
"sub": "d9cf7779-744a-407d-a846-36e0570d70d9",
"iss": "https://<uaa-url>.predix-uaa.run.asv-pr.ice.predix.io/oauth/token",
"iat": 1462878162,
"cid": "client-id",
"grant_type": "authorization_code",
"azp": "client_id",
"auth_time": 1462878076,
"zid": "489afafd-c6b4-4d81-ae52-e51116af4597",
"rev_sig": "d8ddc2e6",
"origin": "gefssstg"
}
Here I have replaced uaa-url, sso and client_id with its actual values.
You can notice that I am not even receiving correct mail id. it gives @unknown.org
In this json nothing except sso seems to be useful to me.
Please guide me how to get the exact user details from UAA that SAML is passing to it.
Let me know if anything else you need.
Regards,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- 9586581656
Mailto: dax.joshi(a)tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
From: Sree Tummidi <stummidi(a)pivotal.io>
To: Jonathan Lo <jlo(a)us.ibm.com>
Cc: Dax Joshi <dax.joshi(a)tcs.com>
Date: 05/14/2016 04:01 AM
Subject: Re: Regarding UAA service
Hi Dax,
Nice to meet you virtually. In the future you can use the cf-dev(a)lists.cloudfoundry.org for any UAA & CloudFoundry related questions.
Can you elaborate on what you are trying to achieve with UAA & SAML Integration and the use-case you have in mind
Thanks,
Sree Tummidi
Sr. Product Manager
Identity - Pivotal Cloud Foundry
On Fri, May 13, 2016 at 10:50 AM, Jonathan Lo <jlo(a)us.ibm.com> wrote:
Hi Dax,
Could you provide a bit more detail so that I can better direct your query? As far as I know, you would be able to decode your access token in order to obtain a user id, with which you could then get more user information.
I've CCed Sree, our UAA PM, on the email.
Regards,
Jonathan
Sent from my iPhone
On May 13, 2016, 4:30:16 AM, dax.joshi(a)tcs.com wrote:
From: dax.joshi(a)tcs.com
To: jlo(a)us.ibm.com
Cc:
Date: May 13, 2016 4:30:16 AM
Subject: Regarding UAA service
Hi Jonathan,
This is Dax Joshi From TCS.
I found you email from https://github.com/GESoftware-CF/uaa.
I have a query regarding getting loggedin
user details in case of UAA service and SAML Integration.
Please let me know if we can talk over
phone at your convenient time.
Please include other persons in this
loop if they can help me.
Thanks,
Dax Joshi
Systems Engineer
Tata Consultancy Services
GARIMA PARK,IT/ITES SEZ,
PLOT # 41,
Gandhinagar - 382007,Gujarat
India
Cell:- 9586581656
Mailto: dax.joshi(a)tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
|
|
|
