Re: can't login with cf CLI but the UAAC tool works
kyle havlovitz <kylehav@...>
Following up on this, I fixed the issue by using a verification key between the CC and the UAA instead of a token secret
toggle quoted messageShow quoted text
On Fri, Sep 4, 2015 at 6:51 PM, Filip Hanik <fhanik(a)pivotal.io> wrote: those urls do not look right. but they are dependent on what url you deployed the uaa under. ( ie / or /uaa )
what's returned by uaac info is dependent on your uaa config.
I'd have to get back to you in CC config. not my area of expertise. but yes I believe the CC will have an oauth client registered with the UAA
Filip
On Friday, September 4, 2015, kyle havlovitz <kylehav(a)gmail.com> wrote:
I realize it's a pain, but I'm setting these up without bosh. I'm just unfamiliar with how the config between the CC and the UAA needs to be set.
The http://localhost:8080/login and http://localhost:8080/uaa seem to be the correct URLs (they're whats returned by 'uaac info') Likewise, the CLI seems to be pointed at the right places, it's just getting this invalid token error, as if the CC can't correctly talk to the uaa or something.
what should the uaa.resource_id and uaa.symmetric_secret fields in the CC config be set to if I'm using the default config/clients? Are there any other values in the CC config that might be the issue here?
On Fri, Sep 4, 2015 at 6:26 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:
ok, is that the correct URL?
you're attempting to configure a very large eco system by hand. That can be a bit difficult. If you want a local cloud foundry, I would suggest bosh-lite
basically, clone cloudfoundry/cf-release and cloudfoundry/bosh-lite
cd bosh-lite vagrant up (this launches a VM with bosh director on it) bin/add-route (sets up network routing) bin/provision-cf (builds and publishing cloud foundry to the VM
cf api api.10.244.0.34.xip.io cf login
On Fri, Sep 4, 2015 at 4:18 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
The cloud controller logs have "Invalid bearer token: #<CF::UAA::InvalidSignature: Signature verification failed>" and the 401 invalid auth message.
On Fri, Sep 4, 2015 at 6:14 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
Ok, I set those 2 properties to http://localhost:8080 and it looks identical; same error, same endpoints requested. Could something be wrong with the cloud controller config?
On Fri, Sep 4, 2015 at 5:58 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:
yes, this is the URL misconfiguration I was talking about.
In your uaa.yml you should have two properties
login.url: http://localhost:8080 uaa.url: http://localhost:8080
set those two, and let's see that trace again
On Fri, Sep 4, 2015 at 2:58 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
The CLI seems to be able to get a token now though, it's failing for a different reason:
cf login API endpoint: http://localhost:8181 REQUEST: [2015-09-04T20:46:51Z] GET /v2/info HTTP/1.1 Host: localhost:8181 Accept: application/json Content-Type: application/json User-Agent: go-cli 6.12.3-c0c9a03 / linux
RESPONSE: [2015-09-04T20:46:51Z] HTTP/1.1 200 OK Content-Length: 406 Connection: keep-alive Content-Type: application/json;charset=utf-8 Server: thin X-Content-Type-Options: nosniff X-Vcap-Request-Id: d44503ef-3b2c-4340-9958-ad91daf3706c {"name":"vcap","build":"2222","support":" http://support.local.example.com","version":2,"description":"CF v2 test environment","authorization_endpoint":"http://localhost:8080 ","token_endpoint":"http://localhost:8080/uaa ","min_cli_version":null,"min_recommended_cli_version":null,"api_version":"2.34.0","app_ssh_endpoint":null,"app_ssh_host_key_fingerprint":null,"logging_endpoint":"ws:// 127.0.0.1:9090"} Warning: Insecure http API endpoint detected: secure https API endpoints are recommended
REQUEST: [2015-09-04T20:46:51Z] GET /login HTTP/1.1 Host: localhost:8080 Accept: application/json Content-Type: application/json User-Agent: go-cli 6.12.3-c0c9a03 / linux
RESPONSE: [2015-09-04T20:46:51Z] HTTP/1.1 200 OK Transfer-Encoding: chunked Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store, max-age=0, must-revalidate Cache-Control: no-cache, no-store, max-age=0 Content-Language: en-US Content-Type: application/json;charset=UTF-8 Date: Fri, 04 Sep 2015 20:46:51 GMT Expires: 0 Expires: Thu, 01 Jan 1970 00:00:00 GMT Pragma: no-cache Pragma: no-cache Server: Apache-Coyote/1.1 Strict-Transport-Security: max-age=31536000 X-Content-Type-Options: nosniff X-Frame-Options: DENY X-Xss-Protection: 1; mode=block 1fd
{"timestamp":"2015-08-05T00:00:41+0000","app":{"version":"2.5.1"},"idpDefinitions":[],"fieldUsernameShow":true,"zone_name":"uaa","commit_id":"eae6724","prompts":{"username":["text","Email"],"password":["password","Password"]},"forgotPasswordLink":"/forgot_password","createAccountLink":"/create_account","links":{"register":"/create_account","passwd":"/forgot_password","login":" http://localhost:8080/login","uaa":"http://localhost:8080/uaa "},"entityID":"cloudfoundry-saml-login","linkCreateAccountShow":true} 0
Email> admin Password> Authenticating... REQUEST: [2015-09-04T20:46:58Z] POST /oauth/token HTTP/1.1 Host: localhost:8080 Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/x-www-form-urlencoded User-Agent: go-cli 6.12.3-c0c9a03 / linux grant_type=password&password=[PRIVATE DATA HIDDEN]&scope=&username=admin RESPONSE: [2015-09-04T20:46:58Z] HTTP/1.1 200 OK Transfer-Encoding: chunked Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store, max-age=0, must-revalidate Cache-Control: no-store Content-Type: application/json;charset=UTF-8 Date: Fri, 04 Sep 2015 20:46:58 GMT Expires: 0 Pragma: no-cache Pragma: no-cache Server: Apache-Coyote/1.1 X-Content-Type-Options: nosniff X-Frame-Options: DENY X-Xss-Protection: 1; mode=block 738 {"access_token":"[PRIVATE DATA HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":43199,"scope":"scim.read scim.userids cloud_controller.admin scim.write cloud_controller.write password.write openid cloud_controller.read","jti":"cbda4e10-cf04-4696-a560-2e1f4d2c610c"} 0
OK
REQUEST: [2015-09-04T20:46:58Z] GET /v2/organizations HTTP/1.1 Host: localhost:8181 Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.3-c0c9a03 / linux
RESPONSE: [2015-09-04T20:46:58Z] HTTP/1.1 401 Unauthorized Content-Length: 97 Connection: keep-alive Content-Type: application/json;charset=utf-8 Server: thin X-Content-Type-Options: nosniff X-Vcap-Request-Id: b7658709-8145-4e31-a7ed-12a7831e99da { "code": 1000, "description": "Invalid Auth Token", "error_code": "CF-InvalidAuthToken" }
REQUEST: [2015-09-04T20:46:58Z] POST /oauth/token HTTP/1.1 Host: localhost:8080 Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/x-www-form-urlencoded User-Agent: go-cli 6.12.3-c0c9a03 / linux
grant_type=refresh_token&refresh_token=eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiJlNzFmOTNmZS0yMmEyLTQ3ZjgtODgwNC0xN2ZmNmU1YzM1NmMiLCJzdWIiOiIzMDBkZTc1YS1mN2RhLTRjMWQtYjA0Yi02YWZhZjI1ZmE5MjgiLCJzY29wZSI6WyJzY2ltLnJlYWQiLCJzY2ltLnVzZXJpZHMiLCJjbG91ZF9jb250cm9sbGVyLmFkbWluIiwic2NpbS53cml0ZSIsImNsb3VkX2NvbnRyb2xsZXIud3JpdGUiLCJwYXNzd29yZC53cml0ZSIsIm9wZW5pZCIsImNsb3VkX2NvbnRyb2xsZXIucmVhZCJdLCJpYXQiOjE0NDEzOTk2MTgsImV4cCI6MTQ0Mzk5MTYxOCwiY2lkIjoiY2YiLCJjbGllbnRfaWQiOiJjZiIsImlzcyI6Imh0dHA6Ly9sb2NhbGhvc3Q6ODA4MC91YWEvb2F1dGgvdG9rZW4iLCJ6aWQiOiJ1YWEiLCJncmFudF90eXBlIjoicGFzc3dvcmQiLCJ1c2VyX25hbWUiOiJhZG1pbiIsInVzZXJfaWQiOiIzMDBkZTc1YS1mN2RhLTRjMWQtYjA0Yi02YWZhZjI1ZmE5MjgiLCJyZXZfc2lnIjoiOTAyODliNjgiLCJhdWQiOlsiY2YiLCJzY2ltIiwiY2xvdWRfY29udHJvbGxlciIsInBhc3N3b3JkIiwib3BlbmlkIl19.-eGB2RWZfYVZkTSvT7c4lUzsY5QZMWgXFHMGGx4HEN8&scope= RESPONSE: [2015-09-04T20:46:58Z] HTTP/1.1 200 OK Transfer-Encoding: chunked Access-Control-Allow-Origin: * Cache-Control: no-cache, no-store, max-age=0, must-revalidate Cache-Control: no-store Content-Type: application/json;charset=UTF-8 Date: Fri, 04 Sep 2015 20:46:58 GMT Expires: 0 Pragma: no-cache Pragma: no-cache Server: Apache-Coyote/1.1 X-Content-Type-Options: nosniff X-Frame-Options: DENY X-Xss-Protection: 1; mode=block 738 {"access_token":"[PRIVATE DATA HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA HIDDEN]","expires_in":43199,"scope":"scim.userids scim.read cloud_controller.admin password.write scim.write openid cloud_controller.write cloud_controller.read","jti":"e62d3265-9ab7-441e-b2b2-69ca92d81d7c"} 0
REQUEST: [2015-09-04T20:46:58Z] GET /v2/organizations HTTP/1.1 Host: localhost:8181 Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.3-c0c9a03 / linux
RESPONSE: [2015-09-04T20:46:58Z] HTTP/1.1 401 Unauthorized Content-Length: 97 Connection: keep-alive Content-Type: application/json;charset=utf-8 Server: thin X-Content-Type-Options: nosniff X-Vcap-Request-Id: 7b07e39c-60f0-4db4-9274-6a59e23b8f29 { "code": 1000, "description": "Invalid Auth Token", "error_code": "CF-InvalidAuthToken" } FAILED Error finding available orgs Invalid auth token: Invalid Auth Token FAILED Error finding available orgs Invalid auth token: Invalid Auth Token
API endpoint: http://localhost:8181 (API version: 2.34.0) User: admin No org or space targeted, use 'cf target -o ORG -s SPACE'
On Fri, Sep 4, 2015 at 4:49 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
Ok, thanks for the helpful links. I replaced my config with the uaa.yml and login.yml from there and now the uaac commands from above work and I can run 'uaac token owner get'. I still can't login to cf with the cli though.
On Fri, Sep 4, 2015 at 4:15 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:
Minimalist defaults are in the UAA repo (uaa.yml and login.yml)
https://github.com/cloudfoundry/uaa/tree/master/uaa/src/main/resources
Yaml is very sensitive to indentation. So hand crafting it may become a bit difficult.
If you want the UAA to provide all default values (including admin/adminsecret client and cf/<blank password> client, then don't add any uaa.yml config file at all. Just start up UAA with it's defaults. It will suck in client defaults from
https://github.com/cloudfoundry/uaa/blob/feature/invitations_flow_by_email_domain/uaa/src/main/webapp/WEB-INF/spring/oauth-clients.xml#L47-L172
Filip
On Fri, Sep 4, 2015 at 2:05 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
is there an example somewhere of a minimalist working config for them? I'm going through at the moment and trying to make mine resemble the config here: https://github.com/cloudfoundry/cf-release/blob/master/jobs/uaa/templates/uaa.yml.erb
I'm also defining a test admin user in the scim users section
On Fri, Sep 4, 2015 at 4:00 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:
ok, that tells me that your configuration of the UAA clients is incorrect
On Fri, Sep 4, 2015 at 1:44 PM, kyle havlovitz < kylehav(a)gmail.com> wrote:
ok so the 'uaac token client get' fails, and the error is 'Bad credentials'
On Fri, Sep 4, 2015 at 3:33 PM, Filip Hanik <fhanik(a)pivotal.io> wrote:
ok, so we can validate that
uaac target http://localhost:8080 uaac token client get admin -s <your admin client secret> uaac clients
Should show your 'cf' client in the list
then we can do
uaac token owner get cf <username> -s "" -p <user password>
and if that works, we can take it to the next step
On Fri, Sep 4, 2015 at 1:26 PM, kyle havlovitz < kylehav(a)gmail.com> wrote:
I started the uaa by building from the tagged version in cf-release v215 and running it via tomcat with a custom config file, but I didn't specify a database. I have both a cf and admin section in the uaa clients config:
cf:
id: cf override: true authorized-grant-types: password,implicit,refresh_token authorities: uaa.none scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write secret: 'xxxxxxxxxx'
admin:
id: admin authorized-grant-types: client_credentials authorities: clients.read,clients.write,clients.secret,password.write,scim.read,uaa.admin scope: read,write,password resource-ids: clients secret: 'xxxxxxxxxx'
On Fri, Sep 4, 2015 at 3:09 PM, Filip Hanik < fhanik(a)pivotal.io> wrote:
ok, so the URL you have is /oauth/token, that's fine. your trace returns
"authorization_endpoint":"http://localhost:8080 ","token_endpoint":"http://localhost:8080/uaa"
indicating that there is a misconfiguration somewhere, but we can fix that later.
How did you start the UAA? Are you sure that your UAA has a client named 'cf' in its database?
On Fri, Sep 4, 2015 at 1:05 PM, kyle havlovitz < kylehav(a)gmail.com> wrote:
Running that command against /uaa/oauth/token gives just a redirect to /login. Doing it with /oauth/token gives a 401 unauthorized, same as the cf cli.
What do you mean by deploy it as root "/"? As in, a override the url it hosts the endpoints at?
|
|
Re: Security Question --- Securely wipe data on warden container removal / destruction???
Will Pragnell <wpragnell@...>
In Diego/Garden, container files are stored on btrfs subvolumes. When a container is destroyed, the subvolume is removed with btrfs subvolume delete. I don’t think this does anything particularly fancy, and I don’t think it classifies as “secure deletion”.
toggle quoted messageShow quoted text
On 17 September 2015 at 11:53, Chris K <christopherkugler2(a)yahoo.de> wrote: Hello again,
I'm sorry for having to revive this topic, but I'm still unaware about the deletion process.
[... ] i believe with standard removal tools. Could you please specify the term "standard removal tool". What's the standard? Is standard just the deletion of the pointer pointing on files / segments, or is secure deletion state of the art? I'd be thankful for any reference on documentation regarding this topic.
Thanks in advance.
Cheers Chris
|
|
Re: Security Question --- Securely wipe data on warden container removal / destruction???
Hello again, I'm sorry for having to revive this topic, but I'm still unaware about the deletion process. [... ] i believe with standard removal tools.
Could you please specify the term "standard removal tool". What's the standard? Is standard just the deletion of the pointer pointing on files / segments, or is secure deletion state of the art? I'd be thankful for any reference on documentation regarding this topic. Thanks in advance. Cheers Chris
|
|
Re: Relationship between HM9000 and router jobs
Thank you for your reply. True i think it was a coincidence ... I've try on a playground config to shutdown hm9000 and etd, and I did not lose routes at all, apps were still running.
|
|
Re: User cannot do CF login when UAA is being updated
Yunata, Ricky <rickyy@...>
Thank you Joseph, I will pass it to Dies Ricky Yunata Software & Solution Specialist Fujitsu Australia Software Technology Pty Ltd 14 Rodborough Road, Frenchs Forest NSW 2086, Australia T +61 2 9452 9128 M +61 433 977 739 F +61 2 9975 2899 rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com> fastware.com.au< http://fastware.com.au> [cid:image001.jpg(a)01D0F138.DC30E980] [cid:image002.jpg(a)01D0F138.DC30E980] From: CF Runtime [mailto:cfruntime(a)gmail.com] Sent: Wednesday, 16 September 2015 7:08 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Re: Re: Re: User cannot do CF login when UAA is being updated If you can't get the list to accept the attachment, you can give it to Dies and he should be able to get it to us. Joseph OSS Release Integration Team On Tue, Sep 15, 2015 at 7:19 PM, Yunata, Ricky <rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com>> wrote: Hi Joseph, Yes that is the case. I have sent my test result but it seems that my e-mail does not get through. How can I sent attachment in this mailing list? Regards, Ricky From: CF Runtime [mailto:cfruntime(a)gmail.com<mailto:cfruntime(a)gmail.com>] Sent: Tuesday, 15 September 2015 8:10 PM To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Re: User cannot do CF login when UAA is being updated Couple of updates here for clarity. No databases are stored on NFS in any default installation. NFS is only used to store blobstore data. If you are using the postgres job from cf-release, since it is single node there will be downtime during a stemcell deploy. I talked with Dies from Fujitsu earlier and confirmed they are NOT using the postgres job but an external non-cf deployed postgres instance. So during a deploy, the UAA db should be up and available the entire time. The issue they are seeing is that even though the database is up, and I'm guessing there is at least a single node of UAA up during the deploy, there are still login failures. Joseph OSS Release Integration Team On Mon, Sep 14, 2015 at 6:39 PM, Filip Hanik <fhanik(a)pivotal.io<mailto:fhanik(a)pivotal.io>> wrote: Amit, see previous comment. Postgresql database is stored on NFS that is restarted during nfs job update. UAA, while being up, is non functional while the NFS job is updated because it can't get to the DB. On Mon, Sep 14, 2015 at 5:09 PM, Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote: Hi Ricky, My understanding is that you still need help, and the issues Jiang and Alexander raised are different. To avoid confusion, let's keep this thread focused on your issue. Can you confirm that you have two UAA VMs in separate bosh jobs, separate AZs, etc. Can you confirm that when you roll the UAAs, only one goes down at a time? The simplest way to affect a roll is to change some trivial property in the manifest for your UAA jobs. If you're using v215, any of the properties referenced here will do: https://github.com/cloudfoundry/cf-release/blob/v215/jobs/uaa/spec#L321-L335You should confirm that only one UAA is down at a time, and comes back up before bosh moves on to updating the other UAA. While this roll is happening, can you just do `CF_TRACE=true cf auth USERNAME PASSWORD` in a loop, and if you see one that fails, post the output, along with noting the state of the bosh deploy when the error happens. Thanks, Amit On Mon, Sep 14, 2015 at 10:51 AM, Amit Gupta <agupta(a)pivotal.io<mailto:agupta(a)pivotal.io>> wrote: Ricky, Jiang, Alexander, are the three of you working together? It's hard to tell since you've got Fujitsu, Gmail, and Altoros email addresses. Are you folks talking about the same issue with the same deployment, or three separate issues. Ricky, if you still need assistance with your issue, please let us know. On Mon, Sep 14, 2015 at 10:16 AM, Lomov Alexander <alexander.lomov(a)altoros.com<mailto:alexander.lomov(a)altoros.com>> wrote: Yes, the problem is that postgresql database is stored on NFS that is restarted during nfs job update. I’m sure that you’ll be able to run updates without outage with several customizations. It is hard to tell without knowing your environment, but in common case steps will be following: 1. Add additional instances to nfs job and customize it to make replications (for instance use this docs for release customization [1]) 2. Make your NFS job to update sequently without our jobs updates in parallel (like it is done for postgresql [2]) 3. Check your options in update section [3]. [1] https://help.ubuntu.com/community/HighlyAvailableNFS[2] https://github.com/cloudfoundry/cf-release/blob/master/example_manifests/minimal-aws.yml#L115-L116[3] https://github.com/cloudfoundry/cf-release/blob/master/example_manifests/minimal-aws.yml#L57-L62On Sep 14, 2015, at 9:47 AM, Yitao Jiang <jiangyt.cn(a)gmail.com<mailto:jiangyt.cn(a)gmail.com>> wrote: On upgrading the deployment, the uaa not working due the uaadb filesystem hangup.Under my environment , the nfs-wal-server's ip changed which causing uaadb,ccdb hang up. Hard reboot the uaadb, restart uaa service solve the issue. Hopes can help you. On Mon, Sep 14, 2015 at 2:13 PM, Yunata, Ricky <rickyy(a)fast.au.fujitsu.com<mailto:rickyy(a)fast.au.fujitsu.com>> wrote: Hello, I have a question regarding UAA in Cloud Foundry. I’m currently running Cloud Foundry on Openstack. I have 2 availability zones and redundancy of the important VMs including UAA. Whenever I do an upgrade of either stemcell or CF release, user will not be able to do CF login when when CF is updating UAA VM. My question is, is this a normal behaviour? If I have redundant UAA VM, shouldn’t user still be able to still login to the apps even though it’s being updated? I’ve done this test a few times, with different CF version and stemcells and all of them are giving me the same result. The latest test that I’ve done was to upgrade CF version from 212 to 215. Has anyone experienced the same issue? Regards, Ricky Disclaimer The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof. Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached. If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com> -- Regards, Yitao jiangyt.github.io< http://jiangyt.github.io/> Disclaimer The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000<tel:%2B%2061%202%209452%209000> or by reply e-mail to the sender and delete the document and all copies thereof. Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached. If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com<mailto:unsubscribe(a)fast.au.fujitsu.com> Disclaimer The information in this e-mail is confidential and may contain content that is subject to copyright and/or is commercial-in-confidence and is intended only for the use of the above named addressee. If you are not the intended recipient, you are hereby notified that dissemination, copying or use of the information is strictly prohibited. If you have received this e-mail in error, please telephone Fujitsu Australia Software Technology Pty Ltd on + 61 2 9452 9000 or by reply e-mail to the sender and delete the document and all copies thereof. Whereas Fujitsu Australia Software Technology Pty Ltd would not knowingly transmit a virus within an email communication, it is the receiver’s responsibility to scan all communication and any files attached for computer viruses and other defects. Fujitsu Australia Software Technology Pty Ltd does not accept liability for any loss or damage (whether direct, indirect, consequential or economic) however caused, and whether by negligence or otherwise, which may result directly or indirectly from this communication or any files attached. If you do not wish to receive commercial and/or marketing email messages from Fujitsu Australia Software Technology Pty Ltd, please email unsubscribe(a)fast.au.fujitsu.com
|
|
Re: Relation between Network property in Resource pool block and Network property in Actual Job block
A lot of this is set to change in the near future to make things more clear. But here is what is currently going on. BOSH will pre-create machines it is going to need, and it does that without any input from the jobs that will eventually run on them. A machine needs a network to be created, so that is the reason it is needed in the resource pool section. Later when BOSH configures the machine for running a job on it, it will update the network to match the details provided in the job. You'll probably get a faster response to questions like this in the cf-bosh mailing list in the future. Joseph OSS Release Integration Team On Tue, Sep 8, 2015 at 6:09 PM, ronak banka <ronakbanka.cse(a)gmail.com> wrote: Hello All,
I have a resource pool , let say small_z1
- name: small_z1 network: cf1 stemcell: stemcell-xyz cloud_properties: instance_type: m1.small availability_zone: zone1
and a Job , router having two networks assigned to it
- name: router instances: 1 networks: - name: router_internal default: [dns, gateway] static_ips: - xy.xy.xy.xy - name: router_external static_ips: - yz.yz.yz.yz gateway: yy.yy.yy.yy networks: apps: router_internal management: router_internal resource_pool: small_z1
With these properties there are no issues anywhere.
what is the network property in resource pool responsible for, if the created job networks and not linked to the one in pool??
Regards, Ronak
-- View this message in context: http://cf-dev.70369.x6.nabble.com/Relation-between-Network-property-in-Resource-pool-block-and-Network-property-in-Actual-Job-block-tp1562.html Sent from the CF Dev mailing list archive at Nabble.com.
|
|
Re: tcp-routing in Lattice
That's true. This is one of the limitations of pure tcp (layer 4) routing. We will be hit by scalability limits in terms of public IPs. However, per public IP we can theoretically provide 64K ports and so can potentially accommodate many apps with tcp routing requirement (provided these apps and their clients can work with non-standard ports).
One alternative to this is to use SNI based solution where host header is provided in initial TLS handshake by client, which will help router to decide how to route the connection request. However, this will be limited to be used only over TLS, which may not be a very big limitation :)
|
|
Re: Proposal: Decomposing cf-release and Extracting Deployment Strategies
Another situation we have that you may want to keep in mind while developing cf-deployment: * We are using vsphere and currently we have a cf installation with 2 AZ using 2 separate vsphere "Datacenters" (more details: https://github.com/cloudfoundry/bosh-notes/issues/7). This means we have a CF installation that is actually made up of 2 deployments. So, we need to generate a manifest for az1 and another for az2. The job names in each deployment must be unique across the installation (e.g. cloud_controller_az1 and cloud_controller_az2) would be the cc job names in each deployment. Mike
toggle quoted messageShow quoted text
On Wed, Sep 16, 2015 at 3:38 PM, Mike Youngstrom <youngm(a)gmail.com> wrote: Here are some of the examples:
* Sensitive property management as part of manifest generation (encrypted or acquired from an outside source)
* We have our own internal bosh releases and config that we'll need to merge in with the things cf-deployment is doing. For example, if cf-deployment tags v250 as including Diego 3333 and etcd 34 with given templates perhaps we'd like to augment this with our own release jobs and config that we know to work with cf-deployment 250's and perhaps tag it as v250.lds and that becomes what we use to generate our manifests and upload releases.
* Occasionally we may wish to use some config from a stock release not currently exposed in a cf-deployment template. I'd like to be sure there is a way we can add that config, in a not hacky way, without waiting for a PR to be accepted and subsequent release.
* If for some reason we are forced to fork a stock release we'd like to be able to use that forked release we are building instead of the publicly available one for manifest generation and release uploads, etc.
Does that help?
Mike
On Tue, Sep 15, 2015 at 9:50 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Thanks for the feedback Mike!
Can you tell us more specifically what sort of extensions you need? It would be great if cf-deployment provided an interface that could serve the needs of essentially all operators of CF.
Thanks, Amit
On Tue, Sep 15, 2015 at 4:02 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:
This is great stuff! My organization currently maintains our own custom ways to generate manifests, include secure properties, and manage release versions.
We would love to base the next generation of our solution on cf-deployment. Have you put any thought into how others might customize or extend cf-deployment? Our needs are very similar to yours just sometimes a little different.
Perhaps a private fork periodically merged with a known good release combination (tag) might be appropriate? Or some way to include the same tools into a wholly private repo?
Mike
On Tue, Sep 8, 2015 at 1:22 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Hi all,
The CF OSS Release Integration team (casually referred to as the "MEGA team") is trying to solve a lot of tightly interrelated problems, and make many of said problems less interrelated. It is difficult to address just one issue without touching the others, so the following proposal addresses several issues, but the most important ones are:
* decompose cf-release into many independently manageable, independently testable, independently usable releases * separate manifest generation strategies from the release source, paving the way for Diego to be part of the standard deployment
This proposal will outline a picture of how manifest generation will work in a unified manner in development, test, and integration environments. It will also outline a picture of what each release’s test pipelines will look like, how they will feed into a common integration environment, and how feedback from the integration environment will feed back into the test environments. Finally, it will propose a picture for what the integration environment will look like, and how we get from the current integration environment to where we want to be.
For further details, please feel free to view and comment here:
https://docs.google.com/document/d/1Viga_TzUB2nLxN_ILqksmUiILM1hGhq7MBXxgLaUOkY
Thanks, Amit, CF OSS Release Integration team
|
|
Re: [vcap-dev] Re: Proposal for Context Path Routing
Stevo, Just a confirmation that I also saw your suggestion for defining the http method for a route. Thank you,
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
toggle quoted messageShow quoted text
On Wed, Sep 16, 2015 at 2:55 PM, Shannon Coen <scoen(a)pivotal.io> wrote: Hello Stevo,
Thank you for your suggestion of support for wildcards in route paths. I have noted the request and will keep an ear out for the same request from others in order to help me prioritize.
Best,
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
On Mon, Sep 14, 2015 at 8:34 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:
Thanks for heads up and fast reply!
Will read spec routing services/API specs in more detail. For the moment, for use cases I mentioned, one can use/deploy something like Netflix Zuul (Spring Boot) CF app, as a routing service. Covers a lot of use cases, but I expected support in CF itself.
Thanks once more!
Kind regards, Stevo Slavic.
On Mon, Sep 14, 2015 at 4:47 PM, Dieu Cao <dcao(a)pivotal.io> wrote:
Shannon, the Product Manager for the Routing team, has this story [1] in the CF Routing tracker project for doing a PR to the CLI to add support for context path routes.
Yes, that's correct that the context path route support does not currently support the particular example you've specified with wildcards or routing based on HTTP methods. If I recall correctly, we wanted to add some basic support for context path routes without adding too much overhead in what gorouter needed to reason about, as any logic added could impact every request to the gorouter.
There's a currently in progress tracks of work to support Route Services [2] [3]. That would allow you to insert you're own proxy to support all sorts of use cases and could be used to support what you've described.
Shannon is currently on vacation, but should be returning later this week. I'll forward this request to him for consideration. You might also consider opening up an issue against the gorouter with these enhancement requests.
[1] https://www.pivotaltracker.com/story/show/93368928 [2] https://www.pivotaltracker.com/epic/show/2031344 [3] https://www.pivotaltracker.com/epic/show/1884060
On Mon, Sep 14, 2015 at 6:44 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:
In this proposal and current Routes API docs <http://apidocs.cloudfoundry.org/217/routes/creating_a_route.html> I don't see support for wildcards when defining context path based route. Am I missing something? E.g. for resources like
/foos/bar/1/doSomeAction /foos/baz/2/doSomeAction
and
/foos/bar/1/doSomeOtherAction /foos/baz/2/doSomeOtherAction
I'd like all /foos/*/*/doSomeAction to be routed to one CF app, and all /foos/*/*/doSomeOtherAction to be routed to a different CF app.
Did you consider supporting HTTP methods to be used for routing as well? E.g. similar to previous example, I'd like some HTTP methods on particular resources to be handled by separate app then the rest of the requests on same resources.
DELETE /foos -> route to one cf app * /foos -> route to another cf app, where * means any HTTP method, most specific rule wins strategy
Ideally one would be able to combine both wildcards in HTTP methods and in path segments to define a routing rule.
Also I do not see context path routing support in CF CLI (v6.12.3) or a ticket in CF CLI issues to add the support. Did I miss something there? If not should I create new ticket?
Kind regards, Stevo Slavic.
On Tue, May 19, 2015 at 4:41 PM, sabith ks <sabithksme(a)gmail.com> wrote:
Thanks
On Tue, May 19, 2015 at 7:16 AM James Bayer <jbayer(a)pivotal.io> wrote:
context path based routing is currently experimental and mostly complete: http://apidocs.cloudfoundry.org/208/routes/creating_a_route.html
see this epic: https://www.pivotaltracker.com/epic/show/1808212
On Tue, May 19, 2015 at 5:55 AM, sabith ks <sabithksme(a)gmail.com> wrote:
Hi all,
Was there any follow ups on this proposal
-- Thanks Sabith
On Fri, Apr 24, 2015 at 08:35 Mike Youngstrom <youngm(a)gmail.com> wrote:
+1 couldn't have designed the feature better myself. :)
Mike
On Fri, Apr 24, 2015 at 3:52 AM, Dieu Cao <dcao(a)pivotal.io> wrote:
Ooops!
The link to the proposal is here:
[1] https://docs.google.com/document/d/1H_adSiY7wGR85av9YfxxPRylSO8Q8U0ANJJTg6wpYRQ/edit?usp=sharing
On Friday, April 24, 2015 at 1:11:15 AM UTC-7, Johannes Hiemer wrote:
Hi Dieu, link missing? :-/
On Friday, April 24, 2015 at 10:04:39 AM UTC+2, Dieu Cao wrote:
Hi All,
We've put together a proposal for Context Path Routing [1].
The goal of this feature is to enhance the gorouter, the cloud controller, the routing api, and the cf cli to support the ability for a space developer to create and map a route with a path to app1 and a route with a different path to app2.
We'd like to be able to iterate on this document quickly, so please keep your input and feedback in the comments of the document.
Thanks! -Dieu CF Runtime PM
--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org?utm_medium=email&utm_source=footer> .
To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
-- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com?utm_medium=email&utm_source=footer> .
To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org ---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com?utm_medium=email&utm_source=footer> .
-- Thank you,
James Bayer
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org --- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com?utm_medium=email&utm_source=footer> .
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org --- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com?utm_medium=email&utm_source=footer> .
|
|
Re: [vcap-dev] Re: Proposal for Context Path Routing
Hello Stevo,
Thank you for your suggestion of support for wildcards in route paths. I have noted the request and will keep an ear out for the same request from others in order to help me prioritize.
Best,
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
toggle quoted messageShow quoted text
On Mon, Sep 14, 2015 at 8:34 AM, Stevo Slavić <sslavic(a)gmail.com> wrote: Thanks for heads up and fast reply!
Will read spec routing services/API specs in more detail. For the moment, for use cases I mentioned, one can use/deploy something like Netflix Zuul (Spring Boot) CF app, as a routing service. Covers a lot of use cases, but I expected support in CF itself.
Thanks once more!
Kind regards, Stevo Slavic.
On Mon, Sep 14, 2015 at 4:47 PM, Dieu Cao <dcao(a)pivotal.io> wrote:
Shannon, the Product Manager for the Routing team, has this story [1] in the CF Routing tracker project for doing a PR to the CLI to add support for context path routes.
Yes, that's correct that the context path route support does not currently support the particular example you've specified with wildcards or routing based on HTTP methods. If I recall correctly, we wanted to add some basic support for context path routes without adding too much overhead in what gorouter needed to reason about, as any logic added could impact every request to the gorouter.
There's a currently in progress tracks of work to support Route Services [2] [3]. That would allow you to insert you're own proxy to support all sorts of use cases and could be used to support what you've described.
Shannon is currently on vacation, but should be returning later this week. I'll forward this request to him for consideration. You might also consider opening up an issue against the gorouter with these enhancement requests.
[1] https://www.pivotaltracker.com/story/show/93368928 [2] https://www.pivotaltracker.com/epic/show/2031344 [3] https://www.pivotaltracker.com/epic/show/1884060
On Mon, Sep 14, 2015 at 6:44 AM, Stevo Slavić <sslavic(a)gmail.com> wrote:
In this proposal and current Routes API docs <http://apidocs.cloudfoundry.org/217/routes/creating_a_route.html> I don't see support for wildcards when defining context path based route. Am I missing something? E.g. for resources like
/foos/bar/1/doSomeAction /foos/baz/2/doSomeAction
and
/foos/bar/1/doSomeOtherAction /foos/baz/2/doSomeOtherAction
I'd like all /foos/*/*/doSomeAction to be routed to one CF app, and all /foos/*/*/doSomeOtherAction to be routed to a different CF app.
Did you consider supporting HTTP methods to be used for routing as well? E.g. similar to previous example, I'd like some HTTP methods on particular resources to be handled by separate app then the rest of the requests on same resources.
DELETE /foos -> route to one cf app * /foos -> route to another cf app, where * means any HTTP method, most specific rule wins strategy
Ideally one would be able to combine both wildcards in HTTP methods and in path segments to define a routing rule.
Also I do not see context path routing support in CF CLI (v6.12.3) or a ticket in CF CLI issues to add the support. Did I miss something there? If not should I create new ticket?
Kind regards, Stevo Slavic.
On Tue, May 19, 2015 at 4:41 PM, sabith ks <sabithksme(a)gmail.com> wrote:
Thanks
On Tue, May 19, 2015 at 7:16 AM James Bayer <jbayer(a)pivotal.io> wrote:
context path based routing is currently experimental and mostly complete: http://apidocs.cloudfoundry.org/208/routes/creating_a_route.html
see this epic: https://www.pivotaltracker.com/epic/show/1808212
On Tue, May 19, 2015 at 5:55 AM, sabith ks <sabithksme(a)gmail.com> wrote:
Hi all,
Was there any follow ups on this proposal
-- Thanks Sabith
On Fri, Apr 24, 2015 at 08:35 Mike Youngstrom <youngm(a)gmail.com> wrote:
+1 couldn't have designed the feature better myself. :)
Mike
On Fri, Apr 24, 2015 at 3:52 AM, Dieu Cao <dcao(a)pivotal.io> wrote:
Ooops!
The link to the proposal is here:
[1] https://docs.google.com/document/d/1H_adSiY7wGR85av9YfxxPRylSO8Q8U0ANJJTg6wpYRQ/edit?usp=sharing
On Friday, April 24, 2015 at 1:11:15 AM UTC-7, Johannes Hiemer wrote:
Hi Dieu, link missing? :-/
On Friday, April 24, 2015 at 10:04:39 AM UTC+2, Dieu Cao wrote:
Hi All,
We've put together a proposal for Context Path Routing [1].
The goal of this feature is to enhance the gorouter, the cloud controller, the routing api, and the cf cli to support the ability for a space developer to create and map a route with a path to app1 and a route with a different path to app2.
We'd like to be able to iterate on this document quickly, so please keep your input and feedback in the comments of the document.
Thanks! -Dieu CF Runtime PM
--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/6f24eea9-7d1c-4547-8ed8-4a4643f1329b%40cloudfoundry.org?utm_medium=email&utm_source=footer> .
To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
-- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAEoPEDpqCbFDS3Ogn%2B%3D_Q7FzX6Nr9OnO5_T0o1eYkBOmpEpUxQ%40mail.gmail.com?utm_medium=email&utm_source=footer> .
To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+unsubscribe(a)cloudfoundry.org.
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org ---
You received this message because you are subscribed to the Google
Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit
https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJd%2BmiWb5apYUKjS3rVF7sr8vpedKFLQJS5XT5N25HCAcA%40mail.gmail.com?utm_medium=email&utm_source=footer> .
-- Thank you,
James Bayer
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org --- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAB%3Dt-sWEN8jaxrfRa3zRBbt-AyjaULBnTPHb0%2BpoboqhYp06jQ%40mail.gmail.com?utm_medium=email&utm_source=footer> .
-- This mailing list is for closed, and is available for archival purposes only. For active discussion, please visit https://lists.cloudfoundry.org/mailman/listinfo/cf-dev or email cf-dev(a)lists.cloudfoundry.org --- You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group. To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com <https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/CAHCF%3DJdyN5BtFJbH0e8sMqgWJ4Z89tx%3D8Ou1kBju%2BWQ3AGAfVw%40mail.gmail.com?utm_medium=email&utm_source=footer> .
|
|
Re: DEA/Warden staging error
Also worth noting: if you're cloning from the ssh endpoint at github you must have an ssh key in your ssh agent that authenticates against a real github user, even if the repo is public. It seems as though you're using the https endpoint so theoretically this should not affect you.
Zak + Natalie, CF OSS Integration
toggle quoted messageShow quoted text
On Wed, Sep 16, 2015 at 8:55 AM, kyle havlovitz <kylehav(a)gmail.com> wrote: It's just the staticfile buildpack from https://github.com/cloudfoundry/staticfile-buildpack.git, I can try git cloning it from inside the warden container though
On Wed, Sep 16, 2015 at 9:30 AM, Mike Dalessio <mdalessio(a)pivotal.io> wrote:
Worth noting that the git repo also needs to allow anonymous access. If it's a private repo, then the 'git clone' is going to fail.
Can you verify that you can download the buildpack from your repo without authenticating?
On Tue, Sep 15, 2015 at 7:43 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
It's not something we've ever seen before.
In theory, the warden container needs the git binary, which I think it gets from the cflinuxfs2 stack; and internet access to wherever the git repo lives.
If the warden container has both of those things, I can't think of any reason why it wouldn't work.
Joseph OSS Release Integration Team
On Tue, Sep 15, 2015 at 2:06 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
I tried deploying via uploading a buildpack to the CC (had to set up nginx first, I didnt have it running/configured before) and that worked! So that's awesome, but I'm not sure what the problem with using a remote buildpack is. Even with nginx, I still get the exact same error as before when pushing using a remote buildpack from git.
On Tue, Sep 15, 2015 at 6:57 AM, CF Runtime <cfruntime(a)gmail.com> wrote:
Looking at the logs, we can see it finishing downloading the app package. The next step should be to download and run the buildpack. Since you mention there is no output after this, I'm guessing it doesn't get that far.
It might be having trouble downloading the buildpack from the remote git url. Could you try uploading the buildpack to Cloud Controller and then having it use that buildpack to see if that makes a difference?
http://apidocs.cloudfoundry.org/217/buildpacks/creates_an_admin_buildpack.html
http://apidocs.cloudfoundry.org/217/buildpacks/upload_the_bits_for_an_admin_buildpack.html
Joseph OSS Release Integration Team
On Mon, Sep 14, 2015 at 5:37 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
Here's the full dea_ng and warden debug logs: https://gist.github.com/MrEnzyme/6dcc74174482ac62c1cf
Are there any other places I should look for logs?
On Mon, Sep 14, 2015 at 8:14 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
That's not an error we normally get. It's not clear if the staging_info.yml error is the source of the problem or an artifact of it. Having more logs would allow us to speculate more.
Joseph & Dan OSS Release Integration Team
On Mon, Sep 14, 2015 at 2:24 PM, kyle havlovitz <kylehav(a)gmail.com> wrote:
I have the cloudfoundry components built, configured and running on one VM (not in BOSH), and when I push an app I'm getting a generic 'FAILED StagingError' message after '-----> Downloaded app package (460K)'.
There's nothing in the logs for the dea/warden that seems suspect other than these 2 things:
{ "timestamp": 1441985105.8883495,
"message": "Exited with status 1 (35.120s):
[[\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\", \"/opt/cloudfoundry/warden/warden/src/closefds/closefds\"], \"/var/warden/containers/18vf956il5v/bin/iomux-link\", \"-w\", \"/var/warden/containers/18vf956il5v/jobs/8/cursors\", \"/var/warden/containers/18vf956il5v/jobs/8\"]", "log_level": "warn",
"source": "Warden::Container::Linux",
"data": {
"handle": "18vf956il5v",
"stdout": "",
"stderr": ""
},
"thread_id": 69890836968240,
"fiber_id": 69890849112480,
"process_id": 17063,
"file":
"/opt/cloudfoundry/warden/warden/lib/warden/container/spawn.rb", "lineno": 135,
"method": "set_deferred_success"
}
{ "timestamp": 1441985105.94083,
"message": "Exited with status 23 (0.023s):
[[\"/opt/cloudfoundry/warden/warden/src/closefds/closefds\", \"/opt/cloudfoundry/warden/warden/src/closefds/closefds\"], \"rsync\", \"-e\", \"/var/warden/containers/18vf956il5v/bin/wsh --socket /var/warden/containers/18vf956il5v/run/wshd.sock --rsh\", \"-r\", \"-p\", \"--links\", \"vcap(a)container:/tmp/staged/staging_info.yml\", \"/tmp/dea_ng/staging/d20150911-17093-1amg6y8\"]", "log_level": "warn",
"source": "Warden::Container::Linux",
"data": {
"handle": "18vf956il5v",
"stdout": "",
"stderr": "rsync: link_stat
\"/tmp/staged/staging_info.yml\" failed: No such file or directory (2)\nrsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1655) [Receiver=3.1.0]\nrsync: [Receiver] write error: Broken pipe (32)\n" },
"thread_id": 69890836968240,
"fiber_id": 69890849112480,
"process_id": 17063,
"file":
"/opt/cloudfoundry/warden/warden/lib/warden/container/spawn.rb", "lineno": 135,
"method": "set_deferred_success"
}
And I think the second error is just during cleanup, only failing because the staging process didn't get far enough in to create the 'staging_info.yml'. The one about iomux-link exiting with status 1 is pretty mysterious though and I have no idea what caused it. Does anyone know why this might be happening?
|
|
Re: Proposal: Decomposing cf-release and Extracting Deployment Strategies
Here are some of the examples:
* Sensitive property management as part of manifest generation (encrypted or acquired from an outside source)
* We have our own internal bosh releases and config that we'll need to merge in with the things cf-deployment is doing. For example, if cf-deployment tags v250 as including Diego 3333 and etcd 34 with given templates perhaps we'd like to augment this with our own release jobs and config that we know to work with cf-deployment 250's and perhaps tag it as v250.lds and that becomes what we use to generate our manifests and upload releases.
* Occasionally we may wish to use some config from a stock release not currently exposed in a cf-deployment template. I'd like to be sure there is a way we can add that config, in a not hacky way, without waiting for a PR to be accepted and subsequent release.
* If for some reason we are forced to fork a stock release we'd like to be able to use that forked release we are building instead of the publicly available one for manifest generation and release uploads, etc.
Does that help?
Mike
toggle quoted messageShow quoted text
On Tue, Sep 15, 2015 at 9:50 PM, Amit Gupta <agupta(a)pivotal.io> wrote: Thanks for the feedback Mike!
Can you tell us more specifically what sort of extensions you need? It would be great if cf-deployment provided an interface that could serve the needs of essentially all operators of CF.
Thanks, Amit
On Tue, Sep 15, 2015 at 4:02 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:
This is great stuff! My organization currently maintains our own custom ways to generate manifests, include secure properties, and manage release versions.
We would love to base the next generation of our solution on cf-deployment. Have you put any thought into how others might customize or extend cf-deployment? Our needs are very similar to yours just sometimes a little different.
Perhaps a private fork periodically merged with a known good release combination (tag) might be appropriate? Or some way to include the same tools into a wholly private repo?
Mike
On Tue, Sep 8, 2015 at 1:22 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Hi all,
The CF OSS Release Integration team (casually referred to as the "MEGA team") is trying to solve a lot of tightly interrelated problems, and make many of said problems less interrelated. It is difficult to address just one issue without touching the others, so the following proposal addresses several issues, but the most important ones are:
* decompose cf-release into many independently manageable, independently testable, independently usable releases * separate manifest generation strategies from the release source, paving the way for Diego to be part of the standard deployment
This proposal will outline a picture of how manifest generation will work in a unified manner in development, test, and integration environments. It will also outline a picture of what each release’s test pipelines will look like, how they will feed into a common integration environment, and how feedback from the integration environment will feed back into the test environments. Finally, it will propose a picture for what the integration environment will look like, and how we get from the current integration environment to where we want to be.
For further details, please feel free to view and comment here:
https://docs.google.com/document/d/1Viga_TzUB2nLxN_ILqksmUiILM1hGhq7MBXxgLaUOkY
Thanks, Amit, CF OSS Release Integration team
|
|
Re: Environment variable names (was RE: Environment variable changes in DIEGO)
john mcteague <john.mcteague@...>
Add another +1. We are implementing something akin to this within out environment, having it standardised would be a big win.
toggle quoted messageShow quoted text
On Wed, Sep 16, 2015 at 8:49 PM, Mike Youngstrom <youngm(a)gmail.com> wrote: +1 We'd love a metadata service as well when the dust settles.
Mike
On Wed, Sep 16, 2015 at 1:17 PM, Christopher B Ferris <chrisfer(a)us.ibm.com
wrote: Onsi,
I can tell you that we (IBM) would welcome a secure metadata service for retrieving credentials, etc. I'm sure we'd help make that happen.
Cheers,
Christopher Ferris IBM Distinguished Engineer, CTO Open Technology IBM Cloud, Open Technologies email: chrisfer(a)us.ibm.com twitter: @christo4ferris blog: http://thoughtsoncloud.com/index.php/author/cferris/ phone: +1 508 667 0402
----- Original message ----- From: Onsi Fakhouri <ofakhouri(a)pivotal.io> To: "Discussions about Cloud Foundry projects and the system overall." < cf-dev(a)lists.cloudfoundry.org> Cc: Subject: [cf-dev] Re: Environment variable names (was RE: Environment variable changes in DIEGO) Date: Wed, Sep 16, 2015 11:40 AM
Would if we could! Environment variables are part of the surface area that constitutes the contract between the platform and the application. While we could have duplicates with CF_* and eventually deprecate VCAP_* the new runtime needs to support droplets staged by the legacy runtime seamlessly.
Once the dust settles I'd sooner see us step back and reconsider this particular abstraction. For example, instead of env vars an authenticated metadata service a la AWS's http://169.254.169.254/ would give us more flexibility, allow us to dynamically control metadata made available to apps, and even version metadata sanely.
Onsi
Sent from my iPad
On Sep 16, 2015, at 8:30 AM, john mcteague <john.mcteague(a)gmail.com> wrote:
On a related, but slightly off, topic, while renaming the VCAP_* vars would have a big impact, is it not time we thought about renaming these to CF_* ?
John. On 16 Sep 2015 16:09, "Matthew Sykes" <matthew.sykes(a)gmail.com> wrote:
The changes, in general, were intentional. The `application_uris` data was always broken as it didn't reflect route changes. I can't speak directly to the time stamp data.
The host is present still so I don't know why you don't see it.
We also have a migration guide [1]. If you think additional information is needed there, pull requests would be welcome.
[1]: https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md
On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight(a)gmail.com> wrote:
I notice the below changes in the environment variables of DIEGO: 1. VCAP_APP_HOST & VCAP_APP_PORT are removed. 2. These fields are removed from VCAP_APPLICATION value: application_uris, started_at, start, started_at_timestamp, host, state_timestamp
I suppose #1 is intentional. How about #2?
Jack
-- Matthew Sykes matthew.sykes(a)gmail.com
|
|
Re: Environment variable names (was RE: Environment variable changes in DIEGO)
+1 We'd love a metadata service as well when the dust settles. Mike On Wed, Sep 16, 2015 at 1:17 PM, Christopher B Ferris <chrisfer(a)us.ibm.com> wrote: Onsi,
I can tell you that we (IBM) would welcome a secure metadata service for retrieving credentials, etc. I'm sure we'd help make that happen.
Cheers,
Christopher Ferris IBM Distinguished Engineer, CTO Open Technology IBM Cloud, Open Technologies email: chrisfer(a)us.ibm.com twitter: @christo4ferris blog: http://thoughtsoncloud.com/index.php/author/cferris/ phone: +1 508 667 0402
----- Original message ----- From: Onsi Fakhouri <ofakhouri(a)pivotal.io> To: "Discussions about Cloud Foundry projects and the system overall." < cf-dev(a)lists.cloudfoundry.org> Cc: Subject: [cf-dev] Re: Environment variable names (was RE: Environment variable changes in DIEGO) Date: Wed, Sep 16, 2015 11:40 AM
Would if we could! Environment variables are part of the surface area that constitutes the contract between the platform and the application. While we could have duplicates with CF_* and eventually deprecate VCAP_* the new runtime needs to support droplets staged by the legacy runtime seamlessly.
Once the dust settles I'd sooner see us step back and reconsider this particular abstraction. For example, instead of env vars an authenticated metadata service a la AWS's http://169.254.169.254/ would give us more flexibility, allow us to dynamically control metadata made available to apps, and even version metadata sanely.
Onsi
Sent from my iPad
On Sep 16, 2015, at 8:30 AM, john mcteague <john.mcteague(a)gmail.com> wrote:
On a related, but slightly off, topic, while renaming the VCAP_* vars would have a big impact, is it not time we thought about renaming these to CF_* ?
John. On 16 Sep 2015 16:09, "Matthew Sykes" <matthew.sykes(a)gmail.com> wrote:
The changes, in general, were intentional. The `application_uris` data was always broken as it didn't reflect route changes. I can't speak directly to the time stamp data.
The host is present still so I don't know why you don't see it.
We also have a migration guide [1]. If you think additional information is needed there, pull requests would be welcome.
[1]: https://github.com/cloudfoundry-incubator/diego-design-notes/blob/master/migrating-to-diego.md
On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight(a)gmail.com> wrote:
I notice the below changes in the environment variables of DIEGO: 1. VCAP_APP_HOST & VCAP_APP_PORT are removed. 2. These fields are removed from VCAP_APPLICATION value: application_uris, started_at, start, started_at_timestamp, host, state_timestamp
I suppose #1 is intentional. How about #2?
Jack
-- Matthew Sykes matthew.sykes(a)gmail.com
|
|
Re: Packaging CF app as bosh-release
My very limited understanding is that NFS writes to the actual filesystem, and achieves persistence by having centralized NFS servers where it writes to a real mounted device, whereas the clients write to an ephemeral nfs-mount.
My very limited understanding of HDFS is that it's all userland FS, does not write to the actual filesystem, and relies on replication to other nodes in the HDFS cluster. Being a userland FS, you don't have to worry about the data being wiped when a container is shut down, if you were to run it as an app.
I think one main issue is going to be ensuring that you never lose too many instances (whether they are containers or VMs), since you might then lose all replicas of a given data shard. Whether you go with apps or BOSH VMs doesn't make a big difference here.
Deploying as an app may be a better way to go, it's simpler right now to configure and deploy and app, than to configure and deploy a full BOSH release. It's also likely to be a more efficient use of resources, since a BOSH VM can only run one of these spark-job-processors, but a CF container-runner can run lots of other things. That actually brings up a different question: is your compute environment a multi-tenant one that will be running multiple different workloads? E.g. could someone also use the CF to push their own apps? Or is the whole thing just for your spark jobs, in which case you might only be running one container per VM anyways?
Assuming you can make use of the VMs for other workloads, I think this would be an ideal use case for Diego. You probably don't need all the extra logic around apps, like staging and routing, you just need Diego to efficiently schedule containers for you.
toggle quoted messageShow quoted text
On Wed, Sep 16, 2015 at 1:13 PM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote: Thanks Dmitriy,
Just for clarity, are you saying multiple instances of a VM cannot share a single shared filesystem?
On Wed, Sep 16, 2015 at 6:59 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote:
BOSH allocates a persistent disk per instance. It never shares persistent disks between multiple instances at the same time.
If you need a shared file system, you will have to use some kind of a release for it. It's not any different from what people do with nfs server/client.
On Wed, Sep 16, 2015 at 7:09 AM, Amit Gupta <agupta(a)pivotal.io> wrote:
The shared file system aspect is an interesting wrinkle to the problem. Unless you use some network layer to how you write to the shared file system, e.g. SSHFS, I think apps will not work because they get isolated to run in a container, they're given a chroot "jail" for their file system, and it gets blown away whenever the app is stopped or restarted (which will commonly happen, e.g. during a rolling deploy of the container-runner VMs).
Do you have something that currently works? How do your VMs currently access this shared FS? I'm not sure BOSH has the abstractions for choosing a shared, already-existing "persistent disk" to be attached to multiple VMs. I also don't know what happens when you scale your VMs down, because BOSH would generally destroy the associated persistent disk, but you don't want to destroy the shared data.
Dmitriy, any idea how BOSH can work with a shared filesystem (e.g. HDFS)?
Amit
On Wed, Sep 16, 2015 at 6:54 AM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:
On Wed, Sep 16, 2015 at 3:44 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Are the spark jobs tasks that you expect to end, or apps that you expect to run forever?
They are tasks that run forever. The jobs are subscribers to RabbitMQ queues that process messages in batches.
Do your jobs need to write to the file system, or do they access a shared/distributed file system somehow?
The jobs write to shared filesystem.
Do you need things like a static IP allocated to your jobs?
No.
Are your spark jobs serving any web traffic?
No.
|
|
Re: Environment variable names (was RE: Environment variable changes in DIEGO)
Christopher B Ferris <chrisfer@...>
Onsi,
I can tell you that we (IBM) would welcome a secure metadata service for retrieving credentials, etc. I'm sure we'd help make that happen.
Cheers,
Christopher Ferris IBM Distinguished Engineer, CTO Open Technology IBM Cloud, Open Technologies email: chrisfer@... twitter: @christo4ferris blog: http://thoughtsoncloud.com/index.php/author/cferris/ phone: +1 508 667 0402
toggle quoted messageShow quoted text
----- Original message ----- From: Onsi Fakhouri <ofakhouri@...> To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev@...> Cc: Subject: [cf-dev] Re: Environment variable names (was RE: Environment variable changes in DIEGO) Date: Wed, Sep 16, 2015 11:40 AM
Would if we could! Environment variables are part of the surface area that constitutes the contract between the platform and the application. While we could have duplicates with CF_* and eventually deprecate VCAP_* the new runtime needs to support droplets staged by the legacy runtime seamlessly.
Once the dust settles I'd sooner see us step back and reconsider this particular abstraction. For example, instead of env vars an authenticated metadata service a la AWS's http://169.254.169.254/ would give us more flexibility, allow us to dynamically control metadata made available to apps, and even version metadata sanely.
Onsi
Sent from my iPad
On a related, but slightly off, topic, while renaming the VCAP_* vars would have a big impact, is it not time we thought about renaming these to CF_* ?
John.
On 16 Sep 2015 16:09, "Matthew Sykes" < matthew.sykes@...> wrote:
The changes, in general, were intentional. The `application_uris` data was always broken as it didn't reflect route changes. I can't speak directly to the time stamp data.
The host is present still so I don't know why you don't see it.
We also have a migration guide [1]. If you think additional information is needed there, pull requests would be welcome.
On Wed, Sep 16, 2015 at 10:19 AM, Jack Cai <greensight@...> wrote:
I notice the below changes in the environment variables of DIEGO: 1. VCAP_APP_HOST & VCAP_APP_PORT are removed. 2. These fields are removed from VCAP_APPLICATION value: application_uris, started_at, start, started_at_timestamp, host, state_timestamp I suppose #1 is intentional. How about #2? Jack
--
|
|
Re: consolidated routing api
Checking once more that no one is using the Routing API yet, so we can proceed with backward incompatible API changes. I believe we still have this opportunity as we have not yet announced the component is ready for use.
Our current plan is that the first consumers of the Routing API will be: - CC (to register it's own route: api.system-domain) - TCP Route Emitter (to register routes for Diego LRPs with TCP Routes) - TCP Router (to fetch routing table of TCP routes)
Thank you,
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
toggle quoted messageShow quoted text
On Wed, Sep 9, 2015 at 6:05 PM, Shannon Coen <scoen(a)pivotal.io> wrote: Some of the proposed changes to the Routing API are backward incompatible. We don't believe anyone is using it yet, as adoption has generally be blocked on securing connections to Consul, but we'd like to confirm.
Please raise your hand if you're using the routing API.
Thank you!
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
On Wed, Sep 9, 2015 at 12:10 PM, Shannon Coen <scoen(a)pivotal.io> wrote:
We currently have two routing APIs in CF. 1. HTTP Routing API in cf-release: https://github.com/cloudfoundry-incubator/routing-api 2. TCP Routing API in cf-routing-release: https://github.com/cloudfoundry-incubator/cf-routing-release
The TCP Routing API is quite basic and we want to extend it for high availability, authentication, etc. However, instead of enhancing the existing TCP Routing API, we plan to add support for TCP route registration to the Routing API in cf-release, as it already supports many of the desired features. We'll get rid of the current API in cf-routing-release and submodule in the Routing API from cf-release. Eventually we'll move the Routing API (along with GoRouter and HAProxy) from cf-release into cf-routing-release and submodule them into cf-release.
This consolidation, along with our not having any API consumer besides GoRouter yet, gives us the opportunity to consider a common behavior for routing API endpoints. We welcome your comments in our design doc:
https://docs.google.com/document/d/1v941oy3Y7RRI80gmLfhPZlaahElK_Q0C3pCQewK8Z3g/edit?usp=sharing
Thank you,
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
|
|
Re: Relationship between HM9000 and router jobs
The DEAs are responsible for broadcasting the routes for the apps they are running. I can't think of why an hm9000 problem would cause routes to get lost, unless there was some problem with NATS itself. Joseph OSS Release Integration Team On Wed, Sep 16, 2015 at 1:37 AM, Sylvain FAUVE <sylvain.fauve(a)sap.com> wrote: Hello,
My team was working on solving inconsistencies issue on etcd jobs, and realized that two hm9000 jobs were running at same time. When fixing this, we experienced route loss to our apps (then restage apps was needed).
As far as I could read/understand there is no direct communication between router and hm9000... Router is getting info from NATS, and NATS gets it from ...? hm9000 ? I wonder which component is sending routes update to the router to keep them alive ?
Thank you for your help Regards, Sylvain.
|
|
Re: cc metric: total_users
Yeah, just as it is possible to create users in UAA, it is also possible to delete them there, thus orphaning the user accounts in the Cloud Controller. The orphaned users won't cause any problems in the Cloud Controller, but the metrics may not be what you expect at that point. The cf CLI automatically deletes users out of both when it is used. Joseph OSS Release Integration Team On Wed, Sep 16, 2015 at 4:29 AM, Klevenz, Stephan <stephan.klevenz(a)sap.com> wrote: Hi,
I did look deeper into implementation. Actually there are two databases: ccdb and uaadb. Each of them has its own users table. The user count value of the uaadb/users is reported to admin ui and the user count of the ccdb/users is used for total_user metric. A ccdb/users table entry contains just a refid and for getting user details from uaa.
So, we have two totals which can be different if users created for uaa which a not cc users. That's fine and this is what I did get from the first answer.
But there is a remaining open point. The total of our ccdb/users is bigger than total of uaa/users. This is an inconsistency in ccdb/users which contains references to uaa users that do not exist. If this diffs grows over time then this is maybe a problem.
Regards, Stephan
Von: CF Runtime Antworten an: "Discussions about Cloud Foundry projects and the system overall." Datum: Mittwoch, 16. September 2015 10:56 An: "Discussions about Cloud Foundry projects and the system overall." Betreff: [cf-dev] Re: Re: Re: Re: cc metric: total_users
The users reported by CloudController.total_users are the users the Cloud Controller has in its database. This is normally the same set of users that exist in UAA.
However, there is nothing that prevents you from creating users via the UAAC cli tool, or creating new UAA clients that can create users themselves.
Joseph OSS Release Integration Team
On Wed, Sep 16, 2015 at 12:03 AM, Voelz, Marco <marco.voelz(a)sap.com> wrote:
Hi John,
thanks for your answer, however, I don’t understand that completely. For the open source version of CF, what does “register in CF console” mean? And what might be an example of “other applications” you are referring to?
Thanks and warm regards Marco
On 16/09/15 08:29, "Klevenz, Stephan" <stephan.klevenz(a)sap.com> wrote:
Thanks for clarification
-- Stephan
Von: John Liptak Antworten an: "Discussions about Cloud Foundry projects and the system overall." Datum: Dienstag, 15. September 2015 18:17 An: "Discussions about Cloud Foundry projects and the system overall." Betreff: [cf-dev] Re: cc metric: total_users
Cloud Controller reports the number of users register in CF console. UAAC reports additional users who may have access to other applications. So they are both correct, depending on what you need.
For example, if you call the REST API for a UAAC user that isn't in the CF console, but still call the cloud controller REST API, you will get a 404.
On Tue, Sep 15, 2015 at 10:10 AM, Klevenz, Stephan < stephan.klevenz(a)sap.com> wrote:
Hi all,
I have a question, hopefully a small one :-)
The CloudController.total_users metric (/CF\.CloudController\.0\..*\.total_users/) differs from number of users reported by uaac (uaac users command / admin ui). Can someone explain why this differs or which number is the correct one?
Regards, Stephan
|
|
Re: Packaging CF app as bosh-release
Thanks Dmitriy, Just for clarity, are you saying multiple instances of a VM cannot share a single shared filesystem? On Wed, Sep 16, 2015 at 6:59 PM, Dmitriy Kalinin <dkalinin(a)pivotal.io> wrote: BOSH allocates a persistent disk per instance. It never shares persistent disks between multiple instances at the same time.
If you need a shared file system, you will have to use some kind of a release for it. It's not any different from what people do with nfs server/client.
On Wed, Sep 16, 2015 at 7:09 AM, Amit Gupta <agupta(a)pivotal.io> wrote:
The shared file system aspect is an interesting wrinkle to the problem. Unless you use some network layer to how you write to the shared file system, e.g. SSHFS, I think apps will not work because they get isolated to run in a container, they're given a chroot "jail" for their file system, and it gets blown away whenever the app is stopped or restarted (which will commonly happen, e.g. during a rolling deploy of the container-runner VMs).
Do you have something that currently works? How do your VMs currently access this shared FS? I'm not sure BOSH has the abstractions for choosing a shared, already-existing "persistent disk" to be attached to multiple VMs. I also don't know what happens when you scale your VMs down, because BOSH would generally destroy the associated persistent disk, but you don't want to destroy the shared data.
Dmitriy, any idea how BOSH can work with a shared filesystem (e.g. HDFS)?
Amit
On Wed, Sep 16, 2015 at 6:54 AM, Kayode Odeyemi <dreyemi(a)gmail.com> wrote:
On Wed, Sep 16, 2015 at 3:44 PM, Amit Gupta <agupta(a)pivotal.io> wrote:
Are the spark jobs tasks that you expect to end, or apps that you expect to run forever?
They are tasks that run forever. The jobs are subscribers to RabbitMQ queues that process messages in batches.
Do your jobs need to write to the file system, or do they access a shared/distributed file system somehow?
The jobs write to shared filesystem.
Do you need things like a static IP allocated to your jobs?
No.
Are your spark jobs serving any web traffic?
No.
|
|