Re: Trouble enabling diego ssh in cf-release:222 diego:0.1437


Mike Youngstrom
 

This is what I'm now seeing in the logs:

ssh-proxy:
{"timestamp":"1446134601.007453442","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","log_level":2,"data":{"error":"no
auth passed yet","user":"cf:52489e92-11b3-447f-813a-322353996d4a/0"}}
{"timestamp":"1446134601.011086702","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-starting","log_level":1,"data":{"session":"23548"}}
{"timestamp":"1446134601.077978134","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.exchange-access-code-for-token.response-status-not-ok","log_level":2,"data":{"error":"Authentication
failed","session":"23548.1","status-code":400}}
{"timestamp":"1446134601.078182459","source":"ssh-proxy","message":"ssh-proxy.cf-authenticate.authenticate-finished","log_level":1,"data":{"session":"23548"}}
{"timestamp":"1446134601.078270674","source":"ssh-proxy","message":"ssh-proxy.authentication-failed","log_level":2,"data":{"error":"Authentication
failed","user":"cf:52489e92-11b3-447f-813a-322353996d4a/0"}}

uaa:
[2015-10-29T16:03:20.422Z] uaa - 6067 [http-bio-8080-exec-7] .... INFO ---
Audit: TokenIssuedEvent
('["cloud_controller.read","password.write","cloud_controller.write","openid","doppler.firehose","scim.read","cloud_controller.admin","uaa.user"]'):
principal=af373f0b-a193-4434-85ca-692c89e8feab, origin=[caller=cf,
details=(type=UaaAuthenticationDetails)], identityZoneId=[uaa]
[2015-10-29T16:03:21.073Z] uaa - 6067 [http-bio-8080-exec-27] .... INFO
--- TokenEndpoint: Handling error: InvalidGrantException, Invalid
authorization code: nZJfFg

CLI:
ssh: handshake failed: ssh: unable to authenticate, attempted methods [none
password], no supported methods remain

I'll try manual ssh. I'm also going to debug into UAA more see if I can
figure out why it isn't validating the authorization code being sent to it
presumably from ssh-proxy.

Mike

On Thu, Oct 29, 2015 at 3:12 AM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

I did the work on the cli plugin but not on the integration into the cli.
Based on your first error, it looked like we were having a problem getting
the one time code, not authenticating with the ssh proxy. The fact that
you're able to get the code from the UAA manually implies that piece is
working correctly.

The authorization code message could be related but, if it were, I'd
expect some evidence of that in the ssh proxy logs as well.

You can try to manually ssh using the instructions in the diego-ssh repo
[1] and see if you can isolate if the problem is on the cli side or the
server side.

[1]:
https://github.com/cloudfoundry-incubator/diego-ssh#cloud-foundry-via-cloud-controller-and-uaa

On Wed, Oct 28, 2015 at 9:57 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:

I think I'm getting closer. In UAA I now get the error:

TokenEndpoint: Handling error: InvalidGrantException, Invalid
authorization code: ad1o9o

This must be someone trying to redeem the auth code.

Mike

On Wed, Oct 28, 2015 at 7:41 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

That curl command returns what appears to be the correct response:

curl -v -k -H "Authorization: $(cf oauth-token | tail -1)" 'https://uaa.
{redacted}/oauth/authorize?response_type=code&grant_type=authorization_code&client_id=ssh-proxy'
{trim}
GET
/oauth/authorize?response_type=code&grant_type=authorization_code&client_id=ssh-proxy
HTTP/1.1
User-Agent: curl/7.38.0
Host: uaa.cf1-dev.lds.org
Accept: */*
Authorization: bearer {redacted}
< HTTP/1.1 302 Found
< Cache-Control: no-cache
< Cache-Control: no-store
< Content-Language: en-US
< Content-Length: 0
< Date: Thu, 29 Oct 2015 01:32:08 GMT
< Expires: Thu, 01 Jan 1970 00:00:00 GMT
< Location: http://uaa.{redacted}/login?code=huQG3t
< Pragma: no-cache
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< X-Cf-Requestid: 36f6b88e-f8a9-49f1-5f90-ef2b868c266d
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-Xss-Protection: 1; mode=block
< Content-Type: text/plain; charset=utf-8

I never see a call like this in my CF_TRACE.

Mike

On Wed, Oct 28, 2015 at 7:09 PM, Matthew Sykes <matthew.sykes(a)gmail.com>
wrote:

That's not the request that the plugin is making to get the token.
We're using the API that was created for us [1].

If you use straight curl with something like this, what does the flow
really look like? Are there any errors in the uaa's logs?

$ curl -v -k -H "Authorization: $(cf oauth-token | tail -1)" '
https://uaa.bosh-lite.com/oauth/authorize?response_type=code&grant_type=authorization_code&client_id=ssh-proxy
'

The UAA should respond with a 302 and a Location header that includes a
code parameter. If not, can you use a jwt decoder against your bearer token
and verify that there's a `uaa.user` scope in the token?

[1]:
https://github.com/cloudfoundry/uaa/blob/master/docs/UAA-APIs.rst#api-authorization-requests-code-get-oauth-authorize-non-standard-oauth-authorize

On Wed, Oct 28, 2015 at 8:56 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

In case it helps this is the CF_TRACE of the UAA call that the ssh
plugin is expecting to be a redirect.

REQUEST: [2015-10-28T17:25:11-06:00]
POST /oauth/token HTTP/1.1
Host: uaa.{redacted}
Accept: application/json
Authorization: [PRIVATE DATA HIDDEN]
Content-Type: application/x-www-form-urlencoded
User-Agent: go-cli 6.12.3-5364935 / linux

grant_type=refresh_token&refresh_token={token redacted}&scope=

RESPONSE: [2015-10-28T17:25:12-06:00]
HTTP/1.1 200 OK
Connection: close
Transfer-Encoding: chunked
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Cache-Control: no-store
Content-Type: application/json;charset=UTF-8
Date: Wed, 28 Oct 2015 23:25:12 GMT
Expires: 0
Pragma: no-cache
Pragma: no-cache
Server: Apache-Coyote/1.1
X-Cf-Requestid: 4a6ad262-07e6-48a8-4640-271996e9bf64
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Xss-Protection: 1; mode=block

a20
{"access_token":"[PRIVATE DATA
HIDDEN]","token_type":"bearer","refresh_token":"[PRIVATE DATA
HIDDEN]","expires_in":3599,"scope":"scim.userids cloud_controller.read
password.write cloud_controller.write openid doppler.firehose scim.read
cloud_controller.admin","jti":"00e07ad7-5090-42e9-8096-a542dffd6026"}
0

This is the ssh-proxy client info returned from a call to uaac clients:

ssh-proxy
scope: cloud_controller.read cloud_controller.write openid
resource_ids: none
authorized_grant_types: authorization_code refresh_token
redirect_uri: /login
autoapprove: true
action: none
authorities: uaa.none
lastmodified: 1446074693000


Mike

On Wed, Oct 28, 2015 at 6:47 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

Yes /v2/info contains "app_ssh_oauth_client: "ssh-proxy"".

Though I didn't set it. It appears CC sets it by default now.


https://github.com/cloudfoundry/cf-release/blob/master/jobs/cloud_controller_ng/spec#L81

Any other ideas?

Mike
On Oct 28, 2015 6:16 PM, "Matthew Sykes" <matthew.sykes(a)gmail.com>
wrote:

Does /v2/info contain the `app_ssh_auth_client` key? If not, it
should be set to the client ID of the ssh proxy. If it's not set, I think
that's one of the symptom.


https://github.com/cloudfoundry-incubator/diego-release/blob/develop/stubs-for-cf-release/enable_diego_ssh_in_cf.yml#L4

On Wed, Oct 28, 2015 at 7:36 PM, Mike Youngstrom <youngm(a)gmail.com>
wrote:

I'm working on upgrading to latest cf-release+diego and I'm having
trouble getting ssh working.

When attempting to ssh with the latest cli I get the error:

"Authorization server did not redirect with one time code"

The relevant config is:

ssh_proxy.uaa_token_url=https://{uaa server}/oauth/token

uaa.clients.ssh-proxy:
authorized-grant-types: authorization_code
autoapprove: true
override: true
redirect-uri: /login
scope: openid,cloud_controller.read,cloud_controller.write
secret: secret

When tracing the CLI I see a call to "POST /oauth/token" and a
200. It appears that the CLI is expecting a redirect and not a 200.

Is "oauth/token" the correct uaa_token_url endpoint? Any idea why
UAA wouldn't be sending a redirect response from /oauth/token when the
plugin is expecting it?

Mike


--
Matthew Sykes
matthew.sykes(a)gmail.com

--
Matthew Sykes
matthew.sykes(a)gmail.com

--
Matthew Sykes
matthew.sykes(a)gmail.com

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.