Re: Passwords visible in infrastructure logs


Amit Kumar Gupta
 

Hey Momchil,

Do you know whether it's the DEA or Warden that's logging that sensitive
data when you say "runner"?

I would recommended opening issues against the relevant projects:

API: https://github.com/cloudfoundry/cloud_controller_ng/issues
DEA or Warden: https://github.com/cloudfoundry/dea_ng/issues or
https://github.com/cloudfoundry/warden/issues

As for NATS, you may be able to change the logging level? Alternatively,
NATS is not a Cloud Foundry project but you could ask over there about
encrypting log output: https://github.com/nats-io/gnatsd

In Pivotal's production environments, we run 100% on Diego, so we are not
concerned with DEA/Warden logging, and this move also removes NATS from the
flows like create-user-provided-service. CC is likely still an issue, so
it would be a good one to raise against their GitHub project.

Best,
Amit

On Fri, Dec 4, 2015 at 2:09 AM, Momchil Atanassov <momchil.atanassov(a)sap.com
wrote:
Hi,

We are using the `syslog_deamon_config` proprty to stream all of our CF
infrastructure logs to an external stack for later processing.

We have noticed that operations like `cf create-user-provided-service`,
`cf bind-service`, and others are logged by multiple components in CF. That
would normally not be a problem, except that these commands often involve
passwords and those passwords get logged as well, ending up in the log
files on the VM and the target log processing stack, which allows operators
of the system to view end-user passwords.

We have noticed that the following jobs are responsible for the logs:

* api
* runner
* nats

Increasing the log level from the default `debug` / `debug2` to `info`
solves the problem for the first two, at the cost of making troubleshooting
tasks more difficult on the system.
The last one can only be solved by removing the `nats_stream_forwarder`
component from the `nats` job, again making troubleshooting more difficult.

I believe the ideal solution is to have those components not log the
payload of commands holding confidential information. Maybe they could
replace it with some pattern.
This would help for the first two but might not help for nats, where some
other means would be needed (encryption of the private content?).

How are you solving this issue on your productive system? What are your
thoughts on this matter?

Thanks in advance!

Regards,
Momchil Atanassov

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.