Date
1 - 7 of 7
abacus collector doesn't work
|
MaggieMeng
Hi
I am trying to run abacus in my cloudfoundry env. However after successfully push all abacus applications into CF, I found following error from some of the applications: dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf logs abacus-usage-aggregator Connected, tailing logs for app abacus-usage-aggregator in org cf / space space as admin... 2015-11-04T04:33:36.47-0500 [App/0] OUT 2015-11-04T09:33:36.469Z e-abacus-request 46 Request error { message: 'connect ECONNREFUSED', 2015-11-04T04:33:36.47-0500 [App/0] OUT code: 'ECONNREFUSED', 2015-11-04T04:33:36.47-0500 [App/0] OUT errno: 'ECONNREFUSED', 2015-11-04T04:33:36.47-0500 [App/0] OUT syscall: 'connect' } - Error: connect ECONNREFUSED 2015-11-04T04:33:36.47-0500 [App/0] OUT at exports._errnoException (util.js:746:11) 2015-11-04T04:33:36.47-0500 [App/0] OUT at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1010:19) Same as abacus-usage-collector. "npm run demo" also failed which may due to this error. Could it be CF configuration issue? How could I enable verbose log or debug? Any help would be appreciated. Thanks, Maggie
|
|
|
|
Hi,
Do you run this in bosh-lite environment of on hosted CF? In case you have bosh-lite env you have to create a security group that enables the different micro-services to talk to each other. We already provide a "setup" script [1] and sec.group definition [2] Judging from the rights you have (admin) you must be using bosh-lite, but if this is not the case you may need to change some of the collector's environment variables [3]. This is especially useful in case your Abacus pipeline runs on a different sub-domain than the default CF domain. [1] https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/bin/cfsetup [2] https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/etc/secgroup.json [2] https://github.com/cloudfoundry-incubator/cf-abacus/blob/master/lib/metering/collector/manifest.yml#L9-L14
|
|
|
|
MaggieMeng
Hi, Hristo
I think I am using the bosh-lite. And I tried to change the environment variable as below. But I still got the same error. Which application does collector want to connect? dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf env abacus-usage-collector Getting env variables for app abacus-usage-collector in org test / space space as admin... OK System-Provided: ... User-Provided: CONF: default COUCHDB: abacus-dbserver.bjngiscf-dev.dctmlabs.com DEBUG: e-abacus-* EUREKA: abacus-eureka-stub.bjngiscf-dev.dctmlabs.com METER: abacus-usage-meter.bjngiscf-dev.dctmlabs.com PROVISIONING: abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com SECURED: false dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf apps Getting apps in org test / space space as admin... OK name requested state instances memory disk urls abacus-account-stub started 1/1 512M 512M abacus-account-stub.bjngiscf-dev.dctmlabs.com abacus-usage-reporting started 1/1 512M 512M abacus-usage-reporting.bjngiscf-dev.dctmlabs.com abacus-usage-meter started 1/1 512M 512M abacus-usage-meter.bjngiscf-dev.dctmlabs.com abacus-usage-accumulator started 1/1 512M 512M abacus-usage-accumulator.bjngiscf-dev.dctmlabs.com abacus-dbserver started 1/1 1G 512M abacus-dbserver.bjngiscf-dev.dctmlabs.com abacus-eureka-stub started 1/1 512M 512M abacus-eureka-stub.bjngiscf-dev.dctmlabs.com abacus-usage-rate started 1/1 512M 512M abacus-usage-rate.bjngiscf-dev.dctmlabs.com abacus-authserver-stub started 1/1 512M 512M abacus-authserver-stub.bjngiscf-dev.dctmlabs.com abacus-provisioning-stub started 1/1 512M 512M abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com abacus-usage-aggregator started 1/1 512M 512M abacus-usage-aggregator.bjngiscf-dev.dctmlabs.com abacus-usage-collector started 1/1 512M 512M abacus-usage-collector.bjngiscf-dev.dctmlabs.com dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf security-group abacus Getting info for security group abacus as admin OK Name abacus Rules [ { "destination": "0.0.0.0/0", "ports": "1-65535", "protocol": "tcp" }, { "destination": "0.0.0.0/0", "ports": "1-65535", "protocol": "udp" } ] Organization Space #0 test space Thanks, Maggie
|
|
|
|
Jean-Sebastien Delfino
Hi Maggie,
toggle quoted messageShow quoted text
Which level of abacus are you using? Are you using the v0.0.2 release or a specific commit from the Abacus Github repository? You can get a more verbose log with the following env variable: DEBUG=e-abacus-*,abacus-request,abacus-router You can set it like this: cf set-env abacus-usage-collector DEBUG "e-abacus-*,abacus-request,abacus-router" cf restage abacus-usage-collector BTW, with the latest version of the Abacus master branch that DEBUG variable is already set to "e-abacus-*" (log all errors) in our default CF manifest.yml files. Would you mind creating a Github issue including the log from the abacus-usage-collector app with the DEBUG variable set as above? We'll take a look. Thanks! - Jean-Sebastien
On Wed, Nov 4, 2015 at 7:48 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:
Hi, Hristo
|
|
|
|
MaggieMeng
Hi, Jean
The root cause is that abacus takes “https” as default protocol when connect between abacus components like abacus-eureka-stub and abacus-dbserver. But in my CF env, I didn’t have any proxy server. It may not be an issue. But I would like to have the protocol be configurable. May you consider it as an future improvement? Thanks a lot for your and Hristo’s help! Thanks, Maggie From: Jean-Sebastien Delfino [mailto:jsdelfino(a)gmail.com] Sent: 2015年11月6日 0:15 To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: abacus collector doesn't work Hi Maggie, Which level of abacus are you using? Are you using the v0.0.2 release or a specific commit from the Abacus Github repository? You can get a more verbose log with the following env variable: DEBUG=e-abacus-*,abacus-request,abacus-router You can set it like this: cf set-env abacus-usage-collector DEBUG "e-abacus-*,abacus-request,abacus-router" cf restage abacus-usage-collector BTW, with the latest version of the Abacus master branch that DEBUG variable is already set to "e-abacus-*" (log all errors) in our default CF manifest.yml files. Would you mind creating a Github issue including the log from the abacus-usage-collector app with the DEBUG variable set as above? We'll take a look. Thanks! - Jean-Sebastien On Wed, Nov 4, 2015 at 7:48 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote: Hi, Hristo I think I am using the bosh-lite. And I tried to change the environment variable as below. But I still got the same error. Which application does collector want to connect? dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf env abacus-usage-collector Getting env variables for app abacus-usage-collector in org test / space space as admin... OK System-Provided: ... User-Provided: CONF: default COUCHDB: abacus-dbserver.bjngiscf-dev.dctmlabs.com<http://abacus-dbserver.bjngiscf-dev.dctmlabs.com> DEBUG: e-abacus-* EUREKA: abacus-eureka-stub.bjngiscf-dev.dctmlabs.com<http://abacus-eureka-stub.bjngiscf-dev.dctmlabs.com> METER: abacus-usage-meter.bjngiscf-dev.dctmlabs.com<http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com> PROVISIONING: abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com<http://abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com> SECURED: false dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf apps Getting apps in org test / space space as admin... OK name requested state instances memory disk urls abacus-account-stub started 1/1 512M 512M abacus-account-stub.bjngiscf-dev.dctmlabs.com<http://abacus-account-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-reporting started 1/1 512M 512M abacus-usage-reporting.bjngiscf-dev.dctmlabs.com<http://abacus-usage-reporting.bjngiscf-dev.dctmlabs.com> abacus-usage-meter started 1/1 512M 512M abacus-usage-meter.bjngiscf-dev.dctmlabs.com<http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com> abacus-usage-accumulator started 1/1 512M 512M abacus-usage-accumulator.bjngiscf-dev.dctmlabs.com<http://abacus-usage-accumulator.bjngiscf-dev.dctmlabs.com> abacus-dbserver started 1/1 1G 512M abacus-dbserver.bjngiscf-dev.dctmlabs.com<http://abacus-dbserver.bjngiscf-dev.dctmlabs.com> abacus-eureka-stub started 1/1 512M 512M abacus-eureka-stub.bjngiscf-dev.dctmlabs.com<http://abacus-eureka-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-rate started 1/1 512M 512M abacus-usage-rate.bjngiscf-dev.dctmlabs.com<http://abacus-usage-rate.bjngiscf-dev.dctmlabs.com> abacus-authserver-stub started 1/1 512M 512M abacus-authserver-stub.bjngiscf-dev.dctmlabs.com<http://abacus-authserver-stub.bjngiscf-dev.dctmlabs.com> abacus-provisioning-stub started 1/1 512M 512M abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com<http://abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-aggregator started 1/1 512M 512M abacus-usage-aggregator.bjngiscf-dev.dctmlabs.com<http://abacus-usage-aggregator.bjngiscf-dev.dctmlabs.com> abacus-usage-collector started 1/1 512M 512M abacus-usage-collector.bjngiscf-dev.dctmlabs.com<http://abacus-usage-collector.bjngiscf-dev.dctmlabs.com> dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf security-group abacus Getting info for security group abacus as admin OK Name abacus Rules [ { "destination": "0.0.0.0/0<http://0.0.0.0/0>", "ports": "1-65535", "protocol": "tcp" }, { "destination": "0.0.0.0/0<http://0.0.0.0/0>", "ports": "1-65535", "protocol": "udp" } ] Organization Space #0 test space Thanks, Maggie
|
|
|
|
Jean-Sebastien Delfino
Hi Maggie,
toggle quoted messageShow quoted text
Good to hear that you've been able to make progress. You're correct that we're defaulting to https if you don't specify a protocol in the METER, COUCHDB etc env variables. I believe that we'll use the protocol you want if you set one. Can you try to include a protocol in your env variables like this: METER=http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com If that doesn't work, then we'll be happy to work with you to improve this. Thanks! - Jean-Sebastien
On Fri, Nov 6, 2015 at 2:11 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com> wrote:
Hi, Jean
|
|
|
|
MaggieMeng
Hi, Sebastien
Yes, the url with protocol specified works. Thanks a lot for your great help! Regards, Maggie From: Jean-Sebastien Delfino [mailto:jsdelfino(a)gmail.com] Sent: 2015年11月7日 8:12 To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: Re: Re: abacus collector doesn't work Hi Maggie, Good to hear that you've been able to make progress. You're correct that we're defaulting to https if you don't specify a protocol in the METER, COUCHDB etc env variables. I believe that we'll use the protocol you want if you set one. Can you try to include a protocol in your env variables like this: METER=http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com If that doesn't work, then we'll be happy to work with you to improve this. Thanks! - Jean-Sebastien On Fri, Nov 6, 2015 at 2:11 AM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote: Hi, Jean The root cause is that abacus takes “https” as default protocol when connect between abacus components like abacus-eureka-stub and abacus-dbserver. But in my CF env, I didn’t have any proxy server. It may not be an issue. But I would like to have the protocol be configurable. May you consider it as an future improvement? Thanks a lot for your and Hristo’s help! Thanks, Maggie From: Jean-Sebastien Delfino [mailto:jsdelfino(a)gmail.com<mailto:jsdelfino(a)gmail.com>] Sent: 2015年11月6日 0:15 To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Re: Re: abacus collector doesn't work Hi Maggie, Which level of abacus are you using? Are you using the v0.0.2 release or a specific commit from the Abacus Github repository? You can get a more verbose log with the following env variable: DEBUG=e-abacus-*,abacus-request,abacus-router You can set it like this: cf set-env abacus-usage-collector DEBUG "e-abacus-*,abacus-request,abacus-router" cf restage abacus-usage-collector BTW, with the latest version of the Abacus master branch that DEBUG variable is already set to "e-abacus-*" (log all errors) in our default CF manifest.yml files. Would you mind creating a Github issue including the log from the abacus-usage-collector app with the DEBUG variable set as above? We'll take a look. Thanks! - Jean-Sebastien On Wed, Nov 4, 2015 at 7:48 PM, Meng, Xiangyi <xiangyi.meng(a)emc.com<mailto:xiangyi.meng(a)emc.com>> wrote: Hi, Hristo I think I am using the bosh-lite. And I tried to change the environment variable as below. But I still got the same error. Which application does collector want to connect? dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf env abacus-usage-collector Getting env variables for app abacus-usage-collector in org test / space space as admin... OK System-Provided: ... User-Provided: CONF: default COUCHDB: abacus-dbserver.bjngiscf-dev.dctmlabs.com<http://abacus-dbserver.bjngiscf-dev.dctmlabs.com> DEBUG: e-abacus-* EUREKA: abacus-eureka-stub.bjngiscf-dev.dctmlabs.com<http://abacus-eureka-stub.bjngiscf-dev.dctmlabs.com> METER: abacus-usage-meter.bjngiscf-dev.dctmlabs.com<http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com> PROVISIONING: abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com<http://abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com> SECURED: false dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf apps Getting apps in org test / space space as admin... OK name requested state instances memory disk urls abacus-account-stub started 1/1 512M 512M abacus-account-stub.bjngiscf-dev.dctmlabs.com<http://abacus-account-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-reporting started 1/1 512M 512M abacus-usage-reporting.bjngiscf-dev.dctmlabs.com<http://abacus-usage-reporting.bjngiscf-dev.dctmlabs.com> abacus-usage-meter started 1/1 512M 512M abacus-usage-meter.bjngiscf-dev.dctmlabs.com<http://abacus-usage-meter.bjngiscf-dev.dctmlabs.com> abacus-usage-accumulator started 1/1 512M 512M abacus-usage-accumulator.bjngiscf-dev.dctmlabs.com<http://abacus-usage-accumulator.bjngiscf-dev.dctmlabs.com> abacus-dbserver started 1/1 1G 512M abacus-dbserver.bjngiscf-dev.dctmlabs.com<http://abacus-dbserver.bjngiscf-dev.dctmlabs.com> abacus-eureka-stub started 1/1 512M 512M abacus-eureka-stub.bjngiscf-dev.dctmlabs.com<http://abacus-eureka-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-rate started 1/1 512M 512M abacus-usage-rate.bjngiscf-dev.dctmlabs.com<http://abacus-usage-rate.bjngiscf-dev.dctmlabs.com> abacus-authserver-stub started 1/1 512M 512M abacus-authserver-stub.bjngiscf-dev.dctmlabs.com<http://abacus-authserver-stub.bjngiscf-dev.dctmlabs.com> abacus-provisioning-stub started 1/1 512M 512M abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com<http://abacus-provisioning-stub.bjngiscf-dev.dctmlabs.com> abacus-usage-aggregator started 1/1 512M 512M abacus-usage-aggregator.bjngiscf-dev.dctmlabs.com<http://abacus-usage-aggregator.bjngiscf-dev.dctmlabs.com> abacus-usage-collector started 1/1 512M 512M abacus-usage-collector.bjngiscf-dev.dctmlabs.com<http://abacus-usage-collector.bjngiscf-dev.dctmlabs.com> dmadmin(a)dmadmin-Lenovo-Product:~/cloudfoundry/cf-abacus/cf-abacus$ cf security-group abacus Getting info for security group abacus as admin OK Name abacus Rules [ { "destination": "0.0.0.0/0<http://0.0.0.0/0>", "ports": "1-65535", "protocol": "tcp" }, { "destination": "0.0.0.0/0<http://0.0.0.0/0>", "ports": "1-65535", "protocol": "udp" } ] Organization Space #0 test space Thanks, Maggie
|
|
|