Re: AWS deployment manifest with HAproxy
Thanks guys!
I am able to get to the deployment phase now, the VM's bind at least but I hit this error:
Started preparing configuration > Binding configuration. Done (00:00:15)
Started updating job ha_proxy_z1 > ha_proxy_z1/0 (canary). Failed: Action Failed get_task: Task 6d7541c8-0476-4177-436b-c1fc22c9c581 result: Applying: Reloading jobSupervisor: Failed to reload monit: before=1438680699 after=1438680699 (00:01:33)
Error 450001: Action Failed get_task: Task 6d7541c8-0476-4177-436b-c1fc22c9c581 result: Applying: Reloading jobSupervisor: Failed to reload monit: before=1438680699 after=1438680699
When I do a "bosh ssh ha_proxy_z1/0" and try to run "monit summary" I get presented with this error:
root(a)9b91249d-9700-4004-9706-f4d4f793206e:/var/vcap/monit# monit summary
monit: Error: Depend service 'consul_agent' is not defined in the control file
I've checked my manifest has a specification for consul, as the default it is there. But should I be defining something? My Consul parts look like this:
consul:
agent:
log_level: null
servers:
lan: []
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 11:56 AM, CF Runtime <cfruntime(a)gmail.com> wrote: Also Stephen, the templates by default attempt to use an ELB named cfrouter. So if you want to try to use an ELB again, just create one with that name and when you deploy BOSH should connect it to your routers.
In either case, you shouldn't need to worry about the DEAs, those are behind the routers. The only thing the load balancer talks to are the routers.
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 12:50 AM, Amit Gupta <agupta(a)pivotal.io> wrote:
Hi Stephen,
You generally never want to alter the manifest templates unless you're making a PR with general improvements to the templates. Customizing your own deployment should be done either within your stub(s), or modifying your spiff-generated manifest with your desired changes before deploying.
Try making the following additions to your stub:
jobs: - name: ha_proxy_z1 instances: 1 networks: - name: ha_proxy_z1_elastic static_ips: [YOUR_STATIC_IP_HERE] - name: cf1 default: [gateway, dns]
networks: - name: ha_proxy_z1_elastic # Add this network to the existing cf1 and cf2 networks, don't remove those type: vip
resource_pools: - name: router_z1 cloud_properties: elbs: [] - name: router_z2 cloud_properties: elbs: []
See if that works for you. You will need to make sure that you've configured DNS records so that your system domain and app domains point to YOUR_STATIC_IP for ha_proxy. If you're already using YOUR_STATIC_IP.xip.io as your system and apps domains, then this will work without having to create any DNS records.
Best, Amit
On Sun, Aug 2, 2015 at 11:11 PM, Stephen Knight <sknight(a)pivotal.io> wrote:
Hi All,
The default manifests for CF seem to use ELB's on AWS for load balancing, I am still getting to grips with manifests so in the meantime, can anyone tell me how you would insert an HAproxy with an elastic IP into a standard manifest?
And should it go in the stub or in the cf-infrastructure-xxx.yml file? As well as ensuring that the DEA pool goes behind the LB pair?
I'm using spiff as per the deployment instructions on Github, filling out the spec/fixtures/stub but I still can't get HAproxy to deploy despite trying to munge something together from other manifests.
Thx Stephen
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: Application Configuration Management
Ronak Agarwal <roagarwa@...>
Basically I am looking for - CF Integration with any application configuration servers(like Sping-cloud-config-server) to allow central management of application settings for Cloud Foundry apps.
toggle quoted messageShow quoted text
On Tue, Aug 4, 2015 at 2:18 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote: Hi,
I need some help around application configuration management in CF, using something like a Zookeeper or Doozered where I can manage all my pushed application configuration details (app-url, port etc).
So if I make changes to any app configuration through lets say zookeeper it should be taken care by app on cf push before getting staged OR on cf restart app.
Can you please point me to some blog/sample to test it?
Thanks, Ronak
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
|
|
Tangshengjun (A) <tangshengjun@...>
There is an sentence on my office wall, said: “You will never tire of looking at the Garden”, personally, I like Warden because it let me know very detail implement when construct a Container.(although this makes Warden a little primitive compare to Docker). Farewell, Garden-Linux, Thank you very much. (there is a word “pull down” in the article, I think it means Goodbye, sorry if misunderstand for my poor English☺) ________________________________ 唐盛军 华为技术有限公司 Huawei Technologies Co., Ltd. [Company_logo] Phone: 13777864354 Fax: Mobile: 13777864354 Email: tsjsdbd(a)huawei.com 地址:杭州市江虹路410号华为基地 邮编:310052 Huawei Technologies Co., Ltd. JiangHong road 410,BingJiang District,Hangzhou 310052, P.R.China http://www.huawei.com________________________________ 本邮件及其附件含有华为公司的保密信息,仅限于发送给上面地址中列出的个人或群组。禁 止任何其他人以任何形式使用(包括但不限于全部或部分地泄露、复制、或散发)本邮件中 的信息。如果您错收了本邮件,请您立即电话或邮件通知发件人并删除本邮件! This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it! 发件人: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] 代表 Gwenn Etourneau 发送时间: 2015年8月3日 9:14 收件人: Discussions about Cloud Foundry projects and the system overall. 主题: Re: [cf-dev] Garden is Moving! Awesome, yes it's a big win for everyome and according to Solomonstre tweet is just natural things "@chanezon< https://twitter.com/chanezon/> @wattersjames< https://twitter.com/wattersjames/> @charlesfitz< https://twitter.com/charlesfitz/> @diogomonica< https://twitter.com/diogomonica/> @justinjsmith< https://twitter.com/justinjsmith/> nice! We did runC partly based on cloudfoundry feedback."
toggle quoted messageShow quoted text
On Sat, Aug 1, 2015 at 12:47 PM, James Bayer <jbayer(a)pivotal.io<mailto:jbayer(a)pivotal.io>> wrote: thanks julz for summarizing all of this. i'm very excited that cloud foundry will be able to use runc and contribute to the open container initiative. by joining with the other members and working together, we'll be able to use the same base runtime as docker, coreos and others. we'll also preserve the flexibility to do the innovations and user experience we want for CF users above the core container runtime. this seems like a big win for everyone. On Fri, Jul 31, 2015 at 3:06 PM, Deepak Vij (A) <deepak.vij(a)huawei.com<mailto:deepak.vij(a)huawei.com>> wrote: Hi Julz & the whole garden team, it is great to know that Garden Container is moving towards Open-Container-Project (OCP) App-Container specifications. Great work. I am hoping that down the road we will also see App Container Pods (Co-locating Containers) capabilities enabled as well. A pod is a list of apps that will be launched together inside a shared execution context ( single Unit of Deployment, migration etc. sharing IP address Space, Storage etc.). Kubernetes also supports similar Pod concept. Pod architecture allows me to enable design patters such as Sidecar, Ambassador & Adaptor. All of this is really helpful from the standpoint of refactoring the core telecom capabilities such as vEPC (virtual Evolved packet Core network) and many more NFV/telecom capabilities - Network Function Virtualization. - Deepak Vij ---------------------------------------------------------------------- Message: 1 Date: Fri, 31 Jul 2015 18:49:25 +0100 From: Julz Friedman <julz.friedman(a)gmail.com<mailto:julz.friedman(a)gmail.com>> To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>> Subject: [cf-dev] Garden is Moving! Message-ID: <CAHfHzfOrrdEn_QBZwnoq7qQtXbBW1K2fk-NbbqgLSKaacMcPsw(a)mail.gmail.com<mailto:CAHfHzfOrrdEn_QBZwnoq7qQtXbBW1K2fk-NbbqgLSKaacMcPsw(a)mail.gmail.com>> Content-Type: text/plain; charset="utf-8" Hi cf-dev, I?d like to discuss some exciting changes the Garden team is planning to make in Diego?s container subsystem, Garden. Garden? What?s that? Garden is the containerisation layer used by Diego. Garden provides a platform-neutral, lightweight container abstraction that can be backed by multiple backends (most importantly, a Linux backend and a Windows backend). Currently the linux backend is based on our own code which evolved from Warden and which has been used to power Cloud Foundry for many years. Garden enables diego to support buildpack apps and docker apps (via the Linux backend) and windows apps (via the Windows backend). So: What's changing? We're planning to use runC [1] as the Linux backend for Garden. Why? Garden has always been an unopinionated container system - we like to have the opinionation happen at the higher levels (i.e. in Diego). Docker, on the other hand, is quite an opinionated container technology: it tightly couples containerisation and the user experience (which is one of the reasons docker is so great to use, I?m not knocking docker here!). Recently, docker and others (including IBM and Pivotal) have come together under the Open Container Initiative to spin out an unopinionated common containerisation runtime, ?runC?, which gives us a fantastic opportunity to be part of this community while letting us ensure we can retain the flexibility required by our broader use cases. RunC is a reference implementation of the Open Container spec, which means both Docker and Cloud Foundry will be running the same code, and both Docker and Cloud Foundry apps will be using Open Containers. Using runC as the garden backend has two major advantages. Firstly it lets us reuse some awesome code and be part of the Open Container community. Secondly it means CF applications will be using not only the same kernel primitives as docker apps (as they already are today), but also the exact same runtime container engine. This will minimise incompatibility for our docker lifecycle and result in a first class experience for users, as well as letting us reuse and contribute back to a great open-source code base. We have some remaining features in the Garden Linux backend that we?d like to see in RunC, but we?re excited to engage with the Open Container community to close these gaps. What about regular CF buildpack apps and the other nice features of Garden? Moving to runC gives us all the above advantages without compromising our ability to also deliver the buildpack-based platform-centric workflows that make CF great. We will retain the garden abstraction to make it easy for Diego to support both buildpack apps, windows apps and docker apps, and we will maintain a small layer above runC to manage the containers, pull down native warden and docker root filesystems, let us perform live upgrades and so on. Why not use the full docker-engine as the backend? Docker-engine has both more capabilities than we need at the layer Garden runs and different opinions than Cloud Foundry currently requires. This means it?s harder for us to maintain (because it?s larger and does more stuff), harder for us to contribute to (for similar reasons) and for some of our use cases (particularly with Diego?s more generic lifecycles) we?d have to actively work around things that would be quite easy to expose if we use runC directly (for example docker-engine intentionally doesn?t support signalling `docker exec`ed processes, which is required by Diego[2]). Most of the reasons you might want to use docker-engine (e.g. being able to ?docker push?) make much more sense to expose at the platform level in a multi-host environment (you want to push to the cluster, not a single host) or need to be integrated with multi-tenancy (which again should happen at the platform level - you need access control on storage and networks to integrate with the rest of a multi-tenant platform). For these reasons Cloud Foundry prefers to implement many features at the Diego layer whereas docker-engine implements some of these capabilities at the host layer. As the capabilities for running distributed applications in containers continue to evolve, CF prefers the flexibility to implement the opinions of our developers and community for areas like networking and storage even if those may differ from other orchestration solutions like docker-engine, and in turn Garden needs to retain that flexibility. We also note that many new features have come to runC first (e.g. criu snapshot restore and - importantly for us - user namespaces were first available in runC before being added to docker-engine; at the time of writing these are still not fully available in docker-engine). We?d like to be able to consume new features as they come out in runC, rather than waiting for them to make it in to docker-engine. We also hope to be contributing new features of our own and this is much easier for us to accomplish against the smaller surface area of runC, and within the open context of the Open Container Initiative. When will this happen? Our first goal is completing the work of improving Garden?s security profile around supporting docker apps in production, we're about two weeks out from this according to Tracker and plan to do this with the current code. As soon as we hit this milestone we plan to shift our focus to runC. We have an initial prototype working and will iterate quickly to bring this to production quality and switch over when we feel confident. I?m excited to hear the community?s views and input on this, so let us know what you think! Thanks! - Julz, on behalf the Garden Team [1]: https://github.com/opencontainers/runc[2]: https://github.com/docker/docker/pull/9167, https://github.com/docker/docker/pull/9402_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org> https://lists.cloudfoundry.org/mailman/listinfo/cf-dev-- Thank you, James Bayer _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org> https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Application Configuration Management
Ronak Agarwal <roagarwa@...>
Hi,
I need some help around application configuration management in CF, using something like a Zookeeper or Doozered where I can manage all my pushed application configuration details (app-url, port etc).
So if I make changes to any app configuration through lets say zookeeper it should be taken care by app on cf push before getting staged OR on cf restart app.
Can you please point me to some blog/sample to test it?
Thanks, Ronak
|
|
Re: Java Buildpack v3.1.1
Excellent stuff, thanks! From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Christopher Frost Sent: 03 August 2015 18:04 To: Discussions about Cloud Foundry projects and the system overall. Subject: Re: [cf-dev] Java Buildpack v3.1.1 Hi, They package 1.8.0_51 based on build 16. This is true for each of the OSs listed in the release notes. Chris. Christopher Frost - Pivotal UK Java Buildpack Team On Mon, Aug 3, 2015 at 5:37 PM, Jones, Daniel <Daniel.Jones3(a)fil.com<mailto:Daniel.Jones3(a)fil.com>> wrote: Hi Chris, Are you able to confirm which build of OpenJDK has been included for each OS? We’re waiting on a Kerberos bug fix that arrived in OpenJDK 1.8.0_u51b12< https://bugs.openjdk.java.net/browse/JDK-8078439>. Regards, Daniel Jones Daniel Jones I 723 6362 I +44 (0)1732 776362<tel:%2B44%20%280%291732%20776362> I Cell: +44 (0)79 8000 9153<tel:%2B44%20%280%2979%208000%209153> Fidelity Worldwide Investment I 130 Tonbridge Road, Hildenborough, Kent, TN11 9DZ The information transmitted is intended for the person or entity to which it is addressed and may contain confidential, privileged or copyrighted material. If you receive this in error, please contact the sender and delete the material from any computer. Fidelity only gives information on products and services and does not give investment advice to retail clients based on individual circumstances. Any comments or statements made are not necessarily those of Fidelity. All e-mails may be monitored. FIL Investments International (Reg. No.1448245), FIL Investment Services (UK) Limited (Reg. No. 2016555), FIL Pensions Management (Reg. No. 2015142) and Financial Administration Services Limited (Reg. No. 1629709) are authorised and regulated in the UK by the Financial Conduct Authority. FIL Life Insurance Limited (Reg. No. 3406905) is authorised in the UK by the Prudential Regulation Authority and regulated in the UK by the Financial Conduct Authority and the Prudential Regulation Authority. Registered offices at Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11 9DZ. From: cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org> [mailto:cf-dev-bounces(a)lists.cloudfoundry.org<mailto:cf-dev-bounces(a)lists.cloudfoundry.org>] On Behalf Of Christopher Frost Sent: 30 July 2015 17:34 To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Java Buildpack v3.1.1 I'm pleased to announce the release of the java-buildpack, version 3.1.1. This release ensures that the dependencies contained in the offline buildpack are up to date. For a more detailed look at the changes in 3.1.1, please take a look at the commit log< https://github.com/cloudfoundry/java-buildpack/compare/v3.1...v3.1.1>. Packaged versions of the buildpack, suitable for use with create-buildpack and update-buildpack, can be found attached to this release< https://github.com/cloudfoundry/java-buildpack/releases/tag/v3.1.1>. Packaged Dependencies * AppDynamics Agent: 4.0.7_0 * GemFire 8.0.0 * GemFire Modules 8.0.0.1 * GemFire Modules Tomcat7 8.0.0.1 * GemFire Security 8.0.0 * Groovy: 2.4.4 * JRebel 6.2.2 * MariaDB JDBC: 1.1.9 * Memory Calculator (centos6): 1.1.1.RELEASE * Memory Calculator (lucid): 1.1.1.RELEASE * Memory Calculator (mountainlion): 1.1.1.RELEASE * Memory Calculator (precise): 1.1.1.RELEASE * Memory Calculator (trusty): 1.1.1.RELEASE * New Relic Agent: 3.18.0 * OpenJDK JRE (centos6): 1.8.0_51 * OpenJDK JRE (lucid): 1.8.0_51 * OpenJDK JRE (mountainlion): 1.8.0_51 * OpenJDK JRE (precise): 1.8.0_51 * OpenJDK JRE (trusty): 1.8.0_51 * Play Framework JPA Plugin: 1.7.0.RELEASE * PostgreSQL JDBC: 9.4.1201 * RedisStore: 1.2.0_RELEASE * Spring Auto-reconfiguration: 1.7.0_RELEASE * Spring Boot CLI: 1.2.5_RELEASE * Tomcat Access Logging Support: 2.4.0_RELEASE * Tomcat Lifecycle Support: 2.4.0_RELEASE * Tomcat Logging Support: 2.4.0_RELEASE * Tomcat: 8.0.24 Christopher Frost - Pivotal UK Java Buildpack Team _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org> https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Service Discovery on CF for Custom buildpack
Ronak Agarwal <roagarwa@...>
Hi, We are trying to introduce a new component that will work together with our custom buildpack to expose and secure APIs (you can think of it as a Micro Gateway). In order to configure those gateways a new webbased UI (our new web component) will be introduced that should know about all our custom buildpack applications that are pushed to CF and expose APIs. Ideally speaking every time an app gets pushed to CF it should also notify the webbased UI that a new API is available (something like service discovery). I know CF has Etcd component which is used for Service Discovery but can we use it to push application cluster details to this custom webbased UI ? The other part of the question is that the webbased UI will create a new app that will in turn expose an API as well. That new app will be exposed to the outside world and route traffic to the pushed application (our buildpack apps) offcourse after applying some policies. Would that be possible to do with CF? Can we setup a proxy for such requirement? which component of CF has to be configured? Thanks, Ronak
|
|
Re: Soliciting feedback on a UX change for route services
Thank you for the interesting use case for route services, Guillaume! A mechanism to halt idle apps does seem valuable.
I've recorded your request for including app_id and will keep an ear out for other use cases that could leverage it, despite being out of date or incorrect.
In the meantime, couldn't your service put to sleep all apps that share a route, if no requests for the route are received in a given period?
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
toggle quoted messageShow quoted text
On Thu, Jul 30, 2015 at 3:23 AM, Guillaume Berche <bercheg(a)gmail.com> wrote: Thanks Shannon for your feedback.
I understand there is a small window into which the pre-determined app might not exist anymore (e.g. during blue/green deployment traffic shift). The default behavior you're suggesting (picking a different app instance) seems sensible to me, even though it will lead to seldom false associations.
We can imagine to refine this behavior in a second step, when use-cases of router service being sensitive to false associations become more frequent: allow the gorouter to comply to hints provided by the route service to tune the behavior in case the pre-determined app might not exist anymore. The router service could for instance augment the router-service HTTP header with hints fields:
- missing-app-policy with one of the following values: - reassign-app: the router transparently route the request to another app(default) - reject: reject the request (e.g. 502 status code with a json body providing the currently available app ids). In this case the route service may reemit the request to the gorouter, specifying the second param below - route-to-app override the predetermined app to which to route the traffic
I'm currently planning to implement a route-service that would leverage the app_id in the request in an "autosleep", see [1]. The "reassign-app" default policy seems fine as a first step. The reject policy would be a nice refinement to close this corner case.
[1] https://docs.google.com/document/d/1tMhIBX3tw7kPEOMCzKhUgmtmr26GVxyXwUTwMO71THI/edit#
Guillaume.
On Sat, Jul 18, 2015 at 12:08 AM, Shannon Coen <scoen(a)pivotal.io> wrote:
Guillaume,
Including the app_id with the request forwarded to the route service becomes misleading/false when, upon receiving the request back from the route service, the pre-determined app no longer has instances available. At that time GoRouter should be able to choose a different app instance for the route, possibly of a different app, rather than rejecting the request or re-forwarding the request to the route service with a different app id. Otherwise, the route service may be making false associations.
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
On Thu, Jun 25, 2015 at 9:19 PM, Guillaume Berche <bercheg(a)gmail.com> wrote:
I was about to suggest a similar UX for expressing a list of route services, by relying on params ordering
cf update-route DOMAIN [-n HOST] (-s 'service instance' )* cf update-route DOMAIN [-n HOST] -s caching -s https-only -s rate-limiting
Besides, If the MVP does not include support for multiple route services, route service implementers might be able to experiment with supporting arbitrary params as a way for users to specify fine grain options, possibly ordered
cf create-service large-grain-route-service -p '{ "caching": true, "ssl_only": true, "rate_limit": 3 }'
+1 for Mike's suggestions to allow for some route services to be implemented in an upfront LB such as no router. This might address the latency and availability concerns in the initial MVP ("route services to forward requests back through the LB and GoRouter")
Lastly, it seems important that route services be able to output logs that end up being associated with the app that receive the associated traffic (e.g. cache hit or cache miss for a specific incoming request). With route services being associated to routes (and not being bound to app instances anymore), I'd like to re-iterate the suggestion I made in the design document (on Feb 17) to have the gorouter include the app_id in the headers of the signed request it sends to route service(s). This will allow for a route service with log_emiter scope to add entries to the proper app through loggregator/doppler. Of course, this also means that when a route is associated to multiple apps, the load balancing decision among app is made prior to sending traffic to route service(s). I'd imagine the app_id could be propagated in the signed request headers when going through route services and finally reaching the gorouter before hitting the app (as to preserve the stateless nature of gorouter).
Guillaume.
On Fri, Jun 26, 2015 at 12:04 AM, Shannon Coen <scoen(a)pivotal.io> wrote:
This is great. Thank you, Mike.
FWIW, James had the following suggestion update-route could be used to associate multiple routes, and express their chain order. We're not fixed on this UX. We'll consider this more carefully when we get closer to the CF CLI work.
cf update-route DOMAIN [-n HOST] [-s 'list,of,service,instances']
Shannon Coen Product Manager, Cloud Foundry Pivotal, Inc.
On Thu, Jun 25, 2015 at 12:58 PM, Mike Youngstrom <youngm(a)gmail.com> wrote:
This is interesting. Could you flesh this out for me? What use cases do you have in mind for associating a service instance with a route, but not providing a forwarding address?
I would imagine you could bind a service to a route any time you want to customize incoming traffic in some way. But that customization wouldn't necessarily have to be implemented as a proxy.
Here are a few examples:
* A Public facing service as an indicator that a given route should be made public facing. (Would require a broker to orchestrate stuff outside of CF DNS, applying DoS security profiles to the route, force ssl on the front end load balancer, etc.) * A service to apply web front caching to a route. Could be done as a proxy but could also be done by changing config in a front end load balancer that supports caching like an F5 LTM. * Rate limiting. Could be implemented as a proxy, or could be implemented by applying some config in a front end load balancer * A security service to limit client IP addresses allowed to connect on a route. Again could be implemented as a proxy if you trust X-Forwarded-For or simply change some config on a front end load balancer no new proxy needed.
Basically a service applied to a route could trigger and manage all kinds of functionality not necessarily implemented as proxy orchestrated by the GoRouter.
It also occurs to me that the only time chain ordering of route services seems to be an issue is in the case of a proxy url. So, it is unfortunate that I'd be limited to binding only one route service when I may want to apply all kinds of functionality to a route not implemented as a proxy because user defined ordering isn't an issue.
That said I can see how it can be difficult for CF to provide a generic solution to the kind of functionality applied above and that you may not want to distract from the basic Route Services MVP to accommodate these types of use cases. I guess I can only hope that you keep the concept of applying non proxy functionality to a route in mind as you move through your MVP.
Mike
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: App autosleep support
Just FYI: Spindown logic has once been (experimentally) implemented but seemed never used: * https://github.com/cloudfoundry/cloud_controller_ng/commit/e7af12d832b07a56ad34ccc7bb27426d9e3e84bc* https://github.com/cloudfoundry/cloud_controller_ng/commit/e592281d1c3c5a7a359d1a87a8ffb1e74ff44b7f* https://github.com/cloudfoundry/cloud_controller_ng/commit/545cd8b5542a55e04b0c66f7cfc4ea3f55cf4572The model of application has a deprecated flag named "production": * http://apidocs.cloudfoundry.org/214/apps/creating_an_app.htmlIt seems to be intended to distinguish memory usage of "free" apps and "paid" apps: * https://github.com/cloudfoundry/cloud_controller_ng/commit/ad4911f7349d2923acb3c6cc95e00a9fd66577b5* https://github.com/cloudfoundry/cloud_controller_ng/commit/5c07b49a66803c5539eb4be9ed4cbf9530f6303bKlevenz, Stephan wrote Hi Guillaume,
We have a similar motivation for such feature request for saving resources. We tried that our with a quickly implement POC and made similar experience as you have written in the document. Here is in short what we did:
* measure inactivity of an app – we took the time stamp of last log entry for an application and compared it with a fresh-hold – for the POC this was good enough but maybe a better measure is needed * inactive apps – stop application and let all app routes point to a watch dog application * watch dog – if a route is accessed then the watch dog wakes up the app, moves routes back to the app and redirects the request after app is up again
We played also with CPU, number of requests and other measures to scale up and down applications.
Challenges we did identify were:
* find the right measure for inactive apps * start, stop or scale apps require permissions * enable the feature only for dedicated orgs or spaces (e.g. a trial org) * quick wakeup of app * a deeper integration into CF could make sense (get better measures, better permission control, configure thresh hold for apps via CF CLI …)
We did implement this POC on top to the CC API. Using such feature in production will cause more issues and findings.
Regards, Stephan
Von: Guillaume Berche Antworten an: "Discussions about Cloud Foundry projects and the system overall." Datum: Donnerstag, 30. Juli 2015 12:46 An: cf-dev Betreff: [cf-dev] App autosleep support
Hi,
I wonder if there are plans to implement an auto-sleep behavior in cloudfoundry, in which inactive apps would be automatically stopped after a max inactivity threshold, and automatically restart upon arrival of traffic on their routes. Similar to google app engine default behavior [1]
I did not find mentions of this yet in mailing lists and trackers.
We feel at Orange that such feature can improve the density for some of our non-prod use-cases (with environmental and financial benefits).
I'd like to know if someone in the community already worked on such feature or would be interested in collaborating on an opensource implementation.
I drafted some specs for a java-based implementation we're planning to work on [2]. I'd love to hear feedbacks and suggestions on this.
Thanks in advance,
Guillaume.
[1] https://cloud.google.com/appengine/docs/java/modules/ [2] https://docs.google.com/document/d/1tMhIBX3tw7kPEOMCzKhUgmtmr26GVxyXwUTwMO71THI/edit#
_______________________________________________ cf-dev mailing list cf-dev(a).cloudfoundry https://lists.cloudfoundry.org/mailman/listinfo/cf-dev -- View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-App-autosleep-support-tp962p1049.htmlSent from the CF Dev mailing list archive at Nabble.com.
|
|
How to provide personalized service catalogs ?

Guillaume Berche
Hi, The service broker api great to provide access to a common catalog (e.g. mysql, cassandra...). I understand private service brokers [3] will enable to extend the marketplace per space or per org (grant custom services per spaces or per orgs). I'm now looking at exposing enterprise services in the marketplace whose visibility would depend on the user making the request, and therefore looking at ways the services visibility in the catalog could dependent on user permissions (possibly managed outside CF) use-case 1: a user has been enabled a set of APIs in google api manager [1] for a given project, and would like to inject some credentials into his application. The list of google apis to display in the catalog for this "google api" service broker would vary from user to user, or more precisely from user+project basis. use-case 2: similar as use-case 1 but for a corporate api manager system, providing access to internal apis. A user has been granted access a set of restricted apis for the applications he is a member/client of, and would like to inject credentials for these apis in his application without having to create user-provided-services (which would require manual copy/paste and be error prone). What could be possible way to provide such personalized service catalog/offerings ? With private service brokers, I imagine the following scenario A) for use-case #1: The marketplace is exposing a meta service "meta-google-apis" accepting as arbitrary params: i- google authentication keys ii- project id (for which apis were enabled) iii- target org/space into which to expose the personalized catalog. iv- UAA Authorization code for granting the "meta-google-apis" rights to call CC api POST /v2/service_brokers on the user's behalf Upon instanciation of the "meta-google-apis" service instance, a private service broker called "google-apis" appears in the marketplace for the specified org/space. The broker being provided authentication key + project is able to restrict the list in the "catalog" endpoint to the apis that were enabled for this specific user/project. When creating a service instance for one of the services offering exposed by the broker (e.g. "google maps"), an api key for service is generated as returned as credentials using [2] equivalent REST API. A second scenario B) would be to provide this mechanism as a cli plugin instead of a "meta service". This has the advantage of allowing more fine-tuned users interactions, having access to the user's oauth tokens, but requires users to install the plugin and update it when improvements as brought. Is there other ways that can thinked of for providing personalized service catalogs, with a more natural UX ? Could it be considered to have the service broker catalog endpoint be provided some kind of identity of the user invoking the "cf marketplace" command, and possibly some credentials to act on the users's behalf ? The requirements for the personalized service brokers params would be specified at the time the broker is registered [5]. Users would be prompted by the CF CLI to provide autorization codes for the broker to act on their behalf, a bit similar to the "cf login --sso" [6] Thanks in advance for your inputs, Guillaume. [1] https://developers.google.com/console/help/new/#activating-and-deactivating-apis[2] https://developers.google.com/console/help/new/#generatingdevkeys[3] https://docs.google.com/document/d/1azArNcDtOjiq5wHx0BCS3OABfJf1PufPmc0OqfkFq7c/edit#heading=h.kz6110mpkon2[4] http://docs.cloudfoundry.org/services/api.html#catalog-mgmt[5] http://docs.cloudfoundry.org/services/managing-service-brokers.html#register-broker[6] https://www.pivotaltracker.com/story/show/69963432
|
|

Guillaume Berche
Thanks Onsi for these precisions. Similar to Colin's blog question, I'm wondering how much versionning is currently provided to rootfs and buildpacks. In other words, when a rootfs (say cflinuxfs2) gets patched (e.g. following CVE such as in [1]), what support is provided in the platform to identify apps that are still running an old version of cflinuxfs2 rootfs and require restaging ? Am I right to assume that there will be multiple distinct stack instances returned by cc api calls such as [3] (with distinct guid but same entity names) and that stacks are indeed immutable (i.e. the field "updated_at" will remain null in [4] ) ? Writing a cli plugin to identify apps that need restaging following a rootfs patch (similar to [2] but for a minor version of a given stack), would therefore browse all stacks using [5], and order those with the same name to understand minor patch version ? I recall similar discussion related to buildpack versionning, where it was mentionned that the buildpacks were mutable, and the only strategy to support multiple versions of a buildpack is to append version numbers to the buildpack names (and rely on priority ordering to have most recent version be picked first). This has the drawback that an app specifying a buildpack (e.g. to deambiguate or fix an invalid detect phase) will then need to manually update the buildpack reference to benefit from new version (a restaging won't be sufficient). Is this correct ? On CF instances that don't version buildpack names, how can users know whether apps were stage a vulrenable version of an offline buildpack, and require restaging ? Is it by comparing staging date, and known rootfs patch date ? Is there some improvements planned around this stack and buildpack versionning by the CAPI team (in tracker [6] seems related, is there something more specific planned) ? Thanks, Guillaume. [1] https://www.pivotaltracker.com/n/projects/966314/stories/90428236[2] https://github.com/simonleung8/cli-stack-changer[3] http://apidocs.cloudfoundry.org/214/apps/get_app_summary.html[4] http://apidocs.cloudfoundry.org/214/stacks/retrieve_a_particular_stack.html[5] http://apidocs.cloudfoundry.org/214/stacks/list_all_stacks.html[6] https://www.pivotaltracker.com/story/show/91553650
toggle quoted messageShow quoted text
On Wed, Jul 29, 2015 at 7:16 PM, Onsi Fakhouri <ofakhouri(a)pivotal.io> wrote: Hey Colin,
Good stuff. I like to draw a circle around the rootfs, the buildpacks, the generated droplet, the Task/LRP recipes, and the lifecycle binaries that run inside containers to stage and launch droplets. You could label that circle an application lifecycle. Diego currently supports three application lifecycles and is loosely coupled to those lifecycles:
1. The Linux-Buildpack App lifecycle: includes the cflinuxfs2 rootfs, the various buildpacks (including a known interface for building custom buildpacks), the droplets (compiled artifacts guaranteed to run with cflinuxfs2), two binaries: the builder (performs staging) and the launcher (runs applications), and code that can convert CC requests for staging and running instances to Diego Task/LRP recipes.
2. The Windows App lifecycle: includes the notion of a correctly configured windows environment, a windows-compatible droplet, a builder, a launcher, and code that can generate Tasks/LRPs. In this context we do not yet have/need the notion of a buildpack though we are free to add one later. The builder simply prepares the droplet from source and the launcher knows how to invoke it.
3. The Docker App lifecycle: has no rootfs as the docker image provides the entire rootfs, includes a builder to extract docker-metadata and send it back to CC for safe-keeping, and a launcher to launch the requested process *and* present it with a standard CF environment. Again, there's also code that knows how to translate CC requests for a docker-based application into Tasks and LRPs.
The cool thing is that Diego doesn't care about any of these details and you are free to construct your own lifecycles and have your own contracts within each lifecycle. You are spot on in noting that there is an implicit contract between the buildpacks and the rootfs. I'd go further and say that that implicit contract covers everything in the lifecycle circle (e.g. the builder has a contract with the buildpacks, it expects `detect`, `compile` and `release` to work a certain way, the recipes have a contract with the builder/launcher, they expect particular command line arguments, etc...)
This is one reason why we've recently transitioned the ownership of the rootfs from the runtime team to the buildpack team - as the buildpack team is best suited to define and maintain the contract between the buildpacks and the rootfs. Would love to explore ways to make all these contracts more explicit.
One last point. I didn't use the word "stack" in this e-mail until just now. I agree that it's an overloaded concept that is easily and often misunderstood ;)
Onsi
On Wed, Jul 29, 2015 at 9:51 AM, Colin Humphreys <colin(a)cloudcredo.com> wrote:
Hi All,
I wrote a couple of articles about droplets and stacks.
http://www.cloudcredo.com/a-droplet-of-value/
http://www.cloudcredo.com/stacks-of-problems/
The droplet post is fairly self-explanatory, and enabling the choice of shipping droplets or source is well on the way in Cloud Foundry development.
I feel our story around stacks is far less complete. It seems to be an overloaded concept inherited from Heroku and the contract with the stack seems to cause issues for both app and buildpack developers.
I'd like to open the discussion on what the future should be for stacks, or if you think they're perfect as they are.
Cheers, Colin
CloudCredo Cheerleader
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: Php build pack offline version availability
Hey James, Thanks for replying. Yes for the option 1, I did the same thing for the building. I could see the below on the screen: AMISHISH-M-91EG:php-buildpack ashah$ BUNDLE_GEMFILE=cf.Gemfile bundle exec buildpack-packager cached % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 36.8M 100 36.8M 0 0 3451k 0 0:00:10 0:00:10 --:--:— 3874k For the 2nd Option, I uploaded the buildpack just by downloading the buildpack zip file from https://github.com/cloudfoundry/php-buildpack/releases. Thanks, Amishi Shah From: James Bayer <jbayer(a)pivotal.io<mailto:jbayer(a)pivotal.io>> Reply-To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>> Date: Saturday, August 1, 2015 at 7:25 AM To: "Discussions about Cloud Foundry projects and the system overall." <cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org>> Subject: Re: [cf-dev] Php build pack offline version availability did you try building it with the "cached" option as referenced in the readme [1]: BUNDLE_GEMFILE=cf.Gemfile bundle exec buildpack-packager cached [1] https://github.com/cloudfoundry/php-buildpack#building
toggle quoted messageShow quoted text
On Fri, Jul 31, 2015 at 5:25 PM, Amishi Shah (amishish) <amishish(a)cisco.com<mailto:amishish(a)cisco.com>> wrote: Hi team, We are a team at Cisco trying to use the Buildpack for Php-Httpd. We tried couple of ways, 1. Cloning a git repo and creating the build pack, uploading the same and using the same for the app. 2. Downloading the readymade build pack available in the github releases pages and uploading, using the same for the app. What we are looking for is an offline version, both of these build packs connects to internet while we are deploying the app. The details are as below. AMISHISH-M-91EG:webapp ashah$ cf push Using manifest file /Users/ashah/workspace/Symphony/sample_httpd_app/webapp/manifest.yml Using stack cflinuxfs2... OK Creating app sample_app in org admin / space skyfall as admin... OK Using route sample-app.203.35.248.123.xip.io< http://sample-app.203.35.248.123.xip.io> Binding sample-app.203.35.248.123.xip.io< http://sample-app.203.35.248.123.xip.io> to sample_app... OK Uploading sample_app... Uploading app files from: /Users/ashah/workspace/Symphony/sample_httpd_app/webapp Uploading 185, 1 files Done uploading OK Starting app sample_app in org admin / space skyfall as admin... -----> Downloaded app package (4.0K) -------> Buildpack version 4.0.0 Installing HTTPD Downloaded [ https://pivotal-buildpacks.s3.amazonaws.com/php/binaries/trusty/httpd/2.4.12/httpd-2.4.12.tar.gz] to [/tmp] Installing PHP PHP 5.5.27 Downloaded [ https://pivotal-buildpacks.s3.amazonaws.com/php/binaries/trusty/php/5.5.27/php-5.5.27.tar.gz] to [/tmp] Finished: [2015-07-31 21:28:46.871154] -----> Uploading droplet (51M) 0 of 1 instances running, 1 down 0 of 1 instances running, 1 down 0 of 1 instances running, 1 down 0 of 1 instances running, 1 down 1 of 1 instances running App started OK App sample_app was started using this command `$HOME/.bp/bin/start` Showing health and status for app sample_app in org admin / space skyfall as admin... OK requested state: started instances: 1/1 usage: 64M x 1 instances urls:sample-app.203.35.248.123.xip.io< http://sample-app.203.35.248.123.xip.io> last uploaded: Fri Jul 31 21:28:35 UTC 2015 stack: cflinuxfs2 buildpack: readymade_httpd_buildpack state since cpu memory disk details #0 running 2015-07-31 02:29:39 PM 0.0% 40.3M of 64M 136.6M of 1G Do we have any offline Php-httpd buildpack available? We want the build pack which would be sufficient enough and it works even without the internet connectivity. Your timely response will be really appreciated. Thanks, Amishi Shah _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org<mailto:cf-dev(a)lists.cloudfoundry.org> https://lists.cloudfoundry.org/mailman/listinfo/cf-dev-- Thank you, James Bayer
|
|
Re: Java Buildpack v3.1.1
Hi,
They package 1.8.0_51 based on build 16. This is true for each of the OSs listed in the release notes.
Chris.
Christopher Frost - Pivotal UK Java Buildpack Team
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 5:37 PM, Jones, Daniel <Daniel.Jones3(a)fil.com> wrote: Hi Chris,
Are you able to confirm which build of OpenJDK has been included for each OS? We’re waiting on a Kerberos bug fix that arrived in OpenJDK 1.8.0_u51b12 <https://bugs.openjdk.java.net/browse/JDK-8078439>.
Regards,
Daniel Jones
*Daniel Jones* I 723 6362 I +44 (0)1732 776362 I *Cell*: +44 (0)79 8000 9153
*Fidelity Worldwide Investment* I 130 Tonbridge Road, Hildenborough, Kent, TN11 9DZ
The information transmitted is intended for the person or entity to which it is addressed and may contain confidential, privileged or copyrighted material. If you receive this in error, please contact the sender and delete the material from any computer. Fidelity only gives information on products and services and does not give investment advice to retail clients based on individual circumstances. Any comments or statements made are not necessarily those of Fidelity. All e-mails may be monitored. FIL Investments International (Reg. No.1448245), FIL Investment Services (UK) Limited (Reg. No. 2016555), FIL Pensions Management (Reg. No. 2015142) and Financial Administration Services Limited (Reg. No. 1629709) are authorised and regulated in the UK by the Financial Conduct Authority. FIL Life Insurance Limited (Reg. No. 3406905) is authorised in the UK by the Prudential Regulation Authority and regulated in the UK by the Financial Conduct Authority and the Prudential Regulation Authority. Registered offices at Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11 9DZ.
*From:* cf-dev-bounces(a)lists.cloudfoundry.org [mailto: cf-dev-bounces(a)lists.cloudfoundry.org] *On Behalf Of *Christopher Frost *Sent:* 30 July 2015 17:34 *To:* Discussions about Cloud Foundry projects and the system overall. *Subject:* [cf-dev] Java Buildpack v3.1.1
I'm pleased to announce the release of the java-buildpack, version 3.1.1. This release ensures that the dependencies contained in the offline buildpack are up to date.
For a more detailed look at the changes in 3.1.1, please take a look at the commit log <https://github.com/cloudfoundry/java-buildpack/compare/v3.1...v3.1.1>. Packaged versions of the buildpack, suitable for use with create-buildpack and update-buildpack, can be found attached to this release <https://github.com/cloudfoundry/java-buildpack/releases/tag/v3.1.1>.
*Packaged Dependencies*
- AppDynamics Agent: 4.0.7_0 - GemFire 8.0.0 - GemFire Modules 8.0.0.1 - GemFire Modules Tomcat7 8.0.0.1 - GemFire Security 8.0.0 - Groovy: 2.4.4 - JRebel 6.2.2 - MariaDB JDBC: 1.1.9 - Memory Calculator (centos6): 1.1.1.RELEASE - Memory Calculator (lucid): 1.1.1.RELEASE - Memory Calculator (mountainlion): 1.1.1.RELEASE - Memory Calculator (precise): 1.1.1.RELEASE - Memory Calculator (trusty): 1.1.1.RELEASE - New Relic Agent: 3.18.0 - OpenJDK JRE (centos6): 1.8.0_51 - OpenJDK JRE (lucid): 1.8.0_51 - OpenJDK JRE (mountainlion): 1.8.0_51 - OpenJDK JRE (precise): 1.8.0_51 - OpenJDK JRE (trusty): 1.8.0_51 - Play Framework JPA Plugin: 1.7.0.RELEASE - PostgreSQL JDBC: 9.4.1201 - RedisStore: 1.2.0_RELEASE - Spring Auto-reconfiguration: 1.7.0_RELEASE - Spring Boot CLI: 1.2.5_RELEASE - Tomcat Access Logging Support: 2.4.0_RELEASE - Tomcat Lifecycle Support: 2.4.0_RELEASE - Tomcat Logging Support: 2.4.0_RELEASE - Tomcat: 8.0.24
Christopher Frost - Pivotal UK
Java Buildpack Team
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: Java Buildpack v3.1.1
Hi Chris, Are you able to confirm which build of OpenJDK has been included for each OS? We’re waiting on a Kerberos bug fix that arrived in OpenJDK 1.8.0_u51b12< https://bugs.openjdk.java.net/browse/JDK-8078439>. Regards, Daniel Jones Daniel Jones I 723 6362 I +44 (0)1732 776362 I Cell: +44 (0)79 8000 9153 Fidelity Worldwide Investment I 130 Tonbridge Road, Hildenborough, Kent, TN11 9DZ The information transmitted is intended for the person or entity to which it is addressed and may contain confidential, privileged or copyrighted material. If you receive this in error, please contact the sender and delete the material from any computer. Fidelity only gives information on products and services and does not give investment advice to retail clients based on individual circumstances. Any comments or statements made are not necessarily those of Fidelity. All e-mails may be monitored. FIL Investments International (Reg. No.1448245), FIL Investment Services (UK) Limited (Reg. No. 2016555), FIL Pensions Management (Reg. No. 2015142) and Financial Administration Services Limited (Reg. No. 1629709) are authorised and regulated in the UK by the Financial Conduct Authority. FIL Life Insurance Limited (Reg. No. 3406905) is authorised in the UK by the Prudential Regulation Authority and regulated in the UK by the Financial Conduct Authority and the Prudential Regulation Authority. Registered offices at Oakhill House, 130 Tonbridge Road, Hildenborough, Tonbridge, Kent TN11 9DZ. From: cf-dev-bounces(a)lists.cloudfoundry.org [mailto:cf-dev-bounces(a)lists.cloudfoundry.org] On Behalf Of Christopher Frost Sent: 30 July 2015 17:34 To: Discussions about Cloud Foundry projects and the system overall. Subject: [cf-dev] Java Buildpack v3.1.1 I'm pleased to announce the release of the java-buildpack, version 3.1.1. This release ensures that the dependencies contained in the offline buildpack are up to date. For a more detailed look at the changes in 3.1.1, please take a look at the commit log< https://github.com/cloudfoundry/java-buildpack/compare/v3.1...v3.1.1>. Packaged versions of the buildpack, suitable for use with create-buildpack and update-buildpack, can be found attached to this release< https://github.com/cloudfoundry/java-buildpack/releases/tag/v3.1.1>. Packaged Dependencies * AppDynamics Agent: 4.0.7_0 * GemFire 8.0.0 * GemFire Modules 8.0.0.1 * GemFire Modules Tomcat7 8.0.0.1 * GemFire Security 8.0.0 * Groovy: 2.4.4 * JRebel 6.2.2 * MariaDB JDBC: 1.1.9 * Memory Calculator (centos6): 1.1.1.RELEASE * Memory Calculator (lucid): 1.1.1.RELEASE * Memory Calculator (mountainlion): 1.1.1.RELEASE * Memory Calculator (precise): 1.1.1.RELEASE * Memory Calculator (trusty): 1.1.1.RELEASE * New Relic Agent: 3.18.0 * OpenJDK JRE (centos6): 1.8.0_51 * OpenJDK JRE (lucid): 1.8.0_51 * OpenJDK JRE (mountainlion): 1.8.0_51 * OpenJDK JRE (precise): 1.8.0_51 * OpenJDK JRE (trusty): 1.8.0_51 * Play Framework JPA Plugin: 1.7.0.RELEASE * PostgreSQL JDBC: 9.4.1201 * RedisStore: 1.2.0_RELEASE * Spring Auto-reconfiguration: 1.7.0_RELEASE * Spring Boot CLI: 1.2.5_RELEASE * Tomcat Access Logging Support: 2.4.0_RELEASE * Tomcat Lifecycle Support: 2.4.0_RELEASE * Tomcat Logging Support: 2.4.0_RELEASE * Tomcat: 8.0.24 Christopher Frost - Pivotal UK Java Buildpack Team
|
|
Re: CF push error - Not able to resolve - Error dialing loggregator server: Get https://loggregator.systemdomain.com
The buildpacks didn't successfully upload, so you'll probably just need to "bosh restart api_z1/0" which should cause the startup script to upload them.
Joseph CF Release Integration Team
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 4:09 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote: I just disabled - source/destination checks
And able to curl 'google.com' now from private IP.
Should I need to again do - bosh deploy ?
On Mon, Aug 3, 2015 at 4:28 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
If I restart my NAT vm and do curl google.com - I am getting response.
When I try ssh to any of the private VM (api_z1 IP) from this NAT vm -> ssh -i "bosh.pem" vcap(a)10.10.17.0
" It is required that your private key files are NOT accessible by others.
This private key will be ignored. bad permissions: ignore key: bosh.pem Permission denied (publickey). "
So I did - chmod 400 bosh.pem
Then tried again above ssh command and able to ssh that 10.10.17.0
But curl google.com not giving anything.
On Mon, Aug 3, 2015 at 3:50 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Thanks Joseph.
I am not clear at one point - I already have a NAT vm which I had just stopped and again started, so are you asking me to create a new NAT vm because I have stopped it earlier? because it maintains some persistent data.
I just tried pushing a Java app because at api_z1, I can't find Java buildpack error, but I am getting -
RESPONSE: [2015-08-03T10:13:42Z] HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive
FAILED Error uploading application. Server error, status code: 504, error code: 0, message: FAILED Error uploading application. Server error, status code: 504, error code: 0, message:
I checked api_z1 logs and couldn't find any trace for this error, is it moved to Cloud_Controller_Worker ? What could be the error? I have good disk space available in api_z1.
cf logs appname --recent , gives below - RESPONSE: [2015-08-03T10:19:07Z] HTTP/1.1 200 OK Content-Length: 6200 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Mon, 03 Aug 2015 10:19:05 GMT Server: nginx X-Cf-Requestid: 2a67e173-20bf-4f89-7b27-efd5b28461c6 X-Content-Type-Options: nosniff X-Vcap-Request-Id:
f7b15702-3650-4258-5b40-48efbd9ef271::230c52e3-52a7-4a81-82f4-4e1ed445082d
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "2ac4b290-fb0d-4273-940e-b2d1aeb31320", "url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320", "created_at": "2015-08-03T09:26:19Z", "updated_at": "2015-08-03T10:12:40Z" }, "entity": { "name": "hello-spring-cloud", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": { "SPRING_PROFILES_DEFAULT": "cloud" }, "memory": 512, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "3719a87d-98b8-4641-a613-c9730f132b5d", "command": null, "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url":
"/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps",
"routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/events",
"service_bindings_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/routes",
"routes": [ { "metadata": { "guid": "f44c4856-08c1-49f2-8e1b-a51e548adca6", "url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6", "created_at": "2015-08-03T09:27:16Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unabsorbent-masculinization", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0",
"space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89",
"apps_url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6/apps"
} }, { "metadata": { "guid": "47a495d6-a484-48f0-ac12-6157282be1c4", "url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4", "created_at": "2015-08-03T09:58:03Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unascertainable-unmelodiousness",
"path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0",
"space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89",
"apps_url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4/apps"
} }, { "metadata": { "guid": "f51fad6c-fd48-4dfb-b201-a756af14bd00", "url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00", "created_at": "2015-08-03T10:12:40Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-completive-hesperinos", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0",
"space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89",
"apps_url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00/apps"
} } ] } } ] } Connected, dumping recent logs for app hello-spring-cloud in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320 :
EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320 :
EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
On Mon, Aug 3, 2015 at 3:33 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
Yeah, without the NAT vm nothing will be able to get out to the internet.
They will be able to respond to external requests, such as a request
to the
api, or the login server, or an individual app. But if anything on the inside tries to initiate a network connection, it has no route to the internet.
Amazon has good generic documentation on how to create a NAT instance:
http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#NATInstance
BOSH also has some documentation, although it doesn't tell you how to
create
the instance itself, but rather how to configure your private subnet
and
routing table to use the NAT instance: http://bosh.io/docs/setup-aws.html#nat
In both sets of documentation, you may already have security groups,
routes
and subnets created from your previous NAT instance.
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:50 AM, Ronak Agarwal <roagarwa(a)tibco.com>
wrote:
I tried 'curl http://google.com' from my api_z1 instance but was unable to get any response.
I create my NAT vm through 'bosh aws create' command, but by mistake I stopped that VM on AWS. So does stopping NAT VM is causing this issue ? It has any persistent data while setting NAT VM ?
If its because of stopping NAT Vm, so now what is the solution, do I need to destroy everything and create entire environment again ? Please suggest.
On Mon, Aug 3, 2015 at 2:53 PM, CF Runtime <cfruntime(a)gmail.com>
wrote:
It looks like the api instance is unable to upload the buildpacks
to the
blobstore. Perhaps the NAT (network NAT, not message bus NATS)
instance
is not configured correctly.
Are you able to curl an external URL from the api instance?
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:12 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Its a fresh install.
I found few logs may be helpful to resolve this issue --
I found few logs on cloud_controller_ng as below -
Nodejs buildpack install fail :- {"timestamp":1438583731.044699,"message":"Buildpack
nodejs_buildpack
failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue
in perform"} {"timestamp":1438583731.045526,"message":"Request
failed:
500: {\"code\"=>10001, \"description\"=>\"connect timeout
reached\",
\"error_code\"=>\"CF-Timeout\", ------------------------------------------ -> Fails at nodejs_buildpack-cached-v1.4.0.zip
"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583731.0496318,"message":"(0.001478s) UPDATE delayed_jobs SET guid = '82d9ebe1-7af0-4671-89a5-30146a5bb554', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler:
!ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: nodejs_buildpack\n file:
\\"/var/vcap/packages/buildpack_nodejs/nodejs_buildpack-cached-v1.4.0.zip\\"\n
opts: {}\n request_id: \n', last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:31:30',failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An
unknown
error occurred.\ncode: 10001\n' WHERE (id = 971) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}
________________________________
go_buildpack failed to install
{"timestamp":1438583970.3534014,"message":"Buildpack go_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue
in perform"} {"timestamp":1438583970.3541129,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in
rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock
in connect'\"
________________________________
->go_buildpack-cached-v1.4.0.zip ,
"log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583970.358118,"message":"(0.001543s) UPDATE delayed_jobs SET guid = 'd559d276-c5a2-471c-aca1-ba25f4e9cd97', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler:
!ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: go_buildpack\n file:
\\"/var/vcap/packages/buildpack_go/go_buildpack-cached-v1.4.0.zip\\"\n
opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:35:29', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An
unknown
error occurred.\ncode: 10001\n' WHERE (id = 972) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}
________________________________
python_buildpack failed to install
{"timestamp":1438583971.811825,"message":"Buildpack
python_buildpack
failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue
in perform"} {"timestamp":1438583971.8125288,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in
rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock
in connect'\",
________________________________
->python_buildpack-cached-v1.4.0.zip
,"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583971.8163912,"message":"(0.001414s) UPDATE delayed_jobs SET guid = '37f04f8f-2e1e-4096-8f5f-c10d8180a9fc', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0,handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler:
!ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: python_buildpack\n file:
\\"/var/vcap/packages/buildpack_python/python_buildpack-cached-v1.4.0.zip\\"\n
opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26',locked_at = '2015-08-03 06:35:31', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An
unknown
error occurred.\ncode: 10001\n' WHERE (id = 973) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":7030
php_buildpack failed to install
{"timestamp":1438584211.920127,"message":"Buildpack php_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue
in perform"}
________________________________
->php_buildpack-cached-v3.3.0.zip
{"timestamp":1438584211.9250205,"message":"(0.001442s) UPDATE delayed_jobs SET guid = '133a41bd-307f-4e2b-b70e-635f3dec60c4', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler:
!ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: php_buildpack\n file:
\\"/var/vcap/packages/buildpack_php/php_buildpack-cached-v3.3.0.zip\\"\n
opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:30', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An
unknown
error occurred.\ncode: 10001\n' WHERE (id = 974) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,
________________________________
binary_buildpack failed to install
{"timestamp":1438584212.2200441,"message":"Buildpack
binary_buildpack
failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue
in perform"}
________________________________
->binary_buildpack-cached-v1.0.1.zip
{"timestamp":1438584212.2245483,"message":"(0.001381s) UPDATE delayed_jobs SET guid = '7ea6156e-024c-4f2d-b574-ca9b218470cd', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler:
!ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n
name: binary_buildpack\n file:
\\"/var/vcap/packages/buildpack_binary/binary_buildpack-cached-v1.0.1.zip\\"\n
opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:31', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An
unknown
error occurred.\ncode: 10001\n' WHERE (id = 975) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":436
________________________________
This seems to be timeout issue.
{"timestamp":1438585142.846115,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in
`rescue in block in connect'
{"timestamp":1438585142.851403,"message":"(0.001378s) UPDATE delayed_jobs SET guid = '6e4167d9-c7ee-4425-9049-e50971864a64', created_at = '2015-08-03 06:55:01', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler:
!ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::AppBitsPacker\n app_guid: 41657738-cc7c-4237-ab52-e4f9683e16f4\n uploaded_compressed_path: \\"/var/vcap/data/cloud_controller_ng/tmp/uploads/0000000001\\"\n fingerprints: []\n request_id:
64cc342c-9db9-495e-46d9-ce5e9331aa41::d6c111c3-79b1-4fec-8fa9-0d4be759bdd9\n',
last_error = NULL, run_at = '2015-08-03 06:55:01', locked_at = '2015-08-03 06:55:02', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 989) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block
in log_each"}
________________________________
I will try to push Java app to verify if its getting pushed.
On Mon, Aug 3, 2015 at 2:19 PM, Gwenn Etourneau <
getourneau(a)pivotal.io>
wrote:
Btw CF-213 is a fresh install or update ??
On Mon, Aug 3, 2015 at 5:37 PM, Ronak Agarwal <
roagarwa(a)tibco.com>
wrote:
Hi,
Please go through all my instance info below and let me know
what
can be done to resolve this issue??
I have deployed CF-213.yml release through microbosh on AWS. I have installed latest CF CLI v6.12.2 I am able to do cf login using below endpoints - /.cf/config.json "Target": "https://api.cf.ronak.com", "ApiVersion": "2.33.0", "AuthorizationEndpoint": "https://login.cf.ronak.com", "LoggregatorEndPoint": "wss://loggregator.cf.ronak.com:4443", "DopplerEndPoint": "wss://doppler.cf.ronak.com:4443", "UaaEndpoint": "https://uaa.cf.ronak.com",
Now when I am trying to push Nodejs app using - cf push nodeapp -c "node main.js" -m 128M -b node_buildpack OR cf push nodeapp -c "node main.js"
both are failing .. gives unknown error. When I do -> cf logs nodeapp --recent It gives below error - FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=05ea8bda-28c1-49ed-8c65-4625c77423be :
EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
--------------------------------------------------------------------------------
I am using ELB and my HA-proxy job is not running. Please let me know how to overcome this challenge. I found lot of people have
faced it
but all of them I think are using HA-proxy and not ELB.
I have configured 4443 with SSL on ELB -
80 (HTTP) forwarding to 80 (HTTP), 443 (HTTPS, Certificate: cfrouter_cert) forwarding to 80 (HTTP), 4443 (SSL, Certificate: cfrouter_cert) forwarding to 80 (TCP)
----------------------------------------------------------------------------------
Below are the vms/jobs which are running on AWS I dont see login_z1/z2 in them, so do I need these jobs?
Job/index | State | Resource Pool | IPs |
+------------------------------------+---------+---------------+--------------+
| api_worker_z1/0 | running | small_z1 | 10.10.17.3 | | api_worker_z2/0 | running | small_z2 | 10.10.81.0 | | api_z1/0 | running | large_z1 | 10.10.17.1 | | api_z2/0 | running | large_z2 | 10.10.80.255 | | clock_global/0 | running | medium_z1 | 10.10.17.2 | | doppler_z1/0 | running | medium_z1 | 10.10.17.6 | | doppler_z2/0 | running | medium_z2 | 10.10.81.3 | | etcd_z1/0 | running | medium_z1 | 10.10.16.20 | | etcd_z1/1 | running | medium_z1 | 10.10.16.35 | | etcd_z2/0 | running | medium_z2 | 10.10.80.19 | | hm9000_z1/0 | running | medium_z1 | 10.10.17.4 | | hm9000_z2/0 | running | medium_z2 | 10.10.81.1 | | loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.10.17.7 | | loggregator_trafficcontroller_z2/0 | running | small_z2 | 10.10.81.4 | | nats_z1/0 | running | medium_z1 | 10.10.16.11 | | nats_z2/0 | running | medium_z2 | 10.10.80.11 | | nfs_z1/0 | running | medium_z1 | 10.10.16.36 | | router_z1/0 | running | router_z1 | 10.10.16.15 | | router_z2/0 | running | router_z2 | 10.10.80.15 | | runner_z1/0 | running | runner_z1 | 10.10.17.5 | | runner_z2/0 | running | runner_z2 | 10.10.81.2 | | stats_z1/0 | running | small_z1 | 10.10.16.255 | | uaa_z1/0 | running | medium_z1 | 10.10.17.0 | | uaa_z2/0 | running | medium_z2 | 10.10.80.254 |
------------------------------------------------------------------------------------------------------
openssl s_client -connect api.cf.ronak.com:443 CONNECTED(00000003) depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com
verify return:1
Certificate chain 0 s:/C=US/O=Pivotal/CN=*.cf.ronak.com
i:/C=US/O=Pivotal/CN=*.cf.ronak.com
Server certificate -----BEGIN CERTIFICATE----- MIIC6TCCAdGgAwIBAgIBADANBgkqhkiG9w0BAQUFADA4MQswCQYDVQQGEwJVUzEQ MA4GA1UECgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wHhcNMTUw NzE0MTAxNDA1WhcNMTgwNzE0MTAxNDA1WjA4MQswCQYDVQQGEwJVUzEQMA4GA1UE CgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQDi2+NVSsR7/oCh40H+CfFWUS0BczyTubpV72Ir 6TCkjLitLdFgJFqNvaIcT0+2+rozxQs7Zr/IlH5OznYsFYNyOSx2l/xDKCqA7q4U l6aGhFZSodIow6AKCRzVt2y5T/nmeRCWsVPjHMvANvJlM57PdWI1vIlaav3l3enF FUs14MSEuXY+eWfeP74Sprg/Xc5TiD5RQPLZC8h2HeAmIMgTV5TAagWiGK5MUoQ9 hScJvsgBPKQuldRfRsk7NxuXmY+wdfOYKc5CZ7szl7QiLsT7Tdeei+UuhIAh874S HjPP+JlFH4/whRJyDtv/h9rZzW1DJbx5ZeUtzFGv0jxmsvJrAgMBAAEwDQYJKoZI hvcNAQEFBQADggEBAM7S9za5V2sJ8Gwb4QQ29F4bQxVNk5/u7AnbzvfmhcWgO9yt oBUwtc04F/Ycz7sTlXXDe0f+nSo+Q0vowd4NmTSont5g8ZznD18k4krCcfvXCsAL AvJ/yt2n+fW/gwZbZHS9kb4Esxu4SmjU8GFN10+DMRFaB9JkbRpJmRLsxZzE8w+E MW7gi0V+IycOLyeJsfwXtjGZhKsLSq6fg/mwVax9Z0MsqVRbzO61RaJjMDiop+aN kf/ktqGVBACwqvG7cxKu4/2cFMzi6dC9xEN7o5spinfdNt/uH3O0JHRcftUCICaL ln6RtW7XXHNWnP2YfMXxPLvhjV/Cg49soz3D+os= -----END CERTIFICATE----- subject=/C=US/O=Pivotal/CN=*.cf.ronak.com
issuer=/C=US/O=Pivotal/CN=*.cf.ronak.com
No client certificate CA names sent
SSL handshake has read 1403 bytes and written 421 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: B845CA20842317DB12FB250C05D40686334A7C85D8BE2E5332B6C70632BF9256 Session-ID-ctx: Master-Key:
3934F0F361C1A407B215D869F8142F3B3180BB8CD232D55D937430CDD3674F047A774594B6A3BA9DB7275D1E507AE0B5
Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 1b 21 b6 45 6b 48 d6 c5-6d 66 e4 97 58 ae 76 91 .!.EkH..mf..X.v. 0010 - 0c 85 3f 3b f8 79 3d 59-b5 55 57 bd a7 98 a2 90 ..?;.y=Y.UW..... 0020 - 20 81 2b 0b 8a 6a 13 8c-1e e7 c4 d2 f7 d1 f6 5b .+..j.........[ 0030 - c7 d6 58 16 42 89 1e 51-01 7d 20 24 3f fa 2e f7
..X.B..Q.}
$?... 0040 - 76 06 d5 61 ff 4f 5c a1-46 77 41 62 fe fc c9 50 v..a.O.FwAb...P 0050 - 55 b6 00 98 15 22 8a 81-e2 e0 0e 9d da 83 fc 7b U....".........{ 0060 - ef 33 16 ec 88 60 ca 38-3f 35 85 6d fe 68 26 f6 .3...`.8?5.m.h&. 0070 - 24 48 65 47 8b c4 f6 20-f9 31 63 74 56 1a 67 d5 $HeG... .1ctV.g. 0080 - 26 d1 4c 0d fa 5f 01 da-ec d9 a6 88 9d ab cd fa &.L.._.......... 0090 - 05 17 bd e4 b4 78 f4 02-6b 5e 86 83 04 b7 f2 33 .....x..k^.....3
Start Time: 1438453185 Timeout : 300 (sec) Verify return code: 18 (self signed certificate)
------------------------------------------------------------------------------------------------------------------------
I am using self signed certificate.. so if I have to continue without --skip-ssl-validation, I should add this ELB certificate in all
the
VMS as trusted keys. Is it correct ? OR anyother option ?
--------------------------------------------------------------------------------------------------------------
In CF-AWS-Stub.yml I found two - a) jwt: signing_key & verification_key b) router . ssl_cert & ssl_key Both should be same as AWS ELB ssl certificate ??
------------------------------------------------------------------------------------------
I think core problem is cf push error which I mentioned above -
1) I can see in Response - "detected_buildpack": null, I am not
sure
why its null because I am uploading system buildpack app which
of
Nodejs and when I do - cf buildpacks I can see 4 buildpacks
below -
buildpack position enabled locked filename staticfile_buildpack 1 true false java_buildpack 2 true false node_buildpack 3 true false php_buildpack 6 true false
I am not sure why filename is null ??
2) Unknown Error Code - 10001 explained more below
---------------------------------------------------------------------------------------
'CF_TRACE=true cf push' gives below error -
REQUEST: [2015-08-02T10:40:17Z] GET /v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5 HTTP/1.1 Host: api.cf.ronak.com Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.2-24abed3 / linux
RESPONSE: [2015-08-02T10:40:17Z] HTTP/1.1 200 OK Content-Length: 491 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:40:17 GMT Server: nginx X-Cf-Requestid: f423947e-8512-41f0-7fd1-21ba6b89178d X-Content-Type-Options: nosniff X-Vcap-Request-Id:
aa782bd8-d758-4c65-505c-af71f2bc212d::a0ac2db0-3f3e-4a49-b011-2f9e7f02660b
{ "metadata": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "created_at": "2015-08-02T10:36:11Z", "url": "/v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5" }, "entity": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "status": "failed", "error": "Use of entity>error is deprecated in favor of entity>error_details.", "error_details": { "error_code": "UnknownError", "description": "An unknown error occurred.", "code": 10001 } } } FAILED Error uploading application. An unknown error occurred. FAILED Error uploading application. An unknown error occurred.
________________________________
I found post in which its mentioned - The point was that S3
buckets
name (specified by following options: resource_directory_key, app_package_directory_key, droplet_directory_key, and buildpack_directory_key) had dots in their names. Removing dots
from
this parameters will solve the problem.
I removing dots and replaced with '-' and again did 'bosh
deploy'
--- I can't find these directories getting created after deployment under S3 ?? is it normal ? I tried cf push anyways to check if its working - but gave me
same
'Unkown Error code 10001.
Please help me to solve the problem.
---------------------------------------------------------------------------------------------------
cf logs nodeapp --recent
RESPONSE: [2015-08-02T10:41:51Z] HTTP/1.1 200 OK Content-Length: 4525 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:41:50 GMT Server: nginx X-Cf-Requestid: c5508a55-a9f4-4be0-49a4-23a4f14fb5d3 X-Content-Type-Options: nosniff X-Vcap-Request-Id:
ae35d7fc-659f-4811-4d7c-077a78963951::4bae904f-6903-4a76-9d0e-314098266502
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "41657738-cc7c-4237-ab52-e4f9683e16f4", "url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4", "created_at": "2015-08-02T10:36:10Z", "updated_at": "2015-08-02T10:36:11Z" }, "entity": { "name": "nodeapp", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": {
}, "memory": 1024, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "d2fa841d-17d6-4bae-9400-409966082443", "command": "node main.js", "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url":
"/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89",
"space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url":
"/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89",
"created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid":
"66e33d43-92fa-4f30-809d-1f5d9adaf921",
"space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url":
"/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances",
"app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url":
"/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups"
} }, "stack_url":
"/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e",
"stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url":
"/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e",
"created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/events", "service_bindings_url":
"/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/service_bindings",
"service_bindings": [
], "routes_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/routes", "routes": [ { "metadata": { "guid": "2489386f-973b-45db-bc86-412b57bfffb0", "url":
"/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0",
"created_at": "2015-07-31T06:53:17Z", "updated_at": null }, "entity": { "host": "nodeapp", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0/apps" } } ] } }
] } Connected, dumping recent logs for app nodeapp in org ronak /
space
ronak as admin...
FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4 :
EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4 :
EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
----------------------------------------------------------------------------------------------------------
Please go through all my above research and let me know what
can be
done to resolve this issue??
Thanks Ronak _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: FYI: CAB call for July 2015 - moved to 7/15 @ 8a PDT

Guillaume Berche
Thanks a lot Amit for this detailed write up. This is much clearer now.
Guillaume.
toggle quoted messageShow quoted text
On Fri, Jul 31, 2015 at 6:01 AM, Amit Gupta <agupta(a)pivotal.io> wrote: Hi Guillaume,
Thanks for your question. The best place to start understanding how diego generates manifests is to look at the manifest generation script in the diego-release repository:
https://github.com/cloudfoundry-incubator/diego-release/blob/develop/scripts/generate-deployment-manifest
It primarily merges data into the main diego template:
https://github.com/cloudfoundry-incubator/diego-release/blob/develop/manifest-generation/diego.yml
There are several principles operating at the heart of this manifest generation strategy:
1. The inputs to the script should be a fixed list of stubs with a specific data structure (not an arbitrary list of stub files which can have all sorts of crazy structures in their YAML). 2. The YAML structure of the stub should make sense for the purpose of the stub, i.e. clearly represent the data it contains, it should not represent the data in some strange contorted format that needs to know where in the upstream manifest template it will be merged into. 3. Avoid (( merge )) within the body of the template, places that need to have values filled in explicitly reference the value from the stub that they need. 4. No nested templates, or "stubs" that have spiff double-parens syntax in them. 5. Each stub should have a single responsibility (e.g. instance count overrides, persistent disk overrides, etc.) which should be small and easy to understand; any two stubs should have orthogonal responsibilities, e.g. you don't specify some credentials in this stub, and some other credentials in another stub. 6. The responsibilities of the stubs should be generalizable to any BOSH release (e.g. "instance count overrides" is a generic enough purpose that would make sense for any BOSH release.) 7. Have an explicit stub for IaaS settings (basically anything that goes into a cloud_properties stanza in a BOSH manifest); the main template should be IaaS agnostic (rather than a template for each IaaS). 8. Cleanliness; do not produce a manifest in the end that has a bunch of unnecessary junk that BOSH doesn't even use (e.g. "meta"). People tend to develop the bad habit of depending on this random data. 9. No default property values in the templates; defaults should live in the job specs, the only properties exposed in the template should be the ones it makes sense to override. 10. Don't expose every property for override. spiff has hard limitations that make it impossible to use if you want to construct any arbitrary deployment manifest for a given release (or set of releases). Given that it already can't solve every problem, understand that the manifest generation tooling also doesn't need to solve every possible problem. Advanced users who want to override obscure properties can use their own advanced tooling. 11. Dovetail with upcoming bosh features: https://github.com/cloudfoundry/bosh-notes/blob/master/links.md https://github.com/cloudfoundry/bosh-notes/blob/master/global-networking.md https://github.com/cloudfoundry/bosh-notes/blob/master/cloud-config.md
https://github.com/cloudfoundry/bosh-notes/blob/master/availability-zones.md
The long-term goal is to make BOSH smarter and ideally obviate the need for complex tooling to generate these unwieldy, several-thousand-line YAML manifests. On the surface, both diego-release and cf-release suggest how to use spiff to generate manifests, but the way the two use spiff is drastically different. With diego, we've started by simplifying how manifests are generated so that the full power of spiff isn't needed, and identifying better abstractions that make sense for BOSH to know about so that eventually none of the power of spiff is needed.
----- Amit, CF OSS Release Integration PM Pivotal Software, Inc. -- View this message in context: http://cf-dev.70369.x6.nabble.com/cf-dev-CAB-call-for-July-2015-moved-to-7-15-8a-PDT-tp645p986.html Sent from the CF Dev mailing list archive at Nabble.com. _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: Is Garden limiting disk IOs ?

Guillaume Berche
Thanks Will. I guess with Garden moving to runC, the migration will bring disk io limits, would be nice to record that somewhere in a story so that this does not get forgotten. I was about to start some quick tests with a disk benchmark tool such as boonie [1], (invoked by the binary buildpack with a custom start cmd using netcat to bing the $PORT, invoke bonnie and return results in the HTTP response), and see how much other containers running the same test would be impacted. A refinement would be to have a regular app (e.g. a ruby app), and mesure response time for startup and response, with and without having one neighbor running bonnie on the same host. [1] http://www.textuality.com/bonnie/intro.htmlGuillaume.
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 11:10 AM, Will Pragnell <wpragnell(a)pivotal.io> wrote: The honest answer there is that I'm not sure. It's conceivable that limiting CPU for a container could prevent it from pummeling disk enough to DOS a garden host, but we've not tested that. I'll talk to the team about investigating this DOS vector.
On 30 July 2015 at 21:03, Guillaume Berche <bercheg(a)gmail.com> wrote:
Thanks Will.
Would other existing limits (e.g. nice, or fair cpu share) prevent one app from using up all of the host disk io, or could such app impact its neighbor containers (and other DEA/Cell process), and possibly up to denial of service (e.g. a compromised app that was to impact the cf platform that host them) ?
Guillaume.
On Thu, Jul 30, 2015 at 6:34 PM, Will Pragnell <wpragnell(a)pivotal.io> wrote:
Hi Guillaume,
No, Garden-Linux does not currently do any disk IO limiting.
Thanks, Will
On 30 July 2015 at 17:24, Guillaume Berche <bercheg(a)gmail.com> wrote:
Out of curiosity, I'd like to understand whether Garden is limiting disk IOs, similar to [2].
I do see disk space, inodes..., and network IOs limits described into [1] but did not find block IOs.
Thanks in advance,
Guillaume.
[1] http://godoc.org/github.com/cloudfoundry-incubator/garden [2] https://github.com/appc/spec/blob/master/spec/ace.md#resourceblock-bandwidth
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
|
|
Re: App autosleep support
Klevenz, Stephan <stephan.klevenz@...>
Hi Guillaume, We have a similar motivation for such feature request for saving resources. We tried that our with a quickly implement POC and made similar experience as you have written in the document. Here is in short what we did: * measure inactivity of an app – we took the time stamp of last log entry for an application and compared it with a fresh-hold – for the POC this was good enough but maybe a better measure is needed * inactive apps – stop application and let all app routes point to a watch dog application * watch dog – if a route is accessed then the watch dog wakes up the app, moves routes back to the app and redirects the request after app is up again We played also with CPU, number of requests and other measures to scale up and down applications. Challenges we did identify were: * find the right measure for inactive apps * start, stop or scale apps require permissions * enable the feature only for dedicated orgs or spaces (e.g. a trial org) * quick wakeup of app * a deeper integration into CF could make sense (get better measures, better permission control, configure thresh hold for apps via CF CLI …) We did implement this POC on top to the CC API. Using such feature in production will cause more issues and findings. Regards, Stephan Von: Guillaume Berche Antworten an: "Discussions about Cloud Foundry projects and the system overall." Datum: Donnerstag, 30. Juli 2015 12:46 An: cf-dev Betreff: [cf-dev] App autosleep support Hi, I wonder if there are plans to implement an auto-sleep behavior in cloudfoundry, in which inactive apps would be automatically stopped after a max inactivity threshold, and automatically restart upon arrival of traffic on their routes. Similar to google app engine default behavior [1] I did not find mentions of this yet in mailing lists and trackers. We feel at Orange that such feature can improve the density for some of our non-prod use-cases (with environmental and financial benefits). I'd like to know if someone in the community already worked on such feature or would be interested in collaborating on an opensource implementation. I drafted some specs for a java-based implementation we're planning to work on [2]. I'd love to hear feedbacks and suggestions on this. Thanks in advance, Guillaume. [1] https://cloud.google.com/appengine/docs/java/modules/[2] https://docs.google.com/document/d/1tMhIBX3tw7kPEOMCzKhUgmtmr26GVxyXwUTwMO71THI/edit#
|
|
Re: CF push error - Not able to resolve - Error dialing loggregator server: Get https://loggregator.systemdomain.com
Ronak Agarwal <roagarwa@...>
I just disabled - source/destination checks
And able to curl 'google.com' now from private IP.
Should I need to again do - bosh deploy ?
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 4:28 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote: If I restart my NAT vm and do curl google.com - I am getting response.
When I try ssh to any of the private VM (api_z1 IP) from this NAT vm -> ssh -i "bosh.pem" vcap(a)10.10.17.0
" It is required that your private key files are NOT accessible by others. This private key will be ignored. bad permissions: ignore key: bosh.pem Permission denied (publickey). "
So I did - chmod 400 bosh.pem
Then tried again above ssh command and able to ssh that 10.10.17.0
But curl google.com not giving anything.
On Mon, Aug 3, 2015 at 3:50 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Thanks Joseph.
I am not clear at one point - I already have a NAT vm which I had just stopped and again started, so are you asking me to create a new NAT vm because I have stopped it earlier? because it maintains some persistent data.
I just tried pushing a Java app because at api_z1, I can't find Java buildpack error, but I am getting -
RESPONSE: [2015-08-03T10:13:42Z] HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive
FAILED Error uploading application. Server error, status code: 504, error code: 0, message: FAILED Error uploading application. Server error, status code: 504, error code: 0, message:
I checked api_z1 logs and couldn't find any trace for this error, is it moved to Cloud_Controller_Worker ? What could be the error? I have good disk space available in api_z1.
cf logs appname --recent , gives below - RESPONSE: [2015-08-03T10:19:07Z] HTTP/1.1 200 OK Content-Length: 6200 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Mon, 03 Aug 2015 10:19:05 GMT Server: nginx X-Cf-Requestid: 2a67e173-20bf-4f89-7b27-efd5b28461c6 X-Content-Type-Options: nosniff X-Vcap-Request-Id: f7b15702-3650-4258-5b40-48efbd9ef271::230c52e3-52a7-4a81-82f4-4e1ed445082d
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "2ac4b290-fb0d-4273-940e-b2d1aeb31320", "url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320", "created_at": "2015-08-03T09:26:19Z", "updated_at": "2015-08-03T10:12:40Z" }, "entity": { "name": "hello-spring-cloud", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": { "SPRING_PROFILES_DEFAULT": "cloud" }, "memory": 512, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "3719a87d-98b8-4641-a613-c9730f132b5d", "command": null, "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/events", "service_bindings_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/routes", "routes": [ { "metadata": { "guid": "f44c4856-08c1-49f2-8e1b-a51e548adca6", "url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6", "created_at": "2015-08-03T09:27:16Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unabsorbent-masculinization", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6/apps" } }, { "metadata": { "guid": "47a495d6-a484-48f0-ac12-6157282be1c4", "url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4", "created_at": "2015-08-03T09:58:03Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unascertainable-unmelodiousness", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4/apps" } }, { "metadata": { "guid": "f51fad6c-fd48-4dfb-b201-a756af14bd00", "url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00", "created_at": "2015-08-03T10:12:40Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-completive-hesperinos", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00/apps" } } ] } } ] } Connected, dumping recent logs for app hello-spring-cloud in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
On Mon, Aug 3, 2015 at 3:33 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
Yeah, without the NAT vm nothing will be able to get out to the internet. They will be able to respond to external requests, such as a request to the api, or the login server, or an individual app. But if anything on the inside tries to initiate a network connection, it has no route to the internet.
Amazon has good generic documentation on how to create a NAT instance: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#NATInstance
BOSH also has some documentation, although it doesn't tell you how to create the instance itself, but rather how to configure your private subnet and routing table to use the NAT instance: http://bosh.io/docs/setup-aws.html#nat
In both sets of documentation, you may already have security groups, routes and subnets created from your previous NAT instance.
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:50 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
I tried 'curl http://google.com' from my api_z1 instance but was unable to get any response.
I create my NAT vm through 'bosh aws create' command, but by mistake I stopped that VM on AWS. So does stopping NAT VM is causing this issue ? It has any persistent data while setting NAT VM ?
If its because of stopping NAT Vm, so now what is the solution, do I need to destroy everything and create entire environment again ? Please suggest.
On Mon, Aug 3, 2015 at 2:53 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
It looks like the api instance is unable to upload the buildpacks to the blobstore. Perhaps the NAT (network NAT, not message bus NATS) instance is not configured correctly.
Are you able to curl an external URL from the api instance?
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:12 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Its a fresh install.
I found few logs may be helpful to resolve this issue --
I found few logs on cloud_controller_ng as below -
Nodejs buildpack install fail :- {"timestamp":1438583731.044699,"message":"Buildpack nodejs_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583731.045526,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\", ------------------------------------------ -> Fails at nodejs_buildpack-cached-v1.4.0.zip
"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583731.0496318,"message":"(0.001478s) UPDATE delayed_jobs SET guid = '82d9ebe1-7af0-4671-89a5-30146a5bb554', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: nodejs_buildpack\n file:
\\"/var/vcap/packages/buildpack_nodejs/nodejs_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n', last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:31:30',failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 971) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
go_buildpack failed to install
{"timestamp":1438583970.3534014,"message":"Buildpack go_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583970.3541129,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\"
________________________________
->go_buildpack-cached-v1.4.0.zip ,
"log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583970.358118,"message":"(0.001543s) UPDATE delayed_jobs SET guid = 'd559d276-c5a2-471c-aca1-ba25f4e9cd97', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: go_buildpack\n file: \\"/var/vcap/packages/buildpack_go/go_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:35:29', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 972) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
python_buildpack failed to install
{"timestamp":1438583971.811825,"message":"Buildpack python_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583971.8125288,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\",
________________________________
->python_buildpack-cached-v1.4.0.zip
,"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583971.8163912,"message":"(0.001414s) UPDATE delayed_jobs SET guid = '37f04f8f-2e1e-4096-8f5f-c10d8180a9fc', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0,handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: python_buildpack\n file:
\\"/var/vcap/packages/buildpack_python/python_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26',locked_at = '2015-08-03 06:35:31', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 973) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":7030
php_buildpack failed to install
{"timestamp":1438584211.920127,"message":"Buildpack php_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->php_buildpack-cached-v3.3.0.zip
{"timestamp":1438584211.9250205,"message":"(0.001442s) UPDATE delayed_jobs SET guid = '133a41bd-307f-4e2b-b70e-635f3dec60c4', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: php_buildpack\n file:
\\"/var/vcap/packages/buildpack_php/php_buildpack-cached-v3.3.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:30', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 974) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,
________________________________
binary_buildpack failed to install
{"timestamp":1438584212.2200441,"message":"Buildpack binary_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->binary_buildpack-cached-v1.0.1.zip
{"timestamp":1438584212.2245483,"message":"(0.001381s) UPDATE delayed_jobs SET guid = '7ea6156e-024c-4f2d-b574-ca9b218470cd', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: binary_buildpack\n file:
\\"/var/vcap/packages/buildpack_binary/binary_buildpack-cached-v1.0.1.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:31', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 975) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":436
________________________________
This seems to be timeout issue.
{"timestamp":1438585142.846115,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in `rescue in block in connect'
{"timestamp":1438585142.851403,"message":"(0.001378s) UPDATE delayed_jobs SET guid = '6e4167d9-c7ee-4425-9049-e50971864a64', created_at = '2015-08-03 06:55:01', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::AppBitsPacker\n app_guid: 41657738-cc7c-4237-ab52-e4f9683e16f4\n uploaded_compressed_path: \\"/var/vcap/data/cloud_controller_ng/tmp/uploads/0000000001\\"\n fingerprints: []\n request_id:
64cc342c-9db9-495e-46d9-ce5e9331aa41::d6c111c3-79b1-4fec-8fa9-0d4be759bdd9\n', last_error = NULL, run_at = '2015-08-03 06:55:01', locked_at = '2015-08-03 06:55:02', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 989) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
I will try to push Java app to verify if its getting pushed.
On Mon, Aug 3, 2015 at 2:19 PM, Gwenn Etourneau <getourneau(a)pivotal.io> wrote:
Btw CF-213 is a fresh install or update ??
On Mon, Aug 3, 2015 at 5:37 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Hi,
Please go through all my instance info below and let me know what can be done to resolve this issue??
I have deployed CF-213.yml release through microbosh on AWS. I have installed latest CF CLI v6.12.2 I am able to do cf login using below endpoints - /.cf/config.json "Target": "https://api.cf.ronak.com", "ApiVersion": "2.33.0", "AuthorizationEndpoint": "https://login.cf.ronak.com", "LoggregatorEndPoint": "wss://loggregator.cf.ronak.com:4443", "DopplerEndPoint": "wss://doppler.cf.ronak.com:4443", "UaaEndpoint": "https://uaa.cf.ronak.com",
Now when I am trying to push Nodejs app using - cf push nodeapp -c "node main.js" -m 128M -b node_buildpack OR cf push nodeapp -c "node main.js"
both are failing .. gives unknown error. When I do -> cf logs nodeapp --recent It gives below error - FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=05ea8bda-28c1-49ed-8c65-4625c77423be: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
--------------------------------------------------------------------------------
I am using ELB and my HA-proxy job is not running. Please let me know how to overcome this challenge. I found lot of people have faced it but all of them I think are using HA-proxy and not ELB.
I have configured 4443 with SSL on ELB -
80 (HTTP) forwarding to 80 (HTTP), 443 (HTTPS, Certificate: cfrouter_cert) forwarding to 80 (HTTP), 4443 (SSL, Certificate: cfrouter_cert) forwarding to 80 (TCP)
----------------------------------------------------------------------------------
Below are the vms/jobs which are running on AWS I dont see login_z1/z2 in them, so do I need these jobs?
Job/index | State | Resource Pool | IPs |
+------------------------------------+---------+---------------+--------------+ | api_worker_z1/0 | running | small_z1 | 10.10.17.3 | | api_worker_z2/0 | running | small_z2 | 10.10.81.0 | | api_z1/0 | running | large_z1 | 10.10.17.1 | | api_z2/0 | running | large_z2 | 10.10.80.255 | | clock_global/0 | running | medium_z1 | 10.10.17.2 | | doppler_z1/0 | running | medium_z1 | 10.10.17.6 | | doppler_z2/0 | running | medium_z2 | 10.10.81.3 | | etcd_z1/0 | running | medium_z1 | 10.10.16.20 | | etcd_z1/1 | running | medium_z1 | 10.10.16.35 | | etcd_z2/0 | running | medium_z2 | 10.10.80.19 | | hm9000_z1/0 | running | medium_z1 | 10.10.17.4 | | hm9000_z2/0 | running | medium_z2 | 10.10.81.1 | | loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.10.17.7 | | loggregator_trafficcontroller_z2/0 | running | small_z2 | 10.10.81.4 | | nats_z1/0 | running | medium_z1 | 10.10.16.11 | | nats_z2/0 | running | medium_z2 | 10.10.80.11 | | nfs_z1/0 | running | medium_z1 | 10.10.16.36 | | router_z1/0 | running | router_z1 | 10.10.16.15 | | router_z2/0 | running | router_z2 | 10.10.80.15 | | runner_z1/0 | running | runner_z1 | 10.10.17.5 | | runner_z2/0 | running | runner_z2 | 10.10.81.2 | | stats_z1/0 | running | small_z1 | 10.10.16.255 | | uaa_z1/0 | running | medium_z1 | 10.10.17.0 | | uaa_z2/0 | running | medium_z2 | 10.10.80.254 |
------------------------------------------------------------------------------------------------------
openssl s_client -connect api.cf.ronak.com:443 CONNECTED(00000003) depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com
verify return:1
Certificate chain 0 s:/C=US/O=Pivotal/CN=*.cf.ronak.com
i:/C=US/O=Pivotal/CN=*.cf.ronak.com
Server certificate -----BEGIN CERTIFICATE----- MIIC6TCCAdGgAwIBAgIBADANBgkqhkiG9w0BAQUFADA4MQswCQYDVQQGEwJVUzEQ MA4GA1UECgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wHhcNMTUw NzE0MTAxNDA1WhcNMTgwNzE0MTAxNDA1WjA4MQswCQYDVQQGEwJVUzEQMA4GA1UE CgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQDi2+NVSsR7/oCh40H+CfFWUS0BczyTubpV72Ir 6TCkjLitLdFgJFqNvaIcT0+2+rozxQs7Zr/IlH5OznYsFYNyOSx2l/xDKCqA7q4U l6aGhFZSodIow6AKCRzVt2y5T/nmeRCWsVPjHMvANvJlM57PdWI1vIlaav3l3enF FUs14MSEuXY+eWfeP74Sprg/Xc5TiD5RQPLZC8h2HeAmIMgTV5TAagWiGK5MUoQ9 hScJvsgBPKQuldRfRsk7NxuXmY+wdfOYKc5CZ7szl7QiLsT7Tdeei+UuhIAh874S HjPP+JlFH4/whRJyDtv/h9rZzW1DJbx5ZeUtzFGv0jxmsvJrAgMBAAEwDQYJKoZI hvcNAQEFBQADggEBAM7S9za5V2sJ8Gwb4QQ29F4bQxVNk5/u7AnbzvfmhcWgO9yt oBUwtc04F/Ycz7sTlXXDe0f+nSo+Q0vowd4NmTSont5g8ZznD18k4krCcfvXCsAL AvJ/yt2n+fW/gwZbZHS9kb4Esxu4SmjU8GFN10+DMRFaB9JkbRpJmRLsxZzE8w+E MW7gi0V+IycOLyeJsfwXtjGZhKsLSq6fg/mwVax9Z0MsqVRbzO61RaJjMDiop+aN kf/ktqGVBACwqvG7cxKu4/2cFMzi6dC9xEN7o5spinfdNt/uH3O0JHRcftUCICaL ln6RtW7XXHNWnP2YfMXxPLvhjV/Cg49soz3D+os= -----END CERTIFICATE----- subject=/C=US/O=Pivotal/CN=*.cf.ronak.com
issuer=/C=US/O=Pivotal/CN=*.cf.ronak.com
No client certificate CA names sent
SSL handshake has read 1403 bytes and written 421 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: B845CA20842317DB12FB250C05D40686334A7C85D8BE2E5332B6C70632BF9256 Session-ID-ctx: Master-Key:
3934F0F361C1A407B215D869F8142F3B3180BB8CD232D55D937430CDD3674F047A774594B6A3BA9DB7275D1E507AE0B5 Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 1b 21 b6 45 6b 48 d6 c5-6d 66 e4 97 58 ae 76 91 .!.EkH..mf..X.v. 0010 - 0c 85 3f 3b f8 79 3d 59-b5 55 57 bd a7 98 a2 90 ..?;.y=Y.UW..... 0020 - 20 81 2b 0b 8a 6a 13 8c-1e e7 c4 d2 f7 d1 f6 5b .+..j.........[ 0030 - c7 d6 58 16 42 89 1e 51-01 7d 20 24 3f fa 2e f7 ..X.B..Q.} $?... 0040 - 76 06 d5 61 ff 4f 5c a1-46 77 41 62 fe fc c9 50 v..a.O.FwAb...P 0050 - 55 b6 00 98 15 22 8a 81-e2 e0 0e 9d da 83 fc 7b U....".........{ 0060 - ef 33 16 ec 88 60 ca 38-3f 35 85 6d fe 68 26 f6 .3...`.8?5.m.h&. 0070 - 24 48 65 47 8b c4 f6 20-f9 31 63 74 56 1a 67 d5 $HeG... .1ctV.g. 0080 - 26 d1 4c 0d fa 5f 01 da-ec d9 a6 88 9d ab cd fa &.L.._.......... 0090 - 05 17 bd e4 b4 78 f4 02-6b 5e 86 83 04 b7 f2 33 .....x..k^.....3
Start Time: 1438453185 Timeout : 300 (sec) Verify return code: 18 (self signed certificate)
------------------------------------------------------------------------------------------------------------------------
I am using self signed certificate.. so if I have to continue without --skip-ssl-validation, I should add this ELB certificate in all the VMS as trusted keys. Is it correct ? OR anyother option ?
--------------------------------------------------------------------------------------------------------------
In CF-AWS-Stub.yml I found two - a) jwt: signing_key & verification_key b) router . ssl_cert & ssl_key Both should be same as AWS ELB ssl certificate ??
------------------------------------------------------------------------------------------
I think core problem is cf push error which I mentioned above -
1) I can see in Response - "detected_buildpack": null, I am not sure why its null because I am uploading system buildpack app which of Nodejs and when I do - cf buildpacks I can see 4 buildpacks below -
buildpack position enabled locked filename staticfile_buildpack 1 true false java_buildpack 2 true false node_buildpack 3 true false php_buildpack 6 true false
I am not sure why filename is null ??
2) Unknown Error Code - 10001 explained more below
---------------------------------------------------------------------------------------
'CF_TRACE=true cf push' gives below error -
REQUEST: [2015-08-02T10:40:17Z] GET /v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5 HTTP/1.1 Host: api.cf.ronak.com Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.2-24abed3 / linux
RESPONSE: [2015-08-02T10:40:17Z] HTTP/1.1 200 OK Content-Length: 491 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:40:17 GMT Server: nginx X-Cf-Requestid: f423947e-8512-41f0-7fd1-21ba6b89178d X-Content-Type-Options: nosniff X-Vcap-Request-Id:
aa782bd8-d758-4c65-505c-af71f2bc212d::a0ac2db0-3f3e-4a49-b011-2f9e7f02660b
{ "metadata": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "created_at": "2015-08-02T10:36:11Z", "url": "/v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5" }, "entity": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "status": "failed", "error": "Use of entity>error is deprecated in favor of entity>error_details.", "error_details": { "error_code": "UnknownError", "description": "An unknown error occurred.", "code": 10001 } } } FAILED Error uploading application. An unknown error occurred. FAILED Error uploading application. An unknown error occurred.
________________________________
I found post in which its mentioned - The point was that S3 buckets name (specified by following options: resource_directory_key, app_package_directory_key, droplet_directory_key, and buildpack_directory_key) had dots in their names. Removing dots from this parameters will solve the problem.
I removing dots and replaced with '-' and again did 'bosh deploy' --- I can't find these directories getting created after deployment under S3 ?? is it normal ? I tried cf push anyways to check if its working - but gave me same 'Unkown Error code 10001.
Please help me to solve the problem.
---------------------------------------------------------------------------------------------------
cf logs nodeapp --recent
RESPONSE: [2015-08-02T10:41:51Z] HTTP/1.1 200 OK Content-Length: 4525 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:41:50 GMT Server: nginx X-Cf-Requestid: c5508a55-a9f4-4be0-49a4-23a4f14fb5d3 X-Content-Type-Options: nosniff X-Vcap-Request-Id:
ae35d7fc-659f-4811-4d7c-077a78963951::4bae904f-6903-4a76-9d0e-314098266502
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "41657738-cc7c-4237-ab52-e4f9683e16f4", "url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4", "created_at": "2015-08-02T10:36:10Z", "updated_at": "2015-08-02T10:36:11Z" }, "entity": { "name": "nodeapp", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": {
}, "memory": 1024, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "d2fa841d-17d6-4bae-9400-409966082443", "command": "node main.js", "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/events", "service_bindings_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/routes", "routes": [ { "metadata": { "guid": "2489386f-973b-45db-bc86-412b57bfffb0", "url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0", "created_at": "2015-07-31T06:53:17Z", "updated_at": null }, "entity": { "host": "nodeapp", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0/apps" } } ] } }
] } Connected, dumping recent logs for app nodeapp in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
----------------------------------------------------------------------------------------------------------
Please go through all my above research and let me know what can be done to resolve this issue??
Thanks Ronak _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
|
|
Re: CF push error - Not able to resolve - Error dialing loggregator server: Get https://loggregator.systemdomain.com
Ronak Agarwal <roagarwa@...>
If I restart my NAT vm and do curl google.com - I am getting response.
When I try ssh to any of the private VM (api_z1 IP) from this NAT vm -> ssh -i "bosh.pem" vcap(a)10.10.17.0
" It is required that your private key files are NOT accessible by others. This private key will be ignored. bad permissions: ignore key: bosh.pem Permission denied (publickey). "
So I did - chmod 400 bosh.pem
Then tried again above ssh command and able to ssh that 10.10.17.0
But curl google.com not giving anything.
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 3:50 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote: Thanks Joseph.
I am not clear at one point - I already have a NAT vm which I had just stopped and again started, so are you asking me to create a new NAT vm because I have stopped it earlier? because it maintains some persistent data.
I just tried pushing a Java app because at api_z1, I can't find Java buildpack error, but I am getting -
RESPONSE: [2015-08-03T10:13:42Z] HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive
FAILED Error uploading application. Server error, status code: 504, error code: 0, message: FAILED Error uploading application. Server error, status code: 504, error code: 0, message:
I checked api_z1 logs and couldn't find any trace for this error, is it moved to Cloud_Controller_Worker ? What could be the error? I have good disk space available in api_z1.
cf logs appname --recent , gives below - RESPONSE: [2015-08-03T10:19:07Z] HTTP/1.1 200 OK Content-Length: 6200 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Mon, 03 Aug 2015 10:19:05 GMT Server: nginx X-Cf-Requestid: 2a67e173-20bf-4f89-7b27-efd5b28461c6 X-Content-Type-Options: nosniff X-Vcap-Request-Id: f7b15702-3650-4258-5b40-48efbd9ef271::230c52e3-52a7-4a81-82f4-4e1ed445082d
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "2ac4b290-fb0d-4273-940e-b2d1aeb31320", "url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320", "created_at": "2015-08-03T09:26:19Z", "updated_at": "2015-08-03T10:12:40Z" }, "entity": { "name": "hello-spring-cloud", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": { "SPRING_PROFILES_DEFAULT": "cloud" }, "memory": 512, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "3719a87d-98b8-4641-a613-c9730f132b5d", "command": null, "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/events", "service_bindings_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/routes", "routes": [ { "metadata": { "guid": "f44c4856-08c1-49f2-8e1b-a51e548adca6", "url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6", "created_at": "2015-08-03T09:27:16Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unabsorbent-masculinization", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6/apps" } }, { "metadata": { "guid": "47a495d6-a484-48f0-ac12-6157282be1c4", "url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4", "created_at": "2015-08-03T09:58:03Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unascertainable-unmelodiousness", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4/apps" } }, { "metadata": { "guid": "f51fad6c-fd48-4dfb-b201-a756af14bd00", "url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00", "created_at": "2015-08-03T10:12:40Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-completive-hesperinos", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00/apps" } } ] } } ] } Connected, dumping recent logs for app hello-spring-cloud in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
On Mon, Aug 3, 2015 at 3:33 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
Yeah, without the NAT vm nothing will be able to get out to the internet. They will be able to respond to external requests, such as a request to the api, or the login server, or an individual app. But if anything on the inside tries to initiate a network connection, it has no route to the internet.
Amazon has good generic documentation on how to create a NAT instance: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#NATInstance
BOSH also has some documentation, although it doesn't tell you how to create the instance itself, but rather how to configure your private subnet and routing table to use the NAT instance: http://bosh.io/docs/setup-aws.html#nat
In both sets of documentation, you may already have security groups, routes and subnets created from your previous NAT instance.
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:50 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
I tried 'curl http://google.com' from my api_z1 instance but was unable to get any response.
I create my NAT vm through 'bosh aws create' command, but by mistake I stopped that VM on AWS. So does stopping NAT VM is causing this issue ? It has any persistent data while setting NAT VM ?
If its because of stopping NAT Vm, so now what is the solution, do I need to destroy everything and create entire environment again ? Please suggest.
On Mon, Aug 3, 2015 at 2:53 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
It looks like the api instance is unable to upload the buildpacks to the blobstore. Perhaps the NAT (network NAT, not message bus NATS) instance is not configured correctly.
Are you able to curl an external URL from the api instance?
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:12 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Its a fresh install.
I found few logs may be helpful to resolve this issue --
I found few logs on cloud_controller_ng as below -
Nodejs buildpack install fail :- {"timestamp":1438583731.044699,"message":"Buildpack nodejs_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583731.045526,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\", ------------------------------------------ -> Fails at nodejs_buildpack-cached-v1.4.0.zip
"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583731.0496318,"message":"(0.001478s) UPDATE delayed_jobs SET guid = '82d9ebe1-7af0-4671-89a5-30146a5bb554', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: nodejs_buildpack\n file:
\\"/var/vcap/packages/buildpack_nodejs/nodejs_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n', last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:31:30',failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 971) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
go_buildpack failed to install
{"timestamp":1438583970.3534014,"message":"Buildpack go_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583970.3541129,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\"
________________________________
->go_buildpack-cached-v1.4.0.zip ,
"log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583970.358118,"message":"(0.001543s) UPDATE delayed_jobs SET guid = 'd559d276-c5a2-471c-aca1-ba25f4e9cd97', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: go_buildpack\n file: \\"/var/vcap/packages/buildpack_go/go_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:35:29', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 972) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
python_buildpack failed to install
{"timestamp":1438583971.811825,"message":"Buildpack python_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583971.8125288,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\",
________________________________
->python_buildpack-cached-v1.4.0.zip
,"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583971.8163912,"message":"(0.001414s) UPDATE delayed_jobs SET guid = '37f04f8f-2e1e-4096-8f5f-c10d8180a9fc', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0,handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: python_buildpack\n file:
\\"/var/vcap/packages/buildpack_python/python_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26',locked_at = '2015-08-03 06:35:31', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 973) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":7030
php_buildpack failed to install
{"timestamp":1438584211.920127,"message":"Buildpack php_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->php_buildpack-cached-v3.3.0.zip
{"timestamp":1438584211.9250205,"message":"(0.001442s) UPDATE delayed_jobs SET guid = '133a41bd-307f-4e2b-b70e-635f3dec60c4', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: php_buildpack\n file:
\\"/var/vcap/packages/buildpack_php/php_buildpack-cached-v3.3.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:30', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 974) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,
________________________________
binary_buildpack failed to install
{"timestamp":1438584212.2200441,"message":"Buildpack binary_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->binary_buildpack-cached-v1.0.1.zip
{"timestamp":1438584212.2245483,"message":"(0.001381s) UPDATE delayed_jobs SET guid = '7ea6156e-024c-4f2d-b574-ca9b218470cd', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: binary_buildpack\n file:
\\"/var/vcap/packages/buildpack_binary/binary_buildpack-cached-v1.0.1.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:31', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 975) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":436
________________________________
This seems to be timeout issue.
{"timestamp":1438585142.846115,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in `rescue in block in connect'
{"timestamp":1438585142.851403,"message":"(0.001378s) UPDATE delayed_jobs SET guid = '6e4167d9-c7ee-4425-9049-e50971864a64', created_at = '2015-08-03 06:55:01', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::AppBitsPacker\n app_guid: 41657738-cc7c-4237-ab52-e4f9683e16f4\n uploaded_compressed_path: \\"/var/vcap/data/cloud_controller_ng/tmp/uploads/0000000001\\"\n fingerprints: []\n request_id:
64cc342c-9db9-495e-46d9-ce5e9331aa41::d6c111c3-79b1-4fec-8fa9-0d4be759bdd9\n', last_error = NULL, run_at = '2015-08-03 06:55:01', locked_at = '2015-08-03 06:55:02', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 989) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
I will try to push Java app to verify if its getting pushed.
On Mon, Aug 3, 2015 at 2:19 PM, Gwenn Etourneau <getourneau(a)pivotal.io> wrote:
Btw CF-213 is a fresh install or update ??
On Mon, Aug 3, 2015 at 5:37 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Hi,
Please go through all my instance info below and let me know what can be done to resolve this issue??
I have deployed CF-213.yml release through microbosh on AWS. I have installed latest CF CLI v6.12.2 I am able to do cf login using below endpoints - /.cf/config.json "Target": "https://api.cf.ronak.com", "ApiVersion": "2.33.0", "AuthorizationEndpoint": "https://login.cf.ronak.com", "LoggregatorEndPoint": "wss://loggregator.cf.ronak.com:4443", "DopplerEndPoint": "wss://doppler.cf.ronak.com:4443", "UaaEndpoint": "https://uaa.cf.ronak.com",
Now when I am trying to push Nodejs app using - cf push nodeapp -c "node main.js" -m 128M -b node_buildpack OR cf push nodeapp -c "node main.js"
both are failing .. gives unknown error. When I do -> cf logs nodeapp --recent It gives below error - FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=05ea8bda-28c1-49ed-8c65-4625c77423be: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
--------------------------------------------------------------------------------
I am using ELB and my HA-proxy job is not running. Please let me know how to overcome this challenge. I found lot of people have faced it but all of them I think are using HA-proxy and not ELB.
I have configured 4443 with SSL on ELB -
80 (HTTP) forwarding to 80 (HTTP), 443 (HTTPS, Certificate: cfrouter_cert) forwarding to 80 (HTTP), 4443 (SSL, Certificate: cfrouter_cert) forwarding to 80 (TCP)
----------------------------------------------------------------------------------
Below are the vms/jobs which are running on AWS I dont see login_z1/z2 in them, so do I need these jobs?
Job/index | State | Resource Pool | IPs |
+------------------------------------+---------+---------------+--------------+ | api_worker_z1/0 | running | small_z1 | 10.10.17.3 | | api_worker_z2/0 | running | small_z2 | 10.10.81.0 | | api_z1/0 | running | large_z1 | 10.10.17.1 | | api_z2/0 | running | large_z2 | 10.10.80.255 | | clock_global/0 | running | medium_z1 | 10.10.17.2 | | doppler_z1/0 | running | medium_z1 | 10.10.17.6 | | doppler_z2/0 | running | medium_z2 | 10.10.81.3 | | etcd_z1/0 | running | medium_z1 | 10.10.16.20 | | etcd_z1/1 | running | medium_z1 | 10.10.16.35 | | etcd_z2/0 | running | medium_z2 | 10.10.80.19 | | hm9000_z1/0 | running | medium_z1 | 10.10.17.4 | | hm9000_z2/0 | running | medium_z2 | 10.10.81.1 | | loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.10.17.7 | | loggregator_trafficcontroller_z2/0 | running | small_z2 | 10.10.81.4 | | nats_z1/0 | running | medium_z1 | 10.10.16.11 | | nats_z2/0 | running | medium_z2 | 10.10.80.11 | | nfs_z1/0 | running | medium_z1 | 10.10.16.36 | | router_z1/0 | running | router_z1 | 10.10.16.15 | | router_z2/0 | running | router_z2 | 10.10.80.15 | | runner_z1/0 | running | runner_z1 | 10.10.17.5 | | runner_z2/0 | running | runner_z2 | 10.10.81.2 | | stats_z1/0 | running | small_z1 | 10.10.16.255 | | uaa_z1/0 | running | medium_z1 | 10.10.17.0 | | uaa_z2/0 | running | medium_z2 | 10.10.80.254 |
------------------------------------------------------------------------------------------------------
openssl s_client -connect api.cf.ronak.com:443 CONNECTED(00000003) depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com
verify return:1
Certificate chain 0 s:/C=US/O=Pivotal/CN=*.cf.ronak.com
i:/C=US/O=Pivotal/CN=*.cf.ronak.com
Server certificate -----BEGIN CERTIFICATE----- MIIC6TCCAdGgAwIBAgIBADANBgkqhkiG9w0BAQUFADA4MQswCQYDVQQGEwJVUzEQ MA4GA1UECgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wHhcNMTUw NzE0MTAxNDA1WhcNMTgwNzE0MTAxNDA1WjA4MQswCQYDVQQGEwJVUzEQMA4GA1UE CgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQDi2+NVSsR7/oCh40H+CfFWUS0BczyTubpV72Ir 6TCkjLitLdFgJFqNvaIcT0+2+rozxQs7Zr/IlH5OznYsFYNyOSx2l/xDKCqA7q4U l6aGhFZSodIow6AKCRzVt2y5T/nmeRCWsVPjHMvANvJlM57PdWI1vIlaav3l3enF FUs14MSEuXY+eWfeP74Sprg/Xc5TiD5RQPLZC8h2HeAmIMgTV5TAagWiGK5MUoQ9 hScJvsgBPKQuldRfRsk7NxuXmY+wdfOYKc5CZ7szl7QiLsT7Tdeei+UuhIAh874S HjPP+JlFH4/whRJyDtv/h9rZzW1DJbx5ZeUtzFGv0jxmsvJrAgMBAAEwDQYJKoZI hvcNAQEFBQADggEBAM7S9za5V2sJ8Gwb4QQ29F4bQxVNk5/u7AnbzvfmhcWgO9yt oBUwtc04F/Ycz7sTlXXDe0f+nSo+Q0vowd4NmTSont5g8ZznD18k4krCcfvXCsAL AvJ/yt2n+fW/gwZbZHS9kb4Esxu4SmjU8GFN10+DMRFaB9JkbRpJmRLsxZzE8w+E MW7gi0V+IycOLyeJsfwXtjGZhKsLSq6fg/mwVax9Z0MsqVRbzO61RaJjMDiop+aN kf/ktqGVBACwqvG7cxKu4/2cFMzi6dC9xEN7o5spinfdNt/uH3O0JHRcftUCICaL ln6RtW7XXHNWnP2YfMXxPLvhjV/Cg49soz3D+os= -----END CERTIFICATE----- subject=/C=US/O=Pivotal/CN=*.cf.ronak.com
issuer=/C=US/O=Pivotal/CN=*.cf.ronak.com
No client certificate CA names sent
SSL handshake has read 1403 bytes and written 421 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: B845CA20842317DB12FB250C05D40686334A7C85D8BE2E5332B6C70632BF9256 Session-ID-ctx: Master-Key:
3934F0F361C1A407B215D869F8142F3B3180BB8CD232D55D937430CDD3674F047A774594B6A3BA9DB7275D1E507AE0B5 Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 1b 21 b6 45 6b 48 d6 c5-6d 66 e4 97 58 ae 76 91 .!.EkH..mf..X.v. 0010 - 0c 85 3f 3b f8 79 3d 59-b5 55 57 bd a7 98 a2 90 ..?;.y=Y.UW..... 0020 - 20 81 2b 0b 8a 6a 13 8c-1e e7 c4 d2 f7 d1 f6 5b .+..j.........[ 0030 - c7 d6 58 16 42 89 1e 51-01 7d 20 24 3f fa 2e f7 ..X.B..Q.} $?... 0040 - 76 06 d5 61 ff 4f 5c a1-46 77 41 62 fe fc c9 50 v..a.O.FwAb...P 0050 - 55 b6 00 98 15 22 8a 81-e2 e0 0e 9d da 83 fc 7b U....".........{ 0060 - ef 33 16 ec 88 60 ca 38-3f 35 85 6d fe 68 26 f6 .3...`.8?5.m.h&. 0070 - 24 48 65 47 8b c4 f6 20-f9 31 63 74 56 1a 67 d5 $HeG... .1ctV.g. 0080 - 26 d1 4c 0d fa 5f 01 da-ec d9 a6 88 9d ab cd fa &.L.._.......... 0090 - 05 17 bd e4 b4 78 f4 02-6b 5e 86 83 04 b7 f2 33 .....x..k^.....3
Start Time: 1438453185 Timeout : 300 (sec) Verify return code: 18 (self signed certificate)
------------------------------------------------------------------------------------------------------------------------
I am using self signed certificate.. so if I have to continue without --skip-ssl-validation, I should add this ELB certificate in all the VMS as trusted keys. Is it correct ? OR anyother option ?
--------------------------------------------------------------------------------------------------------------
In CF-AWS-Stub.yml I found two - a) jwt: signing_key & verification_key b) router . ssl_cert & ssl_key Both should be same as AWS ELB ssl certificate ??
------------------------------------------------------------------------------------------
I think core problem is cf push error which I mentioned above -
1) I can see in Response - "detected_buildpack": null, I am not sure why its null because I am uploading system buildpack app which of Nodejs and when I do - cf buildpacks I can see 4 buildpacks below -
buildpack position enabled locked filename staticfile_buildpack 1 true false java_buildpack 2 true false node_buildpack 3 true false php_buildpack 6 true false
I am not sure why filename is null ??
2) Unknown Error Code - 10001 explained more below
---------------------------------------------------------------------------------------
'CF_TRACE=true cf push' gives below error -
REQUEST: [2015-08-02T10:40:17Z] GET /v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5 HTTP/1.1 Host: api.cf.ronak.com Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.2-24abed3 / linux
RESPONSE: [2015-08-02T10:40:17Z] HTTP/1.1 200 OK Content-Length: 491 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:40:17 GMT Server: nginx X-Cf-Requestid: f423947e-8512-41f0-7fd1-21ba6b89178d X-Content-Type-Options: nosniff X-Vcap-Request-Id:
aa782bd8-d758-4c65-505c-af71f2bc212d::a0ac2db0-3f3e-4a49-b011-2f9e7f02660b
{ "metadata": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "created_at": "2015-08-02T10:36:11Z", "url": "/v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5" }, "entity": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "status": "failed", "error": "Use of entity>error is deprecated in favor of entity>error_details.", "error_details": { "error_code": "UnknownError", "description": "An unknown error occurred.", "code": 10001 } } } FAILED Error uploading application. An unknown error occurred. FAILED Error uploading application. An unknown error occurred.
________________________________
I found post in which its mentioned - The point was that S3 buckets name (specified by following options: resource_directory_key, app_package_directory_key, droplet_directory_key, and buildpack_directory_key) had dots in their names. Removing dots from this parameters will solve the problem.
I removing dots and replaced with '-' and again did 'bosh deploy' --- I can't find these directories getting created after deployment under S3 ?? is it normal ? I tried cf push anyways to check if its working - but gave me same 'Unkown Error code 10001.
Please help me to solve the problem.
---------------------------------------------------------------------------------------------------
cf logs nodeapp --recent
RESPONSE: [2015-08-02T10:41:51Z] HTTP/1.1 200 OK Content-Length: 4525 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:41:50 GMT Server: nginx X-Cf-Requestid: c5508a55-a9f4-4be0-49a4-23a4f14fb5d3 X-Content-Type-Options: nosniff X-Vcap-Request-Id:
ae35d7fc-659f-4811-4d7c-077a78963951::4bae904f-6903-4a76-9d0e-314098266502
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "41657738-cc7c-4237-ab52-e4f9683e16f4", "url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4", "created_at": "2015-08-02T10:36:10Z", "updated_at": "2015-08-02T10:36:11Z" }, "entity": { "name": "nodeapp", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": {
}, "memory": 1024, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "d2fa841d-17d6-4bae-9400-409966082443", "command": "node main.js", "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/events", "service_bindings_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/routes", "routes": [ { "metadata": { "guid": "2489386f-973b-45db-bc86-412b57bfffb0", "url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0", "created_at": "2015-07-31T06:53:17Z", "updated_at": null }, "entity": { "host": "nodeapp", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0/apps" } } ] } }
] } Connected, dumping recent logs for app nodeapp in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
----------------------------------------------------------------------------------------------------------
Please go through all my above research and let me know what can be done to resolve this issue??
Thanks Ronak _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
|
|
Re: CF push error - Not able to resolve - Error dialing loggregator server: Get https://loggregator.systemdomain.com
Ronak Agarwal <roagarwa@...>
Thanks Joseph. I am not clear at one point - I already have a NAT vm which I had just stopped and again started, so are you asking me to create a new NAT vm because I have stopped it earlier? because it maintains some persistent data. I just tried pushing a Java app because at api_z1, I can't find Java buildpack error, but I am getting - RESPONSE: [2015-08-03T10:13:42Z] HTTP/1.1 504 GATEWAY_TIMEOUT Content-Length: 0 Connection: keep-alive FAILED Error uploading application. Server error, status code: 504, error code: 0, message: FAILED Error uploading application. Server error, status code: 504, error code: 0, message: I checked api_z1 logs and couldn't find any trace for this error, is it moved to Cloud_Controller_Worker ? What could be the error? I have good disk space available in api_z1. cf logs appname --recent , gives below - RESPONSE: [2015-08-03T10:19:07Z] HTTP/1.1 200 OK Content-Length: 6200 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Mon, 03 Aug 2015 10:19:05 GMT Server: nginx X-Cf-Requestid: 2a67e173-20bf-4f89-7b27-efd5b28461c6 X-Content-Type-Options: nosniff X-Vcap-Request-Id: f7b15702-3650-4258-5b40-48efbd9ef271::230c52e3-52a7-4a81-82f4-4e1ed445082d { "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "2ac4b290-fb0d-4273-940e-b2d1aeb31320", "url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320", "created_at": "2015-08-03T09:26:19Z", "updated_at": "2015-08-03T10:12:40Z" }, "entity": { "name": "hello-spring-cloud", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": { "SPRING_PROFILES_DEFAULT": "cloud" }, "memory": 512, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "3719a87d-98b8-4641-a613-c9730f132b5d", "command": null, "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/events", "service_bindings_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/service_bindings", "service_bindings": [ ], "routes_url": "/v2/apps/2ac4b290-fb0d-4273-940e-b2d1aeb31320/routes", "routes": [ { "metadata": { "guid": "f44c4856-08c1-49f2-8e1b-a51e548adca6", "url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6", "created_at": "2015-08-03T09:27:16Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unabsorbent-masculinization", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f44c4856-08c1-49f2-8e1b-a51e548adca6/apps" } }, { "metadata": { "guid": "47a495d6-a484-48f0-ac12-6157282be1c4", "url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4", "created_at": "2015-08-03T09:58:03Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-unascertainable-unmelodiousness", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/47a495d6-a484-48f0-ac12-6157282be1c4/apps" } }, { "metadata": { "guid": "f51fad6c-fd48-4dfb-b201-a756af14bd00", "url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00", "created_at": "2015-08-03T10:12:40Z", "updated_at": null }, "entity": { "host": "hello-spring-cloud-completive-hesperinos", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/f51fad6c-fd48-4dfb-b201-a756af14bd00/apps" } } ] } } ] } Connected, dumping recent logs for app hello-spring-cloud in org ronak / space ronak as admin... FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320:EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get https://loggregator.cf.ronak.com:4443/recent?app=2ac4b290-fb0d-4273-940e-b2d1aeb31320:EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
toggle quoted messageShow quoted text
On Mon, Aug 3, 2015 at 3:33 PM, CF Runtime <cfruntime(a)gmail.com> wrote: Yeah, without the NAT vm nothing will be able to get out to the internet. They will be able to respond to external requests, such as a request to the api, or the login server, or an individual app. But if anything on the inside tries to initiate a network connection, it has no route to the internet.
Amazon has good generic documentation on how to create a NAT instance: http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_NAT_Instance.html#NATInstance
BOSH also has some documentation, although it doesn't tell you how to create the instance itself, but rather how to configure your private subnet and routing table to use the NAT instance: http://bosh.io/docs/setup-aws.html#nat
In both sets of documentation, you may already have security groups, routes and subnets created from your previous NAT instance.
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:50 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
I tried 'curl http://google.com' from my api_z1 instance but was unable to get any response.
I create my NAT vm through 'bosh aws create' command, but by mistake I stopped that VM on AWS. So does stopping NAT VM is causing this issue ? It has any persistent data while setting NAT VM ?
If its because of stopping NAT Vm, so now what is the solution, do I need to destroy everything and create entire environment again ? Please suggest.
On Mon, Aug 3, 2015 at 2:53 PM, CF Runtime <cfruntime(a)gmail.com> wrote:
It looks like the api instance is unable to upload the buildpacks to the blobstore. Perhaps the NAT (network NAT, not message bus NATS) instance is not configured correctly.
Are you able to curl an external URL from the api instance?
Joseph CF Release Integration Team
On Mon, Aug 3, 2015 at 2:12 AM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Its a fresh install.
I found few logs may be helpful to resolve this issue --
I found few logs on cloud_controller_ng as below -
Nodejs buildpack install fail :- {"timestamp":1438583731.044699,"message":"Buildpack nodejs_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583731.045526,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\", ------------------------------------------ -> Fails at nodejs_buildpack-cached-v1.4.0.zip
"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583731.0496318,"message":"(0.001478s) UPDATE delayed_jobs SET guid = '82d9ebe1-7af0-4671-89a5-30146a5bb554', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: nodejs_buildpack\n file:
\\"/var/vcap/packages/buildpack_nodejs/nodejs_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n', last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:31:30',failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 971) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
go_buildpack failed to install
{"timestamp":1438583970.3534014,"message":"Buildpack go_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583970.3541129,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\"
________________________________
->go_buildpack-cached-v1.4.0.zip ,
"log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"} {"timestamp":1438583970.358118,"message":"(0.001543s) UPDATE delayed_jobs SET guid = 'd559d276-c5a2-471c-aca1-ba25f4e9cd97', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: go_buildpack\n file: \\"/var/vcap/packages/buildpack_go/go_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:35:29', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 972) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
python_buildpack failed to install
{"timestamp":1438583971.811825,"message":"Buildpack python_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"} {"timestamp":1438583971.8125288,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in rescue in block in connect'\",
\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:114:inblock in connect'\",
________________________________
->python_buildpack-cached-v1.4.0.zip
,"log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/exception_catching_job.rb","lineno":24,"method":"log_error"}
{"timestamp":1438583971.8163912,"message":"(0.001414s) UPDATE delayed_jobs SET guid = '37f04f8f-2e1e-4096-8f5f-c10d8180a9fc', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0,handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: python_buildpack\n file:
\\"/var/vcap/packages/buildpack_python/python_buildpack-cached-v1.4.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26',locked_at = '2015-08-03 06:35:31', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 973) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":7030
php_buildpack failed to install
{"timestamp":1438584211.920127,"message":"Buildpack php_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->php_buildpack-cached-v3.3.0.zip
{"timestamp":1438584211.9250205,"message":"(0.001442s) UPDATE delayed_jobs SET guid = '133a41bd-307f-4e2b-b70e-635f3dec60c4', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: php_buildpack\n file:
\\"/var/vcap/packages/buildpack_php/php_buildpack-cached-v3.3.0.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:30', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 974) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,
________________________________
binary_buildpack failed to install
{"timestamp":1438584212.2200441,"message":"Buildpack binary_buildpack failed to install or update. Error:
#","log_level":"error","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":4360,"file":"/var/vcap/data/packages/cloud_controller_ng/a60643ffa860f4f065c25b6b99687b9d55067757.1-0e8fbd36253d4107c466281136ad267d0660bc51/cloud_controller_ng/app/jobs/runtime/buildpack_installer.rb","lineno":38,"method":"rescue in perform"}
________________________________
->binary_buildpack-cached-v1.0.1.zip
{"timestamp":1438584212.2245483,"message":"(0.001381s) UPDATE delayed_jobs SET guid = '7ea6156e-024c-4f2d-b574-ca9b218470cd', created_at = '2015-08-03 06:27:26', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::BuildpackInstaller\n name: binary_buildpack\n file:
\\"/var/vcap/packages/buildpack_binary/binary_buildpack-cached-v1.0.1.zip\\"\n opts: {}\n request_id: \n',last_error = NULL, run_at = '2015-08-03 06:27:26', locked_at = '2015-08-03 06:39:31', failed_at = NULL, locked_by = 'cc_api_worker.api_z1.0.1', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 975) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70305412338480,"fiber_id":70305448717440,"process_id":436
________________________________
This seems to be timeout issue.
{"timestamp":1438585142.846115,"message":"Request failed: 500: {\"code\"=>10001, \"description\"=>\"connect timeout reached\", \"error_code\"=>\"CF-Timeout\",
\"backtrace\"=>[\"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/excon-0.44.1/lib/excon/socket.rb:135:in `rescue in block in connect'
{"timestamp":1438585142.851403,"message":"(0.001378s) UPDATE delayed_jobs SET guid = '6e4167d9-c7ee-4425-9049-e50971864a64', created_at = '2015-08-03 06:55:01', updated_at = CURRENT_TIMESTAMP, priority = 0, attempts = 0, handler = '---
!ruby/object:VCAP::CloudController::Jobs::ExceptionCatchingJob\nhandler: !ruby/object:VCAP::CloudController::Jobs::RequestJob\n handler: !ruby/object:VCAP::CloudController::Jobs::TimeoutJob\n handler: !ruby/object:VCAP::CloudController::Jobs::Runtime::AppBitsPacker\n app_guid: 41657738-cc7c-4237-ab52-e4f9683e16f4\n uploaded_compressed_path: \\"/var/vcap/data/cloud_controller_ng/tmp/uploads/0000000001\\"\n fingerprints: []\n request_id:
64cc342c-9db9-495e-46d9-ce5e9331aa41::d6c111c3-79b1-4fec-8fa9-0d4be759bdd9\n', last_error = NULL, run_at = '2015-08-03 06:55:01', locked_at = '2015-08-03 06:55:02', failed_at = NULL,locked_by = 'cc_api_worker.api_z1.0.2', queue = 'cc-api_z1-0', cf_api_error = '---\nerror_code: UnknownError\ndescription: An unknown error occurred.\ncode: 10001\n' WHERE (id = 989) LIMIT
1","log_level":"debug2","source":"cc.background","data":{},"thread_id":70006300463920,"fiber_id":70006336842960,"process_id":4397,"file":"/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.1.0/gems/sequel-4.21.0/lib/sequel/database/logging.rb","lineno":70,"method":"block in log_each"}
________________________________
I will try to push Java app to verify if its getting pushed.
On Mon, Aug 3, 2015 at 2:19 PM, Gwenn Etourneau <getourneau(a)pivotal.io> wrote:
Btw CF-213 is a fresh install or update ??
On Mon, Aug 3, 2015 at 5:37 PM, Ronak Agarwal <roagarwa(a)tibco.com> wrote:
Hi,
Please go through all my instance info below and let me know what can be done to resolve this issue??
I have deployed CF-213.yml release through microbosh on AWS. I have installed latest CF CLI v6.12.2 I am able to do cf login using below endpoints - /.cf/config.json "Target": "https://api.cf.ronak.com", "ApiVersion": "2.33.0", "AuthorizationEndpoint": "https://login.cf.ronak.com", "LoggregatorEndPoint": "wss://loggregator.cf.ronak.com:4443", "DopplerEndPoint": "wss://doppler.cf.ronak.com:4443", "UaaEndpoint": "https://uaa.cf.ronak.com",
Now when I am trying to push Nodejs app using - cf push nodeapp -c "node main.js" -m 128M -b node_buildpack OR cf push nodeapp -c "node main.js"
both are failing .. gives unknown error. When I do -> cf logs nodeapp --recent It gives below error - FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=05ea8bda-28c1-49ed-8c65-4625c77423be: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
--------------------------------------------------------------------------------
I am using ELB and my HA-proxy job is not running. Please let me know how to overcome this challenge. I found lot of people have faced it but all of them I think are using HA-proxy and not ELB.
I have configured 4443 with SSL on ELB -
80 (HTTP) forwarding to 80 (HTTP), 443 (HTTPS, Certificate: cfrouter_cert) forwarding to 80 (HTTP), 4443 (SSL, Certificate: cfrouter_cert) forwarding to 80 (TCP)
----------------------------------------------------------------------------------
Below are the vms/jobs which are running on AWS I dont see login_z1/z2 in them, so do I need these jobs?
Job/index | State | Resource Pool | IPs |
+------------------------------------+---------+---------------+--------------+ | api_worker_z1/0 | running | small_z1 | 10.10.17.3 | | api_worker_z2/0 | running | small_z2 | 10.10.81.0 | | api_z1/0 | running | large_z1 | 10.10.17.1 | | api_z2/0 | running | large_z2 | 10.10.80.255 | | clock_global/0 | running | medium_z1 | 10.10.17.2 | | doppler_z1/0 | running | medium_z1 | 10.10.17.6 | | doppler_z2/0 | running | medium_z2 | 10.10.81.3 | | etcd_z1/0 | running | medium_z1 | 10.10.16.20 | | etcd_z1/1 | running | medium_z1 | 10.10.16.35 | | etcd_z2/0 | running | medium_z2 | 10.10.80.19 | | hm9000_z1/0 | running | medium_z1 | 10.10.17.4 | | hm9000_z2/0 | running | medium_z2 | 10.10.81.1 | | loggregator_trafficcontroller_z1/0 | running | small_z1 | 10.10.17.7 | | loggregator_trafficcontroller_z2/0 | running | small_z2 | 10.10.81.4 | | nats_z1/0 | running | medium_z1 | 10.10.16.11 | | nats_z2/0 | running | medium_z2 | 10.10.80.11 | | nfs_z1/0 | running | medium_z1 | 10.10.16.36 | | router_z1/0 | running | router_z1 | 10.10.16.15 | | router_z2/0 | running | router_z2 | 10.10.80.15 | | runner_z1/0 | running | runner_z1 | 10.10.17.5 | | runner_z2/0 | running | runner_z2 | 10.10.81.2 | | stats_z1/0 | running | small_z1 | 10.10.16.255 | | uaa_z1/0 | running | medium_z1 | 10.10.17.0 | | uaa_z2/0 | running | medium_z2 | 10.10.80.254 |
------------------------------------------------------------------------------------------------------
openssl s_client -connect api.cf.ronak.com:443 CONNECTED(00000003) depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com verify error:num=18:self signed certificate verify return:1 depth=0 C = US, O = Pivotal, CN = *.cf.ronak.com
verify return:1
Certificate chain 0 s:/C=US/O=Pivotal/CN=*.cf.ronak.com
i:/C=US/O=Pivotal/CN=*.cf.ronak.com
Server certificate -----BEGIN CERTIFICATE----- MIIC6TCCAdGgAwIBAgIBADANBgkqhkiG9w0BAQUFADA4MQswCQYDVQQGEwJVUzEQ MA4GA1UECgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wHhcNMTUw NzE0MTAxNDA1WhcNMTgwNzE0MTAxNDA1WjA4MQswCQYDVQQGEwJVUzEQMA4GA1UE CgwHUGl2b3RhbDEXMBUGA1UEAwwOKi5jZi5yb25hay5jb20wggEiMA0GCSqGSIb3 DQEBAQUAA4IBDwAwggEKAoIBAQDi2+NVSsR7/oCh40H+CfFWUS0BczyTubpV72Ir 6TCkjLitLdFgJFqNvaIcT0+2+rozxQs7Zr/IlH5OznYsFYNyOSx2l/xDKCqA7q4U l6aGhFZSodIow6AKCRzVt2y5T/nmeRCWsVPjHMvANvJlM57PdWI1vIlaav3l3enF FUs14MSEuXY+eWfeP74Sprg/Xc5TiD5RQPLZC8h2HeAmIMgTV5TAagWiGK5MUoQ9 hScJvsgBPKQuldRfRsk7NxuXmY+wdfOYKc5CZ7szl7QiLsT7Tdeei+UuhIAh874S HjPP+JlFH4/whRJyDtv/h9rZzW1DJbx5ZeUtzFGv0jxmsvJrAgMBAAEwDQYJKoZI hvcNAQEFBQADggEBAM7S9za5V2sJ8Gwb4QQ29F4bQxVNk5/u7AnbzvfmhcWgO9yt oBUwtc04F/Ycz7sTlXXDe0f+nSo+Q0vowd4NmTSont5g8ZznD18k4krCcfvXCsAL AvJ/yt2n+fW/gwZbZHS9kb4Esxu4SmjU8GFN10+DMRFaB9JkbRpJmRLsxZzE8w+E MW7gi0V+IycOLyeJsfwXtjGZhKsLSq6fg/mwVax9Z0MsqVRbzO61RaJjMDiop+aN kf/ktqGVBACwqvG7cxKu4/2cFMzi6dC9xEN7o5spinfdNt/uH3O0JHRcftUCICaL ln6RtW7XXHNWnP2YfMXxPLvhjV/Cg49soz3D+os= -----END CERTIFICATE----- subject=/C=US/O=Pivotal/CN=*.cf.ronak.com
issuer=/C=US/O=Pivotal/CN=*.cf.ronak.com
No client certificate CA names sent
SSL handshake has read 1403 bytes and written 421 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: B845CA20842317DB12FB250C05D40686334A7C85D8BE2E5332B6C70632BF9256 Session-ID-ctx: Master-Key:
3934F0F361C1A407B215D869F8142F3B3180BB8CD232D55D937430CDD3674F047A774594B6A3BA9DB7275D1E507AE0B5 Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - 1b 21 b6 45 6b 48 d6 c5-6d 66 e4 97 58 ae 76 91 .!.EkH..mf..X.v. 0010 - 0c 85 3f 3b f8 79 3d 59-b5 55 57 bd a7 98 a2 90 ..?;.y=Y.UW..... 0020 - 20 81 2b 0b 8a 6a 13 8c-1e e7 c4 d2 f7 d1 f6 5b .+..j.........[ 0030 - c7 d6 58 16 42 89 1e 51-01 7d 20 24 3f fa 2e f7 ..X.B..Q.} $?... 0040 - 76 06 d5 61 ff 4f 5c a1-46 77 41 62 fe fc c9 50 v..a.O.FwAb...P 0050 - 55 b6 00 98 15 22 8a 81-e2 e0 0e 9d da 83 fc 7b U....".........{ 0060 - ef 33 16 ec 88 60 ca 38-3f 35 85 6d fe 68 26 f6 .3...`.8?5.m.h&. 0070 - 24 48 65 47 8b c4 f6 20-f9 31 63 74 56 1a 67 d5 $HeG... .1ctV.g. 0080 - 26 d1 4c 0d fa 5f 01 da-ec d9 a6 88 9d ab cd fa &.L.._.......... 0090 - 05 17 bd e4 b4 78 f4 02-6b 5e 86 83 04 b7 f2 33 .....x..k^.....3
Start Time: 1438453185 Timeout : 300 (sec) Verify return code: 18 (self signed certificate)
------------------------------------------------------------------------------------------------------------------------
I am using self signed certificate.. so if I have to continue without --skip-ssl-validation, I should add this ELB certificate in all the VMS as trusted keys. Is it correct ? OR anyother option ?
--------------------------------------------------------------------------------------------------------------
In CF-AWS-Stub.yml I found two - a) jwt: signing_key & verification_key b) router . ssl_cert & ssl_key Both should be same as AWS ELB ssl certificate ??
------------------------------------------------------------------------------------------
I think core problem is cf push error which I mentioned above -
1) I can see in Response - "detected_buildpack": null, I am not sure why its null because I am uploading system buildpack app which of Nodejs and when I do - cf buildpacks I can see 4 buildpacks below -
buildpack position enabled locked filename staticfile_buildpack 1 true false java_buildpack 2 true false node_buildpack 3 true false php_buildpack 6 true false
I am not sure why filename is null ??
2) Unknown Error Code - 10001 explained more below
---------------------------------------------------------------------------------------
'CF_TRACE=true cf push' gives below error -
REQUEST: [2015-08-02T10:40:17Z] GET /v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5 HTTP/1.1 Host: api.cf.ronak.com Accept: application/json Authorization: [PRIVATE DATA HIDDEN] Content-Type: application/json User-Agent: go-cli 6.12.2-24abed3 / linux
RESPONSE: [2015-08-02T10:40:17Z] HTTP/1.1 200 OK Content-Length: 491 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:40:17 GMT Server: nginx X-Cf-Requestid: f423947e-8512-41f0-7fd1-21ba6b89178d X-Content-Type-Options: nosniff X-Vcap-Request-Id:
aa782bd8-d758-4c65-505c-af71f2bc212d::a0ac2db0-3f3e-4a49-b011-2f9e7f02660b
{ "metadata": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "created_at": "2015-08-02T10:36:11Z", "url": "/v2/jobs/fd682f50-c857-40dc-a190-abc0f59492c5" }, "entity": { "guid": "fd682f50-c857-40dc-a190-abc0f59492c5", "status": "failed", "error": "Use of entity>error is deprecated in favor of entity>error_details.", "error_details": { "error_code": "UnknownError", "description": "An unknown error occurred.", "code": 10001 } } } FAILED Error uploading application. An unknown error occurred. FAILED Error uploading application. An unknown error occurred.
________________________________
I found post in which its mentioned - The point was that S3 buckets name (specified by following options: resource_directory_key, app_package_directory_key, droplet_directory_key, and buildpack_directory_key) had dots in their names. Removing dots from this parameters will solve the problem.
I removing dots and replaced with '-' and again did 'bosh deploy' --- I can't find these directories getting created after deployment under S3 ?? is it normal ? I tried cf push anyways to check if its working - but gave me same 'Unkown Error code 10001.
Please help me to solve the problem.
---------------------------------------------------------------------------------------------------
cf logs nodeapp --recent
RESPONSE: [2015-08-02T10:41:51Z] HTTP/1.1 200 OK Content-Length: 4525 Connection: keep-alive Content-Type: application/json;charset=utf-8 Date: Sun, 02 Aug 2015 10:41:50 GMT Server: nginx X-Cf-Requestid: c5508a55-a9f4-4be0-49a4-23a4f14fb5d3 X-Content-Type-Options: nosniff X-Vcap-Request-Id:
ae35d7fc-659f-4811-4d7c-077a78963951::4bae904f-6903-4a76-9d0e-314098266502
{ "total_results": 1, "total_pages": 1, "prev_url": null, "next_url": null, "resources": [ { "metadata": { "guid": "41657738-cc7c-4237-ab52-e4f9683e16f4", "url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4", "created_at": "2015-08-02T10:36:10Z", "updated_at": "2015-08-02T10:36:11Z" }, "entity": { "name": "nodeapp", "production": false, "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "stack_guid": "612874d8-144e-4599-88a7-e38e333ca93e", "buildpack": null, "detected_buildpack": null, "environment_json": {
}, "memory": 1024, "instances": 1, "disk_quota": 1024, "state": "STOPPED", "version": "d2fa841d-17d6-4bae-9400-409966082443", "command": "node main.js", "console": false, "debug": null, "staging_task_id": null, "package_state": "PENDING", "health_check_type": "port", "health_check_timeout": null, "staging_failed_reason": null, "staging_failed_description": null, "diego": false, "docker_image": null, "package_updated_at": null, "detected_start_command": "", "enable_ssh": true, "docker_credentials_json": { "redacted_message": "[PRIVATE DATA HIDDEN]" }, "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "space": { "metadata": { "guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "created_at": "2015-07-30T09:08:36Z", "updated_at": null }, "entity": { "name": "ronak", "organization_guid": "66e33d43-92fa-4f30-809d-1f5d9adaf921", "space_quota_definition_guid": null, "allow_ssh": true, "organization_url": "/v2/organizations/66e33d43-92fa-4f30-809d-1f5d9adaf921", "developers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/developers", "managers_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/managers", "auditors_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/auditors", "apps_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/apps", "routes_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/routes", "domains_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/domains", "service_instances_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/service_instances", "app_events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/app_events", "events_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/events", "security_groups_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89/security_groups" } }, "stack_url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "stack": { "metadata": { "guid": "612874d8-144e-4599-88a7-e38e333ca93e", "url": "/v2/stacks/612874d8-144e-4599-88a7-e38e333ca93e", "created_at": "2015-07-23T12:10:53Z", "updated_at": null }, "entity": { "name": "cflinuxfs2", "description": "Cloud Foundry Linux-based filesystem" } }, "events_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/events", "service_bindings_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/service_bindings", "service_bindings": [
], "routes_url": "/v2/apps/41657738-cc7c-4237-ab52-e4f9683e16f4/routes", "routes": [ { "metadata": { "guid": "2489386f-973b-45db-bc86-412b57bfffb0", "url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0", "created_at": "2015-07-31T06:53:17Z", "updated_at": null }, "entity": { "host": "nodeapp", "path": "", "domain_guid": "c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_guid": "6ffcaf95-f30b-41b6-88ce-d67f59584e89", "domain_url": "/v2/domains/c3c27b28-9fef-4130-8942-41dbeb8220a0", "space_url": "/v2/spaces/6ffcaf95-f30b-41b6-88ce-d67f59584e89", "apps_url": "/v2/routes/2489386f-973b-45db-bc86-412b57bfffb0/apps" } } ] } }
] } Connected, dumping recent logs for app nodeapp in org ronak / space ronak as admin...
FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443). FAILED Error dialing loggregator server: Get
https://loggregator.cf.ronak.com:4443/recent?app=41657738-cc7c-4237-ab52-e4f9683e16f4: EOF. Please ask your Cloud Foundry Operator to check the platform configuration (loggregator endpoint is wss://loggregator.cf.ronak.com:4443).
----------------------------------------------------------------------------------------------------------
Please go through all my above research and let me know what can be done to resolve this issue??
Thanks Ronak _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com _______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
_______________________________________________ cf-dev mailing list cf-dev(a)lists.cloudfoundry.org https://lists.cloudfoundry.org/mailman/listinfo/cf-dev
-- Regards, Ronak Agarwal (M) +91 8446044994 (O) +91 20 4150 7935 roagarwa(a)tibco.com
|
|