I have cloud foundry deployed on azure machines using bosh. For adding another node to the cluster I made changes in the cloud foundry deployment manifest file and redeployed cloud foundry using 'bosh deploy'.
The deployment failed in-between but bosh deployment lock didn't get released. When I do 'bosh locks' I see the lock is acquired by deployment and the expiring time is slightly more than current time and it keeps increasing endlessly.
bosh locks Acting as user 'admin' on 'bosh'
+------------+-----------------------+-------------------------+ | Type | Resource | Expires at | +------------+-----------------------+-------------------------+ | deployment | single-vm-cf-on-azure | 2017-05-23 10:27:59 UTC | +------------+-----------------------+-------------------------+ I tried cancelling the deployment task (bosh cancel task #task-number). The status of deployment task changed to 'cancelling' but it didn't get cancelled.
bosh tasks Acting as user 'admin' on 'bosh'
+----+------------+-------------------------+-------+-------------------+--------+ | # | State | Timestamp | User | Description | Result | +----+------------+-------------------------+-------+-------------------+--------+ | 38 | cancelling | 2017-05-23 08:40:12 UTC | admin | create deployment | | +----+------------+-------------------------+-------+-------------------+--------+ The issue I am facing is bosh has acquired deployment lock and everytime I try to start deployment again or try to delete deployment I get an error saying:
Error 100: Unable to get deployment lock, maybe a deployment is in progress. Try again later. 1. Can I delete the acquired deployment lock information to release the lock? If yes, where is the information stored and how to delete it?
2. If a task (example, bosh deploy) fails does it keep an infinite hold on the bosh locks? Is there a way of handling task failure gracefully?
3. How to do "bosh deploy" with changes in deployment manifest file to avoid getting into infinite deployment lock acquired situation?