Re: Any news on Tasks in CF Diego?


Andreas Mueller
 

Here are the details, mentioned before:

[1]

- This document describes the very low level technical/internal details of the CF task lifecycle, i.e. it does not necessarily match with the V3 API for consumption (e.g. the internal task states are PENDING -> CLAIMED -> RUNNING -> COMPLETED -> RESOLVING, while the states in V3 are PENDING, RUNNING, SUCCEEDED, CANCELING, and FAILED).

- The document is quite interesting, but it not intended as end-user documentation, as it deals with the intrinsic mechanisms, which guarantee the at “most once” execution guarantee of tasks (via BBs, etcd's atomic and consistent CompareAndSwap, claiming and resolving).

- An interesting detail is the fact, that the internal v1 task description still has some (redundant) “failed” Boolean and a “result” field next to the “failure_reason”. “The result is the output of the last FetchResult action.” – whatever this means.

-Also interesting: “If the Task is PENDING: And the Task was created more than 30 minutes ago (this is configurable via –timeToClaimTask in executor): then the task is marked as failed.” -> later documents have more/different information about this aspect


[2]

- This document seems to be some very internal discussion state.

- I found the following non-goals interesting:
a) Specifying a path to a (small) file to store and return result of the task
b) Specifying a custom callback web url to be notified when the task is completed

- And:
a) Only X number of tasks are kept on an app before rolling off older, completed tasks.
b) Stdout/stderr from the task will be available on the apps firehose logs.


[3]

- Very nice and understandable but still internal (BBS) overview document on tasks.

- “When the Task completes, the Cell sets the Failed, FailureReason, and Result fields on the Task as appropriate, and sets the Task's state to COMPLETED.” – “Result field” obviously refers to the internal JSON description of a task. How does that correlate to the (external) Result JSON object in the V3 API?

- “At this point it is up to the Diego client to detect and resolve the completed Task. It can do this either by having set a completion callback URL on the Task when defined, or by polling for the Task and resolving and deleting it itself.” – So the callback URL is only intended for client impls? Seems so, since otherwise there would need to be an array of URLs, if the actual task creator/starter could also provide such an callback URL, in addition. [4] seems to confirm this assumption.

- “If ResultFile was specified and the Task has completed successfully, Result will include the first 10KB of the ResultFile.” – How does this result file relate to the log file of the app/task and how can the task impl write to such an result file? Also, such a result file cannot be specified, when creating a task via V3 API.

- “Diego will automatically delete completed Tasks that remain unresolved after 2 minutes.” – How does that fit to the “Only X number of tasks are kept on an app before rolling off older, completed tasks.” mentioned in some other/previous document?


[5] and [6]

- Seem to be the most recent and official external documentation
- Unclear, how the client/task starter is supposed to retrieve the task result, other than the “failure_message”


[1] https://github.com/cloudfoundry/diego-release/wiki/Diego-Task-Lifecycle (2014/04)
[2] https://docs.google.com/document/d/1CCHDUa2UWRjXkxEdksX4M9BGQ8hBqiMys46wxeF5XE4/edit# (2016/04)
[3] https://github.com/cloudfoundry/bbs/blob/master/doc/tasks.md (2016/06)
[4] https://github.com/cloudfoundry/bbs/blob/master/doc/defining-tasks.md
[5] http://v3-apidocs.cloudfoundry.org/version/release-candidate/#the-task-object
[6] https://github.com/cloudfoundry/v3-cli-plugin/tree/master/commands

Join cf-dev@lists.cloudfoundry.org to automatically receive all group messages.