Hi all,

we at SAP are very interested in the new task feature, because we currently run some dummy/fake apps, in order to simulate task-like behavior, which includes some cumbersome app log parsing. The promises of the tasks are exactly what we are looking for, since the starter of the task would just like to see the result of the task action in some convenient and easy to consume way.

Unfortunately we currently know of a number of older and newer resources (URLs) with information on the task topic, which do not give much information on the details of the result handling and seem to even be slightly contradicting in some low level details.

As a starting point, it would be good to know the most recent official documentation on the tasks (for the task implementor and consumer).

To shortly summarize, our most important questions from task implementation and consumption perspective are:

1) How does the task implementation provide its result - either through process exit code and stderr/out or via tracing (+ optional/internal result file)? Is there some definite contract or at least best-practice for the task implementation side?

2) How does the infrastructure collect the task result and especially how is it exposed to the consumer? In the V3 API docu there’s just a result JSON object, with only one “failure_reason” property. We’d also expect the process exit code and maybe stdout/err to be part of the result JSON. Or in case the (optional) ResultFile and callback URL are the way to go, there should be some env var exposed to the task implementation, so that one knows to which file the result data shall be written.

3) It seems like the result from the V3 REST API is not exposed via the V3 tasks client command (at least in its current state)?

4) Is the “Task Completion Callback URL” mechanism that is mentioned in several sources only intended for internal use by client implementations, or also for external consumers?

It would also be interesting, when the tasks will be ready to use.

In the talk on V3 given on European CF Summit (in Sept. in Frankfurt) the following statement on tasks was given ([1] – slides 46/47):
“When? Very soon (~ next few weeks)”

I’ll add some more detailed observations about the potential result handing as a second post.

Thanks a lot and kind regards,
Andreas Mueller – SAP SE

[1] http://schd.ws/hosted_files/cfsummiteu2016/2a/Unleasing%20the%20Power%20of%20the%20V3%20API_Frankfurt.pdf

Here are the details, mentioned before:

[1]

- This document describes the very low level technical/internal details of the CF task lifecycle, i.e. it does not necessarily match with the V3 API for consumption (e.g. the internal task states are PENDING -> CLAIMED -> RUNNING -> COMPLETED -> RESOLVING, while the states in V3 are PENDING, RUNNING, SUCCEEDED, CANCELING, and FAILED).

- The document is quite interesting, but it not intended as end-user documentation, as it deals with the intrinsic mechanisms, which guarantee the at “most once” execution guarantee of tasks (via BBs, etcd's atomic and consistent CompareAndSwap, claiming and resolving).

- An interesting detail is the fact, that the internal v1 task description still has some (redundant) “failed” Boolean and a “result” field next to the “failure_reason”. “The result is the output of the last FetchResult action.” – whatever this means.

-Also interesting: “If the Task is PENDING: And the Task was created more than 30 minutes ago (this is configurable via –timeToClaimTask in executor): then the task is marked as failed.” -> later documents have more/different information about this aspect


[2]

- This document seems to be some very internal discussion state.

- I found the following non-goals interesting:
a) Specifying a path to a (small) file to store and return result of the task
b) Specifying a custom callback web url to be notified when the task is completed

- And:
a) Only X number of tasks are kept on an app before rolling off older, completed tasks.
b) Stdout/stderr from the task will be available on the apps firehose logs.


[3]

- Very nice and understandable but still internal (BBS) overview document on tasks.

- “When the Task completes, the Cell sets the Failed, FailureReason, and Result fields on the Task as appropriate, and sets the Task's state to COMPLETED.” – “Result field” obviously refers to the internal JSON description of a task. How does that correlate to the (external) Result JSON object in the V3 API?

- “At this point it is up to the Diego client to detect and resolve the completed Task. It can do this either by having set a completion callback URL on the Task when defined, or by polling for the Task and resolving and deleting it itself.” – So the callback URL is only intended for client impls? Seems so, since otherwise there would need to be an array of URLs, if the actual task creator/starter could also provide such an callback URL, in addition. [4] seems to confirm this assumption.

- “If ResultFile was specified and the Task has completed successfully, Result will include the first 10KB of the ResultFile.” – How does this result file relate to the log file of the app/task and how can the task impl write to such an result file? Also, such a result file cannot be specified, when creating a task via V3 API.

- “Diego will automatically delete completed Tasks that remain unresolved after 2 minutes.” – How does that fit to the “Only X number of tasks are kept on an app before rolling off older, completed tasks.” mentioned in some other/previous document?


[5] and [6]

- Seem to be the most recent and official external documentation
- Unclear, how the client/task starter is supposed to retrieve the task result, other than the “failure_message”


[1] https://github.com/cloudfoundry/diego-release/wiki/Diego-Task-Lifecycle (2014/04)
[2] https://docs.google.com/document/d/1CCHDUa2UWRjXkxEdksX4M9BGQ8hBqiMys46wxeF5XE4/edit# (2016/04)
[3] https://github.com/cloudfoundry/bbs/blob/master/doc/tasks.md (2016/06)
[4] https://github.com/cloudfoundry/bbs/blob/master/doc/defining-tasks.md
[5] http://v3-apidocs.cloudfoundry.org/version/release-candidate/#the-task-object
[6] https://github.com/cloudfoundry/v3-cli-plugin/tree/master/commands

Hi Andreas,

This is a lot, but I’ll try my best to help. I encourage you to stop by our
[1] Slack channel anytime for a quick chat.

1. The V3 Task resource should eventually end up in either
SUCCEEDED or FAILED
state. If the state is FAILED, the result.failure_reason will have a human
readable message, e.g. "Exited with status 1”. Task logs will appear in
the log stream of the app. Otherwise, state is left up to the developer.
2. I think this is the same question and answer as #1. If not, maybe
let’s chat in Slack and update this thread.
3. I think you are asking about the “V3 CLI Plugin”. This was only
intended to help developers validate the API, and is definitely not
full-featured. We are working with the CLI team to properly expose a
coherent Tasks UX.
4. The Task Completion Callback URL is an internal Diego Tasks feature
and is not exposed to the enduser.

As for when the Tasks feature will be ready to use, we are working very
closely with the CLI team to expose an MVP of Tasks. Aside from a few minor
changes, the API in CF-245 appears to be supporting this effort.


Thanks,

Nick


[1] https://cloudfoundry.slack.com/messages/capi/

--
Nicholas Calugar
Product Manager - Cloud Foundry API
Pivotal Software, Inc.

Hi Nicholas,

thanks a lot for the quick and helpful response, which is in line with the latetest "release candidate" version of the V3 REST API docu.

So, the app log is the only channel for the task starter to consume the task results and multiple tasks with the same name can be started in parallel on a single app droplet.

Based on these facts I'd assume that the log lines belonging to the task can be easily identified in the app's log, by means of the task name and newly introduced sequenceId.

Could you please elaborate a bit more on the detailed format (source tagging) of log lines produced by an task belonging to an app?

I've found a hint in [1] (at that time there was no sequenceId, yet), but there does not seem to be some official documentation yet, about how exactly log lines produced by an app's task can be identified and grouped together.

Thanks and kind regards,
Andreas

[1] https://www.pivotaltracker.com/n/projects/966314/stories/111472198

Hi Andreas,

My apologies, I didn’t intend to imply that app logs were the only channel
for the task starter to consume the task results. The V3 Task resource will
contain the result in the form of state of SUCCEEDED or FAILED. What I
meant by, "state is left up to the developer”, is the task code should
offload anything that needs persisted from the task run. For example, the
task could stream data to a service, write to a database, write a file to a
blobstore, etc. The platform is just responsible for running the task and
logs shouldn’t be relied upon for state or results.

The log source for tasks are “APP/TASK/<task_name>”. You’ll see something
like “APP/TASK/0E858E94/0” in the log stream. Please note, the 0 is the
instance id which will always be 0 as there is never more than 1 task.

The sequence id was added for the CLI UX and is never sent to the backend.
This allows the CLI to uniquely identify a task within an app without
exposing the GUID or the non-unique Task name.

I think this will all become a lot more clear when we complete the CLI UX,
we are hoping to finish that work in the coming weeks.


Thanks,

Nick

--
Nicholas Calugar
Product Manager - Cloud Foundry API
Pivotal Software, Inc.

Hi Andreas,

If you’re interested in the CLI UX, refer to the stories in the following epic:
https://www.pivotaltracker.com/epic/show/3028607

In this iteration, there is no CLI command to retrieve the logs of certain task – mainly because in the log stream we have no identifier unique to a single task.

Regards,
Dies Koper
Cloud Foundry Product Manager - CLI


From: Nicholas Calugar [mailto:ncalugar(a)pivotal.io]
Sent: Saturday, October 22, 2016 12:12 PM
To: Andreas Mueller; Discussions about Cloud Foundry projects and the system overall.
Subject: [cf-dev] Re: Re: Re: Any news on Tasks in CF Diego?

Hi Andreas,

My apologies, I didn’t intend to imply that app logs were the only channel for the task starter to consume the task results. The V3 Task resource will contain the result in the form of state of SUCCEEDED or FAILED. What I meant by, "state is left up to the developer”, is the task code should offload anything that needs persisted from the task run. For example, the task could stream data to a service, write to a database, write a file to a blobstore, etc. The platform is just responsible for running the task and logs shouldn’t be relied upon for state or results.

The log source for tasks are “APP/TASK/<task_name>”. You’ll see something like “APP/TASK/0E858E94/0” in the log stream. Please note, the 0 is the instance id which will always be 0 as there is never more than 1 task.

The sequence id was added for the CLI UX and is never sent to the backend. This allows the CLI to uniquely identify a task within an app without exposing the GUID or the non-unique Task name.

I think this will all become a lot more clear when we complete the CLI UX, we are hoping to finish that work in the coming weeks.


Thanks,

Nick

--
Nicholas Calugar
Product Manager - Cloud Foundry API
Pivotal Software, Inc.


On October 21, 2016 at 12:55:55 AM, Andreas Mueller (andr.mueller(a)sap.com<mailto:andr.mueller(a)sap.com>) wrote:
Hi Nicholas,

thanks a lot for the quick and helpful response, which is in line with the latetest "release candidate" version of the V3 REST API docu.

So, the app log is the only channel for the task starter to consume the task results and multiple tasks with the same name can be started in parallel on a single app droplet.

Based on these facts I'd assume that the log lines belonging to the task can be easily identified in the app's log, by means of the task name and newly introduced sequenceId.

Could you please elaborate a bit more on the detailed format (source tagging) of log lines produced by an task belonging to an app?

I've found a hint in [1] (at that time there was no sequenceId, yet), but there does not seem to be some official documentation yet, about how exactly log lines produced by an app's task can be identified and grouped together.

Thanks and kind regards,
Andreas

[1] https://www.pivotaltracker.com/n/projects/966314/stories/111472198