Task

[bjornbytes]

This is an experiment to add a coroutine-based task system to LÖVR, with the main goal of making it easier for projects to scale to multiple CPU cores and run expensive work in the background.

Design

The basic idea is to create lots of tasks (coroutines) that are each running independent work. Lots of tasks can all run more or less at the same time, cooperatively.

Normally, Lua coroutines are still single threaded, and don't allow for parallelism. However, LÖVR can achieve parallelism with them by running asynchronous functions.

When a coroutine calls an asynchronous function, it will start work on a thread pool, yield itself, and wake up automatically once the work is complete. While the coroutine is yielded, other coroutines get a chance to run.

In this way, many coroutines can all run work "at the same time", with all of them doing work on a thread pool, making use of all of the CPU cores.

API

Everything is in a new module, lovr.task:

success, results = lovr.task.resume(co, ...) --> starts/resumes a task
lovr.task.poll() --> iterate over ready tasks (for co in lovr.task.poll() do ... end)
lovr.task.wait(...tasks) --> wait for tasks to finish.  returns results/error
lovr.task.isWaiting(task) --> check if a task can be resumed, or its its work is still pending

-- New callback: called when a task is ready
-- Default behavior is to just resume the task and propagate errors,
-- but you can override it to implement custom scheduling/error behavior
-- boot.lua polls tasks before lovr.update automatically
function lovr.taskready(task)
  assert(lovr.task.resume(task))
end

Notes:

• Create a task with coroutine.create. Note that task functions share the global scope with the caller and other tasks. You don't have to use Channel to communicate with the task.
• success, ...results = lovr.task.wait(...tasks). Blocks until all the tasks are complete, returning their results or the first error that was encountered.
  • Handling multiple return values: Results are collapsed similar to Lua's existing convention: All tasks return their first return value, except for the last task, which returns all of its results. So if you have 3 tasks that return 1, 2, 3, lovr.task.wait(a, b, c) returns 1, 1, 1, 2, 3.
  • Note that this is an asynchronous function! This means:
    • If you call it from within a task, it will yield, and become ready once the tasks are complete.
    • If you call from outside a task, this function will block, running tasks and jobs until all the tasks finish.
  • Note that if you wait on a task, it will not show up in lovr.taskready anymore, because it's already finished. You can choose to "fire and forget" a task and let lovr.taskready take care of running it on a future frame, or explicitly wait on the task if you need its results sooner.
• success, ...results = lovr.task.resume(task, ...args). Resumes a task, if possible, otherwise returns false and an error.
  • You'll get an error if the task is already finished, has an error, or isn't ready to run yet.
  • If the task is resuming after a call to an asynchronous function, args are ignored and the results from the async operation are given to the task instead.
  • If the task ran successfully and yielded/finished, success will be true and you'll get any yield/return values.
  • Note that if you resume a task manually, it won't show up in lovr.taskready, since you already ran it.

Asynchronous Functions

Asynchronous functions are where the magic happens. Asynchronous functions behave differently, depending on whether they are called inside of a Task or not:

• When run outside a task, the asynchronous function behaves in a blocking manner, exactly how it would today.
• When called from inside a task, asynchronous functions will yield the task immediately, running the work in the background. When the work finishes, the task will become ready, and it will show up in lovr.taskready on the next frame (unless it gets run sooner than that).

Any LÖVR function can become asynchronous. Let's take lovr.data.newImage, which I've been using as a test async function on this branch:

image = lovr.data.newImage('file.png')

Even though this is an async function, it behaves exactly the same as it does today, and you don't need to change any code. But if you run it on a task, it yields the task:

task = coroutine.create(function()
  return lovr.data.newImage('file.png')
end)

-- runs the task.  it calls newImage and yields
lovr.task.resume(task)

print(lovr.task.isWaiting(task)) --> true, still loading image

-- wait until image is loaded
local image = lovr.task.wait(task)

By itself this isn't interesting. But there are interesting things you can do with asynchronicity and parallelism.

Here's an example that loads an image in the background:

function lovr.load()
  local task = coroutine.create(function()
    local image = lovr.data.newImage('file.png')
    texture = lovr.graphics.newTexture(image)
    loaded = true
  end)

  -- Note: task will automatically resume again once it becomes ready, via lovr.taskready
  lovr.task.resume(task)
end

function lovr.draw(pass)
  if loaded then
    pass:draw(texture, 0, 2, -3)
  else
    pass:text('Loading...', 0, 2, -3)
  end
end

Here's an example that loads 100 images on multiple threads:

local tasks = {}
for i = 1, 100 do
  local task = coroutine.create(function(file)
    return lovr.data.newImage(file)
  end)
  lovr.task.resume(task, 'file' .. i .. '.png')
  table.insert(tasks, task)
end
local images = lovr.task.wait(tasks)

Discussion

• Why isn't taskready just a regular event?
  • Had to remove tasks from queue if they are resumed/waited before being consumed. Felt weird to remove events like this.
• Which functions should/could be asynchronous?
  • Going to start with a small set of async methods, but more can be added in the future:
    • lovr.data.newSound / lovr.audio.newSource
    • lovr.data.newImage / lovr.graphics.newTexture
    • lovr.data.newModelData / lovr.graphics.newModel / lovr.headset.newModel
    • lovr.timer.sleep
    • Buffer:getData and Texture:getPixels (theoretically obviates Readback)
    • Channel:push and Channel:pop
    • lovr.task.wait
    • lovr.thread.run -- note that this can be used to make custom async functions, or even make an async version of an existing LÖVR function
• Will Thread continue to exist?
  • Yes, but it will be much more niche.

TODO

• Make .wait work with tables too
• Add more async functions
• Consider adding a way to sleep a task, like lovr.task.sleep (async, wakes up task later, could cause problems if you want the wait time to be based on dt?) Maybe just lovr.timer.sleep is async
• Add an async function that runs Lua code on the thread pool
• Add a parallel-for helper (potentially both task-based and thread-based)
• Profiling/testing
• Add a way of getting the underlying coroutine for a task? Could be useful for using the debug library with the task to inspect its state.

[shakesoda]

for my use cases i tend to want a parallel for (often 2d/3d, too), fwiw

[bjornbytes]

There will be a (async) function to run Lua code on a worker thread, and likely a parallel-for helper on top of this. I haven't figured out the specific API for the parallel-for function yet though.

[bjornbytes]

Decided to change the API back to plain coroutines instead of Task objects. I think it's a little easier to understand (it's more clear that tasks are just coroutines), interops better with other libraries, and has slightly less overhead. Semantics are pretty much exactly the same.

Also, added a way to wait on a polled condition instead of a job. This is good for IO bound stuff like sleeps, readbacks, async IO, condition variables, etc. As a test, I made lovr.timer.sleep async.