Wptrunner example

tools/ci/README.md

@@ -0,0 +1,232 @@
+# Taskgraph Setup
+
+The taskgraph is built from a yaml file. This file has two top-level
+properties: `components` and `tasks`. The full list of tasks is
+defined by the `tasks` object; each task is an object with a single
+property representing the task with the corresponding value an object
+representing the task properties. Each task requires the following
+top-level properties:
+
+* `provisionerId`: String. Name of TaskCluster provisioner
+* `schedulerId`: String. Name of TaskCluster scheduler
+* `deadline`: String. Time until the task expires
+* `image`: String. Name of docker image to use for task
+* `maxRunTime`: Number. Maximum time in seconds for which the task can
+  run.
+* `artifacts`: Object. List of artifacts and directories to upload; see
+  TaskCluster documentation.
+* `command`: String. Command to run. This is automatically wrapped in a
+  run_tc command
+* `options`: Optional Object. Options to pass into run_tc
+  - xvfb: Boolean. Enable Xvfb for run
+  - oom-killer: Boolean. Enable xvfb for run
+  - hosts: Boolean. Update hosts file with wpt hosts before run
+  - install-certificates: Boolean. Install wpt certs into OS
+    certificate store for run
+  - browser: List. List of browser names for run
+  - channel: String. Browser channel for run
+* `trigger`: Object. Conditions on which to consider task. One or more
+  of following properties:
+  - branch: List. List of branch names on which to trigger.
+  - pull-request: No value. Trigger for pull request actions
+* `schedule-if`: Optional Object. Conditions on which task should be
+  scheduled given it meets the trigger conditions.
+  - `run-job`: List. Job names for which this task should be considered,
+    matching the output from ./wpt jobs
+* `env`: Optional Object. Environment variables to set when running task.
+* `require`: Optional list. List of task names that must be complete
+  before the current task is scheduled.
+* `description`: String. Task description.
+* `name`: Optional String. Name to use for the task overriding the
+  property name. This is usful in combination with substitutions
+  described below.
+
+## Task Expansions
+
+Using the above syntax it's possble to describe each task
+directly. But typically in a taskgraph there are many common
+properties between tasks so it's tedious and error prone to repeat
+information that's common to multiple tasks. Therefore the taskgraph
+format provides several mechanisms to reuse partial task definitions
+across multiple tasks.
+
+### Components
+
+The other top-level property in the taskgraph format is
+`components`. The value of this property is an object containing named
+partial task definitions. Each task definition may contain a property called
+`use` which is a list of components to use as the basis for the task
+definition. The components list is evaluated in-order. If a property
+is not previously defined in the output it is added to the output. If
+it was previously defined, the value is updated according to the type:
+ * Strings and numbers are replaced with a new value
+ * Lists are extended with the additional values
+ * Objects are updated recursively following the above rules
+This means that types must always match between components and the
+final value.
+
+For example
+```
+components:
+  example-1:
+    list_prop:
+      - first
+      - second
+    object_prop:
+      key1: value1
+      key2: base_value
+  example-2:
+    list_prop:
+      - third
+      - fourth
+    object_prop:
+      key3:
+        - value3-1
+
+tasks:
+  - example-task:
+      use:
+        - example-1
+        - example-2
+      object_prop:
+        key2: value2
+        key3:
+         - value3-2
+```
+
+will evaluate to the following task:
+
+```
+example-task:
+  list_prop:
+    - first
+    - second
+    - third
+    - fourth
+  object_prop:
+    key1: value1
+    key2: value2
+    key3:
+      - value3-1
+      - value3-2
+```
+
+Note that components cannot currently define `use` properties of their own.
+
+## Substitutions
+
+Components and tasks can define a property `vars` that holds variables
+which are later substituted into the task definition using the syntax
+`${vars.property-name}`. For example:
+
+```
+components:
+  generic-component:
+    prop: ${vars.value}
+
+tasks:
+  - first:
+      use:
+        - generic-component
+      vars:
+        value: value1
+  - second:
+      use:
+        - generic-component
+      vars:
+        value: value2
+```
+
+Results in the following tasks:
+
+```
+first:
+  prop: value1
+second:
+  prop: value2
+```
+
+## Maps
+
+Instead of defining a task directly, an item in the tasks property may
+be an object with a single property `$map`. This object itself has two
+child properties; `for` and `do`. The value of `for` is a list of
+objects, and the value of `do` is either an object or a list of
+objects. For each object in the `for` property, a set of tasks is
+created by taking a copy of that object for each task in the `do`
+property, updating the object with the properties from the
+corresponding `do` object, using the same rules as for components
+above, and then processing as for a normal task. `$map` rules can also
+be nested.
+
+For example
+
+```
+components: {}
+tasks:
+  $map:
+    for:
+      - vars:
+          example: value1
+      - vars:
+          example: value2
+    do:
+      example-${vars.example}
+        prop: ${vars.example}
+```
+
+Results in the tasks
+
+```
+example-value1:
+  prop: value1
+example-value2:
+  prop: value2
+```
+
+Note that in combination with `$map`, variable substitutions are
+applied *twice*; once after the `$map` is evaluated and once after the
+`use` statements are evaluated.
+
+## Chunks
+
+A common requirements for tasks is that they are "chunked" into N
+partial tasks. This is handled specially in the syntax. A top level
+property `chunks` can be used to define the number of individual
+chunks to create for a specific task. Each chunked task is created
+with a `chunks` property set to an object containing an `id` property
+containing the one-based index of the chunk an a `total` property
+containing the total number of chunks. These can be substituted into
+the task definition using the same syntax as for `vars` above
+e.g. `${chunks.id}`. Note that because task names must be unique, it's
+common to specify a `name` property on the task that will override the
+property name e.g.
+
+```
+components: {}
+tasks:
+  - chunked-task:
+      chunks:2
+      command: "task-run --chunk=${chunks.id} --totalChunks=${chunks.total}"
+      name: task-chunk-${chunks.id}
+```
+
+creates tasks:
+
+```
+task-chunk-1:
+  command: "task-run --chunk=1 --totalChunks=2"
+task-chunk-2:
+  command: "task-run --chunk=2 --totalChunks=2"
+```
+
+# Overall processing model
+
+The overall processing model for tasks is as follows:
+ * Evaluate maps
+ * Perform subsitutions
+ * Evaluate use statements
+ * Exapnd chunks
+ * Perform subsitutions
+
+At each point after maps are evaluated tasks must have a unique name.

tools/ci/commands.json

@@ -24,5 +24,27 @@
       "requests",
       "pygithub"
     ]
+  },
+  "taskgraph": {
+    "path": "taskgraph.py",
+    "script": "run",
+    "help": "Build the taskgraph",
+    "virtualenv": true,
+    "install": [
+      "requests",
+      "pyyaml"
+    ]
+  },
+  "decision": {
+    "path": "decision.py",
+    "parser": "get_parser",
+    "script": "run",
+    "help": "Run the decision task",
+    "virtualenv": true,
+    "install": [
+        "requests",
+        "pyyaml",
+        "taskcluster"
+    ]
   }
 }