Standardize long-running/batch processes

[eaton]

We do long-running processes quite frequently. The standout examples right now are Spider.run(), URLHierarchy building, and any of the VerticeWorker implementations. Standardizing these operations would be a big improvement for the general dev experience.

• Standardize ongoing updates around the JobStatus interface
• Use a CancellablePromise — or provide a cancel reference in the status object?
• Add interactive and verbose status updates to SpiderCommand (eliminating a bunch of boilerplate code for long-running processes)

[eaton]

The 0.6.0 release covered some of these, but things like the --verbose flag are manually coded into each CLI command rather than handled uniformly.