Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 18 additions & 2 deletions native/cluster-render/PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ lacks.
| `stop` | `ref`, `anim` |
| `scene` | `ref`, `target`, `file`, `res`, `sha256`, `contend`, `layers`, `layers_order` ([below](#scene-files)); sent again with its ref, another file in place |
| `edit` | `ref` (a scene), `lines` ([below](#scene-edits)) |
| `host` | `ref` (a scene with `layers` 1), `target`, `x`, `y`, `w`, `h`, `src_x`, `src_y`, `src_w`, `src_h`, `shown` ([below](#hosting-another-target)) |
| `host` | `ref` (a scene with `layers` 1), `target`, `x`, `y`, `w`, `h`, `src_x`, `src_y`, `src_w`, `src_h`, `shown`; optional `edge_top`, `edge_bottom`, `edge_fade`, `edge_dim`, `edge_blur` ([below](#hosting-another-target)) |
| `drop` | `ref`: a layer, a scene, or an asset no layer samples (else refused: `asset in use`) |
| `repeat` | `ref` (a video layer): a frame whose atlas is the layer's last frame's, byte for byte, in place of that frame; to a service whose hello has `repeat` 1 |
| `status` | — (answered by a `status` event) |
Expand Down Expand Up @@ -429,13 +429,29 @@ messages and send them again after a reconnect, except one the service
refused or one whose scene was sent again since, and send none for a scene
without `layers` or for the scene's own target.

A host may also ask for an edge on the hosted picture, for a rect shorter
than the shape the scene shows it in: the service draws the rows 2 px in
from the top and from the bottom of each video another client draws inside
the rect (the hosted producer's) on past it, `edge_top` px above and
`edge_bottom` px below (held within the scene's target), blurred along the
row by a Gaussian of sigma `edge_blur` px (0 to 32; 0: not blurred), and
darkened with the distance d from the rect to max(1/4, 1 - d / `edge_dim`)
(0: not darkened); the picture's own rows fade into the edge over
`edge_fade` px inside the rect. All five default to 0; with `edge_top` and
`edge_bottom` 0 there is none. The edge is drawn by the service over the
video, in its fade, and only while the host is shown; the producer sends
nothing more for it, and the `host` event does not carry it. Unturned
videos only, and not for videos the target draws through one composite.
Send them only to a service whose hello has `host_edge` 1 (an older one
ignores them, and draws no edge).

## Service to client

Every JSON message has `ev`.

| ev | Fields | When |
|---|---|---|
| `hello` | `proto`, `backend` (`id7`, or `id8` on iDrive 8), `targets` (`map:1088x480,panel:1920x720,hud:800x480`; on iDrive 8 `map:874x749,hud:353x203,guide:702x213,widget:2240x896,popup:2240x896`, then any a drop-in adds, README.md "iDrive 8"), `repeat` (1: takes `repeat`), `always_packed` (1: takes a video layer's `always_packed`; absent from older services) | after the client's hello |
| `hello` | `proto`, `backend` (`id7`, or `id8` on iDrive 8), `targets` (`map:1088x480,panel:1920x720,hud:800x480`; on iDrive 8 `map:874x749,hud:353x203,guide:702x213,widget:2240x896,popup:2240x896`, then any a drop-in adds, README.md "iDrive 8"), `repeat` (1: takes `repeat`), `always_packed` (1: takes a video layer's `always_packed`; absent from older services), `host_edge` (1: takes a `host`'s `edge_*`; absent from older services) | after the client's hello |
| `layer` | `ref`, `state`, `reason` | the layer's state changed |
| `target` | `target`, `state`, `holder`, `reason`, `retry_in_ms`, `rendering`, `guide_disabled`, `disabled` | a target's state changed |
| `budget` | `cap_kb_per_s`, `max_kb_per_s`, `floor_kb_per_s`, `reason` | the upload cap changed |
Expand Down
5 changes: 4 additions & 1 deletion native/cluster-render/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -230,7 +230,10 @@ Beside the layers:
only from under `/var/opt/`, owned by root and writable by no one else,
with the SHA-256 the client gives. With `layers` 1 the target's layers
are drawn over the file (RAMSES 19 only), and the scene may host another
target's content (PROTOCOL.md "Hosting another target").
target's content (PROTOCOL.md "Hosting another target"), with an
optional edge: the hosted picture's top and bottom rows drawn on past its
rect, blurred and darkened, for a picture shorter than the shape it shows
in.

Each target's changes are flushed together once a tick, so the cluster never
shows a half-written frame: a video or exclusive image left half written
Expand Down
4 changes: 4 additions & 0 deletions native/cluster-render/client/client.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -556,6 +556,10 @@ bool Client::host(uint32_t ref, const Host &h)
if (h.src_w && h.src_h)
kv.insert(kv.end(), {{"src_x", num(h.src_x)}, {"src_y", num(h.src_y)}, {"src_w", num(h.src_w)},
{"src_h", num(h.src_h)}});
if (h.edge_top || h.edge_bottom)
kv.insert(kv.end(), {{"edge_top", num(h.edge_top)}, {"edge_bottom", num(h.edge_bottom)},
{"edge_fade", num(h.edge_fade)}, {"edge_dim", num(h.edge_dim)},
{"edge_blur", num(h.edge_blur)}});
const std::string json = json_object(kv);
it->second.hosts[h.target] = json;
queue(json);
Expand Down
7 changes: 6 additions & 1 deletion native/cluster-render/include/cluster_render/client.h
Original file line number Diff line number Diff line change
Expand Up @@ -167,10 +167,15 @@ struct Host {
int src_x = 0, src_y = 0;
unsigned src_w = 0, src_h = 0; /* 0: the whole target */
bool shown = true;
/* the edge drawn on the hosted picture (sent, all five, with edge_top or
* edge_bottom; the event never carries it) */
unsigned edge_top = 0, edge_bottom = 0, edge_fade = 0, edge_dim = 0, edge_blur = 0;
bool operator==(const Host &o) const
{
return target == o.target && into == o.into && x == o.x && y == o.y && w == o.w && h == o.h &&
src_x == o.src_x && src_y == o.src_y && src_w == o.src_w && src_h == o.src_h && shown == o.shown;
src_x == o.src_x && src_y == o.src_y && src_w == o.src_w && src_h == o.src_h && shown == o.shown &&
edge_top == o.edge_top && edge_bottom == o.edge_bottom && edge_fade == o.edge_fade &&
edge_dim == o.edge_dim && edge_blur == o.edge_blur;
}
bool operator!=(const Host &o) const { return !(*this == o); }
};
Expand Down
13 changes: 9 additions & 4 deletions native/cluster-render/js/cluster_render_client.js
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,7 @@ class ClusterRender extends EventEmitter {
if (ev.ev === 'hello') {
this.backend = ev.backend;
this.targets = ev.targets;
this.hostEdge = ev.host_edge === 1; /* takes a host's edge_* */
this.ready = true;
this.backoff = 0;
this._pump();
Expand Down Expand Up @@ -394,10 +395,12 @@ class ClusterRender extends EventEmitter {
/* The scene (made with layers) hosts another target (PROTOCOL.md
* "host"): opts target, x, y, w, h (in the scene's target), src_x,
* src_y, src_w, src_h (in the hosted target's pixels; sent, all four,
* when src_w and src_h are set, else all of it), shown. Kept by target
* and sent again after a reconnect, until the service refuses it or the
* scene is replaced; false, nothing sent, for a scene without layers or
* a host of its own target. */
* when src_w and src_h are set, else all of it), shown; edge_top,
* edge_bottom, edge_fade, edge_dim, edge_blur (the edge drawn on the
* hosted picture; sent, all five, when edge_top or edge_bottom is set).
* Kept by target and sent again after a reconnect, until the service
* refuses it or the scene is replaced; false, nothing sent, for a scene
* without layers or a host of its own target. */
host(ref, opts) {
const l = this.layers.get(ref);
/* what the service refuses whatever it holds is not sent or kept */
Expand All @@ -411,6 +414,8 @@ class ClusterRender extends EventEmitter {
m.src_w = opts.src_w;
m.src_h = opts.src_h;
}
if (opts.edge_top || opts.edge_bottom)
['edge_top', 'edge_bottom', 'edge_fade', 'edge_dim', 'edge_blur'].forEach((k) => { m[k] = opts[k] || 0; });
l.hosts.set(opts.target, m);
this._send(m);
return true;
Expand Down
11 changes: 11 additions & 0 deletions native/cluster-render/js/test/client_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,18 @@ async function main() {
ovl.host(over, { target: 'map', x: 700, y: 160, w: 560, h: 400, src_x: 264, src_w: 560, src_h: 400, src_y: 40 });
const sent = ovl.layers.get(over).hosts.get('map');
ok('with them, all four', sent.src_x === 264 && sent.src_y === 40 && sent.src_w === 560 && sent.src_h === 400);
ok('no edge unless edge_top or edge_bottom is set; the hello says the service takes one', !('edge_fade' in sent) &&
ovl.hostEdge === true);
ok('it hosts the map: the other client told', await until(() => hosted.indexOf('map:panel:1:560') >= 0, 2000));
ovl.host(over, { target: 'map', x: 700, y: 160, w: 560, h: 400, edge_top: 40, edge_blur: 9 });
const edged = ovl.layers.get(over).hosts.get('map');
ok('an edge: all five sent, kept', edged.edge_top === 40 && edged.edge_bottom === 0 && edged.edge_fade === 0 &&
edged.edge_dim === 0 && edged.edge_blur === 9);
ovl.host(over, { target: 'map', x: 700, y: 160, w: 560, h: 400, src_x: 264, src_w: 560, src_h: 400, src_y: 40, edge_top: 40, edge_blur: 33 });
ok('an edge out of range refused', await until(() =>
eo.refused.some((r) => r.indexOf('host: edge_blur must be a number from 0 to 32') === 0), 2000));
ovl.host(over, { target: 'map', x: 700, y: 160, w: 560, h: 400, src_x: 264, src_w: 560, src_h: 400, src_y: 40 });
ok('hosted again', await until(() => hosted[hosted.length - 1] === 'map:panel:1:560', 2000));
ovl.host(over, { target: 'map', x: 1800, y: 160, w: 560, h: 400 });
ok('a host refused: the client forgets it, the service ends it', await until(() =>
!ovl.layers.get(over).hosts.has('map') && hosted[hosted.length - 1] === 'map:-' &&
Expand Down
55 changes: 55 additions & 0 deletions native/cluster-render/src/compositor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ Compositor::~Compositor()
if (atlas_fx_) gpu_->drop_effect(atlas_fx_);
if (block_fx_) gpu_->drop_effect(block_fx_);
if (image_fx_) gpu_->drop_effect(image_fx_);
for (int fx : edge_fx_)
if (fx) gpu_->drop_effect(fx);
}

bool Compositor::init(std::string *err)
Expand All @@ -126,6 +128,21 @@ bool Compositor::init(std::string *err)
return image_fx_ != 0;
}

/* Compiled at the first edge a host asks for; refused, the service draws
* none (and runs as without them). */
bool Compositor::edge_effects()
{
if (!edge_tried_) {
edge_tried_ = true;
std::string why;
if (!(edge_fx_[0] = gpu_->effect(kQuadVertexShader, kEdgeAtlasFragmentShader, &why)))
log_warn("compositor: no edges for hosts: %s", why.c_str());
else if (!(edge_fx_[1] = gpu_->effect(kQuadVertexShader, kEdgeBlockFragmentShader, &why)))
log_warn("compositor: no edges for hosts' packed videos: %s", why.c_str());
}
return edge_fx_[0] != 0;
}

uint32_t Compositor::admit(uint64_t owner, int target, int64_t pixel_bytes, std::string *err)
{
if (target < 0 || target >= (int)targets_.size()) {
Expand Down Expand Up @@ -671,6 +688,12 @@ std::string Compositor::scene_text(uint32_t id) const
slots_->targets()[kv.first].name.c_str(), h.rect.w, h.rect.h, h.rect.x, h.rect.y, h.src.w,
h.src.h, h.src.x, h.src.y, h.shown ? "" : ", not shown");
text += b;
if (h.edge.on()) {
std::snprintf(b, sizeof b, ", its edge %u up and %u down (fade %u, dim %u, blur %u)%s", h.edge.top,
h.edge.bottom, h.edge.fade, h.edge.dim, h.edge.blur,
edge_tried_ && !edge_fx_[0] ? ", not drawn here" : "");
text += b;
}
}
return text;
}
Expand Down Expand Up @@ -878,12 +901,44 @@ void Compositor::arrange(int64_t now_ms)
t.dirty = true;
mark(e, t.arr.order[pos], t);
}
if (!std::strcmp(e.layer->kind(), "video")) {
VideoEdge ed = edge_for((int)ti, e);
if (ed.on() && !edge_effects())
ed = VideoEdge();
if (static_cast<VideoLayer *>(e.layer.get())->set_edge(ed, edge_fx_[0], edge_fx_[1])) {
t.dirty = true;
mark(e, t.arr.order[pos], t);
}
}
}
if (t.composite && t.composite->arranged())
t.dirty = true;
}
}

VideoEdge Compositor::edge_for(int ti, const Entry &e) const
{
const Rect &r = e.layer->params.rect;
const int64_t x0 = r.x, y0 = r.y, x1 = x0 + r.w, y1 = y0 + r.h, th = slots_->targets()[ti].height;
for (const auto &kv : scenes_) {
const SceneEntry &s = kv.second;
if (s.target != ti || s.owner == e.owner || !s.error.empty())
continue;
for (const auto &hk : s.hosts) {
const Host &h = hk.second;
if (!h.shown || !h.edge.on() || x0 < h.rect.x || y0 < h.rect.y || x1 > (int64_t)h.rect.x + h.rect.w ||
y1 > (int64_t)h.rect.y + h.rect.h)
continue;
VideoEdge v = h.edge;
v.top = (uint32_t)std::min<int64_t>(v.top, y0);
v.bottom = (uint32_t)std::min<int64_t>(v.bottom, std::max<int64_t>(0, th - y1));
v.fade = std::min(v.fade, r.h / 2);
return v;
}
}
return VideoEdge();
}

void Compositor::fade(int ti, Target &t, int64_t now_ms)
{
const int64_t dt = std::max<int64_t>(0, now_ms - t.fade_ms);
Expand Down
9 changes: 8 additions & 1 deletion native/cluster-render/src/compositor.h
Original file line number Diff line number Diff line change
Expand Up @@ -216,10 +216,12 @@ struct SceneLayers {

/* A scene content showing another target's content (PROTOCOL.md "host"): at
* rect in its own target, the part src of the hosted target; shown 0:
* nowhere. */
* nowhere. edge: drawn on the videos other clients draw inside rect there
* (the hosted producer's), held within the target. */
struct Host {
Rect rect, src;
bool shown = true;
VideoEdge edge;
};

class Compositor : public Scenes {
Expand Down Expand Up @@ -654,6 +656,11 @@ class Compositor : public Scenes {
int64_t second_ms_ = 0;
int64_t (*clock_us_)() = nullptr;
int atlas_fx_ = 0, block_fx_ = 0, image_fx_ = 0;
int edge_fx_[2] = {0, 0}; /* kEdgeAtlasFragmentShader, kEdgeBlockFragmentShader; 0: no edges */
bool edge_tried_ = false;
bool edge_effects();
/* The edge of the host whose rect holds e's video on target ti, or none. */
VideoEdge edge_for(int ti, const Entry &e) const;
int atlas_view_fx_ = 0, block_view_fx_ = 0; /* the composite's, made with the first one */
std::string plan_error_; /* why the last reconcile changed nothing; logged once */
std::map<uint32_t, Entry> layers_;
Expand Down
18 changes: 17 additions & 1 deletion native/cluster-render/src/server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -605,7 +605,8 @@ void Server::on_json(Client &c, const std::string &body, int64_t now_ms)
{"backend", cr::json_quote(backend_)},
{"targets", cr::json_quote(targets)},
{"repeat", "1"},
{"always_packed", "1"}}));
{"always_packed", "1"},
{"host_edge", "1"}}));
c.target_sent.clear();
c.host_sent.clear();
c.host_gen = 0;
Expand Down Expand Up @@ -833,10 +834,25 @@ void Server::op_host(Client &c, uint32_t ref, const cr::Json &j)
refuse("src_x, src_y, src_w, src_h go together (none: all of " + from.name + ")");
return;
}
/* the optional edge drawn on the hosted picture (PROTOCOL.md "host") */
static const char *const edge_keys[] = {"edge_top", "edge_bottom", "edge_fade", "edge_dim", "edge_blur"};
static const double edge_max[] = {720, 720, 360, 4096, 32};
double ev[5] = {0, 0, 0, 0, 0};
for (size_t i = 0; i < 5; i++)
if (!num(j, edge_keys[i], 0, edge_max[i], &ev[i], &err)) {
comp_->unhost(c.id, scene, hosted);
refuse(err);
return;
}
Host h;
h.rect = Rect{(int32_t)v[0], (int32_t)v[1], (uint32_t)v[2], (uint32_t)v[3]};
h.src = Rect{(int32_t)v[4], (int32_t)v[5], (uint32_t)v[6], (uint32_t)v[7]};
h.shown = v[8] != 0;
h.edge.top = (uint32_t)ev[0];
h.edge.bottom = (uint32_t)ev[1];
h.edge.fade = (uint32_t)ev[2];
h.edge.dim = (uint32_t)ev[3];
h.edge.blur = (uint32_t)ev[4];
if (!comp_->host(c.id, scene, hosted, h, &err)) {
refuse(err);
return;
Expand Down
Loading