Nvunnam/combined pr795 797 798#800
Open
nvunnam57128 wants to merge 6 commits into
Open
Conversation
…ranspose-optimization
When the quantized eltwise (Add/Mul/Sub/Div/Tanh/Sqrt) feeding an
Relu/LeakyRelu has more than one user, Pattern 2 today clones the
eltwise into the activation slot. The original eltwise survives
because of its other user(s), and Pattern 1 then claims it. The
result is two qlinear-eltwise ops with identical inputs that compute
the same value, e.g. for scene_parser around input.115/input.119:
%140 = xir.qlinear_eltwise %139, %137 ... op_type="ADD"
%141 = xir.qlinear_eltwise %139, %137 ... op_type="ADD",
nonlinear="RELU"
This wastes a kernel invocation. The xmodel-flow on the same graph
produces the equivalent "golden" form:
%143 = xir.qlinear_eltwise %142, %140 op_type="ADD"
%144 = xir.qlinear_eltwise %143 op_type="RELU"
i.e., one bare ADD shared by both consumers and a standalone RELU
chained on top. xmodel achieves this because get_template() picks up
Relu standalone (its elew list includes "relu") whenever the
through-Q/DQ fusion template (get_template_qlinear_eltwise_with_
single_relu) is gated off, or rejects on a multi-fanout filter.
Mirror that behavior here:
* FuseQuantizedEltwiseActivation::matchAndRewrite now requires
eltwiseOp->hasOneUse(). When the eltwise has multiple users we
refuse to fuse, letting Pattern 1 emit a standalone activation op
(FuseQuantizedEltwiseWithoutActivation<ONNXReluOp> is already
registered for this).
* isInputFromPattern2Eltwise, which Pattern 1 uses to *defer* the
standalone Relu/LeakyRelu rewrite when Pattern 2 can fuse, now
returns false for multi-user eltwise so Pattern 1 takes over.
Single-user Add -> Relu is unaffected; the existing single-fused
op is still emitted (perf-optimal). Multi-user cases now match the
xmodel-flow golden form and remove the duplicated eltwise compute.
Co-authored-by: Cursor <cursoragent@cursor.com>
nvunnam57128
force-pushed
the
nvunnam/combined-pr795-797-798
branch
from
e041d3c to
69598d5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.