Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ module objects or loader through `process` taken as a value (`var p = process; p
computed member of `process`, or requests the `module` or `process` built-in is not split, but an
alias is not followed, since most bundles pass `process` around. And
no value the program tests is `document.all`, so `x != null ? x : y` becomes `x ?? y`: the two
differ only for that object, which compares loosely equal to `null` without being nullish. Nor does any code read a function's caller through `.caller`, `arguments.callee.caller` or a stack frame (`Error.prepareStackTrace` call sites with `getFunction()` or `getFunctionName()`) to observe which function called it (the non-standard `Function.prototype.caller`; strict code cannot): inlining a control-flow object's call proxy removes the proxy's frame, so such a read would see the function that held the call instead of the proxy. The default lane also assumes obfuscator.io's self-defending check would pass on the input: removing the trap may make an input that would trap (for example one reformatted after obfuscation) run normally. It does not assume an obfuscator.io domain lock would pass: the lock reads the host's `document` and `location` and enumerates the global object's names, so the default lane keeps it, the protections registered with it and the machinery it can see, and says so (`a domain lock was found …`); `--unsafe` assumes the lock passes and removes it. The default lane keeps a protection that may still run, one it cannot read or prove inert (a self-defending callback is inert only when the built-in scan proves that nothing in the program can replace a built-in member the callback calls), and when that is a block registered through a call controller or a debug-protection timer it keeps every block registered that way and every debug-protection timer with it, including another layer's console silencing, because removing one layer's silencing and keeping another's changes what the program prints, and removing the silencing and keeping a timer, or the reverse, changes what it prints or whether it keeps running. Untwist also assumes no debugger or inspector session changes the program's values while it
differ only for that object, which compares loosely equal to `null` without being nullish. Nor does any code read a function's caller through `.caller`, `arguments.callee.caller` or a stack frame (`Error.prepareStackTrace` call sites with `getFunction()` or `getFunctionName()`) to observe which function called it (the non-standard `Function.prototype.caller`; strict code cannot): inlining a control-flow object's call proxy removes the proxy's frame, so such a read would see the function that held the call instead of the proxy. The default lane also assumes obfuscator.io's self-defending check would pass on the input: removing the trap may make an input that would trap (for example one reformatted after obfuscation) run normally. It does not assume an obfuscator.io domain lock would pass: the lock reads the host's `document` and `location` and enumerates the global object's names, so the default lane keeps it, the protections registered with it and the machinery it can see, and says so (`a domain lock was found …`); `--unsafe` assumes the lock passes and removes it. The default lane keeps a protection that may still run, one it cannot read or prove inert (a self-defending callback is inert only when the built-in scan proves that nothing in the program can replace a built-in member the callback calls or read the RegExp legacy statics its searches change), and when that is a block registered through a call controller or a debug-protection timer it keeps every block registered that way and every debug-protection timer with it, including another layer's console silencing, because removing one layer's silencing and keeping another's changes what the program prints, and removing the silencing and keeping a timer, or the reverse, changes what it prints or whether it keeps running. Untwist also assumes no debugger or inspector session changes the program's values while it
runs; every lane and every proof rests on this.
[`docs/known-divergences/deobfuscate.md`](docs/known-divergences/deobfuscate.md) lists every
premise.
Expand Down Expand Up @@ -267,7 +267,7 @@ or runs code text through any runner, `(0, eval)`,
overwrites a decoder alias, a decoder wrapper or a constant used as a decoder argument through
Annex B hoisting, and no direct `eval` redeclares one in its own scope (`var d = dec;
{ function d() {} } d(1)`, `var d = dec; if (1) function d() {} d(1)` and
`var d = dec; eval("function d() {}"); d(1)` are each decoded as a call of `dec`). A few passes that rest on
`var d = dec; eval("function d() {}"); d(1)` are each decoded as a call of `dec`). No code reads the RegExp legacy statics (`RegExp.$1`, `lastMatch`, `input`, ...) after a removed self-defending callback's regexp searches would have changed them; the default lane keeps the block when the program may read them. A few passes that rest on
heuristics are allowed to run. These are `template-literals` and JSX reconstruction (`jsx`, `jsx-new`).
`dead-code`, which deletes branches behind constant comparisons, runs in both lanes.
`fold-pure-calls` also folds calls to a pure helper that is passed around as a value or that the
Expand Down
4 changes: 3 additions & 1 deletion crates/untwist-core/src/semantic/builtins.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ use crate::semantic::has_module_syntax;

mod scan;

pub use scan::{intact, scan, scan_outside, scan_with_unknown_keys_plain};
pub use scan::{
intact, scan, scan_outside, scan_outside_with_regexp_statics, scan_with_unknown_keys_plain,
};

use oxc_semantic::Scoping;

Expand Down
27 changes: 27 additions & 0 deletions crates/untwist-core/src/semantic/builtins/scan.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ mod modules;
mod numeric;
mod parts;
mod reach;
mod statics;
mod targets;
mod windows;

Expand Down Expand Up @@ -200,6 +201,7 @@ struct Scan<'s> {
writes_only: bool,
unknown_keys_plain: bool,
first_only: bool,
statics: bool,
skipped: Option<&'s FxHashSet<Span>>,
module: bool,
frames: Vec<Frame>,
Expand Down Expand Up @@ -284,9 +286,31 @@ pub fn scan_outside(
module: bool,
reliance: &Reliance,
skipped: &FxHashSet<Span>,
) -> Builtins {
outside(program, scoping, module, reliance, skipped, false)
}

pub fn scan_outside_with_regexp_statics(
program: &Program<'_>,
scoping: &Scoping,
module: bool,
reliance: &Reliance,
skipped: &FxHashSet<Span>,
) -> Builtins {
outside(program, scoping, module, reliance, skipped, true)
}

fn outside(
program: &Program<'_>,
scoping: &Scoping,
module: bool,
reliance: &Reliance,
skipped: &FxHashSet<Span>,
statics: bool,
) -> Builtins {
let mut scan = Scan::new(program, scoping, module, reliance);
scan.skipped = Some(skipped);
scan.statics = statics;
scan.visit_program(program);
scan.settle_window_flows();
scan.verdict
Expand Down Expand Up @@ -320,6 +344,7 @@ impl<'s> Scan<'s> {
writes_only: false,
unknown_keys_plain: false,
first_only: false,
statics: false,
skipped: None,
module,
frames: vec![Frame::of_program(program, module)],
Expand Down Expand Up @@ -938,6 +963,7 @@ impl<'a> VisitJs<'a> for Scan<'_> {

fn visit_static_member_expression(&mut self, it: &StaticMemberExpression<'a>) {
let key = Key::Name(it.property.name.as_str());
self.judge_statics(&it.object, key);
if self.names_a_loader(key) {
self.not_allowed();
}
Expand All @@ -948,6 +974,7 @@ impl<'a> VisitJs<'a> for Scan<'_> {

fn visit_computed_member_expression(&mut self, it: &ComputedMemberExpression<'a>) {
let key = self.sharpen(key_of(&it.expression));
self.judge_statics(&it.object, key);
if self.names_a_loader(key) {
self.not_allowed();
}
Expand Down
55 changes: 39 additions & 16 deletions crates/untwist-core/src/semantic/builtins/scan/numeric.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ use rustc_hash::{FxHashMap, FxHashSet};

use super::reach::{key_of, Key};
use super::GLOBAL_OBJECTS;
use crate::semantic::is_foreign_global;
use crate::host_globals::{is_host_global, is_host_global_property};
use crate::semantic::{is_foreign_global, AnnexBWrites};

pub(super) type NumericBindings = FxHashSet<SymbolId>;

Expand Down Expand Up @@ -139,6 +140,7 @@ const MAX_PASSES: usize = 8;

struct Collect<'s> {
scoping: &'s Scoping,
annex_b: &'s AnnexBWrites,
numeric: &'s NumericBindings,
led: Option<&'s NumericBindings>,
module: bool,
Expand Down Expand Up @@ -296,8 +298,15 @@ impl Collect<'_> {
}
}

fn host_provided(&self, symbol: SymbolId) -> bool {
let name = self.scoping.symbol_name(symbol);
name == ARGUMENTS
|| (self.scoping.symbol_scope_id(symbol) == self.scoping.root_scope_id()
&& (is_host_global(name) || is_host_global_property(name)))
}

fn holds(&self, symbol: SymbolId) -> bool {
if self.direct_eval {
if self.direct_eval || self.annex_b.may_write(symbol) || self.host_provided(symbol) {
return false;
}
let flags = self.scoping.symbol_flags(symbol);
Expand Down Expand Up @@ -415,14 +424,26 @@ impl<'a> VisitJs<'a> for Collect<'_> {
}
}

fn narrowed(
program: &Program<'_>,
scoping: &Scoping,
struct Sources<'s, 'a> {
program: &'s Program<'a>,
scoping: &'s Scoping,
annex_b: &'s AnnexBWrites,
module: bool,
}

fn narrowed(
sources: &Sources<'_, '_>,
(numeric, led): (&NumericBindings, Option<&NumericBindings>),
) -> NumericBindings {
let Sources {
program,
scoping,
annex_b,
module,
} = *sources;
let mut collect = Collect {
scoping,
annex_b,
numeric,
led,
module,
Expand All @@ -448,17 +469,12 @@ fn narrowed(
.collect()
}

fn fixpoint(
program: &Program<'_>,
scoping: &Scoping,
module: bool,
numeric: Option<&NumericBindings>,
) -> NumericBindings {
let mut current: NumericBindings = scoping.symbol_ids().collect();
fn fixpoint(sources: &Sources<'_, '_>, numeric: Option<&NumericBindings>) -> NumericBindings {
let mut current: NumericBindings = sources.scoping.symbol_ids().collect();
for _ in 0..MAX_PASSES {
let next = match numeric {
Some(numeric) => narrowed(program, scoping, module, (numeric, Some(&current))),
None => narrowed(program, scoping, module, (&current, None)),
Some(numeric) => narrowed(sources, (numeric, Some(&current))),
None => narrowed(sources, (&current, None)),
};
if next.len() == current.len() {
return next;
Expand All @@ -469,7 +485,14 @@ fn fixpoint(
}

pub(super) fn key_bindings(program: &Program<'_>, scoping: &Scoping, module: bool) -> KeyBindings {
let numeric = fixpoint(program, scoping, module, None);
let led = fixpoint(program, scoping, module, Some(&numeric));
let annex_b = AnnexBWrites::of(program, scoping);
let sources = Sources {
program,
scoping,
annex_b: &annex_b,
module,
};
let numeric = fixpoint(&sources, None);
let led = fixpoint(&sources, Some(&numeric));
KeyBindings { numeric, led }
}
1 change: 1 addition & 0 deletions crates/untwist-core/src/semantic/builtins/scan/parts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ impl Scan<'_> {
let Some((object, key)) = member_parts(expression) else {
return false;
};
self.judge_member_statics(expression);
self.visit_unless_root(object);
if let Some(key) = key {
self.visit_expression(key);
Expand Down
70 changes: 70 additions & 0 deletions crates/untwist-core/src/semantic/builtins/scan/statics.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
use oxc_ast::ast::*;

use super::reach::{member_of, Key};
use super::Scan;

const REGEXP: &str = "RegExp";

const LEGACY_STATICS: [&str; 21] = [
"$1",
"$2",
"$3",
"$4",
"$5",
"$6",
"$7",
"$8",
"$9",
"$_",
"$&",
"$+",
"$`",
"$'",
"input",
"lastMatch",
"lastParen",
"leftContext",
"rightContext",
"multiline",
"$*",
];

impl Scan<'_> {
pub(super) fn judge_statics(&mut self, object: &Expression<'_>, key: Key<'_>) {
if self.statics
&& !self.writes_only
&& self.may_name_a_static(key)
&& self.may_be_regexp(object)
{
self.aliased();
}
}

pub(super) fn judge_member_statics(&mut self, expression: &Expression<'_>) {
if let Some((object, key)) = member_of(expression) {
self.judge_statics(object, key);
}
}

fn may_name_a_static(&self, key: Key<'_>) -> bool {
match self.sharpen(key) {
Key::Name(name) => LEGACY_STATICS.contains(&name),
Key::Other => false,
Key::Unknown(expression) => self
.known_key(expression)
.is_none_or(|name| LEGACY_STATICS.contains(&name.as_str())),
}
}

fn may_be_regexp(&self, object: &Expression<'_>) -> bool {
let object = object.without_parentheses();
if self.is_global_object(object) || self.is_maybe_global_this(object) {
return false;
}
match object {
Expression::Identifier(id) => self.global_named(id, &[REGEXP]),
Expression::ThisExpression(_) => false,
other => self.is_root(other),
}
}
}
14 changes: 14 additions & 0 deletions crates/untwist-core/tests/builtins_numeric_keys.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ fn a_key_proven_numeric_reaches_no_prototype() {
"function f(a) { for (let i = 0; i < a.length; i++) g(a[i]); }",
"function f(a, n) { let i = n - 1; i *= 2; g(a[i]); }",
"function f(a, n) { let i = 0; i += 1; g(a[i + 1]); }",
"function f(a) { var k; g(a[k]); }",
"function f(a, n) { g(a[n | 0], a[-n], a[~n], a[!n], a[n++]); }",
"function f(p, q) { p = p - 277; var c = d(); return c[p]; }",
"function f(a, c) { g(a[c -= 0], a[c = +c]); }",
Expand Down Expand Up @@ -50,6 +51,19 @@ fn a_key_that_may_be_a_string_still_counts() {
"function f(a, s, n) { g(a[s + (n | 0)]); }",
"var a = [1]; var i = 0; window.i = k; g(a[i]);",
"var a = [1]; var i = 0; this[k] = 1; g(a[i]);",
"function f(a) { var arguments; g(a[arguments]); }",
"function f(a) { { var arguments; } g(a[arguments]); }",
"function f(a) { for (var arguments; ;) return a[arguments]; }",
"function f(a) { g(a[arguments]); var arguments = 0; }",
"var exports; g(a[exports]);",
"var module; g(a[module]);",
"g(a[require]); var require = 0;",
"function f(a) { var k; { function k() {} } g(a[k]); }",
"function f(a) { var k = 0; if (1) function k() {} g(a[k]); }",
"function f(a) { var k = 0; switch (1) { case 1: function k() {} } g(a[k]); }",
"function f(a) { var k; { { function k() {} } } g(a[k + ''] ); }",
"function f(a) { var k; try {} catch (e) { function k() {} } g(a[k]); }",
"var f = (a) => { var k; { function k() {} } g(a[k]); };",
] {
assert_ne!(verdict(source), Builtins::Intact, "{source}");
}
Expand Down
5 changes: 3 additions & 2 deletions crates/untwist-deobfuscate/src/self_defending.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ use untwist_core::ast_util::matcher::{anonymous_function, call_parts, iife};
use untwist_core::ast_util::naming_slot::without_unnamed;
use untwist_core::ast_util::prop::{identifier_name, member_parts};
use untwist_core::pass::{Cx, Interest, Pass};
use untwist_core::semantic::builtins::{scan_outside, Builtins, Reliance};
use untwist_core::semantic::builtins::{scan_outside_with_regexp_statics, Builtins, Reliance};
use untwist_core::semantic::{has_module_syntax, AnnexBWrites};

use crate::bindings::{
Expand Down Expand Up @@ -1618,7 +1618,8 @@ fn callback_built_ins_intact<'a>(program: &Program<'a>, spans: &[Span], cx: &Cx<
let module = has_module_syntax(program);
let mut skipped: FxHashSet<Span> = spans.iter().copied().collect();
skipped.extend(crate::debug_protection::planned_removal(program, spans, cx));
scan_outside(program, cx.scoping(), module, &CALLBACK_RELIES_ON, &skipped) == Builtins::Intact
scan_outside_with_regexp_statics(program, cx.scoping(), module, &CALLBACK_RELIES_ON, &skipped)
== Builtins::Intact
}

#[derive(Default)]
Expand Down
1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a1-arguments.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a10-number.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a1b-arguments-fn.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a1c-arguments-led.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a2-annexb.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a2b-annexb-if.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/untwist/tests/data/r7nx12/a2c-annexb-noVar.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading