Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,24 @@ accepts PHP named arguments:
Polytypo::transform($input, 'fr', mode: 'html', rules: ['ranges' => true]);
```

`Polytypo::analyze` runs the same pipeline and reports what it would do instead of doing it — one
`Polytypo\Change` per edit, each with the rule that made it and code-point offsets into the input
you passed (into the **document**, in `html` mode, not into a span):

```php
Polytypo::analyze('Wait... "really"?', 'en-US');
// => [ Change { ruleId: "ellipsis", start: 4, end: 7, before: "...", after: "…" },
// Change { ruleId: "quotes", start: 8, end: 9, before: "\"", after: "“" }, ... ]
```

Offsets are code points, not bytes: in `😀 and "this"` the opening quotation mark is reported at
6, where `strpos` would say 9. It is a report, not a patch. The list is empty exactly when
`Polytypo::transform` would return the input unchanged, and every `ruleId` is a rule that was
enabled for that call — but two rules may touch the same original range (French `spaces` deletes
the space before `:` and `nbsp` puts a no-break one back), so replaying the list is not guaranteed
to reproduce the output. Call `Polytypo::transform` for the text. Full contract:
`spec/rules/analyze.md`.

### Errors

Every error `Polytypo::transform` raises is a `Polytypo\PolytypoException` carrying one of seven
Expand Down
23 changes: 23 additions & 0 deletions src/Change.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<?php

declare(strict_types=1);

namespace Polytypo;

/**
* One entry of Polytypo::analyze()'s result, in input coordinates (spec/rules/analyze.md
* section 2): $ruleId is a rule id from spec/rules/order.json, $start/$end are code-point
* offsets into the input (inclusive/exclusive), and $before/$after are the text on either side
* of that one edit. $start === $end is a pure insertion, and $before is then empty.
*/
final class Change
{
public function __construct(
public readonly string $ruleId,
public readonly int $start,
public readonly int $end,
public readonly string $before,
public readonly string $after,
) {
}
}
115 changes: 115 additions & 0 deletions src/Engine/Origin.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
<?php

declare(strict_types=1);

namespace Polytypo\Engine;

use Polytypo\Change;

/**
* analyze.md section 2: every reported offset is a code-point offset into the input the caller
* passed, in every mode. The rules, however, run over an array that is not the input -- in "text"
* mode edits from earlier rules have already shifted it, and in "html" mode it is the
* marker-joined concatenation of the processable spans (modes.md 3.5). This class carries the one
* structure that bridges the two: an origin map, parallel to the current code-point array,
* holding the input offset each code point came from, or NO_ORIGIN for one the pipeline itself
* produced. Mirrors polytypo-js's src/engine/origin.ts.
*/
final class Origin
{
public const NO_ORIGIN = -1;

private function __construct()
{
}

/**
* The input offset an edit boundary at $index addresses. Synthetic code points have no origin
* of their own, so the scan runs forward to the first that has one -- an insertion between two
* earlier insertions still lands where the next real character is. Falling off the end means
* the boundary is at the end of the input.
*
* @param int[] $origin
*/
public static function originAt(array $origin, int $index, int $inputLength): int
{
$count = count($origin);
for ($i = $index; $i < $count; $i++) {
if ($origin[$i] !== self::NO_ORIGIN) {
return $origin[$i];
}
}

return $inputLength;
}

/**
* The origin map for the array Edits::applyEdits is about to produce. A replacement of equal
* length keeps its origins position by position, which is what makes a conversion
* (U+0020 -> U+00A0) still point at the character it converted; anything longer is synthetic
* beyond the positions it covers.
*
* @param int[] $origin
* @param Edit[] $edits
* @return int[]
*/
public static function applyEditsToOrigin(array $origin, array $edits): array
{
if ($edits === []) {
return $origin;
}

$out = [];
$cursor = 0;
foreach ($edits as $edit) {
for ($i = $cursor; $i < $edit->start; $i++) {
$out[] = $origin[$i];
}
$length = count($edit->replacement);
for ($k = 0; $k < $length; $k++) {
$source = $edit->start + $k;
$out[] = $source < $edit->end ? $origin[$source] : self::NO_ORIGIN;
}
$cursor = $edit->end;
}
$total = count($origin);
for ($i = $cursor; $i < $total; $i++) {
$out[] = $origin[$i];
}

return $out;
}

/**
* One rule's edits, in the coordinates that rule saw, rendered as Changes in input
* coordinates. $before is the text this rule replaced and $after what it replaced it with
* (analyze.md section 2), so on a text two rules have both touched, $before is what the second
* rule saw rather than what the caller typed -- section 5 says so and shows the French case
* where it matters.
*
* @param int[] $cp
* @param Edit[] $edits
* @param int[] $origin
* @return Change[]
*/
public static function recordChanges(
array $cp,
array $edits,
array $origin,
int $inputLength,
string $ruleId,
): array {
$changes = [];
foreach ($edits as $edit) {
$changes[] = new Change(
ruleId: $ruleId,
start: self::originAt($origin, $edit->start, $inputLength),
end: self::originAt($origin, $edit->end, $inputLength),
before: Codepoints::fromCodepoints(array_slice($cp, $edit->start, $edit->end - $edit->start)),
after: Codepoints::fromCodepoints($edit->replacement),
);
}

return $changes;
}
}
41 changes: 41 additions & 0 deletions src/Engine/Pipeline.php
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,45 @@ public static function runRules(array $cp, array $plan, array $localeData, RuleC

return $current;
}

/**
* runRules, keeping the edits instead of discarding them (analyze.md section 1: same
* pipeline, same order, reporting rather than applying). The origin map travels alongside the
* array so every change comes back in input coordinates, and $filterEdits is the hook the
* span-runner needs for modes.md 3.4's boundary filters -- text mode passes null and gets the
* identity.
*
* @param int[] $cp
* @param string[] $plan
* @param array<string, mixed> $localeData
* @param int[] $origin
* @param null|callable(int[], Edit[]): Edit[] $filterEdits
* @return \Polytypo\Change[]
*/
public static function runRulesRecording(
array $cp,
array $plan,
array $localeData,
RuleContext $ctx,
array $origin,
int $inputLength,
?callable $filterEdits = null,
): array {
$current = $cp;
$currentOrigin = $origin;
$changes = [];
foreach ($plan as $ruleId) {
$fn = Registry::rule($ruleId);
$produced = $fn($current, $localeData, $ctx);
$edits = $filterEdits === null ? $produced : $filterEdits($current, $produced);
if ($edits === []) {
continue;
}
array_push($changes, ...Origin::recordChanges($current, $edits, $currentOrigin, $inputLength, $ruleId));
$currentOrigin = Origin::applyEditsToOrigin($currentOrigin, $edits);
$current = Edits::applyEdits($current, $edits, $ruleId);
}

return $changes;
}
}
40 changes: 40 additions & 0 deletions src/Modes/Runner.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
namespace Polytypo\Modes;

use Polytypo\Engine\Edits;
use Polytypo\Engine\Pipeline;
use Polytypo\Engine\Registry;
use Polytypo\Engine\RuleContext;

Expand Down Expand Up @@ -90,4 +91,43 @@ public static function runOverSpans(

return \Polytypo\Engine\Codepoints::fromCodepoints($out);
}

/**
* runOverSpans, reporting instead of applying (analyze.md section 1). The span table supplies
* the origin map, so every change comes back in DOCUMENT coordinates -- analyze.md section 6
* names a runtime that reports span-local offsets here as the mistake that passes every
* text-mode test.
*
* @param int[] $sourceCp
* @param Span[] $spans
* @param string[] $plan
* @param array<string, mixed> $localeData
* @return \Polytypo\Change[]
*/
public static function analyzeOverSpans(
array $sourceCp,
array $spans,
array $plan,
array $localeData,
RuleContext $ctx,
): array {
$normalized = Spans::normalizeSpans($spans);
if ($normalized === []) {
return [];
}

return Pipeline::runRulesRecording(
Spans::concatenateSpans($sourceCp, $normalized),
$plan,
$localeData,
$ctx,
Spans::originOfSpans($normalized),
count($sourceCp),
static fn (array $current, array $edits): array => Spans::filterBoundaryEdits(
$current,
$edits,
Spans::spanRangesOf($current),
),
);
}
}
28 changes: 28 additions & 0 deletions src/Modes/Spans.php
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
namespace Polytypo\Modes;

use Polytypo\Engine\Edit;
use Polytypo\Engine\Origin;
use Polytypo\Engine\Sentinels;
use Polytypo\PolytypoException;

Expand Down Expand Up @@ -94,6 +95,33 @@ public static function concatenateSpans(array $sourceCp, array $spans): array
return $cp;
}

/**
* The origin map for concatenateSpans (analyze.md section 2): for every code point of the
* joined array, the code-point offset of the character it came from IN THE DOCUMENT, and
* Origin::NO_ORIGIN for the markers, which came from nowhere. A Span's bounds are already
* code-point offsets (the mode adapter converts its parser's byte offsets away before
* constructing one), so no coordinate conversion belongs here.
*
* @param Span[] $spans
* @return int[]
*/
public static function originOfSpans(array $spans): array
{
$origin = [];
$first = true;
foreach ($spans as $span) {
if (!$first) {
$origin[] = Origin::NO_ORIGIN;
}
for ($i = $span->start; $i < $span->end; $i++) {
$origin[] = $i;
}
$first = false;
}

return $origin;
}

/**
* The span extents of the array as it stands. Recomputed after every rule, because applying
* edits shifts every index after the first one -- the markers themselves always survive,
Expand Down
72 changes: 72 additions & 0 deletions src/Polytypo.php
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,40 @@ public static function transform(
};
}

/**
* Runs the same pipeline as transform() and reports what it would do instead of doing it
* (spec/rules/analyze.md). Offsets are code-point offsets into $input in every mode -- into
* the document, in "html" mode, not into a span.
*
* What it guarantees: the list is empty exactly when transform() would return the input
* unchanged, every ruleId was enabled for the call, and every offset is inside the input.
* What it does not: the list is a report, not a patch -- two rules may touch the same
* original range, so replaying it is not guaranteed to reproduce transform()'s output. Call
* transform() for the text (analyze.md sections 4 and 5).
*
* Pure on the same terms as transform().
*
* @param array<string, bool>|null $rules
* @return Change[]
*/
public static function analyze(
string $input,
string $locale,
string $mode = 'text',
?string $dialect = null,
?array $rules = null,
): array {
$resolvedMode = self::resolveMode($mode);

return match ($resolvedMode) {
'text' => self::analyzeText($input, $locale, $dialect, $rules),
'html' => self::analyzeHtml($input, $locale, $dialect, $rules),
// 'markdown': not implemented by this runtime, exactly as for transform() -- A1
// requires analyze() to reject what transform() rejects, with the same code.
default => self::transformMarkdown($input, $locale, $dialect, $rules),
};
}

private static function resolveMode(string $mode): string
{
return match ($mode) {
Expand Down Expand Up @@ -124,4 +158,42 @@ private static function transformMarkdown(string $input, string $locale, ?string
. 'See spec/CONFORMANCE.md in github.com/polytypo/polytypo for what this runtime implements.',
);
}

/**
* @param array<string, bool>|null $rules
* @return Change[]
*/
private static function analyzeText(string $input, string $locale, ?string $dialect, ?array $rules): array
{
if ($dialect !== null) {
throw new PolytypoException(
PolytypoException::CODE_INVALID_DIALECT,
'"dialect" is only valid when mode is "markdown"',
);
}
[, $localeData, $plan] = Pipeline::prepare($locale, $rules);
$cp = Codepoints::toCodepoints($input);
$ctx = new RuleContext(mode: 'text', dialect: null, locale: $locale);

return Pipeline::runRulesRecording($cp, $plan, $localeData, $ctx, array_keys($cp), count($cp));
}

/**
* @param array<string, bool>|null $rules
* @return Change[]
*/
private static function analyzeHtml(string $input, string $locale, ?string $dialect, ?array $rules): array
{
if ($dialect !== null) {
throw new PolytypoException(
PolytypoException::CODE_INVALID_DIALECT,
'"dialect" is only valid when mode is "markdown"',
);
}
[$resolvedLocale, $localeData, $plan] = Pipeline::prepare($locale, $rules);
$spans = Html::htmlSpans($input);
$ctx = new RuleContext(mode: 'html', dialect: null, locale: $resolvedLocale);

return Runner::analyzeOverSpans(Codepoints::toCodepoints($input), $spans, $plan, $localeData, $ctx);
}
}
Loading
Loading