From 5a68956d0058e635e29ebaf094df4cc9733f6598 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:46:13 +0000
Subject: [PATCH 01/10] Define cross-entropy as the expectation of the
 surprisal measured by q

One definition now covers mass functions and densities. The finite-sum form
becomes a theorem proved by LOTUS, with an integral form and an example for
densities. The exercise that showed the expectation form now shows the sum
form, and the corollary that entropy is the self cross-entropy covers any
random variable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd              | 28 +++++++++------
 .../_cor-cross-entropy-entropy.qmd            |  8 ++---
 .../_exm-cross-entropy-continuous.qmd         | 19 ++++++++++
 ...qmd => _exr-cross-entropy-general-sum.qmd} |  6 ++--
 .../_rem-cross-entropy-expected-surprisal.qmd |  2 +-
 .../_sol-cross-entropy-general-self.qmd       |  7 ++--
 ...qmd => _sol-cross-entropy-general-sum.qmd} | 10 +++---
 .../_thm-cross-entropy-continuous.qmd         | 28 +++++++++++++++
 .../_thm-cross-entropy-discrete.qmd           | 36 +++++++++++++++++++
 .../_thm-cross-entropy-expectation.qmd        | 36 -------------------
 10 files changed, 115 insertions(+), 65 deletions(-)
 create mode 100644 _subfiles/def-cross-entropy/_exm-cross-entropy-continuous.qmd
 rename _subfiles/def-cross-entropy/{_exr-cross-entropy-general-expectation.qmd => _exr-cross-entropy-general-sum.qmd} (56%)
 rename _subfiles/def-cross-entropy/{_sol-cross-entropy-general-expectation.qmd => _sol-cross-entropy-general-sum.qmd} (62%)
 create mode 100644 _subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
 create mode 100644 _subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
 delete mode 100644 _subfiles/def-cross-entropy/_thm-cross-entropy-expectation.qmd

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index 4d225d0d..97ba1b91 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -1,20 +1,20 @@
 :::{#def-cross-entropy}
 #### Cross-entropy
 
-Let $p$ and $q$ be [probability mass functions](random-variables.qmd#def-pmf)
-on the same finite set of outcomes $\set{1, \ldots, K}$,
-with $p_k$ and $q_k$ the probabilities of outcome $k$.
+Let $X$ be a [random variable](random-variables.qmd#def-random-variable) with
+[probability mass function](random-variables.qmd#def-pmf) or [density](random-variables.qmd#def-pdf) $p$,
+and let $q$ be a mass function or density on the same values.
 The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
-is
+is the [expectation](#def-expectation), when $X$ has distribution $p$, of the [surprisal](#def-surprisal) measured by $q$:
 
-$$\crossentropyf{p}{q} \eqdef -\sumto{k}{K} p_k \log q_k.$$
+$$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
 
-We take $-\log 0 = \infty$, so when $q_k = 0$ for an outcome with $p_k > 0$, the cross-entropy is infinite.
-A term with $p_k = 0$ is taken to be $0$ whatever $q_k$ is, as for [entropy](#def-entropy).
+We take $-\log 0 = \infty$, so when $q(x) = 0$ for a value $x$ that $X$ takes with positive probability, the cross-entropy is infinite.
 
 :::
 
+
 {{< slidebreak >}}
 
 {{< include pds/_subfiles/def-cross-entropy/_exr-cross-entropy-numeric-pmf.qmd >}}
@@ -49,11 +49,11 @@ A term with $p_k = 0$ is taken to be $0$ whatever $q_k$ is, as for [entropy](#de
 
 {{< slidebreak >}}
 
-{{< include pds/_subfiles/def-cross-entropy/_exr-cross-entropy-general-expectation.qmd >}}
+{{< include pds/_subfiles/def-cross-entropy/_exr-cross-entropy-general-sum.qmd >}}
 
 {{< slidebreak >}}
 
-{{< include pds/_subfiles/def-cross-entropy/_sol-cross-entropy-general-expectation.qmd >}}
+{{< include pds/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd >}}
 
 {{< slidebreak >}}
 
@@ -73,7 +73,15 @@ A term with $p_k = 0$ is taken to be $0$ whatever $q_k$ is, as for [entropy](#de
 
 {{< slidebreak >}}
 
-{{< include pds/_subfiles/def-cross-entropy/_thm-cross-entropy-expectation.qmd >}}
+{{< include pds/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd >}}
+
+{{< slidebreak >}}
+
+{{< include pds/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd >}}
+
+{{< slidebreak >}}
+
+{{< include pds/_subfiles/def-cross-entropy/_exm-cross-entropy-continuous.qmd >}}
 
 {{< slidebreak >}}
 
diff --git a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
index 98c9e39c..7ed20fa0 100644
--- a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
+++ b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
@@ -1,7 +1,7 @@
 :::{#cor-cross-entropy-entropy}
 #### Entropy is the cross-entropy of a distribution with itself
 
-Let $X$ be a [discrete random variable](random-variables.qmd#def-discrete-rv) with [probability mass function](random-variables.qmd#def-pmf) $p$ on $\set{1, \ldots, K}$.
+Let $X$ be a [random variable](random-variables.qmd#def-random-variable) with [probability mass function](random-variables.qmd#def-pmf) or [density](random-variables.qmd#def-pdf) $p$.
 Then
 
 $$\crossentropyf{p}{p} = \entropyf{X}.$$
@@ -10,10 +10,8 @@ $$\crossentropyf{p}{p} = \entropyf{X}.$$
 
 ::: proof
 
-By @thm-cross-entropy-expectation with $q = p$, $\crossentropyf{p}{p} = \E{-\log p_X}$.
-Since $\P(X = k) = p_k$, the [random variable](random-variables.qmd#def-random-variable) $-\log p_X$ is the [surprisal](#def-surprisal) $\surprisalf{X}$.
-The surprisal $\surprisalf{X}$ is finite with [probability](probability-basics.qmd#def-probability) 1, because $X$ takes only values with $p_k > 0$,
-and the values with $p_k = 0$ contribute $0$ to both $\crossentropyf{p}{p}$ and $\entropyf{X}$ by the convention in their definitions.
+By the definition of [cross-entropy](#def-cross-entropy) with $q = p$, $\crossentropyf{p}{p} = \E{-\log p(X)}$.
+The random variable $-\log p(X)$ is the [surprisal](#def-surprisal) $\surprisalf{X}$.
 So $\crossentropyf{p}{p} = \E{\surprisalf{X}} = \entropyf{X}$ by the definition of [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_exm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_exm-cross-entropy-continuous.qmd
new file mode 100644
index 00000000..46fd58af
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_exm-cross-entropy-continuous.qmd
@@ -0,0 +1,19 @@
+:::{#exm-cross-entropy-continuous}
+#### Cross-entropy of two densities on the unit interval
+
+Let $p(x) = 1$ and $q(x) = 2x$ for $0 < x < 1$ be [densities](random-variables.qmd#def-pdf).
+By @thm-cross-entropy-continuous, using the natural logarithm,
+
+$$
+\ba
+\crossentropyf{p}{q}
+&= -\int_0^1 \log (2x) \, dx \\
+&= -\log 2 - \int_0^1 \log x \, dx \\
+&= -\log 2 + 1 \\
+&\approx 0.307 \text{ nats}.
+\ea
+$$
+
+<!-- no-figure: a single closed-form value -->
+
+:::
diff --git a/_subfiles/def-cross-entropy/_exr-cross-entropy-general-expectation.qmd b/_subfiles/def-cross-entropy/_exr-cross-entropy-general-sum.qmd
similarity index 56%
rename from _subfiles/def-cross-entropy/_exr-cross-entropy-general-expectation.qmd
rename to _subfiles/def-cross-entropy/_exr-cross-entropy-general-sum.qmd
index 58cd8ff9..17b1e2d3 100644
--- a/_subfiles/def-cross-entropy/_exr-cross-entropy-general-expectation.qmd
+++ b/_subfiles/def-cross-entropy/_exr-cross-entropy-general-sum.qmd
@@ -1,9 +1,9 @@
-:::{#exr-cross-entropy-general-expectation}
+:::{#exr-cross-entropy-general-sum}
 
-#### Cross-entropy as an expectation when $q$ is positive where $p$ is
+#### Cross-entropy as a sum when $q$ is positive where $p$ is
 
 Let $p$ and $q$ be [probability mass functions](random-variables.qmd#def-pmf) on $\set{1, \ldots, K}$ with $q_k > 0$ whenever $p_k > 0$.
 Let $X$ have [mass function](random-variables.qmd#def-pmf) $p$, and write $q_X$ for the [random variable](random-variables.qmd#def-random-variable) that equals $q_k$ when $X = k$.
-Show that the [cross-entropy](#def-cross-entropy) $\crossentropyf{p}{q}$ equals the [expectation](#def-expectation) $\E{-\log q_X}$.
+Show that the [cross-entropy](#def-cross-entropy), the [expectation](#def-expectation) $\E{-\log q_X}$, equals $-\sumto{k}{K} p_k \log q_k$.
 
 :::
diff --git a/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd b/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
index 32588152..c669f3e6 100644
--- a/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
+++ b/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
@@ -1,7 +1,7 @@
 :::{#rem-cross-entropy-expected-surprisal}
 #### Cross-entropy is an expected surprisal
 
-By @thm-cross-entropy-expectation, the [cross-entropy](#def-cross-entropy) $\crossentropyf{p}{q}$ is the average [surprisal](#def-surprisal) of the values drawn from $p$,
+By definition, the [cross-entropy](#def-cross-entropy) $\crossentropyf{p}{q}$ is the average [surprisal](#def-surprisal) of the values drawn from $p$,
 with each surprisal measured by $q$ as $-\log q_k$.
 By @cor-cross-entropy-entropy, measuring with $p$ itself gives the [entropy](#def-entropy).
 
diff --git a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-self.qmd b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-self.qmd
index 64cf4311..1510ca89 100644
--- a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-self.qmd
+++ b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-self.qmd
@@ -1,10 +1,7 @@
 :::{#sol-cross-entropy-general-self}
 
-By @thm-cross-entropy-expectation with $q = p$,
-$\crossentropyf{p}{p} = \E{-\log p_X}$.
-Since $\P(X = k) = p_k$, we have $-\log p_X = \surprisalf{X}$, the [surprisal](#def-surprisal) of $X$.
-The surprisal $\surprisalf{X}$ is finite with [probability](probability-basics.qmd#def-probability) 1, because $X$ takes only values with $p_k > 0$,
-and the values with $p_k = 0$ contribute $0$ to both $\crossentropyf{p}{p}$ and $\entropyf{X}$ by the convention in their definitions.
+By definition, $\crossentropyf{p}{p} = \E{-\log p(X)}$.
+Since $\P(X = k) = p_k$, we have $-\log p(X) = \surprisalf{X}$, the [surprisal](#def-surprisal) of $X$.
 So $\crossentropyf{p}{p} = \E{\surprisalf{X}} = \entropyf{X}$ by the definition of [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-expectation.qmd b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
similarity index 62%
rename from _subfiles/def-cross-entropy/_sol-cross-entropy-general-expectation.qmd
rename to _subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
index b4fd4c08..48ef7aeb 100644
--- a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-expectation.qmd
+++ b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
@@ -1,4 +1,4 @@
-:::{#sol-cross-entropy-general-expectation}
+:::{#sol-cross-entropy-general-sum}
 
 Let $g(k) \eqdef -\log q_k$, so that $-\log q_X = g(X)$.
 Since $q_k > 0$ whenever $p_k > 0$,
@@ -7,15 +7,15 @@ By [LOTUS](#thm-lotus),
 
 $$
 \ba
-\E{-\log q_X}
+\crossentropyf{p}{q}
+&= \E{-\log q_X} \\
 &= \E{g(X)} \\
 &= \sum_{k : p_k > 0} g(k) \cd \P(X = k) \\
 &= -\sum_{k : p_k > 0} p_k \log q_k \\
-&= -\sumto{k}{K} p_k \log q_k \\
-&= \crossentropyf{p}{q}.
+&= -\sumto{k}{K} p_k \log q_k.
 \ea
 $$
 
-The fourth line adds the terms with $p_k = 0$, which are $0$ by the convention in the definition.
+The last line adds the terms with $p_k = 0$, which are $0$ by the convention in @thm-cross-entropy-discrete.
 
 :::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
new file mode 100644
index 00000000..74a2c34a
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -0,0 +1,28 @@
+:::{#thm-cross-entropy-continuous}
+#### Cross-entropy of two densities
+
+Let $p$ and $q$ be [probability density functions](random-variables.qmd#def-pdf) on the same [range](random-variables.qmd#def-range) $\rangef{X}$,
+with $q(x) > 0$ wherever $p(x) > 0$.
+Then the [cross-entropy](#def-cross-entropy) is
+
+$$\crossentropyf{p}{q} = -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx.$$
+
+The integrand is taken to be $0$ where $p(x) = 0$.
+
+:::
+
+::: proof
+
+Let $X$ be a [continuous random variable](random-variables.qmd#def-continuous-rv) with density $p$, and let $g(x) \eqdef -\log q(x)$, so that $-\log q(X) = g(X)$.
+Then [LOTUS](#thm-lotus) gives
+
+$$
+\ba
+\crossentropyf{p}{q}
+&= \E{g(X)} && \text{(definition)} \\
+&= \int_{x \in \rangef{X}} g(x) \cd p(x) \, dx && \text{(LOTUS)} \\
+&= -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx. && \text{(definition of } g \text{)}
+\ea
+$$
+
+:::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
new file mode 100644
index 00000000..41437d4a
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
@@ -0,0 +1,36 @@
+:::{#thm-cross-entropy-discrete}
+#### Cross-entropy of two mass functions
+
+Let $p$ and $q$ be [probability mass functions](random-variables.qmd#def-pmf) on $\set{1, \ldots, K}$,
+with $p_k$ and $q_k$ the probabilities of outcome $k$.
+Then the [cross-entropy](#def-cross-entropy) is
+
+$$\crossentropyf{p}{q} = -\sumto{k}{K} p_k \log q_k.$$
+
+A term with $p_k = 0$ is taken to be $0$ whatever $q_k$ is, as for [entropy](#thm-entropy-discrete).
+
+:::
+
+::: proof
+
+Let $X$ have [mass function](random-variables.qmd#def-pmf) $p$, and let $g(k) \eqdef -\log q_k$, so that $-\log q(X) = g(X)$.
+First suppose $q_k > 0$ whenever $p_k > 0$.
+Then $g(k)$ is finite for every value $k$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability),
+and [LOTUS](#thm-lotus) gives
+
+$$
+\ba
+\crossentropyf{p}{q}
+&= \E{g(X)} && \text{(definition)} \\
+&= \sum_{k : p_k > 0} g(k) \cd \P(X = k) && \text{(LOTUS)} \\
+&= -\sum_{k : p_k > 0} p_k \log q_k && \text{(definition of } g \text{ and } \P(X = k) = p_k \text{)} \\
+&= -\sumto{k}{K} p_k \log q_k. && \text{(terms with } p_k = 0 \text{ are } 0 \text{)}
+\ea
+$$
+
+Now suppose $q_k = 0$ for some $k$ with $p_k > 0$.
+Then $-\log q(X) \ge 0$, and $-\log q(X) = \infty$ with probability at least $p_k > 0$,
+so $\crossentropyf{p}{q} = \infty$.
+The sum is also $\infty$, because its term for $k$ is $-p_k \log 0 = \infty$ and its other terms are finite or $\infty$ but never $-\infty$.
+
+:::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-expectation.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-expectation.qmd
deleted file mode 100644
index c4fc9583..00000000
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-expectation.qmd
+++ /dev/null
@@ -1,36 +0,0 @@
-:::{#thm-cross-entropy-expectation}
-#### Cross-entropy as an expected negative log probability
-
-Let $p$ and $q$ be [probability mass functions](random-variables.qmd#def-pmf) on $\set{1, \ldots, K}$,
-and let $X$ have [mass function](random-variables.qmd#def-pmf) $p$.
-Write $q_X$ for the [random variable](random-variables.qmd#def-random-variable) that equals $q_k$ when $X = k$.
-Then the [cross-entropy](#def-cross-entropy) is the [expectation](#def-expectation) of $-\log q_X$:
-
-$$\crossentropyf{p}{q} = \E{-\log q_X}.$$
-
-:::
-
-::: proof
-
-Let $g(k) \eqdef -\log q_k$, so that $-\log q_X = g(X)$.
-First suppose $q_k > 0$ whenever $p_k > 0$.
-Then $g(k)$ is finite for every value $k$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability),
-and [LOTUS](#thm-lotus) gives
-
-$$
-\ba
-\E{-\log q_X}
-&= \E{g(X)} && \text{(definition of } g \text{)} \\
-&= \sum_{k : p_k > 0} g(k) \cd \P(X = k) && \text{(LOTUS)} \\
-&= -\sum_{k : p_k > 0} p_k \log q_k && \text{(definition of } g \text{ and } \P(X = k) = p_k \text{)} \\
-&= -\sumto{k}{K} p_k \log q_k && \text{(terms with } p_k = 0 \text{ are } 0 \text{)} \\
-&= \crossentropyf{p}{q}. && \text{(definition)}
-\ea
-$$
-
-Now suppose $q_k = 0$ for some $k$ with $p_k > 0$.
-Then $\crossentropyf{p}{q} = \infty$ by definition.
-Also $-\log q_X \ge 0$, and $-\log q_X = \infty$ with probability at least $p_k > 0$,
-so $\E{-\log q_X} = \infty$.
-
-:::

From f56c22f20af36e9665fd10dd51d86486fc23f8f1 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:46:30 +0000
Subject: [PATCH 02/10] Link first uses of defined terms in the cross-entropy
 divs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd                            | 6 +++---
 _subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd  | 2 +-
 .../def-cross-entropy/_thm-cross-entropy-continuous.qmd     | 2 +-
 _subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd | 2 +-
 4 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index 97ba1b91..ecc84b74 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -3,14 +3,14 @@
 
 Let $X$ be a [random variable](random-variables.qmd#def-random-variable) with
 [probability mass function](random-variables.qmd#def-pmf) or [density](random-variables.qmd#def-pdf) $p$,
-and let $q$ be a mass function or density on the same values.
+and let $q$ be a [mass function](random-variables.qmd#def-pmf) or density on the same values.
 The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
-is the [expectation](#def-expectation), when $X$ has distribution $p$, of the [surprisal](#def-surprisal) measured by $q$:
+is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$, of the [surprisal](#def-surprisal) measured by $q$:
 
 $$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
 
-We take $-\log 0 = \infty$, so when $q(x) = 0$ for a value $x$ that $X$ takes with positive probability, the cross-entropy is infinite.
+We take $-\log 0 = \infty$, so when $q(x) = 0$ for a value $x$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability), the cross-entropy is infinite.
 
 :::
 
diff --git a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
index 7ed20fa0..30bfaea4 100644
--- a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
+++ b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
@@ -11,7 +11,7 @@ $$\crossentropyf{p}{p} = \entropyf{X}.$$
 ::: proof
 
 By the definition of [cross-entropy](#def-cross-entropy) with $q = p$, $\crossentropyf{p}{p} = \E{-\log p(X)}$.
-The random variable $-\log p(X)$ is the [surprisal](#def-surprisal) $\surprisalf{X}$.
+The [random variable](random-variables.qmd#def-random-variable) $-\log p(X)$ is the [surprisal](#def-surprisal) $\surprisalf{X}$.
 So $\crossentropyf{p}{p} = \E{\surprisalf{X}} = \entropyf{X}$ by the definition of [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index 74a2c34a..0bb5fa4b 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -13,7 +13,7 @@ The integrand is taken to be $0$ where $p(x) = 0$.
 
 ::: proof
 
-Let $X$ be a [continuous random variable](random-variables.qmd#def-continuous-rv) with density $p$, and let $g(x) \eqdef -\log q(x)$, so that $-\log q(X) = g(X)$.
+Let $X$ be a [continuous random variable](random-variables.qmd#def-continuous-rv) with [density](random-variables.qmd#def-pdf) $p$, and let $g(x) \eqdef -\log q(x)$, so that $-\log q(X) = g(X)$.
 Then [LOTUS](#thm-lotus) gives
 
 $$
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
index 41437d4a..b0ebfaae 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
@@ -31,6 +31,6 @@ $$
 Now suppose $q_k = 0$ for some $k$ with $p_k > 0$.
 Then $-\log q(X) \ge 0$, and $-\log q(X) = \infty$ with probability at least $p_k > 0$,
 so $\crossentropyf{p}{q} = \infty$.
-The sum is also $\infty$, because its term for $k$ is $-p_k \log 0 = \infty$ and its other terms are finite or $\infty$ but never $-\infty$.
+The right side is also $\infty$, because its term for $k$ is $-p_k \log 0 = \infty$ and its other terms are finite or $\infty$ but never $-\infty$.
 
 :::

From 2831f23be200ccd123824cc487c957b8ebff397e Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:48:50 +0000
Subject: [PATCH 03/10] Give each div in the cross-entropy page its own subfile

The one-div-per-subfile check examines every div in a file that a commit touches,
so the inline KL divergence, remark, example and source divs move out unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd              | 56 ++-----------------
 .../def-cross-entropy/_def-kl-divergence.qmd  | 10 ++++
 .../def-cross-entropy/_exm-cross-entropy.qmd  | 24 ++++++++
 .../_rem-cross-entropy-source.qmd             | 11 ++++
 .../def-cross-entropy/_rem-kl-asymmetric.qmd  |  7 +++
 5 files changed, 58 insertions(+), 50 deletions(-)
 create mode 100644 _subfiles/def-cross-entropy/_def-kl-divergence.qmd
 create mode 100644 _subfiles/def-cross-entropy/_exm-cross-entropy.qmd
 create mode 100644 _subfiles/def-cross-entropy/_rem-cross-entropy-source.qmd
 create mode 100644 _subfiles/def-cross-entropy/_rem-kl-asymmetric.qmd

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index ecc84b74..7c91f467 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -125,60 +125,16 @@ We take $-\log 0 = \infty$, so when $q(x) = 0$ for a value $x$ that $X$ takes wi
 
 {{< slidebreak >}}
 
-:::{#def-kl-divergence}
-#### Kullback--Leibler divergence (KL divergence, relative entropy)
-
-For $p$ and $q$ as in @def-cross-entropy,
-the **Kullback--Leibler divergence** (**KL divergence**, or **relative entropy**)
-of $q$ from $p$ is
-
-$$\kldivf{p}{q} \eqdef \sumto{k}{K} p_k \log \frac{p_k}{q_k} = \crossentropyf{p}{q} - \crossentropyf{p}{p}.$$
-
-:::
-
-::: {#rem-kl-asymmetric}
-#### The divergence is not symmetric
-
-In general, $\kldivf{p}{q} \ne \kldivf{q}{p}$,
-so KL divergence is not a distance between $p$ and $q$.
-
-:::
+{{< include pds/_subfiles/def-cross-entropy/_def-kl-divergence.qmd >}}
 
 {{< slidebreak >}}
 
-:::{#exm-cross-entropy}
-#### Cross-entropy and divergence for two coins
+{{< include pds/_subfiles/def-cross-entropy/_rem-kl-asymmetric.qmd >}}
 
-Let $p = (\tfrac{1}{2}, \tfrac{1}{2})$ be a fair coin
-and $q = (\tfrac{1}{4}, \tfrac{3}{4})$ a coin biased toward the second outcome.
-Using logarithms to base 2, $\log_2 \tfrac{1}{4} = -2$
-and $\log_2 \tfrac{3}{4} = \log_2 3 - 2$, so
-
-$$
-\ba
-\crossentropyf{p}{q} &= -\tfrac{1}{2} \log_2 \tfrac{1}{4} - \tfrac{1}{2} \log_2 \tfrac{3}{4} \\
-&= 1 + \tfrac{1}{2} \paren{2 - \log_2 3} \\
-&= 2 - \tfrac{1}{2} \log_2 3 \\
-&\approx 1.208, \\
-\crossentropyf{p}{p} &= -\tfrac{1}{2} \log_2 \tfrac{1}{2} - \tfrac{1}{2} \log_2 \tfrac{1}{2} \\
-&= 1, \\
-\kldivf{p}{q} &= \crossentropyf{p}{q} - \crossentropyf{p}{p} = 1 - \tfrac{1}{2} \log_2 3 \approx 0.208.
-\ea
-$$
-
-Here the cross-entropy is larger than the entropy of $p$,
-and the divergence is the difference.
-
-:::
+{{< slidebreak >}}
 
-::: {#rem-cross-entropy-source}
-#### Where the definitions come from
+{{< include pds/_subfiles/def-cross-entropy/_exm-cross-entropy.qmd >}}
 
-@murphy2012mlpp [section 2.8.2, pp. 57--58] defines the KL divergence
-and the cross-entropy as above,
-and gives the coding interpretation:
-the cross-entropy is the average number of bits needed
-to encode data from a source with distribution $p$
-using a code designed for $q$.
+{{< slidebreak >}}
 
-:::
+{{< include pds/_subfiles/def-cross-entropy/_rem-cross-entropy-source.qmd >}}
diff --git a/_subfiles/def-cross-entropy/_def-kl-divergence.qmd b/_subfiles/def-cross-entropy/_def-kl-divergence.qmd
new file mode 100644
index 00000000..8324e907
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_def-kl-divergence.qmd
@@ -0,0 +1,10 @@
+:::{#def-kl-divergence}
+#### Kullback--Leibler divergence (KL divergence, relative entropy)
+
+For $p$ and $q$ as in @def-cross-entropy,
+the **Kullback--Leibler divergence** (**KL divergence**, or **relative entropy**)
+of $q$ from $p$ is
+
+$$\kldivf{p}{q} \eqdef \sumto{k}{K} p_k \log \frac{p_k}{q_k} = \crossentropyf{p}{q} - \crossentropyf{p}{p}.$$
+
+:::
diff --git a/_subfiles/def-cross-entropy/_exm-cross-entropy.qmd b/_subfiles/def-cross-entropy/_exm-cross-entropy.qmd
new file mode 100644
index 00000000..099fcd58
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_exm-cross-entropy.qmd
@@ -0,0 +1,24 @@
+:::{#exm-cross-entropy}
+#### Cross-entropy and divergence for two coins
+
+Let $p = (\tfrac{1}{2}, \tfrac{1}{2})$ be a fair coin
+and $q = (\tfrac{1}{4}, \tfrac{3}{4})$ a coin biased toward the second outcome.
+Using logarithms to base 2, $\log_2 \tfrac{1}{4} = -2$
+and $\log_2 \tfrac{3}{4} = \log_2 3 - 2$, so
+
+$$
+\ba
+\crossentropyf{p}{q} &= -\tfrac{1}{2} \log_2 \tfrac{1}{4} - \tfrac{1}{2} \log_2 \tfrac{3}{4} \\
+&= 1 + \tfrac{1}{2} \paren{2 - \log_2 3} \\
+&= 2 - \tfrac{1}{2} \log_2 3 \\
+&\approx 1.208, \\
+\crossentropyf{p}{p} &= -\tfrac{1}{2} \log_2 \tfrac{1}{2} - \tfrac{1}{2} \log_2 \tfrac{1}{2} \\
+&= 1, \\
+\kldivf{p}{q} &= \crossentropyf{p}{q} - \crossentropyf{p}{p} = 1 - \tfrac{1}{2} \log_2 3 \approx 0.208.
+\ea
+$$
+
+Here the cross-entropy is larger than the entropy of $p$,
+and the divergence is the difference.
+
+:::
diff --git a/_subfiles/def-cross-entropy/_rem-cross-entropy-source.qmd b/_subfiles/def-cross-entropy/_rem-cross-entropy-source.qmd
new file mode 100644
index 00000000..c6717a2c
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_rem-cross-entropy-source.qmd
@@ -0,0 +1,11 @@
+::: {#rem-cross-entropy-source}
+#### Where the definitions come from
+
+@murphy2012mlpp [section 2.8.2, pp. 57--58] defines the KL divergence
+and the cross-entropy as above,
+and gives the coding interpretation:
+the cross-entropy is the average number of bits needed
+to encode data from a source with distribution $p$
+using a code designed for $q$.
+
+:::
diff --git a/_subfiles/def-cross-entropy/_rem-kl-asymmetric.qmd b/_subfiles/def-cross-entropy/_rem-kl-asymmetric.qmd
new file mode 100644
index 00000000..5847a95d
--- /dev/null
+++ b/_subfiles/def-cross-entropy/_rem-kl-asymmetric.qmd
@@ -0,0 +1,7 @@
+::: {#rem-kl-asymmetric}
+#### The divergence is not symmetric
+
+In general, $\kldivf{p}{q} \ne \kldivf{q}{p}$,
+so KL divergence is not a distance between $p$ and $q$.
+
+:::

From 355e1fec7ec14480bac1b3a0956b2d2410fc868a Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:50:42 +0000
Subject: [PATCH 04/10] Address review: clause line breaks, introduce X in the
 continuous theorem, link continuous surprisal

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd                      |  8 +++++---
 .../def-cross-entropy/_cor-cross-entropy-entropy.qmd  |  2 +-
 .../_rem-cross-entropy-expected-surprisal.qmd         |  2 +-
 .../_sol-cross-entropy-general-sum.qmd                |  3 ++-
 .../_thm-cross-entropy-continuous.qmd                 |  8 ++++++--
 .../def-cross-entropy/_thm-cross-entropy-discrete.qmd | 11 ++++++++---
 6 files changed, 23 insertions(+), 11 deletions(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index 7c91f467..5625b3f9 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -6,15 +6,17 @@ Let $X$ be a [random variable](random-variables.qmd#def-random-variable) with
 and let $q$ be a [mass function](random-variables.qmd#def-pmf) or density on the same values.
 The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
-is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$, of the [surprisal](#def-surprisal) measured by $q$:
+is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$,
+of the surprisal ([discrete](#def-surprisal) or [continuous](#def-surprisal-continuous)) measured by $q$:
 
 $$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
 
-We take $-\log 0 = \infty$, so when $q(x) = 0$ for a value $x$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability), the cross-entropy is infinite.
+We take $-\log 0 = \infty$,
+so when $q(x) = 0$ for a value $x$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability),
+the cross-entropy is infinite.
 
 :::
 
-
 {{< slidebreak >}}
 
 {{< include pds/_subfiles/def-cross-entropy/_exr-cross-entropy-numeric-pmf.qmd >}}
diff --git a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
index 30bfaea4..adec2171 100644
--- a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
+++ b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
@@ -11,7 +11,7 @@ $$\crossentropyf{p}{p} = \entropyf{X}.$$
 ::: proof
 
 By the definition of [cross-entropy](#def-cross-entropy) with $q = p$, $\crossentropyf{p}{p} = \E{-\log p(X)}$.
-The [random variable](random-variables.qmd#def-random-variable) $-\log p(X)$ is the [surprisal](#def-surprisal) $\surprisalf{X}$.
+The [random variable](random-variables.qmd#def-random-variable) $-\log p(X)$ is the surprisal ([discrete](#def-surprisal) or [continuous](#def-surprisal-continuous)) $\surprisalf{X}$.
 So $\crossentropyf{p}{p} = \E{\surprisalf{X}} = \entropyf{X}$ by the definition of [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd b/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
index c669f3e6..f813fc93 100644
--- a/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
+++ b/_subfiles/def-cross-entropy/_rem-cross-entropy-expected-surprisal.qmd
@@ -2,7 +2,7 @@
 #### Cross-entropy is an expected surprisal
 
 By definition, the [cross-entropy](#def-cross-entropy) $\crossentropyf{p}{q}$ is the average [surprisal](#def-surprisal) of the values drawn from $p$,
-with each surprisal measured by $q$ as $-\log q_k$.
+with each surprisal measured by $q$ as $-\log q(x)$.
 By @cor-cross-entropy-entropy, measuring with $p$ itself gives the [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
index 48ef7aeb..4e82cd50 100644
--- a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
+++ b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
@@ -16,6 +16,7 @@ $$
 \ea
 $$
 
-The last line adds the terms with $p_k = 0$, which are $0$ by the convention in @thm-cross-entropy-discrete.
+The last line adds the terms with $p_k = 0$,
+which are $0$ by the convention in @thm-cross-entropy-discrete.
 
 :::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index 0bb5fa4b..ba1cab95 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -1,19 +1,23 @@
 :::{#thm-cross-entropy-continuous}
 #### Cross-entropy of two densities
 
-Let $p$ and $q$ be [probability density functions](random-variables.qmd#def-pdf) on the same [range](random-variables.qmd#def-range) $\rangef{X}$,
+Let $X$ be a [continuous random variable](random-variables.qmd#def-continuous-rv) with [density](random-variables.qmd#def-pdf) $p$ on its [range](random-variables.qmd#def-range) $\rangef{X}$,
+and let $q$ be a density on $\rangef{X}$,
 with $q(x) > 0$ wherever $p(x) > 0$.
 Then the [cross-entropy](#def-cross-entropy) is
 
 $$\crossentropyf{p}{q} = -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx.$$
 
 The integrand is taken to be $0$ where $p(x) = 0$.
+If instead $q(x) = 0$ on a set to which $X$ belongs with positive [probability](probability-basics.qmd#def-probability),
+both sides are $\infty$, by the argument for mass functions in @thm-cross-entropy-discrete.
 
 :::
 
 ::: proof
 
-Let $X$ be a [continuous random variable](random-variables.qmd#def-continuous-rv) with [density](random-variables.qmd#def-pdf) $p$, and let $g(x) \eqdef -\log q(x)$, so that $-\log q(X) = g(X)$.
+Let $g(x) \eqdef -\log q(x)$,
+so that $-\log q(X) = g(X)$.
 Then [LOTUS](#thm-lotus) gives
 
 $$
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
index b0ebfaae..e9089475 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-discrete.qmd
@@ -13,7 +13,9 @@ A term with $p_k = 0$ is taken to be $0$ whatever $q_k$ is, as for [entropy](#th
 
 ::: proof
 
-Let $X$ have [mass function](random-variables.qmd#def-pmf) $p$, and let $g(k) \eqdef -\log q_k$, so that $-\log q(X) = g(X)$.
+Let $X$ have [mass function](random-variables.qmd#def-pmf) $p$,
+and let $g(k) \eqdef -\log q_k$,
+so that $-\log q(X) = g(X)$.
 First suppose $q_k > 0$ whenever $p_k > 0$.
 Then $g(k)$ is finite for every value $k$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability),
 and [LOTUS](#thm-lotus) gives
@@ -29,8 +31,11 @@ $$
 $$
 
 Now suppose $q_k = 0$ for some $k$ with $p_k > 0$.
-Then $-\log q(X) \ge 0$, and $-\log q(X) = \infty$ with probability at least $p_k > 0$,
+Then $-\log q(X) \ge 0$,
+and $-\log q(X) = \infty$ with probability at least $p_k > 0$,
 so $\crossentropyf{p}{q} = \infty$.
-The right side is also $\infty$, because its term for $k$ is $-p_k \log 0 = \infty$ and its other terms are finite or $\infty$ but never $-\infty$.
+The right side is also $\infty$,
+because its term for $k$ is $-p_k \log 0 = \infty$
+and its other terms are finite or $\infty$ but never $-\infty$.
 
 :::

From 461c892be543f119ba68dd469b9f7fb699c74b02 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:50:55 +0000
Subject: [PATCH 05/10] Link first uses of surprisal and drop unlinked terms

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd                              | 2 +-
 _subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd    | 2 +-
 _subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index 5625b3f9..8677d38b 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -7,7 +7,7 @@ and let $q$ be a [mass function](random-variables.qmd#def-pmf) or density on the
 The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
 is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$,
-of the surprisal ([discrete](#def-surprisal) or [continuous](#def-surprisal-continuous)) measured by $q$:
+of the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuous) form) measured by $q$:
 
 $$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
 
diff --git a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
index adec2171..b7f68031 100644
--- a/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
+++ b/_subfiles/def-cross-entropy/_cor-cross-entropy-entropy.qmd
@@ -11,7 +11,7 @@ $$\crossentropyf{p}{p} = \entropyf{X}.$$
 ::: proof
 
 By the definition of [cross-entropy](#def-cross-entropy) with $q = p$, $\crossentropyf{p}{p} = \E{-\log p(X)}$.
-The [random variable](random-variables.qmd#def-random-variable) $-\log p(X)$ is the surprisal ([discrete](#def-surprisal) or [continuous](#def-surprisal-continuous)) $\surprisalf{X}$.
+The [random variable](random-variables.qmd#def-random-variable) $-\log p(X)$ is the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuous) form) $\surprisalf{X}$.
 So $\crossentropyf{p}{p} = \E{\surprisalf{X}} = \entropyf{X}$ by the definition of [entropy](#def-entropy).
 
 :::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index ba1cab95..1921f21a 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -9,8 +9,8 @@ Then the [cross-entropy](#def-cross-entropy) is
 $$\crossentropyf{p}{q} = -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx.$$
 
 The integrand is taken to be $0$ where $p(x) = 0$.
-If instead $q(x) = 0$ on a set to which $X$ belongs with positive [probability](probability-basics.qmd#def-probability),
-both sides are $\infty$, by the argument for mass functions in @thm-cross-entropy-discrete.
+If instead $X$ takes a value with $q(x) = 0$ with positive [probability](probability-basics.qmd#def-probability),
+both sides are $\infty$, by the same argument as in @thm-cross-entropy-discrete.
 
 :::
 

From e74f0bdff45821dacf56a2a9ca46ce9df8a61f17 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:53:05 +0000
Subject: [PATCH 06/10] State the infinite cross-entropy case as P(q(X) = 0) >
 0 and prove it for densities

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd                    |  2 +-
 .../_thm-cross-entropy-continuous.qmd               | 13 +++++++++++--
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index 8677d38b..b2c232d8 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -12,7 +12,7 @@ of the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuou
 $$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
 
 We take $-\log 0 = \infty$,
-so when $q(x) = 0$ for a value $x$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability),
+so when $\P(q(X) = 0) > 0$,
 the cross-entropy is infinite.
 
 :::
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index 1921f21a..04454357 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -9,8 +9,8 @@ Then the [cross-entropy](#def-cross-entropy) is
 $$\crossentropyf{p}{q} = -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx.$$
 
 The integrand is taken to be $0$ where $p(x) = 0$.
-If instead $X$ takes a value with $q(x) = 0$ with positive [probability](probability-basics.qmd#def-probability),
-both sides are $\infty$, by the same argument as in @thm-cross-entropy-discrete.
+If instead $\P(q(X) = 0) > 0$,
+both sides are $\infty$.
 
 :::
 
@@ -29,4 +29,13 @@ $$
 \ea
 $$
 
+Now suppose $\P(q(X) = 0) > 0$.
+Then $-\log q(X) \ge 0$,
+and $-\log q(X) = \infty$ with positive probability,
+so $\crossentropyf{p}{q} = \infty$.
+The integrand $-p(x) \log q(x)$ is $\infty$ on the set where $q(x) = 0$ and $p(x) > 0$,
+which has positive integral of $p$,
+and it is never $-\infty$,
+so the integral is also $\infty$.
+
 :::

From 77a10403a96cf6fe885a98074f64c34b08fd6e28 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:53:23 +0000
Subject: [PATCH 07/10] Link probability and avoid unlinked terms in the
 infinite-case proof

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 .../def-cross-entropy/_thm-cross-entropy-continuous.qmd   | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index 04454357..3ae17d79 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -31,11 +31,11 @@ $$
 
 Now suppose $\P(q(X) = 0) > 0$.
 Then $-\log q(X) \ge 0$,
-and $-\log q(X) = \infty$ with positive probability,
+and $-\log q(X) = \infty$ with positive [probability](probability-basics.qmd#def-probability),
 so $\crossentropyf{p}{q} = \infty$.
-The integrand $-p(x) \log q(x)$ is $\infty$ on the set where $q(x) = 0$ and $p(x) > 0$,
-which has positive integral of $p$,
+The integrand $-p(x) \log q(x)$ is $\infty$ wherever $q(x) = 0$ and $p(x) > 0$,
+which is a region of positive probability under $p$,
 and it is never $-\infty$,
-so the integral is also $\infty$.
+so the right side is also $\infty$.
 
 :::

From 6e0f1fc1eb6cd34f28716b3546e3b9e6f5bd5dd5 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 22:55:51 +0000
Subject: [PATCH 08/10] Break remaining multi-clause lines and say Otherwise in
 the continuous theorem

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 .../def-cross-entropy/_sol-cross-entropy-general-sum.qmd     | 3 ++-
 .../def-cross-entropy/_thm-cross-entropy-continuous.qmd      | 5 +++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
index 4e82cd50..1e2c297b 100644
--- a/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
+++ b/_subfiles/def-cross-entropy/_sol-cross-entropy-general-sum.qmd
@@ -1,6 +1,7 @@
 :::{#sol-cross-entropy-general-sum}
 
-Let $g(k) \eqdef -\log q_k$, so that $-\log q_X = g(X)$.
+Let $g(k) \eqdef -\log q_k$,
+so that $-\log q_X = g(X)$.
 Since $q_k > 0$ whenever $p_k > 0$,
 $g(k)$ is finite for every value $k$ that $X$ takes with positive [probability](probability-basics.qmd#def-probability).
 By [LOTUS](#thm-lotus),
diff --git a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
index 3ae17d79..cc145340 100644
--- a/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
+++ b/_subfiles/def-cross-entropy/_thm-cross-entropy-continuous.qmd
@@ -9,7 +9,7 @@ Then the [cross-entropy](#def-cross-entropy) is
 $$\crossentropyf{p}{q} = -\int_{x \in \rangef{X}} p(x) \log q(x) \, dx.$$
 
 The integrand is taken to be $0$ where $p(x) = 0$.
-If instead $\P(q(X) = 0) > 0$,
+Otherwise, if $\P(q(X) = 0) > 0$,
 both sides are $\infty$.
 
 :::
@@ -33,7 +33,8 @@ Now suppose $\P(q(X) = 0) > 0$.
 Then $-\log q(X) \ge 0$,
 and $-\log q(X) = \infty$ with positive [probability](probability-basics.qmd#def-probability),
 so $\crossentropyf{p}{q} = \infty$.
-The integrand $-p(x) \log q(x)$ is $\infty$ wherever $q(x) = 0$ and $p(x) > 0$,
+The integrand $-p(x) \log q(x)$ is $\infty$
+wherever $q(x) = 0$ and $p(x) > 0$,
 which is a region of positive probability under $p$,
 and it is never $-\infty$,
 so the right side is also $\infty$.

From f5929a3100e4ce1bcb148371031c1d85a95591a2 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 23:42:45 +0000
Subject: [PATCH 09/10] Write the cross-entropy definition with the surprisal
 macro

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index b2c232d8..c2f20604 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -7,9 +7,10 @@ and let $q$ be a [mass function](random-variables.qmd#def-pmf) or density on the
 The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
 is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$,
-of the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuous) form) measured by $q$:
+of the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuous) form) $\surprisal_q$ measured by $q$,
+where $\surprisal_q(x) \eqdef -\log q(x)$:
 
-$$\crossentropyf{p}{q} \eqdef \E{-\log q(X)}.$$
+$$\crossentropyf{p}{q} \eqdef \E{\surprisal_q\paren{X}}.$$
 
 We take $-\log 0 = \infty$,
 so when $\P(q(X) = 0) > 0$,

From 6aebc565290e150e00bc493744e8942f1365a793 Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Fri, 30 Oct 2026 23:45:23 +0000
Subject: [PATCH 10/10] Use the paren macro in the surprisal-measured-by-q
 definition

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019xfc9mmEbJ5cfXnwuhpxdh
---
 _subfiles/_def-cross-entropy.qmd | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/_subfiles/_def-cross-entropy.qmd b/_subfiles/_def-cross-entropy.qmd
index c2f20604..a54ae416 100644
--- a/_subfiles/_def-cross-entropy.qmd
+++ b/_subfiles/_def-cross-entropy.qmd
@@ -8,7 +8,7 @@ The **cross-entropy** of $q$ relative to $p$
 [@wikipedia_cross_entropy]
 is the [expectation](#def-expectation), when $X$ has [distribution](random-variables.qmd#def-cdf) $p$,
 of the [surprisal](#def-surprisal) (or its [continuous](#def-surprisal-continuous) form) $\surprisal_q$ measured by $q$,
-where $\surprisal_q(x) \eqdef -\log q(x)$:
+where $\surprisal_q\paren{x} \eqdef -\log q\paren{x}$:
 
 $$\crossentropyf{p}{q} \eqdef \E{\surprisal_q\paren{X}}.$$