diff --git a/manuscript/narrative-review/abstract.md b/manuscript/narrative-review/abstract.md index 6ff600f..11382b8 100644 --- a/manuscript/narrative-review/abstract.md +++ b/manuscript/narrative-review/abstract.md @@ -1,16 +1,8 @@ - + # Abstract -[Opening sentence: the field has moved from whole-clip ISC to event-locked methods.] - -[Sentence 2: most empirical evidence is adult fMRI / iEEG / scalp-EEG ISC; per-shot ERSP in a developmental cohort viewing silent character animation has no precedent.] - -[Sentence 3: we review the four-perspective corpus that constrains the design space.] - -[Sentence 4: psychophysics, action, language, and emotion make divergent and partly-falsifiable predictions about the 0 to 500 ms post-shot-onset window.] - -[Closing sentence: HBN-EEG Release 3 viewing *The Present* (Pixar 2014) sits at this empty intersection; the 100 Hz local working set caps beta-band claims until a 500 Hz validation pass.] - - +Naturalistic-stimulus neuroscience has moved from whole-clip inter-subject correlation (ISC) to event-locked methods that interrogate individual shots. Most empirical evidence is adult functional magnetic resonance imaging (fMRI), adult intracranial electroencephalography (iEEG), or adult scalp electroencephalography (EEG) ISC. Per-shot event-related spectral perturbation (ERSP) in a developmental cohort viewing silent character animation has no published precedent. We review the corpus that constrains this design space and argue that psychophysics, action, language, and emotion make divergent, partly-falsifiable predictions about the 0 to 500 ms post-shot-onset window. The Healthy Brain Network EEG Release 3 cohort viewing *The Present* (Pixar 2014) sits at this empty intersection; the 100 Hz local working set caps beta-band claims until a 500 Hz validation pass. diff --git a/manuscript/narrative-review/boxes/box1_anchor.md b/manuscript/narrative-review/boxes/box1_anchor.md index a316bc5..1768832 100644 --- a/manuscript/narrative-review/boxes/box1_anchor.md +++ b/manuscript/narrative-review/boxes/box1_anchor.md @@ -1,18 +1,9 @@ # Box 1: HBN-EEG Release 3 as the anchor cohort -[Paragraph (~180 words). Describe the HBN-EEG R3 cohort: -- 5-21-year-old participants in a developmental research setting. -- 128-channel HydroCel Geodesic Sensor Net. -- Passive viewing of the 3.5-minute Pixar short *The Present* (2014). -- Local working set: 184 subjects at 100 Hz BDF (Nyquist-aware downsample of original 500 Hz). -- 56 stimulus-side shots with per-shot `onset`, `duration`, `LLR`, `has_boy`, `has_puppy` annotations. -- After invalidating 3 high-drift rows (`match_diff_s > 1.0 s`), 49 trusted rows: 20 boy-only + 15 puppy-only shots for mutually exclusive single-agent contrast. -- Pipeline: BIDS import, 1 Hz HPF, conditional cleanline (gated by Nyquist), `clean_rawdata` channel rejection, AMICA, ICLabel (brain threshold 0.69), dipfit5, std_precomp ERSP. -- F4 carry-forward statement: the anchor case rests on Petroni-Cohen 2018 (external precedent) + this partly-validated pipeline (internal feasibility). The six-feature uniqueness conjunction is *descriptive*, not evidential.] +The Healthy Brain Network EEG (HBN-EEG) Release 3 cohort recruits 5- to 21-year-old participants in a developmental research setting and records 128-channel HydroCel Geodesic Sensor Net during passive viewing of the 3.5-minute Pixar short *The Present* (2014). The local working set used in our pipeline development is 184 subjects at 100 Hz Biosignal Data Format (BDF), a Nyquist-aware downsample of the original 500 Hz. The 56 stimulus-side shots carry per-shot `onset`, `duration`, `LLR`, `has_boy`, and `has_puppy` annotations. After invalidating 3 high-drift rows (`match_diff_s > 1.0 s`), 49 rows are trusted, yielding 20 boy-only and 15 puppy-only shots for the mutually exclusive single-agent contrast. The pipeline runs BIDS import, 1 Hz high-pass filter, conditional cleanline (gated by Nyquist), `clean_rawdata` channel rejection, AMICA, ICLabel (brain threshold 0.69), dipfit5, and `std_precomp` ERSP. The anchor case rests on Petroni and colleagues 2018 [Petroni2018TheVO] as the external precedent and this partly-validated pipeline as the internal feasibility proof. diff --git a/manuscript/narrative-review/boxes/outstanding-questions.md b/manuscript/narrative-review/boxes/outstanding-questions.md index e8d4df4..e3d0f99 100644 --- a/manuscript/narrative-review/boxes/outstanding-questions.md +++ b/manuscript/narrative-review/boxes/outstanding-questions.md @@ -1,22 +1,14 @@ # Outstanding Questions Box -[Budget: 5-7 questions. Each question is one sentence; forward-looking; adjudicates between perspectives or names a gap the field could fill. - -1. [Q1: per-shot ERSP in developmental cohort survives LLR and motion-energy partialling at 0-500 ms?] - -2. [Q2: mu-band ERD over central rolandic clusters elicited by animated-character action observation as it is by adult hand-action observation?] - -3. [Q3: cuteness-driven affective response in children produces a sub-second EEG signature distinguishable from generic arousal? And is the signature compatible with frontal asymmetry at sub-second timescales given reliability concerns?] - -4. [Q4: can the topographic-and-band rejection region for the four-perspective ranking be pre-registered before group analysis? Is the central-rolandic vs frontal-asymmetric vs language-network discrimination operationalisable from EEG IC clusters?] - -5. [Q5: can a multimodal vision-language embedding regressor substitute for language-model surprisal on silent stimuli?] - -6. [Q6: does within-stimulus narrative position (three-act trajectory) explain condition-level effects that survive low-level partialling in single-stimulus designs?] - -7. [Q7: residual saccade-locked variance contamination without synchronous eye tracker, and the cohort size at which ICA-only artifact rejection becomes sufficient?]] +1. Does per-shot EEG spectral perturbation in a developmental cohort viewing silent animation survive partialling for log luminance ratio and motion energy at the 0 to 500 ms window? +2. Is mu-band ERD over central rolandic clusters elicited by animated-character action observation, as it is by hand-action observation in adults? +3. Does cuteness-driven affective response in children produce a sub-second EEG signature distinguishable from generic arousal in the alpha band, and is the signature compatible with frontal asymmetry at sub-second timescales given the meta-analytic reliability concerns? +4. Can a topographic-and-band rejection region for the four-perspective ranking be pre-registered before group analysis, and is the central-rolandic-versus-frontal-asymmetric-versus-language-network discrimination operationalisable from EEG IC clusters? +5. Can a multimodal vision-language embedding regressor substitute for language-model surprisal on silent stimuli? +6. Does within-stimulus narrative position (three-act trajectory) explain condition-level effects that survive low-level partialling in single-stimulus designs? +7. What is the residual saccade-locked variance contamination in shot-onset EEG ERSP without a synchronous eye tracker, and at what cohort size does ICA-only artifact rejection become sufficient? diff --git a/manuscript/narrative-review/boxes/trends.md b/manuscript/narrative-review/boxes/trends.md index bed9b87..d71e8c7 100644 --- a/manuscript/narrative-review/boxes/trends.md +++ b/manuscript/narrative-review/boxes/trends.md @@ -1,20 +1,15 @@ - # Trends Box: recent developments enabling the per-shot framing -[Budget: ~200-250 words. Bullet list of 5-7 recent developments. +Recent advances make the per-shot framing newly tractable. -- **Whole-brain shot-cut response in adult intracranial EEG.** Nentwich et al. 2023: 6328 contacts in 23 patients across 43.6 minutes of film clips; triple-regressor against optical-flow magnitude, saccade onsets, and film-cut onsets; whole-brain saccade- and cut-locked responses with motion concentrated in occipitoparietal cortex. - -- **Hidden-Markov-model recovery of event states from fMRI.** Baldassano et al. 2017: hierarchy of event boundaries from Sherlock-movie fMRI; hippocampal boundary signals predict subsequent free recall. - -- **Cross-sectional developmental EEG-ISC.** Petroni et al. 2018: whole-clip EEG-ISC reliability across ages 6-44 during passive viewing of six naturalistic videos, peaking in childhood. - -- **Silent abstract animation for MRI compliance.** Vanderwal et al. 2015: Inscapes, used by HBN itself; reliable network-level activity from a silent stimulus. - -- **Multi-level cinematic-feature regression.** Kauttonen et al. 2015: multi-level cinematic features regressed against fMRI ISC; methodological template for shot-level feature annotation. - -- **Open developmental EEG releases.** HBN-EEG and Studyforrest (Hanke et al. 2014) make large-N developmental datasets available for naturalistic-stimulus analysis at unprecedented scale.] +- **Whole-brain shot-cut response in adult intracranial EEG.** Nentwich and colleagues 2023 recorded 6328 contacts in 23 patients across 43.6 minutes of film clips and regressed responses against optical-flow magnitude, saccade onsets, and film-cut onsets simultaneously, finding whole-brain saccade- and cut-locked responses with motion concentrated in occipitoparietal cortex [Nentwich2023SemanticNM]. +- **Hidden Markov model recovery of event states from fMRI.** Baldassano and colleagues 2017 recovered a hierarchy of event boundaries from Sherlock-movie fMRI, with hippocampal boundary signals predicting subsequent free recall [baldassano2017event]. +- **Cross-sectional developmental EEG-ISC.** Petroni and colleagues 2018 reported whole-clip EEG-ISC reliability across ages 6 to 44 during passive viewing of six naturalistic videos, peaking in childhood [Petroni2018TheVO]. +- **Silent abstract animation for MRI compliance.** Vanderwal and colleagues 2015 built Inscapes, used by HBN itself, with reliable network-level activity [Vanderwal2015InscapesAM]. +- **Multi-level cinematic-feature regression.** Kauttonen and colleagues 2015 regressed multi-level cinematic features against fMRI ISC, supplying a methodological template for shot-level feature annotation [Kauttonen2015OptimizingMF]. +- **Open developmental EEG releases.** HBN-EEG and Studyforrest [Hanke2014AH7] make large-N developmental datasets available for naturalistic-stimulus analysis at unprecedented scale. diff --git a/manuscript/narrative-review/figures.md b/manuscript/narrative-review/figures.md index 37e80d6..5c45ee8 100644 --- a/manuscript/narrative-review/figures.md +++ b/manuscript/narrative-review/figures.md @@ -1,31 +1,31 @@ # Figure legends ## Figure 1. Four-perspective strand map -[Caption placeholder. Compose in Phase 2 after the Phase 3 figure-qa report is in. Reference the four perspectives, the 15 corpus themes, the colour code (Okabe-Ito), and the legend's redundant shape-and-colour encoding.] +Four research perspectives (psychophysics, action, language, emotion) mapped against 15 corpus themes. Filled coloured circles indicate substantial contribution from the perspective to the theme; outlined circles indicate absence or peripheral relevance. The four columns are colour-coded by perspective and the legend doubles as a colour key. Theme overlap is intentional: the perspectives interact at the per-shot ERSP level rather than partitioning variance cleanly. ## Figure 2. Naturalness gradient and developmental cohort coverage -[Caption placeholder. X-axis: stimulus naturalness gradient. Y-axis: participant cohort. Markers: size by number of corpus cards, shape by modality. Highlight the (child, character animation, EEG ERSP) empty cell.] +Stimulus naturalness on the x-axis (controlled gratings, static photographs, Heider-Simmel triangles, abstract animation, character animation, live-action film) versus participant cohort on the y-axis (adult, adolescent, child). Markers are sized by number of corpus cards and shaped and coloured by modality (fMRI as circle, EEG as square, MEG as triangle, intracranial EEG as diamond; behavioural-only entries as the letter b). The dashed yellow rectangle at (child, character animation) marks the target cell for per-shot EEG ERSP at the 0 to 500 ms window: existing coverage is whole-clip ISC, not per-shot ERSP. ## Figure 3. Gap matrix -[Caption placeholder. 8 named gaps from `research/synthesis/gap-analysis.md` versus 4 prior-effort axes. Filled cells contain representative card slugs; empty cells highlighted.] +Eight named gaps from the four-strand corpus (rows) versus four prior-effort axes (cinematic fMRI, naturalistic scalp EEG, intracranial and MEG, behavioural and eye-tracking; columns). Filled cells list a representative card slug; cells marked "no coverage" with a vermillion dashed border indicate uncovered combinations. Thirteen cells across the eight rows carry no coverage, defining the design space for per-shot developmental EEG ERSP. ## Figure 4. Predictions and falsification regions, per perspective -[Caption placeholder. Each perspective (row) gives topography (with head schematic), band, latency, pre-registered falsification region. Reference the F5 carry-forward.] +Each perspective (row) is named with its predicted topography (with a head schematic showing the topographic focus), band, latency, and pre-registered falsification region. Psychophysics is the covariate, not the prediction. Action predicts central-rolandic mu-band (8 to 13 Hz) ERD over electrodes C3, Cz, and C4, with possible beta rebound (15 to 25 Hz). Language predicts no signal locally; a surviving cluster in left-frontotemporal IC space (Lipkin atlas negative-control mask) falsifies the four-perspective ranking. Emotion predicts early occipital alpha desynchronisation (80 to 300 ms) and later frontal F3/F4 asymmetry (200 to 500 ms), at incompatible latencies and topographies. The cluster-level alpha for falsification is p < 0.05 corrected by mass-univariate permutation. ## Figure assembly notes (for Phase 3) - **Composer**: `/figures:scientific-figure` (multi-panel; the recommended composer per its skill description). -- **Panel sources**: - - `/figures:svg-figure` for the matrix-style schematic panels (Figs 1, 3). - - `/figures:transparent-icons` for stimulus thumbnails (Fig 2 x-axis: gratings, photographs, Heider-Simmel triangles, abstract animation, character animation, live-action film) and brain-topo icons (Fig 4 topography column). - - `/figures:plot-styling` only if any panel needs data plotting; the current 4 figures are conceptual schematics, so it may not be needed. +- **Panel sources**: `/figures:svg-figure` for matrix-style schematic panels (Figs 1, 3); `/figures:transparent-icons` for stimulus thumbnails (Fig 2 x-axis) and brain-topo icons (Fig 4 topography column); `/figures:plot-styling` only if any panel needs data plotting. - **QA**: `/figures:figure-qa` on every panel and on the composed figure. Address all findings before completion. No deferrals. diff --git a/manuscript/narrative-review/glossary.md b/manuscript/narrative-review/glossary.md index a40166f..d79cefc 100644 --- a/manuscript/narrative-review/glossary.md +++ b/manuscript/narrative-review/glossary.md @@ -1,38 +1,36 @@ # Glossary -[Each entry follows the pattern: **Term (acronym if any).** Definition sentence(s), <=50 words.] +**Event-related spectral perturbation (ERSP).** A time-frequency representation of the change in spectral power (and optionally inter-trial phase coherence) at each frequency and latency relative to an event, computed by averaging single-trial power spectrograms after subtracting a baseline window. -**Event-related spectral perturbation (ERSP).** [Definition placeholder.] +**Inter-subject correlation (ISC).** The Pearson correlation between time courses of different participants viewing the same stimulus, computed voxel-wise (fMRI) or component-wise (EEG and MEG); a stimulus-locked reliability metric. -**Inter-subject correlation (ISC).** [Definition placeholder.] +**Log luminance ratio (LLR).** The base-10 logarithm of the ratio of mean luminance in the first post-shot frame to the mean luminance in the last pre-shot frame; a per-shot stimulus-side regressor of the visual transient at shot onset. -**Log luminance ratio (LLR).** [Definition placeholder.] +**Adaptive mixture independent component analysis (AMICA).** A multi-model extension of ICA that estimates a mixture of ICA decompositions, used in EEGLAB-style pipelines for artifact-resistant source separation. -**Adaptive mixture independent component analysis (AMICA).** [Definition placeholder.] +**IC classification (ICLabel).** An automated classifier that labels independent components as brain, muscle, eye, heart, line noise, channel noise, or other. -**IC classification (ICLabel).** [Definition placeholder.] +**Mu rhythm.** An 8 to 13 Hz oscillation over central rolandic electrodes (C3, Cz, C4) that desynchronises during motor execution and during observation of others' actions. -**Mu rhythm.** [Definition placeholder.] +**Event-related desynchronisation (ERD).** A decrease in spectral power in a specific frequency band time-locked to an event, interpreted as cortical activation in the band's reference resting state. -**Event-related desynchronisation (ERD).** [Definition placeholder.] +**Frontal alpha asymmetry.** The difference between right and left frontal alpha-band (8 to 13 Hz) power, traditionally framed as an approach-withdrawal index; recent meta-analyses report smaller effects and reliability concerns. -**Frontal alpha asymmetry.** [Definition placeholder.] +**Default-mode network (DMN).** A set of cortical regions including medial prefrontal cortex, posterior cingulate cortex, and lateral parietal cortex that show coordinated activity during internally directed cognition, narrative comprehension, and rest. -**Default-mode network (DMN).** [Definition placeholder.] +**Theory of mind (ToM).** The cognitive capacity to attribute mental states (beliefs, desires, intentions) to self and others. -**Theory of mind (ToM).** [Definition placeholder.] +**Temporal response function (TRF).** A linear filter that maps a continuous stimulus feature to a continuous neural response, fit via regularised regression. -**Temporal response function (TRF).** [Definition placeholder.] +**Baby schema.** A set of infantile physical features (large head, large eyes, round cheeks) that elicit attentional, affective, and caregiving responses. -**Baby schema.** [Definition placeholder.] +**Naturalistic stimulus.** A continuous, ecologically valid stimulus (typically a film, audiobook, or video game) presented without trial-by-trial structuring. Naturalness is a continuum from controlled gratings to live-action film, with character animation and abstract Heider-Simmel triangles as intermediate points (Figure 2). -**Naturalistic stimulus.** [Definition placeholder.] +**Event segmentation.** The cognitive process of parsing continuous experience into discrete events at moments of high prediction error, organised hierarchically. -**Event segmentation.** [Definition placeholder.] - -**Temporal receptive window.** [Definition placeholder.] +**Temporal receptive window.** The span of preceding time over which a brain region integrates information; ranges from milliseconds in primary sensory cortex to tens of seconds in default-mode regions. diff --git a/manuscript/narrative-review/highlights.md b/manuscript/narrative-review/highlights.md index 86277f9..44fc3a1 100644 --- a/manuscript/narrative-review/highlights.md +++ b/manuscript/narrative-review/highlights.md @@ -1,11 +1,10 @@ - + # Highlights -- [bullet 1: 1-line claim about the per-shot turn in naturalistic EEG] -- [bullet 2: 1-line claim that four perspectives diverge in their per-shot predictions] -- [bullet 3: 1-line claim about LM-regressor non-transfer to silent stimuli] -- [bullet 4: 1-line claim about the developmental empty-cell] -- [bullet 5: 1-line claim about HBN-EEG R3 as the testbed] +- Naturalistic EEG has shifted from whole-clip ISC to shot-locked spectral metrics +- Four perspectives diverge on per-shot EEG predictions of naturalistic film +- Language-model regressors cannot transfer to silent character animation +- Per-shot EEG ERSP in children viewing animation has no published precedent +- HBN-EEG Release 3 sits at this empty intersection and can test the predictions diff --git a/manuscript/narrative-review/sections/01_introduction.md b/manuscript/narrative-review/sections/01_introduction.md index 6b3f317..335ce98 100644 --- a/manuscript/narrative-review/sections/01_introduction.md +++ b/manuscript/narrative-review/sections/01_introduction.md @@ -1,24 +1,18 @@ # 1. Introduction: the per-shot turn -[Opening paragraph (~150 words): two-wave history of naturalistic-stimulus neuroscience. -- Wave 1, functional: Hasson et al. 2004 fMRI ISC (45% of cortex synchronises during free viewing). -- Wave 2, electrophysiological: Dmochowski et al. 2012 EEG correlated-component analysis; Ki et al. 2016 attention modulation; Cohen and Parra 2016; Dmochowski et al. 2014; Madsen and Parra 2022. -- Wave 3 (emerging): per-shot / per-event interrogation. Nentwich et al. 2023 whole-brain shot-cut iEEG; Ben-Yakov and Henson 2018 hippocampal cut-vs-boundary; event-segmentation theory: Zacks et al. 2007, Speer et al. 2007, Baldassano et al. 2017.] - -[Paragraph 2 (~100 words): developmental tradition. -- Richardson et al. 2018 pediatric ToM/pain via Pixar shorts; Vanderwal et al. 2015 Inscapes for MRI compliance; Petroni et al. 2018 EEG-ISC across ages 6-44. -- Conclude: none have reported per-shot ERSP at the 0 to 500 ms post-onset window in a child cohort viewing animation.] +Naturalistic-stimulus neuroscience moved from controlled gratings to feature films in two waves. The first wave was functional. Hasson and colleagues showed that voxel-level cortical activity synchronises across viewers of the same audiovisual movie in up to 45 percent of cortex during free functional magnetic resonance imaging (fMRI) viewing [Hasson2004IntersubjectSO]. The second wave was electrophysiological. Correlated-component analysis on scalp electroencephalography (EEG) demonstrated that engagement, attention, memory, and audience preference all scale with the reliability of stimulus-locked variance [dmochowski2012correlated; Ki2016AttentionSM; Cohen2016MemorableAN; Dmochowski2014AudiencePA; Madsen2022CognitivePO]. A third wave is now emerging that interrogates individual events within the continuous stream. Nentwich and colleagues recorded 6328 contacts in 23 patients across 43.6 minutes of film clips, regressed responses against optical-flow magnitude, saccade onsets, and film-cut onsets simultaneously, and found whole-brain shot-cut transients with semantic novelty modulation [Nentwich2023SemanticNM]. The hippocampus distinguishes within-event camera cuts from across-event narrative boundaries [Ben-Yakov2018TheHF], and event-segmentation theory frames boundaries as moments of high prediction error with hierarchical timescales mapped from sensory cortex to default-mode regions [zacks2007event; speer2007narrative; baldassano2017event]. -[Paragraph 3 (~150 words): four-perspective thesis and roadmap. -- Four perspectives (psychophysics, action, language, emotion) make divergent and partly-falsifiable predictions about this empty cell. -- One-sentence methods footprint: LLR partialling; ICA-only artifact rejection (no synchronous eye tracker in HBN-EEG R3). -- Section roadmap: Sections 2-6 develop the perspectives; Section 7 synthesises into a pre-registerable rejection region; Box 1 names the anchor cohort.] +A separate developmental tradition has used Pixar shorts in fMRI to map theory of mind (ToM) and pain networks in children as young as three [Richardson2018DevelopmentOT] and silent abstract animation to improve magnetic resonance imaging (MRI) compliance and reveal reliable network-level activity [Vanderwal2015InscapesAM]. Cross-sectional EEG inter-subject correlation (ISC) across ages 6 to 44 is the closest electrophysiological developmental anchor; ISC is highest in children and declines into adulthood [Petroni2018TheVO]. None of these traditions has reported per-shot event-related spectral perturbation (ERSP) at the 0 to 500 ms post-onset window in a child cohort viewing animation. -[Final transition sentence: explicit bridge into Section 2's four-perspective scaffold. (Per I4 carry-forward.)] +This review argues that four research perspectives, psychophysics, action, language, and emotion, make divergent and partly-falsifiable predictions about this empty cell. The methods footprint is small: partial out log luminance ratio (LLR), accept independent component analysis-only artifact rejection because the Healthy Brain Network EEG (HBN-EEG) Release 3 cohort has no synchronous eye tracker, and pre-register a topographic-and-band rejection region before opening the data. Sections 2 to 6 develop the four perspectives in order. Section 7 synthesises them into a pre-registerable rejection region. Box 1 anchors the argument to the HBN-EEG Release 3 cohort viewing *The Present* (Pixar 2014). Section 2 begins with the four-perspective scaffold the rest of the review builds on. diff --git a/manuscript/narrative-review/sections/02_scaffold.md b/manuscript/narrative-review/sections/02_scaffold.md index b68440d..f07f99d 100644 --- a/manuscript/narrative-review/sections/02_scaffold.md +++ b/manuscript/narrative-review/sections/02_scaffold.md @@ -1,27 +1,17 @@ # 2. The four-perspective scaffold -[Paragraph 1 (~200 words): defend the four-perspective scaffold as structural rather than decorative. Each perspective makes a different *kind* of prediction: -- Psychophysics: a regressor of no interest that must be partialled before any social claim can be defended (LLR; motion energy). -- Action: a band-and-topography prediction (mu-band ERD over central rolandic cortex) inherited from adult mirror-system work but never tested in animated agents. -- Language: a method that structurally cannot transfer (LM surprisal aligned to spoken transcripts) plus a positive silent-narrative sub-thread (Castelli, Vanderwal, Naci, Lankinen). -- Emotion: two predictions at incompatible latencies (early occipital alpha; later frontal-asymmetric alpha). -Together the four form a hierarchy of prior-evidence depth that the data can rerank.] +The four-perspective scaffold is structural rather than decorative. Each perspective makes a different *kind* of prediction. Psychophysics names a regressor of no interest that must be partialled before any social claim can be defended; the regressor is LLR, with motion energy as a named follow-up. Action names a band-and-topography prediction: mu-band event-related desynchronisation (ERD) over central rolandic cortex, inherited from adult mirror-system work [hari1998action; pineda2005mu] but never tested in animated agents. Language names a method that structurally cannot transfer (language-model surprisal aligned to spoken transcripts) plus a positive sub-thread of silent-narrative findings that does transfer (Castelli's Heider-Simmel triangle paradigm [castelli2000heider], Vanderwal's Inscapes [Vanderwal2015InscapesAM], Naci's Hitchcock excerpt [Naci2014ACN], Lankinen's silent-visual MEG ISC [Lankinen2014IntersubjectCO]). Emotion names two predictions at incompatible latencies: early occipital alpha desynchronisation and later frontal-asymmetric alpha. The four perspectives form a hierarchy of prior-evidence depth that the data can rerank. -[Paragraph 2 (~250 words): perspective-to-theme mapping (per I4 carry-forward: argumentative, not catalogue). Use the 15 corpus themes from `research/synthesis/science-map.md`. Group themes by which perspective owns them, not by theme number. -- Two cross-perspective backbones: Theme 1 (ISC as reliability metric) and Theme 2 (event segmentation). These anchor every per-shot analysis. -- Psychophysics-owned: Themes 4 (low-level feature regressors), 5 (time-resolved EEG/MEG), 11 (free-viewing EEG with eye coregistration). -- Action-owned: Themes 6 (mu rhythm and action observation), 8 (social cognition through biological motion), with contributions to Themes 2 and 14 (distributed multivariate signatures). -- Language-owned: Theme 9 (LMs as regressors) and Theme 10 (audiovisual integration); silent-narrative sub-thread cuts across Themes 8 and 13 (developmental neuroimaging in cinematic paradigms). -- Emotion-owned: Themes 7 (affective dynamics), 12 (pet/animal/baby-schema), 13. -- Theme 15 (predictive processing) is a cross-perspective unifier. -Forward-reference Figures 1 and 4.] +Two themes anchor the analytic backbone independent of perspective. Theme 1, inter-subject correlation as a reliability metric, originated in fMRI [Hasson2004IntersubjectSO] and migrated to scalp EEG [dmochowski2012correlated], MEG [Lankinen2014IntersubjectCO], peripheral physiology [Madsen2022CognitivePO], and audience prediction [Dmochowski2014AudiencePA]. Theme 2, event segmentation, anchors in event-segmentation theory and hidden-Markov-model event-state recovery [zacks2007event; baldassano2017event; speer2007narrative; Ben-Yakov2018TheHF]. The four perspectives then sit in specific corners of the theme space. Psychophysics owns Themes 4 (low-level feature regressors) [Adelson1985SpatiotemporalEM; Carandini2011NormalizationAA; Nishimoto2011ReconstructingVE], 5 (time-resolved EEG and MEG), and 11 (free-viewing EEG with eye coregistration). Action owns Themes 6 (mu rhythm and action observation) [hari1998action; pineda2005mu] and 8 (social cognition through biological motion), and contributes to Themes 2 and 14 (distributed multivariate signatures). Language owns Theme 9 (LMs as regressors) [Goldstein2022SharedCP; Caucheteux2022BrainsAA] as a structural comparator and Theme 10 (audiovisual integration); the silent-narrative sub-thread cuts across Themes 8 and 13 (developmental neuroimaging in cinematic paradigms). Emotion owns Themes 7 (affective dynamics), 12 (pet, animal, and baby-schema affective response), and 13. Theme 15 (predictive processing) unifies across perspectives: it ties mu-band ERD to mirror-system prediction error, LM surprisal to next-word prediction, and event boundaries to prediction-error transients. Theme 3 (naturalness gradient; Figure 2) places the stimulus on a continuum from controlled gratings to live-action film, with character animation as the intermediate point that motivates the empty-cell framing. -[Paragraph 3 (~150 words): perspective interaction is intentional, not residual. Briefly preview the synthesis in Section 7 (where the four rejection regions combine into a pre-registerable test). Explicit transition to Section 3.] +Perspective overlap is intentional rather than residual. The four perspectives interact at the per-shot ERSP level rather than partitioning variance cleanly. Sections 3 to 6 develop them in order, naming the band-by-topography signature each makes and the falsification region attached to each (Figure 4). Section 7 closes by combining the four rejection regions into a single pre-registerable test before group analysis. Section 3 begins with psychophysics because the bottom-up floor must be cleared before any higher-order claim can be defended. diff --git a/manuscript/narrative-review/sections/03_psychophysics.md b/manuscript/narrative-review/sections/03_psychophysics.md index 64fe565..add19ad 100644 --- a/manuscript/narrative-review/sections/03_psychophysics.md +++ b/manuscript/narrative-review/sections/03_psychophysics.md @@ -1,28 +1,12 @@ # 3. Psychophysics: the bottom-up floor -[Paragraph 1 (~150 words): lineage of bottom-up visual feature processing. -- V1 receptive fields (Hubel and Wiesel 1962). -- Divisive normalisation (Carandini and Heeger 2011). -- Natural-image statistics + spatiotemporal energy (Bell and Sejnowski 1997; Simoncelli and Olshausen 2001; Adelson and Bergen 1985). -- MT/MST motion machinery (Born and Bradley 2005; Bartels and Zeki 2008). -- Nishimoto et al. 2011: movie reconstruction from BOLD via Adelson-Bergen feature bank (existence proof for motion-energy reconstructability). -- Clinical VEP (Tobimatsu and Celesia 2006): luminance/contrast steps produce reliable scalp signatures with magno/parvo pathway assignment.] +Psychophysics anchors the bottom-up floor that every per-shot analysis must clear before claiming a higher-order effect. The lineage runs from primary visual cortex receptive fields [Hubel1962ReceptiveFB] and divisive normalisation [Carandini2011NormalizationAA] through natural-image statistics and spatiotemporal energy [Bell1997TheC; Simoncelli2001NaturalIS; Adelson1985SpatiotemporalEM] to middle-temporal motion machinery [Born2005StructureAF; Bartels2008NaturalVR]. Nishimoto and colleagues reconstructed natural movies from blood-oxygen-level-dependent activity in occipitotemporal cortex using a motion-energy front end derived from Adelson and Bergen [Nishimoto2011ReconstructingVE]. The reconstruction is an existence proof that an Adelson-Bergen feature bank suffices to recover the stimulus from neural activity. Clinical visual evoked potential work supports a reliable scalp signature for luminance and contrast steps with magnocellular and parvocellular pathway assignment [Tobimatsu2006StudiesOH]. -[Paragraph 2 (~150 words): closest electrophysiological analogue. -- Nentwich et al. 2023 iEEG: triple-regressor (motion, saccades, cuts) shows motion outranks luminance for occipitoparietal cortex. -- Per-shot LLR is one of several low-level features that needs accounting; LLR is the minimum partialling for any social claim. -- Whole-clip EEG ISC tracks low-level features at occipital electrodes more strongly than higher-order content (Dmochowski et al. 2012; Madsen and Parra 2022; Cohen and Parra 2016). Attention modulates this baseline (Ki et al. 2016). -- Envelope-only auditory control (Kaneshiro et al. 2021) is the methodological template the LLR-as-covariate plan inherits.] +The closest electrophysiological analogue to per-shot ERSP during naturalistic film is the intracranial electroencephalography (iEEG) study of Nentwich and colleagues, who showed that motion outranks luminance for occipitoparietal cortex when triple-regressed against optical-flow magnitude, saccade onsets, and film-cut onsets [Nentwich2023SemanticNM]. That result establishes a quantitative ranking among low-level regressors: per-shot LLR is one of several low-level features that need accounting. EEG ISC at the whole-clip scale tracks low-level features at occipital electrodes more strongly than higher-order content [dmochowski2012correlated; Madsen2022CognitivePO; Cohen2016MemorableAN], although attention strongly modulates this baseline [Ki2016AttentionSM]. An envelope-only auditory control isolating low-level acoustic structure from higher-level musical structure [Kaneshiro2021InterSubjectEC] is the methodological template the LLR-as-covariate plan inherits. -[Paragraph 3 (~200 words): eye-movement caveat. -- Free-viewing EEG depends on eye-movement coregistration to separate stimulus-onset from saccade-locked and fixation-related potentials (Dimigen et al. 2011; Plöchl et al. 2012). Regression deconvolution is the state of the art (Dimigen and Ehinger 2021). -- Gaze coherence varies with stimulus class: highest on Hollywood trailers, lowest on natural movie clips and static images (Dorr et al. 2010); a Pixar short sits between these extremes. -- HBN-EEG R3 has no synchronous eye tracker. Per-shot ERSP cannot deconvolve overlapping saccade-locked transients from shot-onset responses. -- ICA-based artifact rejection through AMICA and ICLabel is the operating compromise (Bell and Sejnowski 1997 for ICA foundation). -- Motion energy computed offline from the stimulus video is the named first follow-up regressor (Nishimoto et al. 2011; Nentwich et al. 2023). -- mTRF toolbox (Crosse et al. 2016) supplies the production regression framework. -- Forward-reference Figure 2 (naturalness gradient).] +A second class of bottom-up drivers operates through the eye. Free-viewing EEG depends on eye-movement coregistration to separate stimulus-onset responses from saccade-locked and fixation-related potentials [Dimigen2011CoregistrationOE; Plöchl2012CombiningEA], and regression deconvolution of overlapping events is the methodological state of the art [Dimigen2021RegressionbasedAO]. Gaze coherence varies with stimulus class, highest on Hollywood trailers and lowest on natural movie clips and static images [Dorr2010VariabilityOE]; a Pixar short sits between these extremes. The HBN-EEG Release 3 cohort carries no synchronous eye tracker, which means a per-shot analysis cannot deconvolve overlapping saccade-locked transients from shot-onset responses. Independent component analysis (ICA)-based artifact rejection through adaptive mixture ICA (AMICA) and IC classification (ICLabel) is the operating compromise [Bell1997TheC]. The implication for per-shot ERSP is asymmetric: per-shot LLR is the minimum partialling for any social-content claim. Motion energy computed offline from the stimulus video is the named first follow-up regressor [Nishimoto2011ReconstructingVE; Nentwich2023SemanticNM]. The multivariate temporal response function (mTRF) toolbox supplies the production regression framework [Crosse2016TheMT]. Figure 2 places the empty cell on the naturalness gradient. diff --git a/manuscript/narrative-review/sections/04_action.md b/manuscript/narrative-review/sections/04_action.md index 86e071c..2107ee1 100644 --- a/manuscript/narrative-review/sections/04_action.md +++ b/manuscript/narrative-review/sections/04_action.md @@ -1,35 +1,17 @@ # 4. Action: mu-band ERD and event segmentation -[Paragraph 1 (~150 words): mu-band lineage. -- Hari et al. 1998 MEG: primary motor cortex activated during passive observation of hand action via 15-25 Hz rolandic rebound suppression (31-46% of execution-related suppression). -- Pineda 2005: EEG mu rhythm (8-13 Hz over C3, Cz, C4) as non-invasive proxy for human mirror-system engagement. -- Oberman et al. 2007: mu suppression magnitude correlates with self-reported social skill. -- Saygin 2007: lesion-symptom mapping places posterior STS and ventral premotor cortex as causally necessary nodes for biological-motion perception. Johansson 1973 (biological motion). -- Predictive-coding reformulations: Kilner et al. 2007; Rizzolatti and Craighero 2004; Iacoboni 2009. -- **I3 carry-forward**: Hickok-style critiques of one-to-one mirror-system interpretations of mu suppression are not represented in the corpus and temper the weight that the action-perspective prediction can carry.] +The action perspective makes the most specific positive prediction in the 0 to 500 ms ERSP window. Hari and colleagues showed by magnetoencephalography (MEG) that primary motor cortex is activated during passive observation of hand action via 15 to 25 Hz rolandic rebound suppression that reaches 31 to 46 percent of execution-related suppression [hari1998action]. Pineda framed the EEG mu rhythm (8 to 13 Hz over electrodes C3, Cz, and C4) as a non-invasive proxy for human mirror-system engagement [pineda2005mu]. Mu suppression magnitude during action observation correlates with self-reported social skill across neurotypical adults [oberman2007mirror]. Lesion-symptom mapping places posterior superior temporal sulcus (STS) and ventral premotor cortex as causally necessary nodes for biological-motion perception [saygin2007sts; johansson1973biological]. Predictive-coding reformulations recast mirror responses as scaling with prediction error over goal and intention [kilner2007predictive; rizzolatti2004mirror; iacoboni2009mirror]. The mirror-system framing has well-known critiques outside the corpus, in particular Hickok-style objections to one-to-one mirror-interpretations of mu suppression; the absence of these critiques inside the corpus tempers the weight that the action prediction can carry. -[Paragraph 2 (~100 words): animated-agent extrapolation. -- Heider-Simmel triangle animations (Castelli et al. 2000): even abstract triangles recruit posterior STS, mPFC, and temporal poles when motion implies intention. -- Naturalness gradient (Hasson et al. 2010) places character animation between abstract Heider-Simmel and live-action. -- The inferential bridge from triangle-animation fMRI activation to character-animation mu-band EEG ERD is plausible and untested. -- Shots dominated by character action should produce ERD in the mu band over central electrodes, with possible beta-band rebound suppression.] +Even with that tempering, the prediction is specific. Shots dominated by character action should produce ERD in the mu band over central electrodes, with possible beta-band rebound suppression. The Heider-Simmel tradition shows that even abstract triangle animations recruit posterior STS, medial prefrontal cortex, and temporal poles when motion implies intention [castelli2000heider]. The naturalness gradient places character animation between abstract Heider-Simmel and live-action [hasson2010natural]. The inferential bridge from triangle-animation fMRI activation to character-animation mu-band EEG ERD is plausible and untested at scalp-EEG resolution. -[Paragraph 3 (~150 words): event segmentation as second action beat. -- Speer et al. 2007: posterior cingulate, MT+, posterior STS boundary-locked transients in fMRI. -- Baldassano et al. 2017: HMM recovers hierarchy of event boundaries from Sherlock fMRI; hippocampal boundary signals predict subsequent free recall. -- Lerner et al. 2011: temporal receptive windows from sensory cortex (ms) to default-mode (tens of s). -- Chen et al. 2017: event-specific DMN patterns shared across viewers and reactivated at recall. -- Ben-Yakov and Henson 2018: hippocampus distinguishes within-event camera cuts (minimal response) from across-event narrative boundaries (robust response). -- Magliano and Zacks 2011: behavioural foundation for shot-segmentation independent of dialogue.] +The second action beat is event segmentation. Speer and colleagues found posterior cingulate, middle-temporal, and posterior STS boundary-locked transients in fMRI during narrative listening [speer2007narrative]. Baldassano and colleagues recovered a hierarchy of event boundaries from Sherlock-movie fMRI using hidden Markov models, with hippocampal boundary signals predicting subsequent free recall [baldassano2017event]. Lerner and colleagues mapped temporal receptive windows from sensory cortex (milliseconds) to default-mode regions (tens of seconds) [lerner2011temporal]. Chen and colleagues showed event-specific patterns in the default-mode network are shared across viewers and reactivated at recall [chen2017shared]. Ben-Yakov and Henson distinguished within-event camera cuts, which produce minimal hippocampal responses, from across-event narrative boundaries, which produce robust ones [Ben-Yakov2018TheHF]. Magliano and Zacks supplied the behavioural foundation that viewers segment edited films along cuts independent of dialogue [Magliano2011TheIO]. -[Paragraph 4 (~100 words): single-agent vs two-agent contrast. -- Sliwa and Freiwald 2017: dedicated cortical network in macaque for two-agent social interaction, separable from single-agent action perception. -- This motivates excluding two-agent shots from a clean single-agent contrast (the social-interaction network may dominate two-agent variance). -- Note that autism subsample considerations (Klin et al. 2002, 2009) move to Section 7 limitations (per v1 m8 carry-forward to keep Section 4 focused).] +A third action beat concerns single-agent versus two-agent shots. Sliwa and Freiwald documented a dedicated cortical network in macaque for processing two-agent social interaction, separable from single-agent action perception [sliwa2017macaque]. This motivates excluding two-agent shots from a clean single-agent contrast, since the social-interaction network may dominate two-agent variance. diff --git a/manuscript/narrative-review/sections/05_language.md b/manuscript/narrative-review/sections/05_language.md index e18788a..2220502 100644 --- a/manuscript/narrative-review/sections/05_language.md +++ b/manuscript/narrative-review/sections/05_language.md @@ -1,50 +1,22 @@ # 5. Language: comparator of non-transfer plus silent-narrative sub-thread ## 5a. Language-model regressors are structurally non-transferable -[Budget: ~250 words. - -Paragraph (~250 words): LM-as-regressor mainstream and its dependence on word alignment. -- Goldstein et al. 2022: pre-onset prediction, post-onset surprise, and contextual-embedding signatures shared between word-by-word electrocorticography (ECoG) and autoregressive LMs. -- Heilbron et al. 2022: lexical/syntactic/semantic surprisal regressors during MEG audiobook listening, derived from LMs with word-onset alignment. -- Caucheteux et al. 2022, 2023: transformer intermediate layers map to fMRI/MEG; cortical hierarchy of prediction timescales. -- Antonello et al. 2023: log-linear scaling of brain-prediction with LM parameter count up to 30B. -- Schrimpf et al. 2021: next-word-prediction quality drives brain-score on fMRI/ECoG/reading-time benchmarks. -- Toneva and Wehbe 2019: BERT predicts reading fMRI/MEG. -- Huth et al. 2016: voxelwise semantic atlas requires spoken transcripts. -- Nelson et al. 2017: phrase-structure-building intracranial high-gamma, explicitly reading-based. -- N400 family (Kutas and Federmeier 2011; DeLong et al. 2005): bridges to picture-context paradigms at the cost of dynamic stimulus. - -Closing sentence applies F1 carry-forward verbatim: -"All seven Category G language-ontology cards (and 12 cards corpus-wide) carry `transfer-to-silent: no`." +The contemporary methodological mainstream in naturalistic neuroimaging is built around transformer-based language-model (LM) regressors aligned to spoken or read transcripts. Goldstein and colleagues showed pre-onset prediction, post-onset surprise, and contextual-embedding signatures shared between word-by-word electrocorticography (ECoG) and autoregressive LMs [Goldstein2022SharedCP]. Each signature depends on speech-onset alignment. Heilbron and colleagues separated lexical, syntactic, and semantic surprisal regressors during MEG audiobook listening, all derived from LMs with word-onset alignment [Heilbron2020AHO]. Caucheteux and colleagues mapped transformer intermediate layers to fMRI and MEG responses to natural narrative [Caucheteux2022BrainsAA] and a cortical hierarchy of prediction timescales [Caucheteux2023EvidenceOA]. Antonello and colleagues documented log-linear scaling of brain prediction with LM parameter count up to 30B [Antonello2023ScalingLF]. Schrimpf and colleagues showed that next-word-prediction quality drives brain score on fMRI, ECoG, and reading-time benchmarks [schrimpf2021the]. Toneva and Wehbe used BERT to predict reading fMRI and MEG, with attention-head ablations linking brain prediction to natural-language processing performance [toneva2019interpreting]. Huth and colleagues built the canonical voxelwise word-embedding encoding atlas tiling cortex with semantic clusters; this method requires spoken transcripts [Huth2016NaturalSR]. Nelson and colleagues tracked open-node count during syntactic merge using intracranial high-gamma dynamics, explicitly reading-based [Nelson2017NeurophysiologicalDO]. The N400 family bridges to picture-context paradigms at the cost of dynamic stimulus [Kutas2011ThirtyYA; DeLong2005ProbabilisticWP]. -Lipkin et al. 2022 frontotemporal language network atlas: included as negative-control region in the falsification region of Section 7.] +Each method depends on word-level alignment to spoken or read stimuli. *The Present* is wordless. All seven Category G cards in our language ontology (and 12 cards corpus-wide) carry `transfer-to-silent: no`. A vision-side analogue, multimodal vision-language model embeddings or scene-difference deep-network features as continuous regressors, does not yet exist in the corpus for scalp-EEG ERSP. The Lipkin frontotemporal language-network atlas [Lipkin2022ProbabilisticAF] is used as the negative-control region of interest in the falsification region of Section 7. ## 5b. Silent-narrative neural correlates that do transfer -[Budget: ~350 words. - -Paragraph 1 (~200 words): thesis statement + Heider-Simmel/Inscapes/Hitchcock/MEG. -- Open with a thesis sentence parallel to 5a: silent-narrative neural correlates do transfer to scalp-EEG ERSP analysis even when LM regressors cannot. (Per v1 m5 carry-forward.) -- Castelli et al. 2000 (animations mentalising; Heider-Simmel triangles): silent geometric-shape animations engage mPFC, TPJ, STS when motion implies social interaction. -- Castelli et al. 2002: same paradigm in autism shows reduced engagement. -- Vanderwal et al. 2015 Inscapes: purpose-built silent abstract animation improves MRI compliance and produces reliable network-level activity; used by HBN itself. -- Naci et al. 2014: Hitchcock excerpt as covert assessment. -- Lankinen et al. 2014: source-space MEG reliable across viewers in occipital and temporal cortex during silent-visual and audiovisual movie conditions. Closest electrophysiological analogue with a deliberate silent-visual condition. -- Studyforrest (Hanke et al. 2014): audio-only foundation extended to silent-cohort contrasts.] - -[Paragraph 2 (~150 words): cross-modal phase alignment + DMN as narrative integrator. -- Schroeder et al. 2009: modality-general delta- and theta-band phase alignment to attended event onsets; mechanistic frame for shot-onset ERSP independent of speech. -- Senkowski et al. 2008: cross-modal gamma synchronisation and low-frequency phase coupling. -- Van Wassenhove et al. 2005: visual-speech facilitation. Does not transfer because *The Present* has no dialogue. -- DMN as narrative integrator: Buckner et al. 2008; Simony et al. 2016; Yeshurun et al. 2017; Mar 2011; Tamir et al. 2016. +Silent-narrative neural correlates do transfer to scalp-EEG ERSP analysis even when language-model regressors cannot. Castelli and colleagues showed that silent geometric-shape animations engage medial prefrontal cortex, the temporo-parietal junction, and the STS when motion implies social interaction, with no speech required [Castelli2000MovementAM; castelli2000heider]; the same paradigm in autism shows reduced engagement [Castelli2002AutismAS]. Vanderwal and colleagues built Inscapes, a purpose-built silent abstract animation that improves MRI compliance and produces reliable network-level activity, used by the HBN cohort itself [Vanderwal2015InscapesAM]. Naci and colleagues used a Hitchcock excerpt as a covert assessment, showing that high-order cortex can be probed from a near-silent narrative [Naci2014ACN]. Lankinen and colleagues report source-space MEG reliable across viewers in occipital and temporal cortex during silent-visual and audiovisual movie conditions, the closest electrophysiological analogue with a deliberate silent-visual condition [Lankinen2014IntersubjectCO]. The Studyforrest infrastructure provides an audio-only foundation that has been extended to silent-cohort contrasts [Hanke2014AH7]. Schroeder and colleagues described modality-general delta- and theta-band phase alignment to attended event onsets, providing the mechanistic frame for shot-onset ERSP independent of speech [Schroeder2009LowfrequencyNO]. Senkowski and colleagues described transient gamma synchronisation and low-frequency phase coupling for cross-modal binding [Senkowski2008CrossmodalBT]. Van Wassenhove and colleagues showed visible mouth movements speed the auditory N1 and P2 components, an effect that does not transfer because *The Present* contains no dialogue [Wassenhove2005VisualSS]. Buckner, Simony, Yeshurun, Mar, and Tamir developed the default-mode network (DMN) as narrative integrator, with framing context driving within-stimulus divergence [Buckner2008TheBD; Simony2016DynamicRO; Yeshurun2017SameSD; Mar2011TheNB; Tamir2016ReadingFA]. -Closing transition: silent narrative engages mPFC, TPJ, STS, DMN. Their IC-cluster analogues in EEG are the search regions for the per-shot ERSP analysis. Forward-reference Figure 3 (gap matrix).] +The language perspective plays two roles. The 5a sub-thread isolates the silent-stimulus design from the dominant LM-as-regressor framework. The 5b sub-thread supplies the cortical substrates that silent narrative engages: medial prefrontal cortex, the temporo-parietal junction, the STS, and the DMN. Their independent-component-cluster analogues in EEG are the search regions for the per-shot ERSP analysis. Figure 3 makes the gap structure explicit. diff --git a/manuscript/narrative-review/sections/06_emotion.md b/manuscript/narrative-review/sections/06_emotion.md index 851f234..276c3e2 100644 --- a/manuscript/narrative-review/sections/06_emotion.md +++ b/manuscript/narrative-review/sections/06_emotion.md @@ -1,36 +1,17 @@ # 6. Emotion: two predictions at different latencies -[Paragraph 1 (~150 words): early visual-cortex emotion-schema response. -- Kragel et al. 2018 EmoNet: emotion schemas encoded in early visual cortex. -- Saarimaki et al. 2016: fMRI MVPA decoding of six basic emotions. -- Cowen and Keltner 2017: 27-emotion taxonomy from short videos. -- Lindquist et al. 2012: distributed signatures over strict regional localisation. -- Wager et al. 2013 neurologic pain signature (NPS): multivariate signatures of affect. -- Closest EEG correlate at the 0-500 ms scale: early occipital alpha desynchronisation (80-300 ms post-shot-onset, extrapolated from static-picture latencies). -- **F2 carry-forward**: Codispoti, De Cesarei, and Ferrari (2023; Psychophysiology DOI 10.1111/psyp.14438) review alpha-band oscillations during emotional picture perception, concluding alpha desynchronisation is a robust correlate of attentional engagement with parametric arousal modulation. Cite as "Codispoti and colleagues (2023)" in prose. -- Whether this transfers to dynamic naturalistic stimuli at sub-second timescales in a child cohort is untested.] +The emotion perspective makes two predictions with different latencies and different implicated structures. The first is an early visual-cortex emotion-schema response. Kragel and colleagues built EmoNet, a deep-learning model showing that emotion schemas are encoded in early visual cortex, predicting that emotion-tuned visual representations should appear in early-latency occipital ERSP [Kragel2018EmotionSA]. Saarimaki and colleagues decoded six basic emotions during emotional movie viewing using fMRI multi-voxel pattern analysis [Saarimäki2016DiscreteNS]; Cowen and Keltner extended the taxonomy to 27 distinguishable categories from short videos [Cowen2017SelfreportC2]. Distributed-network meta-analysis argues for distributed signatures over strict regional localisation [Lindquist2012TheBB], with the neurologic pain signature as a methodological exemplar of multivariate signatures of affect [Wager2013AnFN]. The closest EEG correlate at the 0 to 500 ms scale is early occipital alpha desynchronisation (80 to 300 ms post-shot-onset, extrapolated from static-picture latencies). Codispoti and colleagues (2023) review the EEG alpha-band literature on emotional picture perception and conclude that alpha desynchronisation is a robust correlate of attentional engagement by emotional stimuli, with parametric arousal modulation [Codispoti2023AlphabandOA]. Whether this transfers to dynamic naturalistic stimuli at sub-second timescales in a child cohort is untested. -[Paragraph 2 (~150 words): cuteness / affiliative response (longer latency). -- Stoeckel et al. 2014: adult mothers viewing own child versus own dog photos. Common activation across emotion, reward, affiliation, visual processing, and social cognition regions. -- Glocker et al. 2009: baby-schema parametric modulation of NAcc reward in nulliparous adults. -- Borgi et al. 2014: children 3-6 show parametric cuteness ratings and gaze bias for human infant, puppy, and kitten faces. -- Interpretation: Stoeckel measures identity-level pair-bonding; Borgi measures generic baby-schema. HBN viewers have no identity-level bond with an animated puppy; the inference is from generic baby-schema rather than pair-bonding circuitry.] +The second prediction is a longer-latency cuteness or affiliative response. Stoeckel and colleagues reported common activation across child and dog spanning emotion, reward, affiliation, visual processing, and social cognition regions in adult mothers viewing photographs of own child versus own dog [Stoeckel2014PatternsOB]. Glocker and colleagues showed that baby schema parametrically modulates nucleus accumbens reward in adults [Glocker2009BabySM]. Borgi and colleagues demonstrated that children aged 3 to 6 already show parametric cuteness ratings and gaze bias for human infant, puppy, and kitten faces [Borgi2014BabySI]; this is the behavioural anchor that the cuteness response is established well before adolescence. The interpretation implication is that Stoeckel measures identity-level pair-bonding and Borgi measures generic baby schema. HBN viewers have no identity-level bond with an animated puppy, so the relevant inference is from generic baby schema rather than pair-bonding circuitry. -[Paragraph 3 (~150 words): EEG routes (alpha desynchronisation + frontal asymmetry). -- Alpha-band desynchronisation as arousal-modulated correlate of attentional engagement (Codispoti et al. 2023). -- Frontal alpha asymmetry as approach-withdrawal index (Davidson 2000; Coan and Allen 2004). -- Reznik and Allen 2018: meta-analytic critique documents smaller effect sizes and reliability concerns. -- Corpus contains no card applying asymmetry analysis to per-event sub-second windows during continuous naturalistic stimulus, and none in developmental cohort viewing film. -- Frontal asymmetry at shot-onset latency: exploratory rather than confirmatory.] +Two EEG routes connect these predictions to observables. The first is early occipital alpha-band desynchronisation (80 to 300 ms) as an arousal-modulated correlate of attentional engagement [Codispoti2023AlphabandOA]. The second is later frontal alpha asymmetry (200 to 500 ms; extrapolated downward from the seconds-to-minutes Davidson tradition) as an approach-withdrawal index [Davidson2000AffectiveSP; Coan2004FrontalEA]. An updated meta-analytic critique documents smaller effect sizes and substantial reliability concerns [Reznik2018FrontalAA]. The corpus contains no card applying asymmetry analysis to per-event sub-second windows during a continuous naturalistic stimulus, and none in a developmental cohort viewing film. Frontal asymmetry at shot-onset latency is therefore exploratory rather than confirmatory. -[Paragraph 4 (~50 words): social cognition / DMN engagement. -- Richardson et al. 2018: ToM and pain networks from age three using Pixar shorts in 122 children. Load-bearing developmental anchor. -- Brief: Saxe and Kanwisher 2003 (TPJ); Mar 2011; Singer et al. 2004; Zaki and Ochsner 2012; Nummenmaa et al. 2012; Schmaelzle and Grall 2020; Kauttonen et al. 2015. -- Closing sentence: two predictions sit at incompatible latencies and topographies; an LLR-partialled per-shot GLM adjudicates between them.] +The third emotion beat is social cognition. Richardson and colleagues documented theory-of-mind and pain networks present from age three and refining with age, using Pixar shorts in 122 children [Richardson2018DevelopmentOT]; this is the load-bearing developmental anchor. Mar synthesised narrative comprehension as a social-cognitive activity [Mar2011TheNB]; Singer and colleagues documented affective pain-region engagement during observed pain [Singer2004EmpathyFP]; Zaki and Ochsner formalised the tripartite empathy model bridging experience sharing and mental-state attribution [Zaki2012TheNO]. Nummenmaa and colleagues showed emotion intensity modulates ISC in midline cortex during film viewing [Nummenmaa2012EmotionsPS]; Schmaelzle and Grall theorised ISC as audience captivation [Schmälzle2020TheCB]. Two predictions sit at incompatible latencies and topographies; an LLR-partialled per-shot generalised linear model (GLM) adjudicates between them. diff --git a/manuscript/narrative-review/sections/07_synthesis.md b/manuscript/narrative-review/sections/07_synthesis.md index 9228815..04e6b0d 100644 --- a/manuscript/narrative-review/sections/07_synthesis.md +++ b/manuscript/narrative-review/sections/07_synthesis.md @@ -1,49 +1,34 @@ # 7. Synthesis: integration, falsifiability, and open questions ## 7.1 Integration -[Budget: ~100 words. -- Restate the four-perspective ranking by depth of prior evidence. -- Psychophysics: deepest precedent, simplest operationalisation (partial LLR, optionally motion energy, before any condition claim). -- Action: deepest specific oscillatory prediction (mu-band ERD over central rolandic clusters), no animated-agent precedent in EEG. -- Language: structurally non-transferable for LM regressors but supplies cortical priors via the 5b sub-thread (mPFC, TPJ, STS, DMN). -- Emotion: two predictions (early occipital alpha; later frontal-asymmetric alpha) with cuteness anchored developmentally by Borgi 2014. -- Forward-reference Figure 4 (predictions table).] - -## 7.2 Anchor case (F4 carry-forward) - -[Budget: ~100 words. -- External precedent: Petroni et al. 2018 recorded 64-channel EEG at 500 Hz from 114 viewers across ages 6-44 during passive viewing of six naturalistic videos including animated and live-action shorts. They did not analyse shot-onset ERSP, but they demonstrated scalp-EEG signal exists during developmental naturalistic viewing. -- Internal feasibility: a partly-validated developmental EEG pipeline on HBN-EEG R3 brings 184 subjects through BIDS import, 1 Hz HPF, conditional cleanline, channel rejection, AMICA, ICLabel, dipfit5, std_precomp ERSP. 100 Hz local working set; 500 Hz S3 validation pass scheduled after pipeline validation. -- The two anchor assertions are independent and not interchangeable.] - -## 7.3 Falsifiability (F5 carry-forward) - -[Budget: ~150 words. -- Pre-registerable topographic-and-band rejection region: - - Surviving central-rolandic mu-band cluster (electrodes C3, Cz, C4; 8-13 Hz) confirms action. - - Surviving frontal-asymmetric alpha cluster (electrodes F3, F4; 8-13 Hz) confirms emotion. - - Surviving cluster in left frontotemporal IC space, overlapping the Lipkin et al. 2022 language-network atlas used as negative-control mask, falsifies the four-perspective ranking. - - Null result on LLR-partialled GLM at pre-registered cluster-level alpha (p < 0.05 corrected by mass-univariate cluster-based permutation; mTRF toolbox precedent: Crosse et al. 2016) falsifies by bottom-up exhaustion. -- Pinning the rejection region before data analysis constrains analyst degrees of freedom.] - -## 7.4 Open questions and limitations (I1 + I2 carry-forward) - -[Budget: ~100 words. -- I2: narrative-position objection. Boy-only and puppy-only shots differ on three-act position (boy-only in early-act setup, puppy-only in late-act resolution). Add shot-index-in-narrative as continuous covariate; fit within-act stratified analysis as named follow-up (Magliano 2011; Baldassano 2017; Chen 2017). -- Hickok-style mu-system critique gap (I3 carry-forward repeated here). -- Klin et al. 2002, 2009 autism subsample considerations (moved from Section 4 per v1 m8 carry-forward). -- Emotion literature predominantly adult; three pet-evoked affective cards are fMRI/behavioural. -- Frontal asymmetry at sub-second timescales is unprecedented and reliability-limited. -- 100 Hz local working set caps beta/gamma claims until 500 Hz pass. -- Outstanding Questions Box collects forward-looking adjudication targets.] +The four perspectives rank by depth of prior evidence. Psychophysics has the deepest precedent and the simplest operationalisation: partial LLR, optionally motion energy, before any condition claim. Action has the deepest specific oscillatory prediction (mu-band ERD over central rolandic clusters) but no animated-agent precedent in EEG. Language is structurally non-transferable for LM regressors but supplies cortical priors for silent narrative through its 5b sub-thread (medial prefrontal cortex, the temporo-parietal junction, the STS, the default-mode network). Emotion supplies two predictions: early occipital alpha desynchronisation [Codispoti2023AlphabandOA; Kragel2018EmotionSA] and later frontal-asymmetric alpha [Davidson2000AffectiveSP], with the cuteness response anchored developmentally by Borgi [Borgi2014BabySI]. Distributed-multivariate-signature framing supports IC-cluster-level analyses over single-IC decoding [Lindquist2012TheBB; chen2017shared]. Figure 4 displays the four predictions in tabular form. + +## 7.2 Anchor case + +External precedent: Petroni and colleagues recorded 64-channel EEG at 500 Hz from 114 viewers across ages 6 to 44 during passive viewing of six naturalistic videos including animated and live-action shorts [Petroni2018TheVO]. They did not analyse shot-onset ERSP and did not factor stimulus-side regressors, but they demonstrated that scalp-EEG signal exists during developmental naturalistic viewing of short videos. They are the closest external existence proof that the measurement class is feasible in adjacent territory. Internal feasibility: a partly-validated developmental EEG pipeline on HBN-EEG Release 3 brings 184 subjects through Brain Imaging Data Structure (BIDS) import, 1 Hz high-pass filtering, conditional cleanline gated by Nyquist, `clean_rawdata` channel rejection, AMICA decomposition, ICLabel classification, dipole fitting, and `std_precomp` ERSP precomputation; the operating constraint is that the local working set is 100 Hz, with a 500 Hz validation pass on the full Amazon S3 R3 release scheduled after pipeline validation. The two anchor assertions are independent and not interchangeable. + +## 7.3 Falsifiability + +A topographic-and-band rejection region for the four-perspective ranking can be pre-registered before group analysis. A surviving central-rolandic mu-band cluster (electrodes C3, Cz, and C4; 8 to 13 Hz) confirms the action prediction. A surviving frontal-asymmetric alpha cluster (electrodes F3 and F4; 8 to 13 Hz) confirms the emotion prediction. A surviving cluster in left frontotemporal IC space, overlapping the Lipkin language-network atlas [Lipkin2022ProbabilisticAF] used as a negative-control mask, falsifies the four-perspective ranking by relocating the surviving signal into a perspective the thesis says should not transfer. A null result on the LLR-partialled GLM at a pre-registered cluster-level alpha (p < 0.05 corrected by mass-univariate cluster-based permutation, with the mTRF toolbox precedent [Crosse2016TheMT]) also falsifies the four-perspective ranking, by localising per-shot ERSP variance entirely to bottom-up features in this cohort. Pinning the rejection region before data analysis is the publication discipline that constrains analyst degrees of freedom. + +## 7.4 Open questions and limitations + +Narrative position is a within-stimulus confound. Boy-only and puppy-only shots in *The Present* differ on three-act position: boy-only clusters in the early-act setup, puppy-only in the late-act resolution. Any boy-vs-puppy ERSP difference may therefore be confounded with prediction-error or arousal trajectories. The response is to add shot-index-in-narrative as a continuous covariate in the group GLM and to fit a within-act stratified analysis as a named follow-up [Magliano2011TheIO; baldassano2017event; chen2017shared]. Beyond narrative position, several gaps in the corpus limit what this review can claim. The Hickok-style mu-system critique is not represented in our cards, which weakens the action prediction. Klin and colleagues showed that toddlers with autism orient to audiovisual contingency rather than upright biological motion [klin2009biological] and that adolescents with autism fixate eyes 50 percent as often during emotionally evocative viewing [klin2002visual]; the HBN cohort includes a substantial autism-spectrum subsample, so autism status is a candidate moderator, but stratified analyses (autism-spectrum, attention, social skill) are exploratory follow-ups rather than primary tests. The emotion literature is predominantly adult; the three pet-evoked affective cards are fMRI or behavioural, not EEG. Frontal asymmetry at sub-second timescales is unprecedented and reliability-limited. The single-stimulus design forbids generalisation beyond *The Present*. The 100 Hz local working set caps beta-band and gamma-band claims until the 500 Hz validation pass. The Outstanding Questions Box collects the forward-looking adjudication targets.