B modes#36
Draft
cgiovanetti wants to merge 29 commits into
Draft
Conversation
Three research reports (ABCMB flat-geometry audit, CLASS curvature equations, CLASS hyperspherical/transfer machinery) plus the condensed design doc for the Omega_k implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s_l hierarchy factors, curved Einstein constraints and ICs - main.py: omega_k (=Omega_k h^2) default + derived K = -omega_k H100^2; closure omega_Lambda subtracts omega_k; expected_keys updated. - species.py: Curvature(BackgroundFluid) pseudo-fluid (rho ~ a^-2, w=-1/3) makes H/aH_prime automatically curved; curvature_s_l() free-streaming coefficients; CLASS-verbatim (arXiv:1305.3261) s_l factors in the photon T/pol, massless-nu and massive-nu hierarchies; cot_K l_max truncations; s2^2 = 1-3K/k^2 factors in all adiabatic ICs. - perturbations.py: (k^2-3K) Einstein constraints + K h'/2 in eta' (both get_derivatives and make_output_table copies); s2^2 in eta_ini. - background.py: sqrt(1-K rs^2) in the rs ODE; rA_rec = sin_K(chi_rec). - ABCMBTools.py: sin_K/cot_K helpers via entire functions of K chi^2 (series at small argument: smooth through K=0, AD-safe branches). - model_specs.py: Curvature registered in LCDM species; static curvature/omega_k_ref specs; curved k_min for the perturbation grid (closed: sqrt(8K) nu=3 cutoff; open: sqrt(|K|)); |K| in scale2. All value-level changes reduce exactly to the flat equations at K = 0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… + curved Limber lensing - spectrum.py: _transfer_sources extracted (shared flat/curved; s_2 factor in the polarization weight, = 1 exactly at K=0); Cl_one_ell consumes prebuilt sources; new _Cl_all_ells_curved generates Phi_l^nu(chi) by the exact dimensionful forward recurrence (seeds Phi_0, Phi_1 in closed form; smooth through K=0) in one chunked, checkpointed lax.scan over ALL integer ells, contracting against the LOS sources at each l — radial functions T0/T1/T2/E per CLASS transfer_radial_function; evanescent region clamped and masked at the same |Phi|~1e-10 threshold as the flat tables (cutoff variable q*S_K(chi), thresholds reused from the tables' lower edges); closed-universe l>=nu termination and open supercurvature q^2<=0 masks. get_Cl: static specs['curvature'] switch (False = existing flat tables). - lensing_Cl: curved-sky Limber (q=(l+1/2)/S_K, sin_K window, 1/(S_K^2 k^3) measure, WKB amplitude (1-K l^2/q^2)^(-1/2)) — reduces exactly to the prior flat integrand at K=0. lensing_power_spectrum: curved Poisson k^3/(k^2-3K)^2 and Omega_k(1+z)^2 in Om(z). - species.py truncations use the dimensionless g = tau*cot_K(tau) factor so the flat path keeps the original floating-point evaluation order. - main.py: pass specs['curvature'] into SpectrumSolver. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n region) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e edges The flat Bessel tables start at x = 0 for l <= 17, so interpolating their lower edges left the low-l mask wide open and clipped deep-evanescent recurrence garbage (|Phi| ~ 1e10) entered the transfer integral (smoke test: ClTT ~ 1e13 vs 1e-12). Use CLASS's hyperspherical_get_xmin_from_approx closed form for every l; it matches the table edges where they exist (l=5000: 4899 vs 4895). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…teger-nu lattice For closed references the angular diameter distance shrinks, so k_max is divided by the fiducial angular rescaling S_K(chi*)/chi* (CLASS does the same); the transfer grid's low end is resampled to the exact Delta nu = 1 integer lattice up to nu = 512 (covers the sharp ell ~ nu C_l onsets and makes the k-trapezoid the exact discrete-mode sum there), then the flat-built grid snapped to distinct integers. Also: Clpp comparison added to the vs-CLASS script. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r grid (1) At exact integer nu the recurrence coefficient q^2-K(l+1)^2 is analytically zero but numerically O(eps q^2) of either sign; a positive sliver divided the advance by ~1e-11 and produced unmasked 1e10 garbage at ell = nu (the -0.05 unlensed control blew up 1e7 at ell 24-54). Threshold at 1e-6 q^2 (legitimate minimum is 2 q^2/nu). (2) Open universes: walk the transfer grid in q (the oscillation variable) and map to k = sqrt(q^2-K), sampling the q->0 region the flat k-walk missed (CLASS densifies low-q for open; EE ell~3 was 3.5e-3 off). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nsition style) The abrupt Delta nu = 1 -> ~7 jump at nu = 512 left a trapezoid boundary artifact in the oscillatory k-integrand, degrading C_l for ell whose support crosses the junction (-0.01: 2e-2 at ell~137; -0.05: 6.5e-2 at ell~237 — both map to nu just below the junction). Ramp the integer step linearly over 250 points instead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…verse crash) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New abcmb/tensors.py isolates all tensor-sector code: - TensorPerturbationEvolver: GW amplitude h + photon tensor temperature/polarization hierarchies + massless-nu tensor hierarchy (synchronous gauge, CLASS conventions, tensor_method=massless_approx). - TensorSpectrumSolver: flat-space T/E/B radial functions integrated against the tensor sources; reuses the existing phi0/phi1/phi2 Bessel tables (j'' = (2 phi2 - phi0)/3), no new tables. - get_tensor_k_axes: scalar k grids truncated at the tensor k_max. Custom species opt into the GW source via a duck-typed tensor_rho_rel hook; species.py is untouched. Minimal touches to existing files (default scalar path unchanged except Output now carries ClBB): - spectrum.py: lensed_Cls does EE/BB mixing (xi+- from EE+-BB), get_Cl returns a 4-tuple and takes optional tensor_cls. - main.py: Model gains TPE/TSS (None when tensors=False), pipeline wiring, Output.ClBB, and r/n_t derived params (n_t default = CLASS scc). - model_specs.py: tensor spec defaults, incl. tensor-specific solver tolerances rtol_ten=1e-5/atol_ten=1e-9 (scalar large-k rtol 1e-4 biased tensor BB low ~0.7% at l~450). Validation (vs classy 3.3.4, fiducial LCDM + r=0.1): at shared computed ell-nodes ABCMB matches converged CLASS sub-permille; scalar accuracy_test.py still passes; lensed TT/EE/BB within 1%. One open item (raw-BB ell-spline residual ~7e-3 at l=477, between sparse tensor nodes) documented in HANDOFF_bmodes.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… test The raw-BB test's ell=477 residual (6.9e-3 vs the 2.5e-3 bar) is CubicSpline interpolation error over the 40-wide tensor node gap (450->490), not transfer physics. Proven by a CLASS-internal self-interpolation test: re-splining CLASS's own node values through the identical interpax.CubicSpline reproduces ~7e-3 at ell=477 with zero error at the nodes. Nodes are already maximally dense (bessel_l_tab; the phi-tables exist only there), so tightening would need new Bessel tables. test_bb_raw restructured to assert honestly (full BB suite passes on GPU): - recomb-tail nodes (l>=100): 2.5e-3 (clean transfer accuracy; max 1.6e-3) - low-l nodes (3<=l<100): 5.0e-3 (known structural diff below; max 4.2e-3) - full dense band 3<=l<=490: 1.0e-2 (interp-limited, the scalar floor; 6.9e-3) Restructuring surfaced a new, converged ~0.4% low-l raw-BB excess vs CLASS (peak 4.2e-3 at l=10, decaying to the recomb floor). Ruled out: solver tol (ladder gives |A/C-1|=1.4e-5, default rtol_ten already converged), interpolation (it is at nodes), tensor_method (CLASS default IS massless_approximation), n_t (identical formula). A structural ABCMB-vs-CLASS difference in the reion-bump / large-scale tensor source; deferred to a follow-up session. Full writeup + remaining suspects + ready diagnostics in NOTE_lowl_bb_excess.md. diag_bb_dense_hp.py rewritten to a fast (default-precision + l_linstep=5) self-interpolation test. New diagnostics: diag_bb_lowl_nodes.py, diag_bb_tol_ladder.py, diag_bb_lowl_3way.py, run_bb_raw_check.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…trum path get_Cl now always uses _Cl_all_ells_curved (smooth through K=0 -> j_l), for flat and curved cosmologies alike. Removes the tabulated flat path entirely: the 79 MB bessel_tab/*.txt, the sparse-ell CubicSpline reconstruction, the Q/J/j large-x asymptotic, the module-level phi0/phi1/phi2, the Cl_one_ell method, the ells_indices/lensing_ells_indices fields, and the scipy.special.spherical_jn import (~256 lines). Drops bessel_tab from setup.cfg package_data. This eliminates the hard l=5000 cap (raise l_top, no re-tabulation) and the flat path's one genuine approximation (the spline); the recurrence matches scipy.special.spherical_jn to ~1e-14 at K=0, so it is the more correct generator. The 'curvature' flag now only gates k-grid construction, not spectrum-path selection. Validated (1xA100): forward finite + sane for lensing off and on (warm 9.96/10.69 s, a wash); K=0 reverse-AD final gradients finite; accuracy_test PASSED vs CLASS at the 1% gate. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…drop dead code Remove the tracked notes_curvature/ directory (test scripts, design .md notes, changelog drafts; ~3700 lines, not production code). Trim verbose inline comments (ABCMBTools header, the recurrence body in spectrum.py, the CLASS-internals blocks in model_specs.py, the repeated s_l hierarchy comments in species.py) while keeping the NumPy-structured docstrings with the project's Parameters:/Returns: colon convention for autodoc. Drop dead code: model_specs angular_rescaling_ref was set but never read, and the angular rescaling is now computed only in the closed branch where it is used. Behavior unchanged: flat/open k-grids bit-identical, closed algebraically identical. Validated flat+open+closed forwards finite and accuracy_test PASSED (vs CLASS, 1% gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root-caused ~half the low-l raw tensor-BB excess vs CLASS to the TensorPerturbationEvolver start-time cap: lna_start was capped at -10 (z~22000), too late for the photon polarization quadrupole to settle before recombination, inflating low-l BB by ~2e-3. Cap -10 -> -14 (BB is start-converged for t0<=-13). Improves every node l=2-450 at zero wall-clock cost; low-l node max 4.2e-3 -> 3.1e-3, recomb max 1.6e-3 -> 0.85e-3 vs high-precision HYREC-2 CLASS. accuracy_test_bb passes. Tightened test node bars (recomb 2.5e-3 -> 1.5e-3, low-l 5e-3 -> 4e-3) and updated comments. Remaining residual (l=2-4 reion region + a converged ~1e-3 recomb Pi difference) is within inter-code scatter for raw tensor BB; full eliminations + diagnostics in NOTE_lowl_bb_excess.md and the diag_*.py scripts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…to kill the l=477 interp residual The l=477 raw-BB residual is sparse-l-node CubicSpline interpolation. Branch worktree-curvature already replaced tables+spline with an exact hyperspherical recurrence (every l computed) for the scalar path. Added a scoping TASK to assess porting it into TensorSpectrumSolver: payoff = remove the l=477 residual + drop the tensor table dependence; cost = the tensor solver imports the phi-tables the curvature branch deleted, and both feature branches rewrite spectrum.py (merge effort to assess). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… Bessel recurrence Brings the curvature branch's hyperspherical-Bessel recurrence into the B-modes branch and re-expresses the tensor (primordial-GW) spectrum on it, eliminating the l=477 CubicSpline interpolation residual and the tensor solver's dependence on the bessel_tab/*.txt tables. origin/curvature (== worktree-curvature @ 3174b25) merged wholesale: curved background, species.Curvature, curved k-grids, ABCMBTools curved helpers, and the 79 MB Bessel-table deletion all applied clean. Conflicts only in: - spectrum.py: kept curvature's recurrence get_Cl, grafted the bmodes tensor_cls add-in and the 4-tuple EE<->BB lensed_Cls. - main.py: unioned expected_keys (omega_k,K + r,n_t). tensors.py TensorSpectrumSolver ported to the recurrence (_tensor_sources + _Cl_all_ells_tensor): every integer ell 2..l_tensor_max computed exactly via the flat (K=0) spherical-Bessel recurrence, forming the tensor T/E/B radials from Phi_l, dPhi/k, d2Phi/k^2. Dropped the phi-table imports and the sparse-ell CubicSpline. Curved geometry + tensors is guarded as unsupported (the tensor radial basis is flat-only). Validation (1xA100, LCDM + r=0.1): - accuracy_test_bb.py PASS: raw BB at CLASS nodes 8.9e-4 (recomb) / 3.1e-3 (low-l), sliver 491-500 8.8e-4 (was 1.4e-2); lensed TT/EE/BB <=1%. The l=477 residual is gone (the ~5e-3 between CLASS nodes is CLASS's own interpolation; node-level 8.9e-4 vs every-l 5.1e-3). - scalar accuracy_test.py PASS (merged scalar path intact). - reverse-AD tensors=True finite on both lensing flags. - Forward ClBB^2 and reverse grads match the old table path to ~1e-6. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… dual-path plan The curvature merge made the every-ell recurrence the scalar get_Cl for everyone -> +0.64s (lensing=False) / +0.78s (lensing=True) on the default tensors=False forward vs the old table+spline. Confirmed intrinsic to the recurrence (sequential every-ell scan walk), not a bmodes bug: the pure curvature A/B (3174b25 vs base 5eabbab) shows the same +0.64s, so the curvature branch carries it too. Agreed fix (not yet implemented): dual scalar path gated on self.curvature (flat->table+spline fast path, curved->recurrence; tensor sector untouched so l=477 stays fixed). Full writeup in HANDOFF_bmodes.md + proposed_diff_dual_scalar_path.md. Adds timing tools prof_default_path.py / prof_scalar.py and a self-contained diag_bb_recurrence_check.py (proves the ~5e-3 mid-node BB residual is CLASS's own interpolation, node-level 8.9e-4 vs every-l 5.1e-3). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…flag The curvature merge made the every-ell hyperspherical-Bessel recurrence the sole scalar get_Cl, adding a node-dependent forward cost (up to +0.64s/+0.78s on slow nodes, ~0.1s on fast ones) to ALL forward calls incl. tensor/B-mode and OLE runs. Gate the scalar get_Cl on the static self.curvature flag: flat (curvature=False, the common path) -> restored tabulated-Bessel Cl_one_ell + sparse-ell CubicSpline (origin/main's long-validated path); curved (omega_k != 0) -> the recurrence (required for curved radials). The tensor sector (tensors.py) is unchanged, so the ell=477 fix is preserved. Restored from 5eabbab: bessel_tab/*.txt, module-level table loaders + j/Q/J lambdas, ells_indices/lensing_ells_indices fields, Cl_one_ell; setup.cfg re-adds bessel_tab/*.txt to package_data. Validated (1xA100, nid001205): flat-table vs curved-K0 agree to the scalar spline floor; dual-path strictly faster than recurrence same-node (9.66 vs 9.78 / 10.39 vs 10.48 s); accuracy_test PASS, accuracy_test_bb PASS (raw BB nodes 8.9e-4/3.1e-3 unchanged), reverse-AD finite both lensings (tensors=True). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds B modes, both tensor perturbations and the lensing spectrum. Folds in the curvature changes from #35, so that one will need to be reviewed first.
I found the slowdown from computing bessel functions was not constant across hardware and could be as large as 0.6s which I didn't think was tolerable, so I also added the tables back in as an option for users who do not need B modes and/or curvature. Will be updated in #35 as well.