Repository navigation
Conversation
…lantine-Community#714) EDEN_NX2_GPU_PROFILE, EDEN_NX2_CMDPAGE_DIAG, EDEN_NX2_DOWNLOAD_TRACE, EDEN_NX2_PB_CHECK, EDEN_NX2_3D_METHODS and EDEN_NX2_SEM_WATCH were read by namespace-scope initializers. Those run before main(), so before LoadNx2Switches puts nx2_switches.txt into the environment: set in that file, the switches did nothing, and only a real environment variable turned them on. The three header flags stay plain globals (Enabled() runs several times per method write) and VideoCore::CreateGPU reads them again. The DmaPusher and Maxwell3D switches become const members read when the channel is made, like Maxwell3D's record_write_history.
…euse, per-method switch reads (Eglantine-Community#778) * tests: a remembered rejection marks its page in the shader cache's filter From zortos293/Lichii#408 (the shader_invalidation.cpp part of ba882ccb55), for the Eglantine-Community#250 port above, which takes the RememberRejection mark from the same commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 16b4992) * tests: cover the K=16 software HMMA loop's operand halves and rounded stores From zortos293/Lichii#408 (the spirv_software_matrix.cpp part of ba882ccb55), for the Eglantine-Community#174 port above. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit e30e573) * tests: cover the page filter and the Ocarina of Time network bypass setting zortos293/Lichii#250 and Eglantine-Community#319, ported above, came without tests. The page filter tests check marks across pages and 64-page words, empty marks, and the conservative answer outside its 64 GiB range. The bypass tests check the tonemap shader hashes and that the bypass follows the setting and the title. Co-authored-by: Cursor <cursoragent@cursor.com> (cherry picked from commit ad1c0f62a97853b138b24e1a3d0b9e9e12419541) Taken from zortos293/Lichii#408 (ad1c0f62a9); applied cleanly. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit c63b0b1) * fence_manager: with the GPU profile on, log why fences are made instead of skipped Ocarina's Castle Town: 6.1-7.7k fences made a second, 4.3-5.4k of them for committed cache flushes (preflush downloads) and 1.8-2.3k for query cache fence work; 6.3-8.4k skipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d6fb9f5) * debugger: speed_limit method lifts or restores the speed limit live Test scripts boot through the logos and menus unlimited, then restore it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 5b562f8) Adapted: nx2 has no live switch method to sit beside, so only speed_limit is added, and docs/nx2/debug-mcp.md documents it. * video_core: read EDEN_NX2_SKIP_SAME and EDEN_NX2_PERF_VERIFY once per Maxwell3D From zortos293/Lichii#488 (9010ecd678, mstampfli), second half: SkipSameValueWrites() and Nx2Perf::Verify() run for every 3D method on the back and kept their value in a function-local static, so each call checked its initialization guard (0.9% and 0.3% of the back thread in their profile). Adapted: the PR reads both into namespace-scope statics at static initialization, which runs before main() loads nx2_switches.txt (LoadNx2Switches), so a switch file setting either would be missed. Here Maxwell3D reads them in its constructor into members. Other Nx2Perf::Verify() callers are unchanged. Not taken from Eglantine-Community#488: its same-value drop of vertex_buffer.first/count (already done by our port of Eglantine-Community#178, bd27dc7) and of index_buffer.first, which the front sends on purpose (it sets Dirty::IndexBuffer; our EDEN_NX2_EXP2 is the guarded version). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 4f9a471) Adapted to Eglantine-Community#714: the two switches are const members with default initializers, read when the engine is made like trace_first_method_writes, instead of assignments in the constructor. nx2 has no FrontDmaReadEarly, so that hunk is dropped. * video_core: bind no texture for a handle word that is a storage buffer address Ocarina of Time's water material (shader 68d269bc849da37c, the Zora's River waterfall and other water) samples a light's texture in a clustered light loop: the handle is loaded per pixel from the light buffer, so no draw-time handle program can compute it. The translation then falls back to the first constant buffer word on the handle's path, c0[0x520], which is the light buffer's address. Its low bits named an unrelated texture (an HDR render target, a zero water simulation map), and the game double-buffers that buffer, so the texture changed every frame and the water flickered every other frame in bursts. A texture whose handle word is where a storage buffer of the same program keeps its address now gets the null view and sampler. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d603d98) Adapted: unconditional. The fork's EDEN_NX2_EXP21 switch (default on) is dropped, since nx2 doesn't merge experiment switches. * video_core: keep two idle upload staging buffers per level up to 32 MiB Walking through Hyrule Field in Ocarina of Time, single uploads of new 2D images held a draw's setup for 2-68 ms. Upload staging buffers are now kept, two idle per level up to 32 MiB (under 128 MiB), instead of being freed at each 64-frame sweep: uploads over 8 MiB miss the stream buffer and got a new dedicated allocation after every pause. This is bit 1 of the fork's EDEN_NX2_EXP8 (7d75f91, on by default there), taken alone and unconditional. Its other bits (skipping uploads a join copy overwrites, banded unswizzling, no HintCold) are not taken. (cherry picked from commit 7d75f91, staging pool part only) * tests: a page filter mark past its space leaves the tracked range untouched The page filter test imported from the fork (c63b0b1) expects the first version of Eglantine-Community#250, where a mark past the 36-bit space set an overflow flag that made every query answer yes. nx2 has Eglantine-Community#250's later commit 65e3a0983c, which leaves such ranges untracked so images without CPU backing don't switch the filter off. Check that behaviour instead. * fence_manager: no branch hint on the fence decision profile check docs/Coding.md leaves branch likelihood to PGO. --------- Co-authored-by: Frizti <FriztiMouche@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: ZeroOneTV <wwwteko@hotmail.com>
…ds (port of Eglantine-Community#805's read-back) (#1028) * video_core: let the reading thread wait for its downloads A guest CPU read of GPU-written buffer memory (reactive flushing) had the GPU thread download it with Scheduler::Finish. The GPU thread waited for the host GPU with the buffer cache locked, and the reading thread waited for the GPU thread. In Ocarina of Time that was about 1,500 flushes every 10 seconds and 80-90 ms of each second on the split's back thread. Meanwhile every guest core that looked up a flush area spun on the buffer cache lock. Now the GPU thread only records the copies and submits them (RasterizerInterface::FlushRegionForRead, BufferCache::DownloadMemoryForRead). The reading thread waits for that submission's tick itself. It then writes the bytes to guest memory before it reads (FinishRead), and holds the lock only while it copies them out. Until the bytes land: - The ranges stay GPU-modified, so nothing treats guest memory there as current. Another read of them waits for the same copies instead of flushing again (RasterizerDownloadArea::in_flight). - CPU writes keep their bytes. The copy-out leaves out the writes that are still collected, which now includes those InvalidateGPUCache has gathered but not applied yet. Applying a write takes its bytes out of the download (ClearDownload). - A newer GPU write keeps its range GPU-modified. The older download still writes what a synchronous flush would have left there. - Downloads are written oldest first, and before a later fence's downloads (PopAsyncBuffers). None lands over newer bytes. - A write to guest memory from the GPU's side first writes the downloads under it. This covers semaphore releases, inline and constant buffer uploads (MemoryManager) and DMA clears (DMAClear). While no download is pending, the check costs one relaxed load. Ported to nx2 (EGLA-91 step 3): GPU::OnCPURead keeps nx2's flush without the guest-wait timing of fc42430f51 (Stats::FlushTiming, RecordReactive, RasterizerDownloadArea::source) and without d86bffda46's GpuSplit::RequestTick, neither of which nx2 has. vk_rasterizer.h keeps nx2's ReadGpuWrittenCommandList signature and IsGpuTickDone. (cherry picked from commit 1072d4a1eded663614a56694456755e90287d737) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit c91a11088bf15aac743f5913006e882f53e9b971) * video_core: compare read downloads with a synchronous flush (EDEN_NX2_READ_CHECK) EDEN_NX2_READ_CHECK=n checks 1 in n read downloads (1 checks them all). The GPU thread copies a second time, in the same submission, what a synchronous flush of the read would have copied. Once the download is written, guest memory is compared with that copy, except where a newer value has replaced it since (CPU writes, ClearDownload). Each byte a synchronous flush would copy must also be in this download or an earlier one. Every 10 seconds the log has a line "Read download check (EDEN_NX2_READ_CHECK): ... bytes differ, ... GPU-modified bytes no download brought (both must stay 0)". Each difference also gets an error line, up to 16 of them. The "GPU waits" line now shows separately from the wake-up how long reading threads spent waiting for their downloads and copying them to guest memory ("then copies on the reading thread"). It also shows how many reads only waited for an earlier read's downloads. "downloaded" counts the read downloads again. Ported to nx2 (EGLA-91 step 3) as the base of the read-back verify mode: without the "GPU waits" line and its download byte count (gpu_wait_stats.h, fc42430f51), which nx2 doesn't have. (cherry picked from commit f6fc343a7732c048292cc65afea9adbd5d55ea87) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 3cbd68a5ed57ada73667c5899d166261ed046f9d) * video_core: land read downloads under query results, outdate them for query copies Two writers went around the read downloads of 6f8c97dd86. The query cache writes report values and semaphore payloads into guest memory through device memory pointers, not through the memory manager: the completion operation, the immediate payload path and SemiFlushQueryDirty. A read's download still on its way over those bytes landed the GPU's older copy over them later. FlushRegionForRead even wrote them right after taking the download, where a synchronous flush had written them after its copy. They now land the downloads first, as MemoryManager::Write does. SyncValues copies query results into the buffer cache's buffers on the GPU without recording a write. A read download taken before the copy kept its bytes as the newest, so a later read skipped them. OutdateReadDownloads, which MarkWrittenBuffer now calls as well, takes them out of the earlier downloads' clean ranges, and the later read downloads them again. Both ask ReadDownloadsMayCover first: whether a read download is pending, and on Switch 2 whether the bytes' pages were ever GPU-written. The memory manager's writes ask it too, so semaphore words and constant buffer data on other pages no longer take the buffer cache's lock while a read is pending. SemiFlushQueryDirty runs with the query cache's cache_mutex held. Nothing that holds the buffer cache's lock takes cache_mutex, so the new lock order cannot deadlock. Tests: the next read downloads a query copy made after an earlier read's download again. On Switch 2, a write to a page no GPU write reached doesn't wait for a pending download, and a write to a GPU-written page lands it first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Restacked onto nx2 after #979 (the Switch 2 semaphore mirror): CounterReport calls FinishReads before the mirror's Diverts check, so a diverted release reads the word's guest value only after any read download under it has landed. The mirror itself is unchanged. (cherry picked from commit 4c6864af91b2dc9501ba29c71ef2ec2636f957b1) (cherry picked from commit d7766acb67f426a050b7d0aebde3ef4c73096b7b) * video_core: read download check: compare kept bytes as they landed The check from a06257737b compared every GPU-modified byte of the read with the GPU's copy at the read, guest memory included where an earlier read's download brought the bytes. After that download lands, the GPU's side can write newer bytes there before the later download lands: constant buffer data or an inline-to-memory write through the memory manager, which lands the earlier download first and tells the buffer cache only at the next FlushCaching, or a query result. Synchronous flushes keep those bytes too: the earlier read's flush had cleaned them, so the later one doesn't copy them. The check reported them as differences. In the harness, constant buffer data written that way gives exactly the test47 line, "guest memory ...+0x30 differs from a synchronous flush", with the newer bytes in guest memory. Each check now notes the bytes earlier downloads bring (kept), and what guest memory holds there right after each of them lands. At its own landing: - Bytes rewritten since are left out and counted ("got newer values after an earlier read's download brought them"). The first 16 are logged with their writer: the memory manager's writes and the query cache's results note themselves while read downloads are pending. - Kept bytes are compared as they landed: a difference there means the GPU wrote them after the earlier download without taking them out of it, and is logged as such. - This download's own bytes are compared as before. The summary line adds the rewritten count. Tests: constant buffer data over an earlier read's download is left out, and a GPU write that skipped OutdateReadDownloads is found. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 7693bb232b824ca2df4e9090af41c35534d907f9) (cherry picked from commit e220fafe5b2783594e68ea37b57a9e903d529c85) * video_core: download read-back pages right behind their GPU writes Ocarina of Time reads back two compute results every frame, one page each. Each read waited 2.6-2.8 ms for its download. The read-trace (test47d) shows that the dispatch writing a page ended on the host GPU 6.8 ms before the read came. The download was recorded only at the read, behind about 770 draws, 95 render passes and 10 submissions recorded since the write. The host GPU had to run all of them before the copy. The buffer cache now learns which pages the guest reads after each GPU write. A page qualifies once two of its writes in a row were read. Four writes in a row without a read stop it, and 64 pages are kept, the one read longest ago making room. When such a page is written, its download is recorded right behind the writing work (RecordEarlyDownloads): - After a dispatch, at once: the scheduler is outside any render pass. Pages written by a draw wait for the next dispatch, a read's flush or the frame's end. - It doesn't submit on its own. The copies go with the GPU thread's next submission, as a fence's do (QueueForHostRead). A thread that must wait for one before it is submitted asks the split's back to submit it. The back submits it itself. An early download is an ordinary read download, recorded before any read asks for it. The read finds it the way it finds an earlier read's download. GetFlushArea now writes such downloads to guest memory itself once the host GPU has run them. The read then waits for nothing. Finished early downloads also land after each dispatch, every 32 draws and at the frame's end, so most reads find the bytes already in guest memory. Everything that keeps read downloads correct applies unchanged: - A newer GPU write before the read leaves its bytes to a download of their own (OutdateReadDownloads). For a learned page, that is the next early download. - CPU writes keep their values (ClearDownload, pending writes at landing). - Writes to guest memory from the GPU's side land the downloads under them first. - Downloads land oldest first, and before a later fence's downloads. Early downloads are bounded to 1 MiB a frame and 32 not yet in guest memory. Past either bound, reads flush on demand as before. EDEN_NX2_EARLY_READ_DOWNLOADS=0 turns them off for comparisons. Tests: a page read after each write gets its download behind the next write. A read before the host GPU ran it waits for that copy alone; after, it waits for nothing. A newer dispatch or draw write gets its own download. CPU writes keep their values. A page no longer read stops after four writes. A semaphore release over a pending early download lands it first. Finished downloads land within 32 draws or at the next dispatch. The 33rd download still on its way, and the frame's downloads past 1 MiB, are left to their reads' flushes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3): DMACopy drops GpuWaitStats::NoteGpuWrite (fc42430f51), which nx2 doesn't have. (cherry picked from commit 783a9a3409cef0a79f8d570ac08d5b261e8a050a) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 6b5292696b85d5fd3dc2b8b394079a96034793ac) * video_core: read download check for early read downloads The read check half of 00a62b9189 by jpkto (Lichii PR Eglantine-Community#805): the read download check (EDEN_NX2_READ_CHECK) extended to the reads early downloads serve. Without it the check never sees them: a landed early download leaves the page clean, and one still on its way covers the read, which then records nothing. An early download that lands records which bytes it wrote to guest memory and what they were. In check mode, a read that an early download serves flushes. Its check copies the read's bytes from the GPU at the read, as a synchronous flush would. The bytes still on their way are compared as kept bytes are. The landed ones are compared against their record: if guest memory changed since the landing they are counted as rewritten and not compared; otherwise they must equal the GPU's copy. Newer downloaded bytes or guest values drop the record (WriteDownloadedMemory, ClearDownload), and each read takes what it compared. A check whose read gets all its bytes from other downloads becomes a download of its own, which the read waits for. Ported to nx2 (EGLA-91 step 3) without the read trace (EDEN_NX2_READ_TRACE: nx2_read_trace.h, the scheduler's device timestamps, the calibrated timestamps extension and the RasterizerInterface::TraceRead hook), which the verify mode doesn't need, and without fc42430f51's GpuWaitStats, which nx2 doesn't have. (hand-ported from commit 00a62b9189ef84ef8c37426bdb1e75ba5e64bea8) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 5fec72c43e472d4757aef24192ea9a6228f71c8a) * core, video_core: let guest writes wait for a download landing under them The read download check of test49b found 11 bytes of one landed download that differed from its check copy: 0x2b648a20, three bytes of each of four words and two of the last, the top bytes equal. The download and its check copy are recorded back to back and read the same bytes, so guest memory changed between the landing and the check. The only writer that neither takes the buffer cache's lock first nor is left out of the check is a guest core. A guest CPU write into GPU-written memory is collected (HandleRasterizerWrite, GPUDirtyMemoryManager::Collect), then stored, without the buffer cache's lock. After a core's first write to such a page, its later writes there skip OnCPUWrite too: Ocarina of Time collects about 1.3 million writes a second this way, and the check sees about 11 KB a second of them over bytes that downloads landed. A download keeps the bytes of the writes collected when it looks at them (PendingCpuWriteRanges), then writes the rest. A write collected and stored in between was handled wrong: - after the landing's look and before its write, the download wrote the GPU's older bytes over the guest's. The guest's write was lost, in what the game reads. - after the check's look and before its read of guest memory, the check compared the guest's newer bytes with the GPU's copy: the test49b lines. The synchronous downloads (DownloadRangesMemory, DownloadBufferMemory) looked before they waited for the host GPU, so a write collected during that wait was overwritten as well. None of this is new in test49b; its verify run only lands more downloads a second. Each landing is now published (VideoCore::DownloadLanding: its range, and a sequence that is odd while it lasts). A download looks at the collected writes only inside its landing, and writes in it. HandleRasterizerWrite calls DownloadLanding::Await after Collect: a write in the range of a landing under way is stored once it ends. Each side fences between its store and its load, so either the landing's look finds the write and keeps its bytes, or the write finds the landing and comes after it. Downloads land under the buffer cache's lock, one at a time. A read download's landing also covers its check (EDEN_NX2_READ_CHECK), which then compares exactly what the download wrote. Every landing takes part: read downloads (WriteReadDownloads, early ones included), fence downloads (PopAsyncBuffers), and the synchronous ones, which now look once the host GPU has run their copies. A guest write pays a fence and a load. It waits only while a landing over its bytes is under way, a few microseconds. Tested in the read download harness (the real BufferCache, device memory manager and MemoryManager, with the fake runtime) with new tests in buffer_cache_ports.cpp. In them a second thread collects and stores a write as HandleRasterizerWrite does, right after a chosen look: - after a read download's look: the write was lost before this change; - after the check's look: before this change the check printed test49b's shape exactly (0x20a20+0x3 (0xcd for 0x68), 0x20a24+0x3, 0x20a28+0x3, 0x20a2c+0x2, 11 bytes differ); - an early download landed at a draw: the write was lost before; - during a synchronous download's wait for the host GPU, staged and mapped: the write was lost before; - 2000 writes racing read downloads at random delays: before, 2 of 3000 were lost and 326 checks reported differences. All 30 harness tests pass in 21 runs, 10 of them pinned to one CPU. Each of these breaks a test: the landing not published, the look before the landing, the check outside it, the synchronous look before the wait, an empty landing range. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3): DownloadRangesMemory and DownloadBufferMemory drop GpuWaitStats::AddDownloadBytes (fc42430f51), which nx2 doesn't have. (cherry picked from commit 4942eb8a398c857a0b214672d8cdb1f56c03b3f6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit f860e1161e3bfef33bd1a8fb50623382cb8dc3c2) * video_core: read download check: leave out landed bytes newer writes replaced The Hyrule Field verify run of 611a09d4a0 reported one byte as Outdated (guest memory 0x30473f8f held 0x46, the value an earlier download landed, the check copy of the host buffer 0xbf). Its log's version line names f3702f3b97 because it is written at configure time; the build included gated on IsFree) and fence manager change (profile logging) don't touch read downloads. The check reaches that result in three ways while guest memory holds what a synchronous flush leaves, all in the check and not in the downloads: - An early download landed the byte, so it was no longer GPU-modified. A guest write applied later (WriteMemory) reached no ClearDownload, so early_landed kept the byte with its landed value. Written to another value that a draw uploaded, then back to the landed one before the next upload, guest memory equals the landed value while the host buffer holds the upload's: the read's check took it for a GPU write the early download missed. Without early downloads that write takes the byte out of the GPU-modified ones and the read's flush leaves it. - A query result copy (SyncValues) gives landed bytes a new value in the host buffer that the cache hears of only as outdating downloads. Guest memory gets the result once the query cache writes it there, and a synchronous flush doesn't download those bytes either. - A later read's check kept bytes an earlier download brings. That download landed alone and kept the bytes of a guest write still collected (PendingCpuWriteRanges), but NoteLandedBytes noted guest memory there as landed. Once the write was applied it no longer counted as collected, so the later check compared the guest's value with the GPU's as if the download had brought it. WriteMemory and OutdateReadDownloads now forget early landed bytes they change, as ClearDownload does for GPU-modified ones, and NoteLandedBytes takes the bytes a collected write kept out of the later check (changed), as CheckReadDownload does for the download's own check. For MarkWrittenBuffer's calls of OutdateReadDownloads nothing changes: its bytes are GPU-modified from then on, which TakeEarlyLanded leaves out, and every way out of GPU-modified (ClearDownload, WriteDownloadedMemory) forgets them too. All of this is EDEN_NX2_READ_CHECK state only. Guest writes over early landed bytes no longer count as "rewritten" (no read compares those bytes any more); that total keeps the writers the buffer cache doesn't hear of, and guest writes over bytes a read's check holds already. Ruled out on the way: the downloads' own copies record their barriers (with d612710df3, b62f49104d and 8b5be87f79 on this branch), uploads leave out GPU-modified bytes, and reordered uploads can't reach bytes still on their way. Tests: three buffer_cache_ports cases, one for each way, each reporting exactly that Outdated line without the change. Read download harness: all cases pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3) with the ReadCheck counters reads_recorded, alone_recorded and early_recorded and their increments from cf3c7f0099, which this port leaves out otherwise: one of the tests here reads alone_recorded. (cherry picked from commit eb64bfbf87dedfd8c6224f0cc78ebb9982eb1ace) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 0103781395442d9068060be43478f663b3087e11) * video_core: read download check: leave out landed bytes an upload took from the guest The first verify run of the merged branch (d8988e7032, Hyrule Field) reported 55 bytes Outdated in 339,618 downloads, with the explanation line from f9f3bc6394. Every one was path (a): early landed bytes, guest memory equal to the snapshot taken at landing, and the check's copy of the buffer different. Every explanation also read "upload (after it landed, before this read's copy)" and "a guest write since it landed: no". An early download's landed bytes are no longer GPU-modified, so a guest write there may never reach WriteMemory, and the buffer cache doesn't hear of it. The next upload sends that write to the buffer. If the guest then writes the landed value back, guest memory equals the snapshot while the buffer holds the write the guest overwrote. A synchronous flush would not download those bytes at all, so the guest has what it would have without early downloads. The check was wrong, not the downloads. An upload now forgets the early landed bytes whose guest value differs from the snapshot (ForgetRewrittenEarlyLanded). Bytes still equal to it are kept: the buffer gets the landed value back, and the read still compares them. This changes only EDEN_NX2_READ_CHECK state, and costs one test of early_landed per upload range otherwise. Tests: a new buffer_cache_ports case, "Read download check leaves out early downloaded bytes an upload replaced". Without the change it reports "0x20040+0x10 is outdated: (a) ... upload (after it landed, before this read's copy) ... a guest write since it landed: no", the shape of the user's lines. The read download harness passes with and without moving each test's DeviceMemory: 14,348 checks, 0 failures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3) without the NoteReadCheckEvent calls of f9f3bc6394 (explanations of Outdated check results), which this port leaves out. The test fixture's Runtime gets FreeDeferredStagingBuffer from 34656e8225, also left out: the new case's WriteMemory reaches CreateBuffer. nx2 has the fixture's ClearBuffer and buffer-to-buffer CopyBuffer already. (cherry picked from commit f01be636def436e5a46da393a96d008698520051) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit ab72c7edfb643dd626e2886fb6221dd88ff62798) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: mark the read check's own download copy as a use of its bytes EDEN_NX2_READ_CHECK copies the bytes of a read a second time, from the same point of the GPU's work, and compares them with what the read's download brought. Unlike the read's own copies, that second copy marked nothing. Once uploads are reordered within a submission, an upload of those bytes recorded later in the submission could run before it, and the check would compare against bytes written after the read. The copy is marked now, as the read's copies are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 3406508a7540025ab1ae98d09b31f8cfbd7322dc) (cherry picked from commit f2ab7144f9c725599498d407d964f604dfe1c69c) * gpu: take the applied writes out of applying_writes on every exit InvalidateGPUCache publishes the address of its local writes vector in applying_writes and took it out again only after OnCacheInvalidations returned. If that call threw, the vector was gone while its address stayed, and the next ForEachPendingCpuWrite read freed memory. A scope guard now takes it out on every exit. Nothing else changes. Found by CodeRabbit's review of Eglantine-Community#805. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 34621dcd6baf7546d68d6d37955c501a14232ca3) (cherry picked from commit 2452f2fe7ce9ee37c9ea3123b65f5d6385aeabb3) * video_core: one test switch for async read-back, EDEN_NX2_ASYNC_READBACK The read-back commits ported from Eglantine-Community#805 (jpkto) ran unconditionally, with EDEN_NX2_READ_CHECK and EDEN_NX2_EARLY_READ_DOWNLOADS as their switches. They now sit behind one switch, read at static init and again in VideoCore::CreateGPU (after nx2_switches.txt is loaded): - unset or 0 (the default): GPU::OnCPURead calls FlushRegion as before, so the GPU thread waits for the host GPU (Scheduler::Finish). No read or early downloads are taken, and core writes don't wait for DownloadLanding. - 1: the GPU thread calls FlushRegionForRead and goes on; the reading thread waits for its downloads in FinishRead and writes them to guest memory. Pages read after each GPU write get early downloads (what EDEN_NX2_EARLY_READ_DOWNLOADS turned on). - verify: 1, and every read download is compared with a synchronous flush's copy (EDEN_NX2_READ_CHECK=1). The mismatch line, still capped at 16, now adds the download's kind (early, read or check only), the read's range and who wrote it to guest memory: the reading thread, a write from the GPU's side, or a thread that found the host GPU had run it. The totals line splits differing bytes the same way. Counters (nx2_async_readback.h): reads by kind (synchronous, async, already in flight) with their latency, the GPU thread's flush time, and host waits split between the reading thread and GPU-side writes, with the longest. With the switch set, or EDEN_NX2_GPU_PROFILE for the unset baseline, they are logged as per-second rates every 10 s ("Async read-back (EDEN_NX2_ASYNC_READBACK=...) per second"). Tests: mode parsing, no early downloads with the switch unset, verify compares every read download, verify names the lander, and the two wait counters kept apart. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 2276bc218072cf4549dd28677a1e99230a072db2) * video_core: wait only for the work that writes what a macro DMA reads When a macro on the split's front reads memory (MME_DMA_READ and MME_DMA_READ_FIFOED, FrontDmaRead), the front drained the whole split (maxwell_3d.cpp:1431 before this change), had the back download what the buffer cache holds of the words (line 1449) and write out the reports the query cache holds there (line 1456), then read them. In Ocarina of Time's Hyrule Field (EDEN_NX2_GPU_PROFILE, about 50 fps) that was 25,500 reads a second, all by macro 0xe2c (24 bytes, a ReportSemaphore::Compare: the render enable condition of two reports), at 9.6 us each: 246 ms of every second of the front, which runs in near lockstep with the back. The words were written long before almost every time: occlusion reports a frame earlier, status reports some objects back. EDEN_NX2_CMDPAGE_DIAG found 0.3% of the reads changed by the drain. What a drained read returns is guest memory once the back ran every entry before it, plus what those two caches write out. Only entries that write guest memory there, make it GPU-written or leave reports for the query cache can change that. The 3D engine's front sends nearly all entries and knows where its own writes go, so it notes them as it sends them (FrontNoteWrites, maxwell_3d.cpp:1429): - constant buffer data at the constant buffer address and offset, a report or semaphore release at the report address (CounterReport writes its value and timestamp, now or once the host result is in), and a pitch inline upload over its lines are placed at their addresses; - a draw, or any entry the back runs while one may be pending in the draw manager (an instance loop's deferred draw is issued by the next register write, maxwell_3d.cpp:361; FrontState::draw_pending, line 1045), a block linear upload, a render enable condition read from memory (ProcessQueryCondition reads it through every cache, line 864) and a syncpoint increment (macros read syncpoints through memory too, memory_manager.cpp:306) count as writes anywhere; - the other executable methods change registers and host state only, and state writes the back sinks run nothing yet. Every other entry, from the other engines, the puller or deferred calls, counts as a write anywhere unless its sender says otherwise (GpuSplit::Writes). GpuSplit::WaitForWrites (gpu_split.cpp:301) waits for the last entry that may write anywhere and the last one placed in the words read. It places writes by the device memory the GPU addresses map to, so the same memory under another GPU address is found: GPU maps drain the split before later commands (GPU::MapInOrder, gpu_thread.cpp:128), and the unmaps it is sent are deferred calls. Past those entries no queued work can make the words GPU-written or add reports there, so the caches hold now what they would after a drain. The front checks them where it can without the back: the buffer cache as before (IsMemoryDirty), and the query cache under its lock for a report whose value or timestamp reaches into the words (HoldsQueries, query_cache_base.h:96). When either holds them, it drains and has them written out as before; otherwise it reads the words at once. The fencing thread writes only reports the cache holds, and those reads drain. The split test checks that a macro's DMA read on the front goes ahead of other queued work and of writes elsewhere, waits for a write of its words under another GPU address and sees it, waits for work that does not note its writes, and where writes are placed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3b). nx2 has the front's semaphore write tracking (NoteSemaphoreWrite and FrontNoteReport) since #949, but not WaitForSyncedWriters/WritersPending (6229487c8c): FrontWrite and FrontCallMultiMethod keep nx2's sink_pending handling, send Writes::Noted and call FrontNoteWrites before FrontNoteReport, as Eglantine-Community#805 does. GpuSplit::Defer keeps nx2's flags (FastDrawNeutral) ahead of the new Writes argument, so the split test passes 0 for them. The test's GpuSplitTestAccess::FrontWaits comes from 6229487c8c; it reads wait_target, which nx2 has. (cherry picked from commit 1d9b6ef31d7c271fcc955094cf37806478e2e3c6) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 3c0121c36155290704f8c64dabcead88279ba407) * video_core: check macro DMA reads against a drained split (EDEN_NX2_DMA_READ_VERIFY) A macro DMA read on the split's front now waits only for the queued work that can write its words and drains only when the caches hold them (FrontDmaRead). EDEN_NX2_DMA_READ_VERIFY=1 checks that against what every read did before: after reading the words without a drain, the front drains anyway, has the back write out what the buffer and query caches hold of them, reads them again and gives the macro those. It counts the reads whose words differ (CheckDmaRead) and logs every 10 s how many reads it made, how many without a drain, how many of those waited for queued writers and how many differed, with the first 16 differing reads and their words. The differences must stay 0; a guest CPU write that lands between the two reads would count too. Off, it reads one constant flag per read. Rebased onto upstream: EDEN_NX2_DMA_READ_VERIFY is a Maxwell3D member read when the engine is made, as Eglantine-Community#714 made EDEN_NX2_3D_METHODS, so it works from nx2_switches.txt too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Ported to nx2 (EGLA-91 step 3b): the header keeps nx2's per-engine switches (skip_same_value_writes, perf_verify) beside dma_read_verify. The next commit replaces EDEN_NX2_DMA_READ_VERIFY with EDEN_NX2_ASYNC_READBACK=verify. (cherry picked from commit 936c05e1492476eb18901ce95313dcfbb670d69b) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit 5f83be850894f3e6ccd3caa91e1fc86c7e0da138) * video_core: put the macro DMA read's narrower wait under EDEN_NX2_ASYNC_READBACK The two commits before this one, ported from Eglantine-Community#805 (jpkto), made a macro DMA read on the split's front wait only for the queued work that can write its words, with no switch, and added EDEN_NX2_DMA_READ_VERIFY. EDEN_NX2_ASYNC_READBACK now selects both: - unset or 0 (the default): FrontDmaRead drains the split and has the back write out what its caches hold, as nx2 did before (the same write_out sequence). FrontNoteWrites returns at once, so the front places no writes. Every sender still records the last entry that may write anywhere, which is one store per entry. - 1: the read waits for GpuSplit::WaitForWrites, and drains only when the buffer cache or the query cache holds its words. - verify: as 1, plus the check EDEN_NX2_DMA_READ_VERIFY turned on. A read made without draining also drains and is compared, and its 10 s line names the switch. EDEN_NX2_DMA_READ_VERIFY is removed. The per-second "Async read-back" line now also reports macro DMA reads and how many of them waited for the split's back (every read without the switch), with the time from the start of the wait until the words could be read, plus how many drained the split. Counting runs with the switch or with EDEN_NX2_GPU_PROFILE, for the baseline. Tests: the ported split test runs with the switch on, checks the new counters, and adds two steps. With the switch unset, the read waits for queued work that writes nothing. With =verify, it waits for that work too. Ignoring the switch in FrontDmaRead fails the unset step. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> (cherry picked from commit f2610d3dfbde8c4dcc007fd58647b1cc032840c5) * video_core: always use async read-back, drop EDEN_NX2_ASYNC_READBACK EDEN_NX2_ASYNC_READBACK=1 measured +9% fps in OoT Kokiri (27.0 to 29.4, p90 39.9 to 36.1 ms, 3+3 launches), so its behaviour is now the only path and the switch is gone: - GPU::OnCPURead has the GPU thread submit the read's downloads (FlushRegionForRead) and the reading thread wait for them and write them to guest memory (FinishRead). - Pages the guest reads after each GPU write get early downloads. - Guest writes wait for a download landing under them (DownloadLanding::Await). - A macro DMA read on the split's front waits only for the queued work that can write its words, and drains only when the buffer or query cache holds them. FrontNoteWrites always places the writes. The checks =verify turned on stay as diagnostics, off unless set: - EDEN_NX2_READ_CHECK=n compares 1 in n read downloads with a synchronous flush's copy (1 checks them all), as in Eglantine-Community#805, except that 0 now checks none. - EDEN_NX2_DMA_READ_VERIFY drains and compares every macro DMA read made without a drain, as in Eglantine-Community#805. Both are read again in VideoCore::CreateGPU, so nx2_switches.txt can set them. The "Async read-back per second" line is logged only with EDEN_NX2_GPU_PROFILE; it no longer has a synchronous read count. AGENTS.md lists the two checks in its debugging table. Tests: the mode-parse test checks EDEN_NX2_READ_CHECK values instead. The switch-off cases (no early downloads, every macro DMA read drains) are removed with the path they covered. The read check and macro DMA verify tests set their own variable. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: drop the [[likely]]/[[unlikely]] attributes the read-back port added docs/Coding.md asks for none; the ones nx2 had before stay. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: keep nx2.h out of rasterizer_download_area.h The Qt frontend reaches rasterizer_download_area.h through gpu.h (src/qt_common/render/emu_thread.cpp), and Qt's slots macro breaks nx2.h (Nx2SemaphoreWaiters::slots). ReadDownloadsMayCover's page check moves out of line into gpu.cpp as MayHoldGpuWrites; it runs only while a read download is outstanding. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: land read downloads before writes guest memory doesn't hold yet ClearDownload took the range out of every read download that had not landed, as if guest memory already held newer bytes. Two callers never write guest memory: an indirect dispatch's arguments (ObtainCPUBuffer with DiscardWrite) and a DMA copy into GPU-written words (DMACopy). A guest read whose download was taken before either then found the bytes from before the GPU write, and DiscardWrite also dropped the range from the GPU-modified ones. On nx2 the read's synchronous flush had written the GPU's bytes before the discard or copy. ClearDownload now lands the read downloads over the range first (FinishReadDownloads). DMAClear already did that itself and no longer needs to. Inline writes and PrepareRegionWrite get the same order, though MemoryManager's writes already landed the downloads before them. ClearReplacedDownload keeps the old behaviour for the callers that put the newer bytes in guest memory before a read can find them: WriteMemory and ClaimCpuWrites (guest CPU writes), and the synchronous downloads that land under the lock (DownloadRangesMemory, DownloadBuffersMemory, DownloadBufferMemory). Test: a read download, then a discarded indirect argument write, or a ClearDownload from the GPU's side, over it. The read finds the GPU's bytes; without the landing it finds 0. * video_core: keep GPU writes after a query report GPU-modified A report's result reaches guest memory at its fence or sync point (QueryCacheBase::CounterReport), after later GPU work. If a dispatch wrote over the result's words before then, and an early download landed those words (the result's own FinishReads or the next landing check), it also took them out of the GPU-modified ranges. The older result then landed over them, and no later read downloaded them again. On nx2 the words stayed GPU-modified, because nothing downloaded them without a read. Bounding the result's FinishReads by the report's position would not cover downloads that land before the callback, and a download recorded after the report can still hold a write from before it. So the buffer cache tracks the GPU writes instead. The query cache notes each deferred result (RasterizerInterface::NoteReportPending), and the buffer cache keeps it while it lies on a page that gets early downloads. NoteReadBackWrite marks it rewritten when a GPU write covers it, and a landing doesn't clean rewritten results' bytes. The callback ends it (FinishReport) after writing or dropping the result. Results written over GPU writes from before their report still clean, as before. Tests: a dispatch after the report, its early download landed by the result's wait or earlier, and then a read downloads the dispatch's words again (without the change the read finds the result). A dispatch before the report: the result stays. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: end only the query results the buffer cache kept NoteReportPending keeps a result only on a page that already gets early downloads, but every result's callback called FinishReport, which ends the first kept result with the same address and size. Say R1 was reported at A before A's page got early downloads, then R2 was reported at A and kept. R1's callback ended R2's entry, so a GPU write over A after R2 cleaned its bytes when an early download landed, and R2's older result was final. NoteReportPending now returns whether it kept the result, and the query cache calls FinishReport only then. Whether a result is written later is known before its callback is made, so the query cache notes it there. Test: R1 not kept, R2 kept, R1's callback, a dispatch over the words, R2's callback. The next read downloads the dispatch's words. With R1's callback ending R2 (FinishReport called for every result), the read finds R2's result. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> * video_core: count ClearDownload's landings, and keep PrepareRegionWrite from waiting ClearDownload lands the read downloads under a write that guest memory doesn't hold yet (8d096867b4), on the GPU thread and with the buffer cache's lock held, and may wait for the host GPU there. The "Async read-back per second" line (EDEN_NX2_GPU_PROFILE) now counts those landings, how many waited for the host GPU, and for how long. FinishReadDownloads returns the wait, or nothing when no download was over the range. The wait can't depend on unsubmitted work. The landing callers (DMA copy and clear, the indirect dispatch's discarded arguments, inline writes) run on the GPU thread, or the split's back, which is the only thread that submits. There, WaitForHostRead calls Scheduler::Wait, which flushes the current command buffer before it waits on the master semaphore, as Scheduler::Finish does for nx2's synchronous downloads under the same lock. The Vulkan worker takes no buffer cache lock. PrepareRegionWrite can run on any thread that writes through MemoryManager. With the split active, a wait there goes through WaitSubmitted, which waits for the back to submit while this thread holds the lock the back needs. It now uses ClearReplacedDownload, which never waits: MemoryManager lands the read downloads over the range just before it (FinishRead, with the lock released for the wait) and writes guest memory right after. Test: the read download test checks that a discarded write's landing is counted once and waited once. Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> --------- Co-authored-by: jpkto <255363096+jpkto@users.noreply.github.com> Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
hi are you able to fix the conflicts on this branch |
|
Included in #825 , still have to check if anything crucial is missing from here before merging/closing the PR. |
|
Merged the current I used Claude Opus 5.5 to resolve the merge conflict. Edit: #825 is a fix for the shader cache created and used by the Linux Nvidia proprietary driver. This PR is for the open source AMD Mesa driver. |
RADV stores compiled shaders in Mesa's on-disk shader cache, like the NVIDIA and AMD proprietary drivers do, so treat it the same way: Eden no longer saves vulkan_pipelines.bin on RADV unless Mesa's cache is turned off with MESA_SHADER_CACHE_DISABLE (or the older MESA_GLSL_CACHE_DISABLE). A log line says when the driver's cache is used instead. Tested on an RX 9070 XT (Mesa 26.2.6) with Ocarina of Time: a 50 minute session left vulkan_pipelines.bin untouched while Mesa's cache grew; with MESA_SHADER_CACHE_DISABLE=1 Eden writes it as before. Assisted-by: Claude Opus 5.5 (implementation, testing, commit message draft) Signed-off-by: Austin <16418832+ALHughes466@users.noreply.github.com> Signed-off-by: Austin <16418832+auz466@users.noreply.github.com>
6026718 to
dab0332
Compare
Summary
DriverCachesPipelines()already stops Eden saving its ownvulkan_pipelines.binon the NVIDIAand AMD proprietary drivers, because they keep a disk cache of their own and Eden's copy only cost
RAM and a stall at each save. RADV has the same thing: Mesa's on-disk shader cache. This adds RADV
to that list, unless the user has turned Mesa's cache off with
MESA_SHADER_CACHE_DISABLE(or theolder
MESA_GLSL_CACHE_DISABLE); then Eden keeps saving its copy as before. The variables are readthe way Mesa reads them (
1/true/t/yes/y, any case).It also logs one line at startup when the driver's cache is used instead, so it's visible in user
logs.
Nothing changes for other drivers, and
vulkan.bin/vulkan_used.bin(Eden's own shadercaches) are untouched.
Type of change
Related issues
None, at least that I could see.
Testing
RX 9070 XT, RADV (Mesa 26.2.6), CachyOS. Tested on
oot-optimization-fixes+ #678 (needed for OoTon RADV), which is now in
nx2-experimental.Games tested:
The Legend of Zelda: Ocarina of Time (NS2), docked, Accurate GPU:
vulkan_pipelines.binstaysbyte-for-byte unchanged, and Mesa's shader cache grows instead. No new errors compared to an
earlier session without the change.
vulkan_pipelines.binis written.With
MESA_SHADER_CACHE_DISABLE=1, Eden writes it (21 MB) after loading and at exit, as before.Checklist
Screenshots / logs
I used Claude Opus 5.5 to help write and test this change and to draft this description.
Summary by CodeRabbit