Skip to content

Added support for named but not file backed memory maps on windows. - #150

Open
Earthmark wants to merge 1 commit into
RazrFalcon:masterfrom
Earthmark:windows_named_map
Open

Earthmark wants to merge 1 commit into
RazrFalcon:masterfrom
Earthmark:windows_named_map

Conversation

@Earthmark

Copy link
Copy Markdown

See https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-createfilemappingw , this is to add some form of accessing memory maps created via the dotnet call https://learn.microsoft.com/en-us/dotnet/api/system.io.memorymappedfiles.memorymappedfile.createnew?view=net-9.0

I kept the existing pattern of implementing both sides of the unix and windows apis, although there is no equivalent in linux so that panics. This isn't a great solution, let me know if you'd prefer a different pattern.

I kept the existing pattern of implementing both sides of the unix and windows apis, although there is no equivalent in linux so that panics. This isn't a great solution, let me know if you'd prefer a different pattern.
@de-vri-es

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

My first thought is that in unix terms, this feels more like an alternative to shmopen than mmap though. So I find it difficult to see if this should be included in memmap2.

@nullstalgia

nullstalgia commented Feb 10, 2026

Copy link
Copy Markdown

Howdy! I hope it's okay if I intrude, but I had both a comment and a question relevant to this PR.

The first thing to mention is that this is actually a feature I was looking for when I found memmap and then later memmap2!

In a personal project in late 2024, I had to open a named, non-file-backed memory-mapped "file" in Windows in order to read the contents, and I ended up using the windows crate directly with the OpenFileMappingW and MapViewOfFile syscalls.

Now that ended up working great for my use case, it doesn't mean that I was a particular fan of writing all the unsafe Windows-related blocks myself, but it still worked.

Fast-forward to today, and I have to do something similar at work, opening such a "file" made from within C#/.NET! I copied the windows-rs code from my project and changed the target file name and size, and it works no problem.

But like I said, I wanted to perhaps use a more battle-tested solution since this isn't just for a slapped-together personal project, so I went digging and now found memmap2 and this PR.... which should work but doesn't for some reason, leading into my question.

Using Sysinternals Process Explorer, I can see each handle my processes have.

My C# process has the handles I expect (some items censored).

image

But using the new method from this PR in Rust, I see no such handle?

image

With the snippet:

    let mut mmap = MmapMut::map_named(&path, 1000000).unwrap(); // Using the correct size of the file
    loop {} // Map shouldn't be dropped during this loop, letting me take my time to look in Process Explorer.
    io::stdout()
        .write_all(&mmap[..])
        .expect("failed to output the file contents");

But that's using CreateFileMappingW, when I only want to open an existing one.

Using OpenFileMappingW, I can never find the the "file" by name??

fn OpenFileMappingW(dwDesiredAccess: DWORD, bInheritHandle: BOOL, lpName: LPCWSTR) -> HANDLE;

// called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "The system cannot find the file specified." }

It's not the absolute end of the world if I don't hear back from either @Earthmark or @de-vri-es, since this isn't exactly a supported use-case, though ideally it would be since we're likely not alone in desiring this functionality.

@de-vri-es

Copy link
Copy Markdown
Collaborator

Hmm.. so to explain a bit more on my earlier comment: this API feels rather similar to using Unix shm_open() followed by mapping the returned file.

So on Unix, this use case is already supported, if you use a crate to wrap the shm_open() for you.

Isn't there a Windows API to open such a shared memory file without directly mapping it? Because that would play nice with this crate: as long as it implements AsRawHandle it would work.

@de-vri-es

Copy link
Copy Markdown
Collaborator

Hmm, I see that CreateFileMappingW doesn't actually map anything, it returns a mappable object, but not a file handle.. ugh...

Anyway... maybe the solution then is to allow mapping those file mapping objects, and then it wouldn't matter how you obtain the file mapping object.

So maybe we can solve this use case and #139 with one solution.

@nullstalgia

nullstalgia commented Feb 10, 2026

Copy link
Copy Markdown

That sounds reasonable at first glace! I've felt similar pains before with how some crates silo their Windows objects despite Rust making passing them around a lot safer, especially with the type system (as long as you're not, say, duplicating Handles like mad, but that's on you at that point).

Although I'm still confused as to why my drop-in replacement of CreateFileMappingW to OpenFileMappingW resulted in the Not Found error, unless I'm misunderstanding some base principle about those methods and how memmap2 creates it's types. (But this can be worried about later, as my windows-rs impl is working fine for now!!)

@nullstalgia

nullstalgia commented Mar 6, 2026

Copy link
Copy Markdown

I had a moment to look back at this PR's differences with how I implemented it manually with windows-rs, and I've spotted the small diversions.

The biggest one is that it should be using OsStrExt::encode_wide instead of String::encode_utf16. The full rationale is detailed here within Rust's Windows FFI docs themselves, but can be simply described as "Windows' UTF-16 is not conformant due to historical reasons." As such, crates that intend to interact with Windows APIs can't expect to send/receive normal UTF-16-containing LPCWSTRs.

The second is mostly a repeat of what I mentioned previously, my use case is to open a memory-mapped file created by another (in this case, non-Rust) application, and this grows into other smaller issues. For one, by using the CreateFileMappingW syscall vs OpenFileMappingW, the "client" application will create the memory-mapped file if it didn't previously exist instead of returning an Err(). And secondly, the extremely permissive access requested has a chance of failing if the pre-existing memory map was made with stricter access settings (i.e., not allowing writing or execution).

I'm still wrapping my head around the usage of MmapRawDescriptor/MmapAsRawDesc within the crate after your suggestion earlier, in order to see if I can implement it myself and close these PRs once and for all.

But if you have an idea of how to do it right, feel free to take a stab at it or let me know what you'd expect from a proper implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants