Hang in System.Buffer Memmove

[clairernovotny]

When trying to convert a particular Windows PDB to portable, the method hangs. Stack trace and repro files attached.
AWSSDK.2.3.55.2.nupkg.zip [rename to remove the .zip, it's a nupkg file].

It's using lib\net35\AWSSDK.dll as the PE stream and lib\net35\AWSSDK.pdb as the PDB.

It's using
.NET Core SDK (reflecting any global.json):
Version: 5.0.100-preview.3.20216.6
Commit: 9f62a32109

 	[Managed to Native Transition]	
 	System.Private.CoreLib.dll!System.Buffer._Memmove(ref byte dest, ref byte src, ulong len)	Unknown
 	System.Private.CoreLib.dll!System.Buffer.Memmove(ref byte dest, ref byte src, ulong len)	Unknown
 	System.Private.CoreLib.dll!System.Runtime.InteropServices.Marshal.CopyToNative<byte>(byte[] source, int startIndex, System.IntPtr destination, int length)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.Internal.StreamExtensions.CopyTo(System.IO.Stream source, byte* destination, int size)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.Internal.StreamMemoryBlockProvider.ReadMemoryBlockNoLock(System.IO.Stream stream, bool isFileStream, long start, int size)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.Internal.StreamMemoryBlockProvider.GetMemoryBlockImpl(int start, int size)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.PortableExecutable.PEReader.GetPESectionBlock(int index)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.PortableExecutable.PEReader.GetSectionData(int relativeVirtualAddress)	Unknown
 	System.Reflection.Metadata.dll!System.Reflection.Metadata.PEReaderExtensions.GetMethodBody(System.Reflection.PortableExecutable.PEReader peReader, int relativeVirtualAddress)	Unknown
 	Microsoft.DiaSymReader.Converter.dll!Microsoft.DiaSymReader.Tools.PdbConverterWindowsToPortable.Convert(Microsoft.DiaSymReader.ISymUnmanagedReader5 symReader, System.Reflection.PortableExecutable.PEReader peReader, System.IO.Stream targetPdbStream, System.Reflection.Metadata.BlobContentId pdbId)	Unknown
 	Microsoft.DiaSymReader.Converter.dll!Microsoft.DiaSymReader.Tools.PdbConverterWindowsToPortable.Convert(System.Reflection.PortableExecutable.PEReader peReader, System.IO.Stream sourcePdbStream, System.IO.Stream targetPdbStream)	Unknown
 	Microsoft.DiaSymReader.Converter.dll!Microsoft.DiaSymReader.Tools.PdbConverter.ConvertWindowsToPortable(System.Reflection.PortableExecutable.PEReader peReader, System.IO.Stream sourcePdbStream, System.IO.Stream targetPdbStream)	Unknown
 	Microsoft.DiaSymReader.Converter.dll!Microsoft.DiaSymReader.Tools.PdbConverter.ConvertWindowsToPortable(System.IO.Stream peStream, System.IO.Stream sourcePdbStream, System.IO.Stream targetPdbStream)	Unknown
>	Core.dll!NuGetPe.AssemblyMetadata.AssemblyDebugParser.AssemblyDebugParser(System.IO.Stream peStream, System.IO.Stream pdbStream) Line 36	C#

Tagging subscribers to this area: @tannergooding
Notify danmosemsft if you want to be subscribed.

[benaadams]

In the native call __Memmove?

/cc @jkotas

[EgorBo]

Are you sure it's not just some infinite loop that just spends a lot of time in Memmove?

[clairernovotny]

@EgorBo I couldn't tell you that, I just know the call never completes.

Easiest repro (for me) is in opening the attached NuGet package in the latest NuGet Package Explorer. Symbol validation never completes and breaking in the debugger shows the above callstack.

[clairernovotny]

I see 02 on the CPU usage for the process out of 56 logical cores. It appears to be burning a core.

[benaadams]

_Memmove doesn't look like the only call stack, if it is the highest

Infinite loop in Microsoft.DiaSymReader.Tools.PdbConverterWindowsToPortable.Convert ?

[benaadams]

Bit more filtered

[EgorBo]

I also tried it locally and if you are lucky enough, the stacktrace starts from Kernel32.LocalFree instead 🙂

[danmoseley]

This should be moved to https://github.com/dotnet/symreader-portable for investigation

[teo-tsirpanis]

Cannot reproduce with the following program. It takes around half a minute to convert the PDB, but it completes.

using Microsoft.DiaSymReader.Tools;
using System.Diagnostics;

// Does not reproduce regardless of whether we read the file in memory or not.
// var peStream = new MemoryStream(File.ReadAllBytes("AWSSDK.dll"), false);
// var pdbStream = new MemoryStream(File.ReadAllBytes("AWSSDK.pdb"), false);
var peStream = File.OpenRead("AWSSDK.dll");
var pdbStream = File.OpenRead("AWSSDK.pdb");
var portablePdbStream = new MemoryStream();

var pdbConverter = new PdbConverter(x => Console.WriteLine(x));
long timestamp = Stopwatch.GetTimestamp();
pdbConverter.ConvertWindowsToPortable(peStream, pdbStream, portablePdbStream);

Console.WriteLine($"Conversion complete after {Stopwatch.GetElapsedTime(timestamp)}. Wrote {portablePdbStream.Length} bytes.");

Project file

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <RootNamespace>symreader_repro</RootNamespace>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.DiaSymReader.Converter" Version="1.1.0-beta2-25209-02" />
  </ItemGroup>
</Project>

I also opened the attached .nupkg in NuGet Package Explorer (version 6.2.19), and analyzing the PDBs eventually completes as well after some time.

[teo-tsirpanis]

Following some profiling, around 60% of the execution time is spent on this method, split almost equally between method.GetSequencePointsCount and method.GetSequencePoints. Looks like they do duplicate work, but I cannot investigate further and don't know where to open an issue about that.

Maybe this issue can be closed.