Coverage is not recorded on some .net framework projects

[OldWarrior3000]

Please provide the following information when submitting an issue.

Where appropriate replace the [] with a [X]

NOTE Support requests should initially be raised on GitHub discussions

My Framework

• .NET 2
• .NET 3.5
• .NET 4
• .NET 4.5
• .NET 4.6
• .NET 4.6.1
• .NET 4.6.2
• .NET 4.7
• .NET 4.7.1
• .NET 4.7.2
• .NET 4.8
• .NET Core 2.2.0
• .NET Core 3.1.0
• .NET 5
• only release and LTS versions of .NET runtimes and SDKs will be supported

My Environment

• Windows 7 or below (not truly supported due to EOL)
• Windows 8
• Windows 8.1
• Windows 10
• Windows 10 IoT Core
• Windows Server 2012
• Windows Server 2012 R2
• Windows Server 2016
• Windows Server 2019

I have already...

• repeated the problem using the latest stable release of OpenCover.
• reviewed the usage guide and usage document.
• have looked at the opencover output xml file in an attempt to resolve the issue.
• reviewed the current issues to check that the issue isn't already known.

My issue is related to (check only those which apply):

• no coverage being recorded
• 32 or 64 bit support

Describe the bug

I am working on a .net framework 4.6 solution and wanted to introduce OpenCover in order to get coverage into the Sonar.
The issue I am experiencing is that for some projects no coverage is detected but for some is. (Though it is just partial not complete)

What I tried to far:

• Upgrading to the latest version of OpenCover
• Changing between vstest and mstest (vstest shows better results. Mstest doesn't return any)
• Experimenting with -targetdir and -searchDir
• Changing to dotnet vstest instead of a direct vstest call
• Creating a new independent solution based on .net framework (for which it works)
• Adding log:all parameter to get more output
• Checking the xml coverage report and the unit test DLL's and referenced project DLL's are not included.
• Different combinations of -register with user, path32 and path64 or without
• Adding MSTest.TestFramework to the test project
• Checked full pdb support
• Numerous recompilations with Debug and Release

With all the above mentioned combinations I get more or less the same result. Strangely it works for another small test project with I created on the side based on .net framework 4.6. I tried with several of the test projects we have and some produce a minor coverage but mostly they produce none. Here an example

Test Run Failed.
Total tests: 709
     Passed: 682
     Failed: 25
    Skipped: 2
 Total time: 15,2156 Minutes
Committing...
Visited Classes 4 of 3583 (0.11)
Visited Methods 7 of 25690 (0.03)
Visited Points 28 of 235467 (0.01)
Visited Branches 11 of 39005 (0.03)

==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 4 of 5438 (0.07)
Alternative Visited Methods 7 of 33052 (0.02)

Steps to reproduce the problem:

I am using the following command to run opencover.

#$OpenCover="C:\TEMP\opencover.4.7.1138\OpenCover.Console.exe"
#$VSTest="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
 & $OpenCover -target:$VSTest -targetargs:"$(Get-Location)\Testing\UnitTestCore\bin\UnitTestCore.dll /logger:trx" -output:"$(Get-Location)\CoverageResults.xml" -register:user -log:All -searchDirs:"$(Get-Location)\Testing\UnitTestCore\bin" -skipautoprops -mergeoutput -targetdir:"$(Get-Location)\Testing\UnitTestCore"

I am not sure what else I can check. All help is greatly appreciated.
Thanks!

• reviewed the usage guide and usage document again because we know you probably didn't do it the first time.
• sample code or link to repository/gist is available (support is much, much faster this way)

[sawilde]

When using vstest (I never use it in the real world as it has always been a pain in the butt) I often have to use -mergebyhash as the same assembly is often loaded by vstest multiple times from several locations. -mergebyoutput only works for merging multiple test runs into one. Occasionally, I find using /noisolation in the -targetargs can also help.

[OldWarrior3000]

Hey Shaun,

thank you for answering. Unfortunately the suggestions didn't help ...
I am really not sure why it is not working as I mentioned above it works perfectly fine with a demo project. But something must be off in the project configuration?

Do you may have any other idea?

Thanks

[sawilde]

Do the tests use Fakes perhaps? OpenCover and Fakes don't play well as Fakes also uses a profiler and they get in each others way.

[OldWarrior3000]

Hey Shaun,

no we are not using Fakes either. It is very strange. I am out of ideas. If you have any other suggestion. I would be grateful. Would it help if I send over a part of the coverage report generated?

Thanks

[sawilde]

it may do

[OldWarrior3000]

Great, here the link to the file. https://file.re/2021/01/28/coverageresults/
I tried to dig into it myself but could not see anything standing out.

[sawilde]

@OldWarrior3000 sorry nothing is shouting out as an issue
Could you possibly download DebugView and have that running when you run the tests - the profiler will dump additional output.

Alternatively - if you are able to share the assembly (we can use a private repo if you wish) or a cut down version that replicates the issue and I can attempt to debug it.

[OldWarrior3000]

Hey Shaun,
sorry for the delay in my response. Unfortunately I cannot share the code out of IP reasons. I will give it a bash with DebugView. If I find anything, I will let you know. Thanks for your help!

[sawilde]

@OldWarrior3000 did you manage to diagnose the issue?

[OldWarrior3000]

Hey @sawilde, the project got de-prioritised for now, so I didn't have time to look into it. I will check as soon as I can. If it suits you, you can close the issue and I will open a new issue once I have some feedback referring to this one.

[Marat-Gumerov]

Hello!
We have same issue on our project. My solution was in splitting coverage to pieces.
For example, my project is too big and coverage is not collected. I have some namespaces in it:

• MyProject.Services
• MyProject.Controllers
• MyProject.Utils
  And I have tests with categories:
• Services - a category for testing classes from MyProject.Services
• Controllers - a category for testing classes from MyProject.Controllers
• Utils....
  Now I can run OpenCover 3 times for every single test category. I also have to set a namespace filter to OpenCover. OpenCover reports should be saved to 3 different files. If every single namespace for splitting is not too big, I will get coverage data correctly.
  Most of coverage report viewers can merge all report files on the fly (for example SonarQube and ReportGenerator).