📚 When running under Azure DevOps, if you have configured your Pipeline to allow access to the Azure DevOps test reporting API, Fixie will report each test result so that they will appear on the build's "Tests" screen. If you forget that configuration, the console output will provide guidance based on the Azure DevOps documentation (https://docs.microsoft.com/en-us/azure/devops/pipelines/build/variables#systemaccesstoken)
✅ When running tests in Visual Studio under Test Explorer:
- 👌 Improved error reporting, both in the Output pane and in the tree of test results, to help with troubleshooting.
- 👌 Improved performance when selecting a large number of individual tests to run in Test Explorer. Previously, we performed an excessive amount of reflection and redundant evaluation of which test classes and test methods need to be executed.
- ↔ Integrate with Test Explorer's test progress icons, so that the user can identify long-running tests.
- ✅ Graceful handling of catastrophic failures such as tests which perform Environment.Exit(...) or which throw StackOverflowException. Naturally such catastrophic errors force the test run to end early, and that is out of our control, but now we accurately report what happened in both the Test Explorer tree and in the 'Tests' Output pane.
v2.1.0July 24, 2019
🚀 Release 2.0.2 attempted to integrate with breaking changes in Visual Studio 15.8. It failed, however, to properly deal with the following combination:
- ✅ Test assembly targets Full .NET Framework
- 🐎 Enabling the new option Tools \ Options \ Tests \ "For improved performance, only use test adapters in test assembly folder..."
✅ In this scenario, Test Explorer would fail to locate a dependency (Mono.Cecil) causing those tests to disappear from Test Explorer.
🚀 With this release, Tests Explorer is expected to behave well regardless of the new Tools \ Options \ Tests setting.
✅ Handling Stale Test Discovery Information
✅ Because test runners like Test Explorer can request specific tests to run using stale discovery data (such as selecting a test to run which has been renamed but not yet recompiled), we now take care to only execute requested tests which in fact exist.
✅ Test Exception Handling
✅ We simplified the way that test exceptions are captured and propagated while preserving original stack traces.
✅ For async tests, this means favoring task.GetAwaiter().GetResult() over task.Wait(), simplifying our implementation and providing a more readable stack trace.
We leverage ExceptionDispatchInfo.Throw(), though we take care to omit the portion of the resulting stack trace that verbosely describes Fixie's own calling of that method.
🐛 Bugs Fixed
- When formatting output under TeamCity, ensure all non-ASCII characters are properly escaped.
- ✅ When asynchronous F# test cases return the F#-specific
Async<'T>type, ensure that execution has completed before determining the case's result.
- ✅ Ensure that execution of any asynchronous test case completes before determining the case's result, whether or not the
asynckeyword is present, even when the
Taskobject is returned from a method declared with
objectas the return type.
- ✅ Asynchronous test cases which return the
nulltask always pass, as there is no more work to perform.
- ✅ Asynchronous test cases fail with a clear explanation, instead of blocking indefinitely, when the
Taskreturned was never started.
Huge thanks to @petejohanson!
✅ Visual Studio 15.8 included an intentional breaking change to the way Test Explorer discovers third party "test adapters" like Fixie.VisualStudio.TestAdapter.dll.
Historically, Test Explorer would search for .NET Core adapters only in the build output folder, while it would search for .NET Framework adapters in both the build output folder and the referenced NuGet packages. When Fixie 2.0.0 was released, we took advantage of the NuGet package search in order to avoid having to wastefully copy the adapter DLL to the build outpuf folder for .NET Framework builds.
🚀 Visual Studio 15.8 includes a breaking change in the interest of performance. The search no longer considers NuGet package contents when discovering test adapters. This release fixes Test Explorer integration by including the same DLL copy for .NET Framework projects as we've been doing for .NET Core projects.
✅ As a side effect,
dotnet testintegration is likewise improved, though we still recommend using
🛠 Fixie's 2.0 release is a major overhaul of both its implementation and its customization API.
- .NET Core and multitargeting: target net452 or higher, netcoreapp2.0 or higher, or a combination.
- ✅ Run your tests with
- 👌 Improved integration with AppVeyor, TeamCity, TestDriven.NET, and Test Explorer.
- 🐎 Performance enhancements.
- ✅ Test Explorer can now run a mix of 32-bit and 64-bit test assemblies.
- ✅ Reimagined customization API: 1.0's customization API was difficult to explain, difficult to use, motivated bad behavior, and could easily be thwarted when combining its features in unanticipated ways. The new API is simpler to reason about, and gives you greater control over how you want to organize your customization of the test discovery and execution phases.
v2.0.0-rcMay 07, 2018