Merge pull request #800 from nunit/runningtests

updating runners a1eeca7

articles/nunit/getting-started/dotnet-core-and-dotnet-standard.html

@@ -87,18 +87,28 @@ <h1 id="net-core-and-net-standard">.NET Core and .NET Standard</h1>
 
 <p>More information and getting started tutorials are available for NUnit and .NET Core targeting <a href="https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-with-nunit">C#</a>, <a href="https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-fsharp-with-nunit">F#</a> and <a href="https://docs.microsoft.com/en-us/dotnet/core/testing/unit-testing-visual-basic-with-nunit">Visual Basic</a> in the .NET Core documentation's <a href="https://docs.microsoft.com/en-us/dotnet/core/testing/">Unit Testing in .NET Core and .NET Standard</a> page.</p>
 <p>The other information on this page is older documentation. If you follow the instructions in the <a class="xref" href="installation.html">Installation section</a> your project will work with .NET Core and .NET Standard.</p>
-<p>The test projects have to be .NET Core or .NET Framework; .NET Standard can't be used as a test project, since it can't be run on its own, but any code in a .NET Standard library can be tested from a .NET Core or .NET Framework test project.</p>
+<p>The test projects have to be .NET (Core) or .NET Framework; .NET Standard can't be used as a test project, since it can't be run on its own, but any code in a .NET Standard library can be tested from a .NET (Core) or .NET Framework test project.</p>
 <h2 id="tldr">TL;DR</h2>
 <p>Adding the adapter and <code>Microsoft.NET.Test.Sdk</code> version <code>17.0.0</code> or greater to your NUnit test projects will also enable the <code>dotnet test</code> command for .NET Core projects.</p>
 <p>Any tests using the new style CSPROJ format, either .NET Core or .NET 4.x, need to add a <code>PackageReference</code> to the NuGet package <code>Microsoft.NET.Test.Sdk</code>. Your test assemblies must also be .NET Core or .NET 4.x, not .NET Standard.</p>
-<pre><code class="lang-xml">&lt;ItemGroup&gt;
-  &lt;PackageReference Include=&quot;Microsoft.NET.Test.Sdk&quot; Version=&quot;17.7.2&quot; /&gt;
-  &lt;PackageReference Include=&quot;NUnit&quot; Version=&quot;3.13.3&quot; /&gt;
-  &lt;PackageReference Include=&quot;NUnit3TestAdapter&quot; Version=&quot;4.5.0&quot; /&gt;
-&lt;/ItemGroup&gt;
+<p>You can create a new NUnit test project using <code>dotnet new nunit</code>. It will create an ItemGroup in the csproj file with the necessary references.</p>
+<pre><code class="lang-xml">  &lt;ItemGroup&gt;
+    &lt;PackageReference Include=&quot;Microsoft.NET.Test.Sdk&quot; Version=&quot;17.6.0&quot; /&gt;
+    &lt;PackageReference Include=&quot;NUnit&quot; Version=&quot;3.13.3&quot; /&gt;
+    &lt;PackageReference Include=&quot;NUnit3TestAdapter&quot; Version=&quot;4.2.1&quot; /&gt;
+    &lt;PackageReference Include=&quot;NUnit.Analyzers&quot; Version=&quot;3.6.1&quot; /&gt;
+    &lt;PackageReference Include=&quot;coverlet.collector&quot; Version=&quot;6.0.0&quot; /&gt;
+  &lt;/ItemGroup&gt;
 </code></pre>
 <p>.NET Core test can be run on the command line with <code>dotnet test</code>, for example,</p>
-<pre><code class="lang-cmd">&gt; dotnet test .\test\NetCore10Tests\NetCore10Tests.csproj
+<p>From the solution root folder</p>
+<pre><code class="lang-cmd">  dotnet test
+</code></pre>
+<p>or from the test project folder</p>
+<pre><code class="lang-cmd">  dotnet test
+</code></pre>
+<p>Or you can specify the csproj file you want to test</p>
+<pre><code class="lang-cmd">  dotnet test .\test\NetCore10Tests\NetCore10Tests.csproj
 </code></pre>
 <p>For a more complete walk-through, please see <a href="https://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/">Testing .NET Core with NUnit in Visual Studio 2017</a></p>
 <h2 id="using-the-nunit-project-templates">Using the NUnit project templates</h2>

articles/nunit/running-tests/Index.html

@@ -85,18 +85,52 @@
             <article class="content wrap" id="_content" data-uid="runningtests">
 <h1 id="running-tests">Running Tests</h1>
 
-<p>NUnit provides three different runners, which may be used to load and
-run your tests.</p>
+<p>There are several ways to run your tests, depending on your needs.
+The most common way is to use one of the common IDEs, such as Visual Studio, Visual Studio Code or Rider, or to use <code>dotnet test</code> from the command line. All of these use different parts of the NUnit ecosystem to run your tests. The Microsoft tools use the NUnit3TestAdapter, whereas Jetbrains Rider use the NUnit.Engine.</p>
+<p>To start using NUnit with these tools, see the <a class="xref" href="../getting-started/installation.html">Installation section</a>.</p>
+<p>Also see the information on the <a href="https://docs.nunit.org/articles/vs-test-adapter/Index.html">NUnit3TestAdapter</a> for more detailed information on how to use it and configure it.</p>
+<p>In addition to these NUnit provides two special runners, the NUnit.Console and the NUnitLite runners, which are described below.</p>
 <ul>
 <li>The <a href="Console-Runner.html">Console Runner</a>, <code>nunit-console.exe</code>, is used for batch execution.</li>
-<li>The Gui Runner, <code>nunit.exe</code>, provides interactive loading and running of tests.</li>
+<li>The <a href="NUnitLite-Runner.html">NUnitLite runner</a>, <code>nunitlite-runner.exe</code>, is a light weight runner originally used internally in the team, but can be used by anyone who wants to have a simple way of running tests from the command line.</li>
 </ul>
-<h2 id="nunit-agent">NUnit Agent</h2>
+<p>Just for fun, to see how the different runners are being used, we can look at the <code>nuget.org</code> download statistics for each of the runners, per Oct 2023:</p>
+<table>
+<thead>
+<tr>
+<th>Runner</th>
+<th>Downloads</th>
+<th>Comment</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td>NUnit3TestAdapter</td>
+<td>211.4 million</td>
+<td>Used by Visual Studio, VS Code and dotnet test</td>
+</tr>
+<tr>
+<td>NUnit.Console</td>
+<td>22.5 million</td>
+<td></td>
+</tr>
+<tr>
+<td>NUnitLite</td>
+<td>4.6 million</td>
+<td></td>
+</tr>
+</tbody>
+</table>
+<h2 id="other-runners">Other runners</h2>
+<p>The <code>VSTest.Console</code> is also a Microsoft runner, but it is less used now as <code>dotnet test</code> has taken over.  It does use the <code>NUnit3TestAdapter</code> as well.</p>
+<p>The <code>Azure Pipelines</code> have some tasks for running tests, like <a href="https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/vstest-v2?view=azure-pipelines">VSTest</a>, which also use the <code>NUnit3TestAdapter</code> under the hood.</p>
+<h2 id="some-information-on-the-internal-working">Some information on the internal working</h2>
+<h3 id="nunitengine">NUnit.Engine</h3>
+<p>All runners except NUnitLite use the NUnit.Engine, including Rider.
+It should not be used alone, as the different runners have different requirements for how the engine is used and which version is being used.
+See <a href="https://docs.nunit.org/articles/vs-test-adapter/Adapter-Engine-Compatibility.html">Compatibility of the Test Adapter with the Test Engine</a> for some more detailed information on this.</p>
+<h3 id="nunit-agent">NUnit Agent</h3>
 <p>When running tests in a separate process, the console and gui runners make use of the NUnit Agent program, nunit-agent.exe. Although not directly run by users, nunit-agent does load and execute tests and users need to be aware of it, especially when debugging is involved.</p>
-<h2 id="third-party-runners">Third-Party Runners</h2>
-<p>Various third-party applications are available for loading and running NUnit tests. Some of these actually use NUnit to load the tests, while others provide their own emulation and may not work in the same way that NUnit does.</p>
-<p>Because the status of such projects may change from time to time, we don't discuss them individually here. For the latest information, consult the manufacturer of any third-party software or ask other users on our
-<a href="https://groups.google.com/group/nunit-discuss">discussion list</a>.</p>
 <h2 id="additional-information">Additional Information</h2>
 <p>For additional general information on how tests are loaded and run, see</p>
 <ul>

articles/nunit/running-tests/Template-Based-Test-Naming.html

@@ -85,7 +85,10 @@
             <article class="content wrap" id="_content" data-uid="templatebasedtestnaming">
 <h1 id="template-based-test-naming">Template-Based Test Naming</h1>
 
-<p>NUnit uses a standard naming convention for all tests - however, this can be overridden by the user if required. TestName generation is driven by a name formatting string, which may contain any of the following format specifiers:</p>
+<p>NUnit uses a standard naming convention for all tests, which in the language described below corresponds to <code>{m}{a}</code>. The most used runners, Visual Studio and dotnet depends on this being the default, and trying to change this will cause display issues in these runners.</p>
+<p><strong>We strongly recommend you to NOT change the test naming.</strong></p>
+<h2 id="if-you-still-want-to-do-it">If you still want to do it</h2>
+<p>However, if you're out on your own, the naming <em>can</em> be overridden by the user if required. TestName generation is driven by a name formatting string, which may contain any of the following format specifiers:</p>
 <ul>
 <li><p><code>{n}</code> The namespace of the test or empty if there is no namespace. If empty, any immediately following '.' is ignored.</p>
 </li>

articles/nunit/running-tests/Test-Selection-Language.html

@@ -86,10 +86,12 @@
 <h1 id="test-selection-language">Test Selection Language</h1>
 
 <p>The console command-line allows you to specify a filter, which will select which tests are executed. This is done using the --where option, followed by an expression in NUnit's Test Selection Language (TSL), a simple domain-specific language designed for this purpose.</p>
+<p>The --where option can be used with the <code>dotnet test</code> as <code>-- NUnit.Where</code> (note the space after the --). This will inject this into the runsettings. You can also add the same to a <code>.runsettings</code> file, see <a href="https://docs.nunit.org/articles/vs-test-adapter/Tips-And-Tricks.html">Adapter options</a> and <a href="https://blog.prokrams.com/2019/12/16/nunit3-filter-dotnet/">this blog post</a>.</p>
 <p>Some of the characters used in the expression, such as space, | or &amp;, may have a special meaning when entered on the command-line. In such a case, you should place the expression in quotation marks.</p>
-<pre><code class="lang-cmd">  nunit3-console mytest.dll --where &quot;cat == Urgent || Priority == High&quot;
+<pre><code class="lang-cmd">  dotnet test -- NUnit.Where=&quot;cat == Urgent or Priority == High&quot;
+  nunit3-console mytest.dll --where &quot;cat == Urgent || Priority == High&quot;
+
 </code></pre>
-<p>Note that TSL is handled by the NUnit engine but requires framework support to actually select the tests. The NUnit 3.0 framework supports it fully. See below for support limitations in NUnit V2 tests.</p>
 <h2 id="simple-expressions">Simple Expressions</h2>
 <p>Simple Expressions are essentially comparisons, consisting of a key word or property name on the left-hand side, an operator and some constant value on the right-hand side. Here are some examples:</p>
 <pre><code class="lang-text">  cat == Data

articles/vs-test-adapter/Adapter-Installation.html

@@ -116,7 +116,7 @@ <h3 id="manually-adding-the-adapter-as-a-package-reference-to-your-test-projects
     &lt;PackageReference Include=&quot;NUnit.Analyzers&quot; Version=&quot;3.5.0&quot; /&gt;
   &lt;/ItemGroup&gt;
 </code></pre>
-<p>Note: <em>You don't need to add any nunit.console or any other runner package</em></p>
+<p>Note: <em>You don't need to, nor should you, add any nunit.console or any other runner package</em></p>
 <h3 id="working-with-the-visual-studio-nuget-manager">Working with the Visual Studio Nuget manager</h3>
 <p>If you have a legacy project type, or prefer working outside of the command line, you can also use the Visual Studio 'Manage NuGet packages in the solution' menu option. It can also be used for the new SDK projects, but you may find it faster to open the <code>csproj</code> and copy/paste the references in.</p>
 <p>With an active solution in Visual Studio, follow these steps:</p>