diff --git a/docs/migrations/breaking-changes-in-v5.mdx b/docs/migrations/breaking-changes-in-v5.mdx index c27a97d..4f6ce32 100644 --- a/docs/migrations/breaking-changes-in-v5.mdx +++ b/docs/migrations/breaking-changes-in-v5.mdx @@ -6,34 +6,36 @@ description: Pester v5 included an all-new runtime which lead to some breaking c ## Actual breaking changes -- The Parameters of `Invoke-Pester` changed significantly, but in 5.0.1, a compatibility parameter set was added. To allow all the v4 parameters to be used, e.g. like this `Invoke-Pester -Script $testFile -PassThru -Verbose -OutputFile $tr -OutputFormat NUnitXml -CodeCoverage "$tmp/*-*.ps1" -CodeCoverageOutputFile $cc -Show All`. The compatibility is not 100%, neither -Script not -CodeCoverage take hashtables, they just take a collection of paths. The `-Strict` parameter and `-PesterOption` are ignored. The `-Output` \ `-Show` parameter takes all the values, but translates only the most used options to Pester 5 compatible options, otherwise it uses `Detailed` output. It also allows all the Pester 5 output options, to allow you to use `Diagnostic` during migration. **This whole Legacy-parameter set is deprecated** and prints a warning when used. For more options and the Advanced interface see [simple and advanced interface](./v4-to-v5#simple-and-advanced-interface) above on how to invoke Pester. +- The Parameters of `Invoke-Pester` changed significantly, but a compatibility parameter set was added to allow all the v4 parameters to be used, e.g. like this `Invoke-Pester -Script $testFile -PassThru -Verbose -OutputFile $tr -OutputFormat NUnitXml -CodeCoverage "$tmp/*-*.ps1" -CodeCoverageOutputFile $cc -Show All`. Be aware of these limitations and notes: + - The compatibility is not 100%, neither `-Script` not `-CodeCoverage` take hashtables, they just take a collection of paths. + - The `-Strict` parameter and `-PesterOption` are ignored. + - The `-Output` \ `-Show` parameter takes all the values, but translates only the most used options to Pester 5 compatible options, otherwise it uses `Detailed` output. It also allows all the Pester 5 output options, to allow you to use `Diagnostic` during migration. + - **This whole Legacy-parameter set is deprecated** and prints a warning when used. For more options and the Advanced interface see [simple and advanced interface](./v4-to-v5#simple-and-advanced-interface) above on how to invoke Pester. - PowerShell 2 is no longer supported - Legacy syntax `Should Be` (without `-`) is removed, see [Migrating from Pester v3 to v4](./v3-to-v4) - Mocks are scoped based on their placement, not in whole `Describe` / `Context`. The count also depends on their placement. See [mock scoping](../usage/mocking#mocks-are-scoped-based-on-their-placement) - `Assert-VerifiableMocks` was removed, see [Should -Invoke](../usage/mocking#should--invoke) - All code placed in the body of `Describe` outside of `It`, `BeforeAll`, `BeforeEach`, `AfterAll`, `AfterEach` will run during discovery and it's state might or might not be available to the test code, see [Discovery and Run](../usage/discovery-and-run) - -- `-Output` parameter has reduced options to `None`, `Normal`, `Detailed` and `Diagnostic`, `-Show` alias is removed +- `-Output` parameter has reduced options to `None`, `Normal`, `Detailed` and `Diagnostic`. `-Show` alias is removed - `-PesterOption` switch is removed - `-TestName` switch is replaced with `-FullNameFilter` switch - `-Script` option was renamed to `-Path` and takes paths only, it does not take hashtables. For parametrized scripts, see [Providing external data to tests](../usage/data-driven-tests#providing-external-data-to-tests) - Using `$MyInvocation.MyCommand.Path` to locate your script in `BeforeAll` does not work. This does not break it for your scripts and modules. Use `$PSScriptRoot` or `$PSCommandPath`. See [Migrating from Pester v4](../usage/importing-tested-functions#migrating-from-pester-v4) or the [importing ps files](https://jakubjares.com/2020/04/11/pester5-importing-ps-files/) article for detailed information. - Should `-Throw` is using `-like` to match the exception message instead of .Contains. Use `*` or any of the other `-like` wildcard to match only part of the message. -- Variables defined during Discovery, are not available in Before*, After* and It. When generating tests via foreach blocks, make sure you pass all variables into the test using -TestCases / -ForEach. -- Gherkin is removed, and will later move to it's own module, please keep using Pester version 4. -- `TestDrive` is defined during Run only, it cannot be used in -TestCases / -ForEach. +- Variables defined during Discovery, are not available in `BeforeAll/-Each`, `AfterAll/-Each` and `It`. When generating tests via foreach blocks, make sure you pass all variables into the test using `-ForEach`. +- Gherkin is removed, please keep using Pester version 4. +- `TestDrive` is defined during Run only, it cannot be used in `-ForEach`. ### Deprecated features -- `Assert-MockCalled` is deprecated, it is recommended to use [Should -Invoke](../usage/mocking) -- `Assert-VerifiableMock` is deprecated, it is recommended to use [Should -InvokeVerifiable](../usage/mocking) +- `Assert-MockCalled` is deprecated, it is recommended to use [Should -Invoke](../usage/mocking). +- `Assert-VerifiableMock` is deprecated, it is recommended to use [Should -InvokeVerifiable](../usage/mocking). +- Pending state (`Set-ItResult -Pending`) is deprecated and will be removed in a future version. ### Additional issues to be solved future releases - `-Strict` switch is not available -- Inconclusive and Pending states are currently no longer available, `-Pending` and `-Inconclusive` are translated to `-Skip` both on test blocks and when using `Set-ItResult` -- Automatic Code coverage via -CI switch currently disabled as it's slow. Can still be enabled using configuration, but is largely untested. -- Generating tests using foreach during discovery time works mostly, generating them from BeforeAll, to postpone expensive work till it is needed in case the test is filtered out also works, but is hacky. Get in touch if you need it and help me refine it. -- Running on huge codebases is largely untested +- Automatic Code coverage via -CI switch currently disabled as it's slow. Can still be enabled using configuraitonis largely untested. + **Noticed more of them? Share please!** diff --git a/docs/migrations/v4-to-v5.mdx b/docs/migrations/v4-to-v5.mdx index 96c0781..dd0eedc 100644 --- a/docs/migrations/v4-to-v5.mdx +++ b/docs/migrations/v4-to-v5.mdx @@ -138,18 +138,18 @@ The following table shows a mapping of v4 Legacy parameters (those which have no -The following table shows the mapping for v4 *Show* property values to the configuration property *Output.Verbosity*: +The following table shows the mapping for v4 `-Show` property values to the configuration property `Output.Verbosity`: -| *Show* value | Configuration Object *Output Verbosity* Property | -| ------------ | ------------------------------------------------ | -| All | Detailed | -| Default | Detailed | -| Detailed | Detailed | -| Fails | Normal | -| Diagnostic | Diagnostic | -| Normal | Normal | -| Minimal | Minimal | -| None | None | +| `-Show` Value | `Output.Verbosity` Property | +| ------------- | ----------------------------| +| All | Detailed | +| Default | Detailed | +| Detailed | Detailed | +| Fails | Normal | +| Diagnostic | Diagnostic | +| Normal | Normal | +| Minimal | Minimal | +| None | None | ### Implicit parameters for TestCases @@ -170,7 +170,7 @@ This also works for Mock. #### Mocks can be debugged -Mocks don't rewrite the scriptblock you provide anymore. You can now set breakpoints into them as well as any of the ParameterFilter or Should -Invoke Parameter filter. +Mocks don't rewrite the scriptblock you provide anymore. You can now set breakpoints into them as well as any of the `-ParameterFilter` used in `Mock` and `Should -Invoke`. #### Avoid putting in InModuleScope around your Describe and It blocks diff --git a/docs/usage/skip.mdx b/docs/usage/skip.mdx index 4be6a77..2a08571 100644 --- a/docs/usage/skip.mdx +++ b/docs/usage/skip.mdx @@ -54,13 +54,9 @@ Tests completed in 1.03s Tests Passed: 1, Failed: 0, Skipped: 4, Total: 5, NotRun: 0 ``` -:::warning -Pending is translated to skipped, Inconclusive does not exist anymore. -::: - ### Conditional skip -Evaluation of the skip-parameter is done as part of the Discovery-phase. This means you can also dynamically skip tests based on values available in that phase. Either as logic done in BeforeDiscovery or based on values passed into a parametrized test. +Evaluation of the skip-parameter is done as part of the Discovery-phase. This means you can also dynamically skip tests based on values available in that phase. Either as logic done in `BeforeDiscovery` or based on values passed into a parametrized test. ```powershell $sb = { @@ -126,3 +122,7 @@ Describing My Describe Block Tests completed in 206ms Tests Passed: 1, Failed: 0, Skipped: 2 NotRun: 0 ``` + +:::tip Skipping during Run-phase +In certain scenarios you might need to mark a test as skipped or inconclusive during execution. This is possible using [`Set-ItResult`](../commands/Set-ItResult). Be aware that setup and teardown-blocks will be executed, so use `-Skip` whenever possible. +::: diff --git a/versioned_docs/version-v5/migrations/breaking-changes-in-v5.mdx b/versioned_docs/version-v5/migrations/breaking-changes-in-v5.mdx index ebf3d5d..4f6ce32 100644 --- a/versioned_docs/version-v5/migrations/breaking-changes-in-v5.mdx +++ b/versioned_docs/version-v5/migrations/breaking-changes-in-v5.mdx @@ -6,34 +6,36 @@ description: Pester v5 included an all-new runtime which lead to some breaking c ## Actual breaking changes -- (❗ new in 5.0.1) The Parameters of `Invoke-Pester` changed significantly, but in 5.0.1, a compatibility parameter set was added. To allow all the v4 parameters to be used, e.g. like this `Invoke-Pester -Script $testFile -PassThru -Verbose -OutputFile $tr -OutputFormat NUnitXml -CodeCoverage "$tmp/*-*.ps1" -CodeCoverageOutputFile $cc -Show All`. The compatibility is not 100%, neither -Script not -CodeCoverage take hashtables, they just take a collection of paths. The `-Strict` parameter and `-PesterOption` are ignored. The `-Output` \ `-Show` parameter takes all the values, but translates only the most used options to Pester 5 compatible options, otherwise it uses `Detailed` output. It also allows all the Pester 5 output options, to allow you to use `Diagnostic` during migration. **This whole Legacy-parameter set is deprecated** and prints a warning when used. For more options and the Advanced interface see [simple and advanced interface](./v4-to-v5#simple-and-advanced-interface) above on how to invoke Pester. +- The Parameters of `Invoke-Pester` changed significantly, but a compatibility parameter set was added to allow all the v4 parameters to be used, e.g. like this `Invoke-Pester -Script $testFile -PassThru -Verbose -OutputFile $tr -OutputFormat NUnitXml -CodeCoverage "$tmp/*-*.ps1" -CodeCoverageOutputFile $cc -Show All`. Be aware of these limitations and notes: + - The compatibility is not 100%, neither `-Script` not `-CodeCoverage` take hashtables, they just take a collection of paths. + - The `-Strict` parameter and `-PesterOption` are ignored. + - The `-Output` \ `-Show` parameter takes all the values, but translates only the most used options to Pester 5 compatible options, otherwise it uses `Detailed` output. It also allows all the Pester 5 output options, to allow you to use `Diagnostic` during migration. + - **This whole Legacy-parameter set is deprecated** and prints a warning when used. For more options and the Advanced interface see [simple and advanced interface](./v4-to-v5#simple-and-advanced-interface) above on how to invoke Pester. - PowerShell 2 is no longer supported - Legacy syntax `Should Be` (without `-`) is removed, see [Migrating from Pester v3 to v4](./v3-to-v4) - Mocks are scoped based on their placement, not in whole `Describe` / `Context`. The count also depends on their placement. See [mock scoping](../usage/mocking#mocks-are-scoped-based-on-their-placement) - `Assert-VerifiableMocks` was removed, see [Should -Invoke](../usage/mocking#should--invoke) - All code placed in the body of `Describe` outside of `It`, `BeforeAll`, `BeforeEach`, `AfterAll`, `AfterEach` will run during discovery and it's state might or might not be available to the test code, see [Discovery and Run](../usage/discovery-and-run) - -- `-Output` parameter has reduced options to `None`, `Normal`, `Detailed` and `Diagnostic`, `-Show` alias is removed +- `-Output` parameter has reduced options to `None`, `Normal`, `Detailed` and `Diagnostic`. `-Show` alias is removed - `-PesterOption` switch is removed - `-TestName` switch is replaced with `-FullNameFilter` switch - `-Script` option was renamed to `-Path` and takes paths only, it does not take hashtables. For parametrized scripts, see [Providing external data to tests](../usage/data-driven-tests#providing-external-data-to-tests) - Using `$MyInvocation.MyCommand.Path` to locate your script in `BeforeAll` does not work. This does not break it for your scripts and modules. Use `$PSScriptRoot` or `$PSCommandPath`. See [Migrating from Pester v4](../usage/importing-tested-functions#migrating-from-pester-v4) or the [importing ps files](https://jakubjares.com/2020/04/11/pester5-importing-ps-files/) article for detailed information. - Should `-Throw` is using `-like` to match the exception message instead of .Contains. Use `*` or any of the other `-like` wildcard to match only part of the message. -- Variables defined during Discovery, are not available in Before*, After* and It. When generating tests via foreach blocks, make sure you pass all variables into the test using -TestCases / -ForEach. -- Gherkin is removed, and will later move to it's own module, please keep using Pester version 4. -- `TestDrive` is defined during Run only, it cannot be used in -TestCases / -ForEach. +- Variables defined during Discovery, are not available in `BeforeAll/-Each`, `AfterAll/-Each` and `It`. When generating tests via foreach blocks, make sure you pass all variables into the test using `-ForEach`. +- Gherkin is removed, please keep using Pester version 4. +- `TestDrive` is defined during Run only, it cannot be used in `-ForEach`. ### Deprecated features -- `Assert-MockCalled` is deprecated, it is recommended to use [Should -Invoke](../usage/mocking) -- `Assert-VerifiableMock` is deprecated, it is recommended to use [Should -InvokeVerifiable](../usage/mocking) +- `Assert-MockCalled` is deprecated, it is recommended to use [Should -Invoke](../usage/mocking). +- `Assert-VerifiableMock` is deprecated, it is recommended to use [Should -InvokeVerifiable](../usage/mocking). +- Pending state (`Set-ItResult -Pending`) is deprecated and will be removed in a future version. ### Additional issues to be solved future releases - `-Strict` switch is not available -- Inconclusive and Pending states are currently no longer available, `-Pending` and `-Inconclusive` are translated to `-Skip` both on test blocks and when using `Set-ItResult` -- Automatic Code coverage via -CI switch currently disabled as it's slow. Can still be enabled using configuration, but is largely untested. -- Generating tests using foreach during discovery time works mostly, generating them from BeforeAll, to postpone expensive work till it is needed in case the test is filtered out also works, but is hacky. Get in touch if you need it and help me refine it. -- Running on huge codebases is largely untested +- Automatic Code coverage via -CI switch currently disabled as it's slow. Can still be enabled using configuraitonis largely untested. + **Noticed more of them? Share please!** diff --git a/versioned_docs/version-v5/migrations/v4-to-v5.mdx b/versioned_docs/version-v5/migrations/v4-to-v5.mdx index 96c0781..dd0eedc 100644 --- a/versioned_docs/version-v5/migrations/v4-to-v5.mdx +++ b/versioned_docs/version-v5/migrations/v4-to-v5.mdx @@ -138,18 +138,18 @@ The following table shows a mapping of v4 Legacy parameters (those which have no -The following table shows the mapping for v4 *Show* property values to the configuration property *Output.Verbosity*: +The following table shows the mapping for v4 `-Show` property values to the configuration property `Output.Verbosity`: -| *Show* value | Configuration Object *Output Verbosity* Property | -| ------------ | ------------------------------------------------ | -| All | Detailed | -| Default | Detailed | -| Detailed | Detailed | -| Fails | Normal | -| Diagnostic | Diagnostic | -| Normal | Normal | -| Minimal | Minimal | -| None | None | +| `-Show` Value | `Output.Verbosity` Property | +| ------------- | ----------------------------| +| All | Detailed | +| Default | Detailed | +| Detailed | Detailed | +| Fails | Normal | +| Diagnostic | Diagnostic | +| Normal | Normal | +| Minimal | Minimal | +| None | None | ### Implicit parameters for TestCases @@ -170,7 +170,7 @@ This also works for Mock. #### Mocks can be debugged -Mocks don't rewrite the scriptblock you provide anymore. You can now set breakpoints into them as well as any of the ParameterFilter or Should -Invoke Parameter filter. +Mocks don't rewrite the scriptblock you provide anymore. You can now set breakpoints into them as well as any of the `-ParameterFilter` used in `Mock` and `Should -Invoke`. #### Avoid putting in InModuleScope around your Describe and It blocks diff --git a/versioned_docs/version-v5/usage/output.mdx b/versioned_docs/version-v5/usage/output.mdx index 72b36be..ca3e45a 100644 --- a/versioned_docs/version-v5/usage/output.mdx +++ b/versioned_docs/version-v5/usage/output.mdx @@ -31,7 +31,7 @@ This option controls the general level of console output. Pester v5 supports the ### StackTraceVerbosity -**New in Pester 5.3!** +**New in Pester 5.3.0!**
When an error is shown due to a failed setup or test, a stacktrace is included to help you identify where the problem occured. Users might not care much about this while developers need it to identify and fix any issues. This option lets you control how much of the stacktrace that will be printed using one of the following levels: @@ -50,7 +50,7 @@ The result-object returned from `Run.PassThru / -PassThru` includes a `ErrorReco ### CIFormat -**New in Pester 5.3!** +**New in Pester 5.3.0!**
Pester supports CI-specific output syntaxes to highlight, log and navigate to errors in your tests for a few popular CI systems. This options lets your control how this behavior should work. The following levels are currently supported: - **None**: Disable all CI-specific syntax. Error messages are presented in the format you're familiar with. @@ -78,7 +78,7 @@ Auto-detection works by checking if `GITHUB_ACTIONS` environment variable is equ ### RenderMode -**New in Pester 5.4!** +**New in Pester 5.4.0!**
Pester supports multiple render modes for console output, including ANSI escape sequences which enables colors in CI-logs. The currently supported modes are: - **Auto (default)**: Automatically enables the recommended mode using the following rules: diff --git a/versioned_docs/version-v5/usage/skip.mdx b/versioned_docs/version-v5/usage/skip.mdx index 4be6a77..2a08571 100644 --- a/versioned_docs/version-v5/usage/skip.mdx +++ b/versioned_docs/version-v5/usage/skip.mdx @@ -54,13 +54,9 @@ Tests completed in 1.03s Tests Passed: 1, Failed: 0, Skipped: 4, Total: 5, NotRun: 0 ``` -:::warning -Pending is translated to skipped, Inconclusive does not exist anymore. -::: - ### Conditional skip -Evaluation of the skip-parameter is done as part of the Discovery-phase. This means you can also dynamically skip tests based on values available in that phase. Either as logic done in BeforeDiscovery or based on values passed into a parametrized test. +Evaluation of the skip-parameter is done as part of the Discovery-phase. This means you can also dynamically skip tests based on values available in that phase. Either as logic done in `BeforeDiscovery` or based on values passed into a parametrized test. ```powershell $sb = { @@ -126,3 +122,7 @@ Describing My Describe Block Tests completed in 206ms Tests Passed: 1, Failed: 0, Skipped: 2 NotRun: 0 ``` + +:::tip Skipping during Run-phase +In certain scenarios you might need to mark a test as skipped or inconclusive during execution. This is possible using [`Set-ItResult`](../commands/Set-ItResult). Be aware that setup and teardown-blocks will be executed, so use `-Skip` whenever possible. +::: diff --git a/versioned_docs/version-v5/usage/test-results.mdx b/versioned_docs/version-v5/usage/test-results.mdx index 4e6a1f3..b3e3ca6 100644 --- a/versioned_docs/version-v5/usage/test-results.mdx +++ b/versioned_docs/version-v5/usage/test-results.mdx @@ -55,8 +55,8 @@ Code coverage metrics are not included in the NUnit xml report so it is necessar ```powershell $testResults = Invoke-Pester -OutputFile Test.xml -OutputFormat NUnitXml -CodeCoverage (Get-ChildItem -Path $PSScriptRoot\*.ps1 -Exclude *.Tests.* ).FullName -PassThru -Write-Output "##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='$($testResults.CodeCoverage.NumberOfCommandsAnalyzed)']" -Write-Output "##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='$($testResults.CodeCoverage.NumberOfCommandsExecuted)']" +Write-Output "##teamcity[buildStatisticValue key='CodeCoverageAbsLTotal' value='$($testResults.CodeCoverage.CommandsAnalyzedCount)']" +Write-Output "##teamcity[buildStatisticValue key='CodeCoverageAbsLCovered' value='$($testResults.CodeCoverage.CommandsExecutedCount)']" ``` ## [AppVeyor](https://appveyor.com)