Skip to content

Document MSTest 4.3 random test order and parallelization MSBuild properties #54480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Evangelink wants to merge 4 commits into
base: main
Choose a base branch
from
+51 −25
Open

Document MSTest 4.3 random test order and parallelization MSBuild properties #54480

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4b01f74
Document MSTest 4.3 random test order settings and parallelization MS…
Evangelink Jun 19, 2026
0ef4f53
Address review: imperative phrasing, code-format Directory.Build.prop…
Evangelink Jun 19, 2026
c22cb9f
Update docs/core/testing/unit-testing-mstest-writing-tests-controllin…
Evangelink Jun 22, 2026
d8d42d4
Use backticks for runsettings setting names to prevent localization
Evangelink Jun 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 26 additions & 1 deletion docs/core/testing/unit-testing-mstest-configure.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ title: Configure MSTest
description: Learn how to configure MSTest.
author: Evangelink
ms.author: amauryleve
ms.date: 04/16/2024
ms.date: 06/19/2026
ai-usage: ai-assisted
---

# Configure MSTest
Expand Down Expand Up @@ -39,6 +40,8 @@ The following runsettings entries let you configure how MSTest behaves.
|**MapNotRunnableToFailed**|true|A value indicating whether a not runnable result is mapped to failed test.|
|**OrderTestsByNameInClass**|false|If you want to run tests by test names both in Test Explorers and on the command line, set this value to **true**.|
|**Parallelize**||Used to set the parallelization settings:<br /><br />**Workers**: The number of threads/workers to be used for parallelization, which is by default **the number of processors on the current machine**.<br /><br />**SCOPE**: The scope of parallelization. You can set it to **MethodLevel**. By default, it's **ClassLevel**.<br /><br />`<Parallelize><Workers>32</Workers><Scope>MethodLevel</Scope></Parallelize>`|
|**RandomizeTestOrder**|false|Starting with MSTest 4.3, set this value to **true** to run tests in a random order, which helps surface hidden ordering dependencies between tests. This setting can't be combined with **OrderTestsByNameInClass**.|

@gewarren gewarren Jun 22, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the setting names should be put into backticks to prevent localization. You can see here in German that a few setting names are being localized.

https://learn.microsoft.com/de-de/dotnet/core/testing/unit-testing-mstest-configure?branch=pr-en-us-54480#runsettings

@Evangelink Evangelink Jun 22, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in d8d42d4 — the runsettings MSTest element table now wraps all setting names (and literal values like MethodLevel/ClassLevel/EndOfAssembly) in backticks instead of bold so they're not localized.

|**RandomTestOrderSeed**||Starting with MSTest 4.3, when **RandomizeTestOrder** is **true**, set an integer seed to make the random order reproducible across runs. When unset, a new seed is used for each run.|
|**SettingsFile**||You can specify a test settings file to use with the MSTest adapter here. You can also specify a test settings file [from the settings menu](/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file#specify-a-run-settings-file-in-the-ide).<br /><br />If you specify this value, you must also set the **ForcedLegacyMode** to **true**.<br /><br />`<ForcedLegacyMode>true</ForcedLegacyMode>`|
|**TestCleanupTimeout**|0|Specify globally the timeout to apply on each instance of test cleanup method. `[Timeout]` attribute specified on the test cleanup method overrides the global timeout.|
|**TestInitializeTimeout**|0|Specify globally the timeout to apply on each instance of test initialize method. `[Timeout]` attribute specified on the test initialize method overrides the global timeout.|
Expand Down Expand Up @@ -215,6 +218,8 @@ All the settings in this section belong to the `execution` element.
| considerFixturesAsSpecialTests | false | To display `AssemblyInitialize`, `AssemblyCleanup`, `ClassInitialize`, `ClassCleanup` as individual entries in Visual Studio and Visual Studio Code `Test Explorer` and _.trx_ log, set this value to **true**. |
| mapInconclusiveToFailed | false | If a test completes with an inconclusive status, it's mapped to the skipped status in **Test Explorer**. If you want inconclusive tests to be shown as failed, set the value to **true**. |
| mapNotRunnableToFailed | true | A value indicating whether a not runnable result is mapped to failed test. |
| randomizeTestOrder | false | Starting with MSTest 4.3, set this value to `true` to run tests in a random order, which helps surface hidden ordering dependencies between tests. This setting can't be combined with `orderTestsByNameInClass`. |
| randomTestOrderSeed | | Starting with MSTest 4.3, when `randomizeTestOrder` is `true`, set an integer seed to make the random order reproducible across runs. When unset, a new seed is used for each run. |
| treatClassAndAssemblyCleanupWarningsAsErrors | false | To see your failures in class cleanups as errors, set this value to **true**. |
| treatDiscoveryWarningsAsErrors | false | To report test discovery warnings as errors, set this value to **true**. |

Expand Down Expand Up @@ -299,3 +304,23 @@ Each element of the file is optional because it has a default value.
}
}
```

## MSBuild properties

Starting with MSTest 4.3, opt in to assembly-level parallelization from your project file or `Directory.Build.props` without authoring an `[assembly: Parallelize]` attribute. These properties emit the corresponding assembly attribute during build, so they require `GenerateAssemblyInfo` to be `true` (the default for SDK-style projects).

| Property | Default | Description |
|----------|---------|-------------|
| `MSTestParallelizeScope` | | The parallelization scope. Set it to `MethodLevel` or `ClassLevel` to emit `[assembly: Parallelize(Scope = ExecutionScope.MethodLevel)]` (or `ExecutionScope.ClassLevel`), or to `None` to emit `[assembly: DoNotParallelize]`. |
| `MSTestParallelizeWorkers` | | The maximum number of worker threads, emitted as the `Workers` value of `[assembly: Parallelize]`. A value of `0` maps to the number of processors on the current machine. This property can't be set when `MSTestParallelizeScope` is `None`. |

The following example enables method-level parallelization with four workers for every test project that imports the `Directory.Build.props` file:

```xml
<Project>
<PropertyGroup>
<MSTestParallelizeScope>MethodLevel</MSTestParallelizeScope>
<MSTestParallelizeWorkers>4</MSTestParallelizeWorkers>
Comment thread
Evangelink marked this conversation as resolved.
</PropertyGroup>
</Project>
```
3 changes: 2 additions & 1 deletion docs/core/testing/unit-testing-mstest-writing-tests-controlling-execution.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ description: Learn how to control test execution in MSTest with parallelization,
author: Evangelink
ms.author: amauryleve
ms.date: 07/15/2025
ai-usage: ai-assisted
---

# Test execution and control in MSTest
Expand Down Expand Up @@ -142,7 +143,7 @@ The `Workers` property specifies the maximum number of threads for parallel exec
```

> [!TIP]
> You can also configure parallelization through [runsettings](unit-testing-mstest-configure.md#mstest-element) or [testconfig.json](unit-testing-mstest-configure.md#testconfigjson) without modifying code.
> Configure parallelization without modifying code through [runsettings](unit-testing-mstest-configure.md#mstest-element), [testconfig.json](unit-testing-mstest-configure.md#testconfigjson), or the [`MSTestParallelizeScope` and `MSTestParallelizeWorkers` MSBuild properties](unit-testing-mstest-configure.md#msbuild-properties).
Comment thread
Evangelink marked this conversation as resolved.
Outdated

> [!TIP]
> Enable parallelization at the assembly level by default, even if many tests currently require sequential execution. This approach encourages writing new tests that support parallel execution from the start. Use the [MSTEST0001](mstest-analyzers/mstest0001.md) analyzer to ensure that the assembly explicitly declares its parallelization intent with `[assembly: Parallelize]` or `[assembly: DoNotParallelize]`. Once parallelization is enabled, review each test class to determine whether it safely supports concurrent execution. Often, excluding just a few classes or methods with `DoNotParallelize` is sufficient, allowing the majority of your tests to run in parallel for significantly faster test execution.
Expand Down
Loading