Skip to content

Commit

Permalink
Update parallel-execution.md with clarifications on Feature behavior …
Browse files Browse the repository at this point in the history
…with method-level parallelism (#364)

* Update parallel-execution.md with clarifications on Feature behavior with method-level parallelism

* Update parallel execution documentation for clarity
  • Loading branch information
DrEsteban authored Jan 10, 2025
1 parent 7de6b84 commit 1ab7ee0
Showing 1 changed file with 5 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/execution/parallel-execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,11 @@ When using Reqnroll we can consider the parallel scheduling on the level of scen
### Execution Behavior

* `[BeforeTestRun]` and `[AfterTestRun]` hooks (events) are executed only once on the first thread that initializes the framework. Executing tests in the other threads is blocked until the hooks have been fully executed on the first thread.
* As a general guideline, **we do not suggest using `[BeforeFeature]` and `[AfterFeature]` hooks and the `FeatureContext` when running the tests parallel**, because in that case it is not guaranteed that these hooks will be executed only once and there will be only one instance of `FeatureContext` per feature. The lifetime of the `FeatureContext` (that starts and finishes by invoking the `[BeforeFeature]` and `[AfterFeature]` hooks) is the consecutive execution of scenarios of a feature on the same parallel execution worker thread. In case of running the scenarios parallel, the scenarios of a feature might be distributed to multiple workers and therefore might have their own dedicated `FeatureContext`. Because of this behavior the `FeatureContext` is never shared between parallel threads so it does not have to be handled in a thread-safe way. If you wish to have a singleton `FeatureContext` and `[BeforeFeature]` and `[AfterFeature]` hook execution, scenarios in a feature must be executed on the **same thread**.
* As a general guideline, **we do not recommend using the `[BeforeFeature]` and `[AfterFeature]` hooks and the `FeatureContext` when running the tests with method-level parallelism**, because in this case there is no guarantee that these hooks will be executed only once per feature and that there will be only one instance of the `FeatureContext` per feature. The lifetime of the `FeatureContext` (that starts and finishes by invoking the `[BeforeFeature]` and `[AfterFeature]` hooks) is controlled by the test runner. So in the case of running scenarios with method-level parallelism, a feature's scenarios could be distributed across multiple workers and run in parallel. Therefore, each scenario could have its own dedicated `FeatureContext`, or some scenarios of a feature could share the same `FeatureContext`. It all depends on how the test runner (e.g. NUnit or MsTest) distributes the scenarios among the worker threads - which is not predictable or controllable. Because of this behavior of the test runner, Reqnroll can't share the `FeatureContext` between parallel threads. If you want to have a truly singleton `FeatureContext`, and `[BeforeFeature]` and `[AfterFeature]` hook execution, you must use either class-level parallelism or disable parallelism entirely so that scenarios of a feature are all executed on the **same worker thread**.
* However, if you still want to use method-level parallelism and a `FeatureContext` in your test suite, then **the following things will be true**:
* The `FeatureContext` and feature-level DI container will remain consistent **per feature, per test thread**. This means that anything you register in the feature container will be resolvable in the `[AfterFeature]` **per test thread**.
* A given `[BeforeFeature]` or `[AfterFeature]` will only be executed once **per test thread** that runs a scenario of a feature.
* Types you register in the feature-level DI container that implement `IDisposable` will still be disposed **per feature, per test thread**. (Keep this in mind if you try to work around this parallelism behavior to regain singleton-like behavior. E.g. by using static instances, `Lazy<>`, thread-safe collections, etc.)
* Scenarios and their related hooks (Before/After scenario, scenario block, step) are isolated in the different threads during execution and do not block each other. Each thread has a separate (and isolated) `ScenarioContext`.
* The test trace listener (that outputs the scenario execution trace to the console by default) is invoked asynchronously from the multiple threads and the trace messages are queued and passed to the listener in serialized form. If the test trace listener implements `Reqnroll.Tracing.IThreadSafeTraceListener`, the messages are sent directly from the threads.

Expand Down

0 comments on commit 1ab7ee0

Please sign in to comment.