Class TestRunnerApi

The TestRunnerApi retrieves and runs tests programmatically from code inside the project, or inside other packages. TestRunnerApi is a ScriptableObject. You can initialize the API like this:

var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();

You can subscribe and receive test results in one instance of the API, even if the run starts from another instance. The TestRunnerApi supports the following workflows:

• Running tests programmatically
• Gettting test results
• Retrieving the list of tests

Note: Non-static methods in this class will become obsolete in future versions so you should use the static methods (e.g. RegisterTestCallback, ExecuteTestRun) rather than their non-static equivalents (e.g. RegisterCallbacks, Execute).

Inheritance

Object

Object

ScriptableObject

TestRunnerApi

Inherited Members

ScriptableObject.SetDirty()

ScriptableObject.CreateInstance(String)

ScriptableObject.CreateInstance(Type)

ScriptableObject.CreateInstance<T>()

Object.GetInstanceID()

Object.GetHashCode()

Object.Equals(Object)

Object.Instantiate(Object, Vector3, Quaternion)

Object.Instantiate(Object, Vector3, Quaternion, Transform)

Object.Instantiate(Object)

Object.Instantiate(Object, Transform)

Object.Instantiate(Object, Transform, Boolean)

Object.Instantiate<T>(T)

Object.Instantiate<T>(T, Vector3, Quaternion)

Object.Instantiate<T>(T, Vector3, Quaternion, Transform)

Object.Instantiate<T>(T, Transform)

Object.Instantiate<T>(T, Transform, Boolean)

Object.Destroy(Object, Single)

Object.Destroy(Object)

Object.DestroyImmediate(Object, Boolean)

Object.DestroyImmediate(Object)

Object.FindObjectsOfType(Type)

Object.DontDestroyOnLoad(Object)

Object.DestroyObject(Object, Single)

Object.DestroyObject(Object)

Object.FindSceneObjectsOfType(Type)

Object.FindObjectsOfTypeIncludingAssets(Type)

Object.FindObjectsOfType<T>()

Object.FindObjectOfType<T>()

Object.FindObjectsOfTypeAll(Type)

Object.FindObjectOfType(Type)

Object.ToString()

Object.name

Object.hideFlags

Namespace: UnityEditor.TestTools.TestRunner.Api

Syntax

public class TestRunnerApi : ScriptableObject, ITestRunnerApi

Examples

Run tests

Run tests by calling ExecuteTestRun and providing some execution settings in the form of a Filter. The Filter specifies what tests to run. The following is an example of how to run all Play Mode tests in a project:

var filter = new Filter()
{
    testMode = TestMode.PlayMode
};
    TestRunnerApi.ExecuteTestRun(new ExecutionSettings(filter));

Multiple filter values

You can specify a more specific filter by filling out the fields on the Filter class in more detail. Many of the fields allow for multiple values. The runner runs any test that matches at least one of the specified values. In this example, the API runs tests with full names that match either of the two names provided:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(new Filter()
{
    testNames = new[] {"MyTestClass.NameOfMyTest", "SpecificTestFixture.NameOfAnotherTest"}
}));

Multiple filter fields

If using multiple different fields on the filter, then the runner runs tests that match each of the different fields. In this example, a test is run if it matches either of the two test names and belongs to an assembly with the specified name:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(new Filter()
{
    assemblyNames = new [] {"MyTestAssembly"},
    testNames = new [] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}));

Multiple constructor filters

ExecutionSettings takes one or more filters in its constructor. If there is no filter provided, then it runs all Edit Mode tests by default. If there are multiple filters provided, then a test runs if it matches any of the filters. In this example, it runs any tests in the assembly named MyTestAssembly and any test with a full name matching either of the two provided test names:

TestRunnerApi.ExecuteTestRun(new ExecutionSettings(
    new Filter()
    {
        assemblyNames = new[] { "MyTestAssembly" },
    },
    new Filter()
    {
        testNames = new[] { "MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest" }
    }
));

Get test results

You can receive callbacks when the active test run, or an individual test, starts and finishes. You can register callbacks by invoking RegisterTestCallback with an instance of a class that implements ICallbacks. There are four ICallbacks methods for the start and finish of both the whole run and each level of the test tree. An example of how listeners can be set up:

Note: Listeners receive callbacks from all test runs, regardless of the registered TestRunnerApi for that instance.

public void SetupListeners()
{
    TestRunnerApi.RegisterTestCallback(new MyCallbacks());
}

private class MyCallbacks : ICallbacks
{
    public void RunStarted(ITestAdaptor testsToRun)
    {

    }

    public void RunFinished(ITestResultAdaptor result)
    {

    }

    public void TestStarted(ITestAdaptor test)
    {

    }

    public void TestFinished(ITestResultAdaptor result)
    {
       if (!result.HasChildren && result.ResultState != "Passed")
       {
           Debug.Log(string.Format("Test {0} {1}", result.Test.Name, result.ResultState));
       }
    }
}

Note: The registered callbacks are not persisted on domain reloads. So it is necessary to re-register the callback after a domain reload, usually with InitializeOnLoad.

It's possible to provide a priority as an integer as the second argument when registering a callback. This influences the invocation order of different callbacks. The default value is zero. It's also possible to provide RegisterTestCallback with a class instance that implements IErrorCallbacks that is an extended version of ICallbacks. IErrorCallbacks also has a callback method for OnError that invokes if the run fails to start, for example, due to compilation errors or if an IPrebuildSetup throws an exception.

Retrieve list of tests

You can retrieve the test tree by invoking RetrieveTestTree with the desired ExecutionSettings and a callback action, with an ITestAdaptor representing the test tree.

var filter = new Filter()
{
    assemblyNames = new [] {"myTestAssembly"}
};
TestRunnerApi.RetrieveTestTree(new ExecutionSettings(filter), (testRoot) =>
{
    Debug.Log(string.Format("Tree contains {0} tests.", testRoot.TestCaseCount));
});

Methods

CancelAllTestRuns()

Cancels all running test runs. Currently only supports EditMode tests.

Declaration

public static void CancelAllTestRuns()

CancelTestRun(String)

Cancel the test run with a given guid. The guid can be retrieved when executing the test run. Currently only supports EditMode tests.

Declaration

public static bool CancelTestRun(string guid)

Parameters

Type	Name	Description
String	guid	Test run GUID.

Returns

Type	Description
Boolean

Execute(ExecutionSettings)

Starts a test run with a given set of executionSettings.

Declaration

public string Execute(ExecutionSettings executionSettings)

Parameters

Type	Name	Description
ExecutionSettings	executionSettings	Set of ExecutionSettings

Returns

Type	Description
String	A GUID that identifies the TestJobData.

ExecuteTestRun(ExecutionSettings)

Starts a test run with a given set of executionSettings.

Declaration

public static string ExecuteTestRun(ExecutionSettings executionSettings)

Parameters

Type	Name	Description
ExecutionSettings	executionSettings	Set of ExecutionSettings

Returns

Type	Description
String	A GUID that identifies the TestJobData.

GetActiveRunGuids()

Provides a list of guids for all test runs that are currently active.

Declaration

public static string[] GetActiveRunGuids()

Returns

Type	Description
String[]	A list of guids of active runs.

GetCustomRunnerNames()

Get all the names of the currently registered custom runners.

Declaration

public static string[] GetCustomRunnerNames()

Returns

Type	Description
String[]	A list of custom runner names.

GetExecutionSettings(String)

Gets the execution settings for a test run with a specific guid.

Declaration

public static ExecutionSettings GetExecutionSettings(string guid)

Parameters

Type	Name	Description
String	guid	The guid of the test run.

Returns

Type	Description
ExecutionSettings	The execution settings for the run.

Exceptions

Type	Condition
Exception	Throws an exception if no run data for the guid could be found.

IsAnyRunActive()

Checks if any current test run is currently running.

Declaration

public static bool IsAnyRunActive()

Returns

Type	Description
Boolean	A boolean indicating if any test run is currently running.

IsRunActive(String)

Checks if a test run with a given guid is active.

Declaration

public static bool IsRunActive(string guid)

Parameters

Type	Name	Description
String	guid	The guid of the test run to check for.

Returns

Type	Description
Boolean	A boolean indicating if the test run is currently running.

Exceptions

Type	Condition
Exception	Throws an exception if no run data for the guid could be found.

RegisterCallbacks<T>(T, Int32)

Sets up a given instance of ICallbacks to be invoked on test runs.

Declaration

public void RegisterCallbacks<T>(T testCallbacks, int priority = 0)
    where T : ICallbacks

Parameters

Type	Name	Description
T	testCallbacks	The test callbacks to be invoked.
Int32	priority	Sets the order in which the callbacks are invoked, starting with the highest value first.

Type Parameters

Name	Description
T	Generic representing a type of callback.

RegisterCustomRunner(CustomRunnerBase)

Registers an implementation of CustomRunnerBase with the test framework, allowing for running custom test frameworks. Since custom runners are identified by name, the provided custom runner must use a name that has not already been registered.

Declaration

public static void RegisterCustomRunner(CustomRunnerBase customRunner)

Parameters

Type	Name	Description
CustomRunnerBase	customRunner	The custom runner to register.

Exceptions

Type	Condition
ArgumentNullException	Thrown if the customRunner argument is null.

RegisterTestCallback<T>(T, Int32)

Sets up a given instance of ICallbacks to be invoked on test runs.

Declaration

public static void RegisterTestCallback<T>(T testCallbacks, int priority = 0)
    where T : ICallbacks

Parameters

Type	Name	Description
T	testCallbacks	The test callbacks to be invoked
Int32	priority	Sets the order in which the callbacks are invoked, starting with the highest value first.

Type Parameters

Name	Description
T	Generic representing a type of callback.

RetrieveTestList(TestMode, Action<ITestAdaptor>)

Retrieve the full test tree as ITestAdaptor for a given test mode.

Declaration

public void RetrieveTestList(TestMode testMode, Action<ITestAdaptor> callback)

Parameters

Type	Name	Description
TestMode	testMode
Action<ITestAdaptor>	callback

RetrieveTestTree(ExecutionSettings, Action<ITestAdaptor>)

Retrieve a list of tests in a tree structure matching a given execution settings.

Declaration

public static void RetrieveTestTree(ExecutionSettings executionSettings, Action<ITestAdaptor> callback)

Parameters

Type	Name	Description
ExecutionSettings	executionSettings	An execution setting to match. The filters in the settings will be checked against the test tree.
Action<ITestAdaptor>	callback	A callback to invoke with the result of the test retrieving.

Exceptions

Type	Condition
ArgumentNullException	An exception is thrown if the executionSettings or the callback arguments are null.

SaveResultToFile(ITestResultAdaptor, String)

Save a given set of ITestResultAdaptor to a file at the provided file path.

Declaration

public static void SaveResultToFile(ITestResultAdaptor results, string filePath)

Parameters

Type	Name	Description
ITestResultAdaptor	results	Test results to write in a file.
String	filePath	Path to file.

UnregisterCallbacks<T>(T)

Unregister an instance of ICallbacks to no longer receive callbacks from test runs.

Declaration

public void UnregisterCallbacks<T>(T testCallbacks)
    where T : ICallbacks

Parameters

Type	Name	Description
T	testCallbacks	The test callbacks to unregister.

Type Parameters

Name	Description
T	Generic representing a type of callback.

UnregisterCustomRunner(CustomRunnerBase)

Unregisters a custom runner from the test framework.

Declaration

public static void UnregisterCustomRunner(CustomRunnerBase customRunner)

Parameters

Type	Name	Description
CustomRunnerBase	customRunner	The custom runner to unregister.

Exceptions

Type	Condition
ArgumentNullException	Thrown if the customRunner argument is null.

UnregisterTestCallback<T>(T)

Unregister an instance of ICallbacks to no longer receive callbacks from test runs.

Declaration

public static void UnregisterTestCallback<T>(T testCallbacks)
    where T : ICallbacks

Parameters

Type	Name	Description
T	testCallbacks	The test callbacks to unregister.

Type Parameters

Name	Description
T	Generic representing a type of callback.