Merge tag 'linux-kselftest-kunit-next-6.2-rc1' of...

subtest within the same test at the same nesting level.

subtest within the same test at the same nesting level.

The description is a description of the test, generally the name of

The description is a description of the test, generally the name of

the test, and can be any string of words (can't include #). The

the test, and can be any string of characters other than # or a

description is optional, but recommended.

newline.  The description is optional, but recommended.

The directive and any diagnostic data is optional. If either are present, they

The directive and any diagnostic data is optional. If either are present, they

must follow a hash sign, "#".

must follow a hash sign, "#".

KUnit Architecture

KUnit Architecture

==================

==================

The KUnit architecture can be divided into two parts:

The KUnit architecture is divided into two parts:

- `In-Kernel Testing Framework`_

- `In-Kernel Testing Framework`_

- `kunit_tool (Command Line Test Harness)`_

- `kunit_tool (Command-line Test Harness)`_

In-Kernel Testing Framework

In-Kernel Testing Framework

===========================

===========================

The kernel testing library supports KUnit tests written in C using

The kernel testing library supports KUnit tests written in C using

KUnit. KUnit tests are kernel code. KUnit does several things:

KUnit. These KUnit tests are kernel code. KUnit performs the following

tasks:

- Organizes tests

- Organizes tests

- Reports test results

- Reports test results

Test Cases

Test Cases

----------

----------

The fundamental unit in KUnit is the test case. The KUnit test cases are

The test case is the fundamental unit in KUnit. KUnit test cases are organised

grouped into KUnit suites. A KUnit test case is a function with type

into suites. A KUnit test case is a function with type signature

signature ``void (*)(struct kunit *test)``.

``void (*)(struct kunit *test)``. These test case functions are wrapped in a

These test case functions are wrapped in a struct called

struct called struct kunit_case.

struct kunit_case.

.. note:

.. note:

	``generate_params`` is optional for non-parameterized tests.

	``generate_params`` is optional for non-parameterized tests.

Each KUnit test case gets a ``struct kunit`` context

Each KUnit test case receives a ``struct kunit`` context object that tracks a

object passed to it that tracks a running test. The KUnit assertion

running test. The KUnit assertion macros and other KUnit utilities use the

macros and other KUnit utilities use the ``struct kunit`` context

``struct kunit`` context object. As an exception, there are two fields:

object. As an exception, there are two fields:

- ``->priv``: The setup functions can use it to store arbitrary test

- ``->priv``: The setup functions can use it to store arbitrary test

  user data.

  user data.

The KUnit executor can list and run built-in KUnit tests on boot.

The KUnit executor can list and run built-in KUnit tests on boot.

The Test suites are stored in a linker section

The Test suites are stored in a linker section

called ``.kunit_test_suites``. For code, see:

called ``.kunit_test_suites``. For the code, see ``KUNIT_TABLE()`` macro

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.

definition in

`include/asm-generic/vmlinux.lds.h <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v6.0#n950>`_.

The linker section consists of an array of pointers to

The linker section consists of an array of pointers to

``struct kunit_suite``, and is populated by the ``kunit_test_suites()``

``struct kunit_suite``, and is populated by the ``kunit_test_suites()``

macro. To run all tests compiled into the kernel, the KUnit executor

macro. The KUnit executor iterates over the linker section array in order to

iterates over the linker section array.

run all the tests that are compiled into the kernel.

.. kernel-figure:: kunit_suitememorydiagram.svg

.. kernel-figure:: kunit_suitememorydiagram.svg

	:alt:	KUnit Suite Memory

	:alt:	KUnit Suite Memory

	KUnit Suite Memory Diagram

	KUnit Suite Memory Diagram

On the kernel boot, the KUnit executor uses the start and end addresses

On the kernel boot, the KUnit executor uses the start and end addresses

of this section to iterate over and run all tests. For code, see:

of this section to iterate over and run all tests. For the implementation of the

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c

executor, see

`lib/kunit/executor.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c>`_.

When built as a module, the ``kunit_test_suites()`` macro defines a

When built as a module, the ``kunit_test_suites()`` macro defines a

``module_init()`` function, which runs all the tests in the compilation

``module_init()`` function, which runs all the tests in the compilation

unit instead of utilizing the executor.

unit instead of utilizing the executor.

In KUnit tests, some error classes do not affect other tests

In KUnit tests, some error classes do not affect other tests

or parts of the kernel, each KUnit case executes in a separate thread

or parts of the kernel, each KUnit case executes in a separate thread

context. For code, see:

context. See the ``kunit_try_catch_run()`` function in

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58

`lib/kunit/try-catch.c <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58>`_.

Assertion Macros

Assertion Macros

----------------

----------------

- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an

- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an

  expectation.

  expectation.

  In the event of a failure, the testing flow differs as follows:

	- For an expectation, if the check fails, marks the test as failed

	- For expectations, the test is marked as failed and the failure is logged.

	  and logs the failure.

	- An assertion, on failure, causes the test case to terminate

	- Failing assertions, on the other hand, result in the test case being

	  immediately.

	  terminated immediately.

		- Assertions call function:

		- Assertions call the function:

		  ``void __noreturn kunit_abort(struct kunit *)``.

		  ``void __noreturn kunit_abort(struct kunit *)``.

		- ``kunit_abort`` calls function:

		- ``kunit_abort`` calls the function:

		  ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.

		  ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.

		- ``kunit_try_catch_throw`` calls function:

		- ``kunit_try_catch_throw`` calls the function:

		  ``void kthread_complete_and_exit(struct completion *, long) __noreturn;``

		  ``void kthread_complete_and_exit(struct completion *, long) __noreturn;``

		  and terminates the special thread context.

		  and terminates the special thread context.

- ``<op>`` denotes a check with options: ``TRUE`` (supplied property

- ``<op>`` denotes a check with options: ``TRUE`` (supplied property

  has the boolean value “true”), ``EQ`` (two supplied properties are

  has the boolean value "true"), ``EQ`` (two supplied properties are

  equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not

  equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not

  contain an “err” value).

  contain an "err" value).

- ``[_MSG]`` prints a custom message on failure.

- ``[_MSG]`` prints a custom message on failure.

Test Result Reporting

Test Result Reporting

---------------------

---------------------

KUnit prints test results in KTAP format. KTAP is based on TAP14, see:

KUnit prints the test results in KTAP format. KTAP is based on TAP14, see

https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.

Documentation/dev-tools/ktap.rst.

KTAP (yet to be standardized format) works with KUnit and Kselftest.

KTAP works with KUnit and Kselftest. The KUnit executor prints KTAP results to

The KUnit executor prints KTAP results to dmesg, and debugfs

dmesg, and debugfs (if configured).

(if configured).

Parameterized Tests

Parameterized Tests

-------------------

-------------------

parameters. The test is invoked multiple times, once for each parameter

parameters. The test is invoked multiple times, once for each parameter

value and the parameter is stored in the ``param_value`` field.

value and the parameter is stored in the ``param_value`` field.

The test case includes a KUNIT_CASE_PARAM() macro that accepts a

The test case includes a KUNIT_CASE_PARAM() macro that accepts a

generator function.

generator function. The generator function is passed the previous parameter

The generator function is passed the previous parameter and returns the next

and returns the next parameter. It also includes a macro for generating

parameter. It also provides a macro to generate common-case generators based on

array-based common-case generators.

arrays.

kunit_tool (Command Line Test Harness)

kunit_tool (Command-line Test Harness)

======================================

======================================

kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``

``kunit_tool`` is a Python script, found in ``tools/testing/kunit/kunit.py``. It

that can be used to configure, build, exec, parse and run (runs other

is used to configure, build, execute, parse test results and run all of the

commands in order) test results. You can either run KUnit tests using

previous commands in correct order (i.e., configure, build, execute and parse).

kunit_tool or can include KUnit in kernel and parse manually.

You have two options for running KUnit tests: either build the kernel with KUnit

enabled and manually parse the results (see

Documentation/dev-tools/kunit/run_manual.rst) or use ``kunit_tool``

(see Documentation/dev-tools/kunit/run_wrapper.rst).

- ``configure`` command generates the kernel ``.config`` from a

- ``configure`` command generates the kernel ``.config`` from a

  ``.kunitconfig`` file (and any architecture-specific options).

  ``.kunitconfig`` file (and any architecture-specific options).

  For some architectures, additional config options are specified in the

  The Python scripts available in ``qemu_configs`` folder

  ``qemu_config`` Python script

  (for example, ``tools/testing/kunit/qemu configs/powerpc.py``) contains

  (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).

  additional configuration options for specific architectures.

  It parses both the existing ``.config`` and the ``.kunitconfig`` files

  It parses both the existing ``.config`` and the ``.kunitconfig`` files

  and ensures that ``.config`` is a superset of ``.kunitconfig``.

  to ensure that ``.config`` is a superset of ``.kunitconfig``.

  If this is not the case, it will combine the two and run

  If not, it will combine the two and run ``make olddefconfig`` to regenerate

  ``make olddefconfig`` to regenerate the ``.config`` file. It then

  the ``.config`` file. It then checks to see if ``.config`` has become a superset.

  verifies that ``.config`` is now a superset. This checks if all

  This verifies that all the Kconfig dependencies are correctly specified in the

  Kconfig dependencies are correctly specified in ``.kunitconfig``.

  file ``.kunitconfig``. The ``kunit_config.py`` script contains the code for parsing

  ``kunit_config.py`` includes the parsing Kconfigs code. The code which

  Kconfigs. The code which runs ``make olddefconfig`` is part of the

  runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can

  ``kunit_kernel.py`` script. You can invoke this command through:

  invoke this command via: ``./tools/testing/kunit/kunit.py config`` and

  ``./tools/testing/kunit/kunit.py config`` and

  generate a ``.config`` file.

  generate a ``.config`` file.

- ``build`` runs ``make`` on the kernel tree with required options

- ``build`` runs ``make`` on the kernel tree with required options

  (depends on the architecture and some options, for example: build_dir)

  (depends on the architecture and some options, for example: build_dir)

  To build a KUnit kernel from the current ``.config``, you can use the

  To build a KUnit kernel from the current ``.config``, you can use the

  ``build`` argument: ``./tools/testing/kunit/kunit.py build``.

  ``build`` argument: ``./tools/testing/kunit/kunit.py build``.

- ``exec`` command executes kernel results either directly (using

- ``exec`` command executes kernel results either directly (using

  User-mode Linux configuration), or via an emulator such

  User-mode Linux configuration), or through an emulator such

  as QEMU. It reads results from the log via standard

  as QEMU. It reads results from the log using standard

  output (stdout), and passes them to ``parse`` to be parsed.

  output (stdout), and passes them to ``parse`` to be parsed.

  If you already have built a kernel with built-in KUnit tests,

  If you already have built a kernel with built-in KUnit tests,

  you can run the kernel and display the test results with the ``exec``

  you can run the kernel and display the test results with the ``exec``

	api/index

	api/index

	style

	style

	faq

	faq

	tips

	running_tips

	running_tips

This section details the kernel unit testing framework.

This section details the kernel unit testing framework.

How do I use it?

How do I use it?

================

================

*   Documentation/dev-tools/kunit/start.rst - for KUnit new users.

You can find a step-by-step guide to writing and running KUnit tests in

*   Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.

Documentation/dev-tools/kunit/start.rst

*   Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool.

*   Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit_tool.

Alternatively, feel free to look through the rest of the KUnit documentation,

*   Documentation/dev-tools/kunit/usage.rst - write tests.

or to experiment with tools/testing/kunit/kunit.py and the example test under

*   Documentation/dev-tools/kunit/tips.rst - best practices with

lib/kunit/kunit-example-test.c

    examples.

*   Documentation/dev-tools/kunit/api/index.rst - KUnit APIs

Happy testing!

    used for testing.

*   Documentation/dev-tools/kunit/faq.rst - KUnit common questions and

    answers.

Next Steps

Next Steps

==========

==========

*   Documentation/dev-tools/kunit/architecture.rst - KUnit architecture.

If you're interested in using some of the more advanced features of kunit.py,

*   Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool.

take a look at Documentation/dev-tools/kunit/run_wrapper.rst

*   Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit_tool.

*   Documentation/dev-tools/kunit/usage.rst - write tests.

If you'd like to run tests without using kunit.py, check out

*   Documentation/dev-tools/kunit/tips.rst - best practices with

Documentation/dev-tools/kunit/run_manual.rst

    examples.

*   Documentation/dev-tools/kunit/api/index.rst - KUnit APIs

For more information on writing KUnit tests (including some common techniques

    used for testing.

for testing different things), see Documentation/dev-tools/kunit/usage.rst

*   Documentation/dev-tools/kunit/faq.rst - KUnit common questions and

    answers.

.. SPDX-License-Identifier: GPL-2.0

============================

Tips For Writing KUnit Tests

============================

Exiting early on failed expectations

------------------------------------

``KUNIT_EXPECT_EQ`` and friends will mark the test as failed and continue

execution.  In some cases, it's unsafe to continue and you can use the

``KUNIT_ASSERT`` variant to exit on failure.

.. code-block:: c

	void example_test_user_alloc_function(struct kunit *test)

	{

		void *object = alloc_some_object_for_me();

		/* Make sure we got a valid pointer back. */

		KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object);

		do_something_with_object(object);

	}

Allocating memory

-----------------

Where you would use ``kzalloc``, you should prefer ``kunit_kzalloc`` instead.

KUnit will ensure the memory is freed once the test completes.

This is particularly useful since it lets you use the ``KUNIT_ASSERT_EQ``

macros to exit early from a test without having to worry about remembering to

call ``kfree``.

Example:

.. code-block:: c

	void example_test_allocation(struct kunit *test)

	{

		char *buffer = kunit_kzalloc(test, 16, GFP_KERNEL);

		/* Ensure allocation succeeded. */

		KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer);

		KUNIT_ASSERT_STREQ(test, buffer, "");

	}

Testing static functions

------------------------

If you don't want to expose functions or variables just for testing, one option

is to conditionally ``#include`` the test file at the end of your .c file, e.g.

.. code-block:: c

	/* In my_file.c */

	static int do_interesting_thing();

	#ifdef CONFIG_MY_KUNIT_TEST

	#include "my_kunit_test.c"

	#endif

Injecting test-only code

------------------------

Similarly to the above, it can be useful to add test-specific logic.

.. code-block:: c

	/* In my_file.h */

	#ifdef CONFIG_MY_KUNIT_TEST

	/* Defined in my_kunit_test.c */

	void test_only_hook(void);

	#else

	void test_only_hook(void) { }

	#endif

This test-only code can be made more useful by accessing the current kunit

test, see below.

Accessing the current test

--------------------------

In some cases, you need to call test-only code from outside the test file, e.g.

like in the example above or if you're providing a fake implementation of an

ops struct.

There is a ``kunit_test`` field in ``task_struct``, so you can access it via

``current->kunit_test``.

Here's a slightly in-depth example of how one could implement "mocking":

.. code-block:: c

	#include <linux/sched.h> /* for current */

	struct test_data {

		int foo_result;

		int want_foo_called_with;

	};

	static int fake_foo(int arg)

	{

		struct kunit *test = current->kunit_test;

		struct test_data *test_data = test->priv;

		KUNIT_EXPECT_EQ(test, test_data->want_foo_called_with, arg);

		return test_data->foo_result;

	}

	static void example_simple_test(struct kunit *test)

	{

		/* Assume priv is allocated in the suite's .init */

		struct test_data *test_data = test->priv;

		test_data->foo_result = 42;

		test_data->want_foo_called_with = 1;

		/* In a real test, we'd probably pass a pointer to fake_foo somewhere

		 * like an ops struct, etc. instead of calling it directly. */

		KUNIT_EXPECT_EQ(test, fake_foo(1), 42);

	}

Note: here we're able to get away with using ``test->priv``, but if you wanted

something more flexible you could use a named ``kunit_resource``, see

Documentation/dev-tools/kunit/api/test.rst.

Failing the current test

------------------------

But sometimes, you might just want to fail the current test. In that case, we

have ``kunit_fail_current_test(fmt, args...)`` which is defined in ``<kunit/test-bug.h>`` and

doesn't require pulling in ``<kunit/test.h>``.

E.g. say we had an option to enable some extra debug checks on some data structure:

.. code-block:: c

	#include <kunit/test-bug.h>

	#ifdef CONFIG_EXTRA_DEBUG_CHECKS

	static void validate_my_data(struct data *data)

	{

		if (is_valid(data))

			return;

		kunit_fail_current_test("data %p is invalid", data);

		/* Normal, non-KUnit, error reporting code here. */

	}

	#else

	static void my_debug_function(void) { }

	#endif

Customizing error messages

--------------------------

Each of the ``KUNIT_EXPECT`` and ``KUNIT_ASSERT`` macros have a ``_MSG`` variant.

These take a format string and arguments to provide additional context to the automatically generated error messages.

.. code-block:: c

	char some_str[41];

	generate_sha1_hex_string(some_str);

	/* Before. Not easy to tell why the test failed. */

	KUNIT_EXPECT_EQ(test, strlen(some_str), 40);

	/* After. Now we see the offending string. */

	KUNIT_EXPECT_EQ_MSG(test, strlen(some_str), 40, "some_str='%s'", some_str);

Alternatively, one can take full control over the error message by using ``KUNIT_FAIL()``, e.g.

.. code-block:: c

	/* Before */

	KUNIT_EXPECT_EQ(test, some_setup_function(), 0);

	/* After: full control over the failure message. */

	if (some_setup_function())

		KUNIT_FAIL(test, "Failed to setup thing for testing");

Next Steps

==========

*   Optional: see the Documentation/dev-tools/kunit/usage.rst page for a more

    in-depth explanation of KUnit.