Merge tag 'linux-kselftest-kunit-6.1-rc1' of...

			0: force disabled

			1: force enabled

	kunit.enable=	[KUNIT] Enable executing KUnit tests. Requires

			CONFIG_KUNIT to be set to be fully enabled. The

			default value can be overridden via

			KUNIT_DEFAULT_ENABLED.

			Default is 1 (enabled)

	kvm.ignore_msrs=[KVM] Ignore guest accesses to unhandled MSRs.

			Default is 0 (don't ignore, but inject #GP)

The KUnit architecture can be divided into two parts:

- Kernel testing library

- kunit_tool (Command line test harness)

- `In-Kernel Testing Framework`_

- `kunit_tool (Command Line Test Harness)`_

In-Kernel Testing Framework

===========================

can compile to any architecture. It compiles like just another part of the

kernel and runs when the kernel boots, or when built as a module, when the

module is loaded.  However, there is infrastructure, like the KUnit Wrapper

(``tools/testing/kunit/kunit.py``) that does not support other architectures.

(``tools/testing/kunit/kunit.py``) that might not support some architectures

(see :ref:`kunit-on-qemu`).

In short, yes, you can run KUnit on other architectures, but it might require

more work than using KUnit on UML.

For more information, see :ref:`kunit-on-non-uml`.

.. _kinds-of-tests:

What is the difference between a unit test and other kinds of tests?

====================================================================

Most existing tests for the Linux kernel would be categorized as an integration

   seeing. When tests are built-in, they will execute when the kernel boots, and

   modules will automatically execute associated tests when loaded. Test results

   can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and

   can be parsed with ``kunit.py parse``. For more details, see "KUnit on

   non-UML architectures" in Documentation/dev-tools/kunit/usage.rst.

   can be parsed with ``kunit.py parse``. For more details, see :ref:`kunit-on-qemu`.

If none of the above tricks help, you are always welcome to email any issues to

kunit-dev@googlegroups.com.

	run_wrapper

	run_manual

	usage

	kunit-tool

	api/index

	style

	faq

unit tests within the Linux kernel. Using KUnit, you can define groups

of test cases called test suites. The tests either run on kernel boot

if built-in, or load as a module. KUnit automatically flags and reports

failed test cases in the kernel log. The test results appear in `TAP

(Test Anything Protocol) format <https://testanything.org/>`_. It is inspired by

JUnit, Python’s unittest.mock, and GoogleTest/GoogleMock (C++ unit testing

framework).

failed test cases in the kernel log. The test results appear in

:doc:`KTAP (Kernel - Test Anything Protocol) format</dev-tools/ktap>`.

It is inspired by JUnit, Python’s unittest.mock, and GoogleTest/GoogleMock

(C++ unit testing framework).

KUnit tests are part of the kernel, written in the C (programming)

language, and test parts of the Kernel implementation (example: a C

restricted to things exposed to user-space.

In addition, KUnit has kunit_tool, a script (``tools/testing/kunit/kunit.py``)

that configures the Linux kernel, runs KUnit tests under QEMU or UML (`User Mode

Linux <http://user-mode-linux.sourceforge.net/>`_), parses the test results and

that configures the Linux kernel, runs KUnit tests under QEMU or UML

(:doc:`User Mode Linux </virt/uml/user_mode_linux_howto_v2>`),

parses the test results and

displays them in a user friendly manner.

Features

- Improves code quality.

- Encourages writing testable code.

Read also :ref:`kinds-of-tests`.

How do I use it?

================

    examples.

*   Documentation/dev-tools/kunit/api/index.rst - KUnit APIs

    used for testing.

*   Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper

    script.

*   Documentation/dev-tools/kunit/faq.rst - KUnit common questions and

    answers.

.. SPDX-License-Identifier: GPL-2.0

=================

kunit_tool How-To

=================

What is kunit_tool?

===================

kunit_tool is a script (``tools/testing/kunit/kunit.py``) that aids in building

the Linux kernel as UML (`User Mode Linux

<http://user-mode-linux.sourceforge.net/>`_), running KUnit tests, parsing

the test results and displaying them in a user friendly manner.

kunit_tool addresses the problem of being able to run tests without needing a

virtual machine or actual hardware with User Mode Linux. User Mode Linux is a

Linux architecture, like ARM or x86; however, unlike other architectures it

compiles the kernel as a standalone Linux executable that can be run like any

other program directly inside of a host operating system. To be clear, it does

not require any virtualization support: it is just a regular program.

What is a .kunitconfig?

=======================

It's just a defconfig that kunit_tool looks for in the build directory

(``.kunit`` by default).  kunit_tool uses it to generate a .config as you might

expect. In addition, it verifies that the generated .config contains the CONFIG

options in the .kunitconfig; the reason it does this is so that it is easy to

be sure that a CONFIG that enables a test actually ends up in the .config.

It's also possible to pass a separate .kunitconfig fragment to kunit_tool,

which is useful if you have several different groups of tests you wish

to run independently, or if you want to use pre-defined test configs for

certain subsystems.

Getting Started with kunit_tool

===============================

If a kunitconfig is present at the root directory, all you have to do is:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run

However, you most likely want to use it with the following options:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run --timeout=30 --jobs=`nproc --all`

- ``--timeout`` sets a maximum amount of time to allow tests to run.

- ``--jobs`` sets the number of threads to use to build the kernel.

.. note::

	This command will work even without a .kunitconfig file: if no

	.kunitconfig is present, a default one will be used instead.

If you wish to use a different .kunitconfig file (such as one provided for

testing a particular subsystem), you can pass it as an option.

.. code-block:: bash

	./tools/testing/kunit/kunit.py run --kunitconfig=fs/ext4/.kunitconfig

For a list of all the flags supported by kunit_tool, you can run:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run --help

Configuring, Building, and Running Tests

========================================

It's also possible to run just parts of the KUnit build process independently,

which is useful if you want to make manual changes to part of the process.

A .config can be generated from a .kunitconfig by using the ``config`` argument

when running kunit_tool:

.. code-block:: bash

	./tools/testing/kunit/kunit.py config

Similarly, if you just want to build a KUnit kernel from the current .config,

you can use the ``build`` argument:

.. code-block:: bash

	./tools/testing/kunit/kunit.py build

And, if you already have a built UML kernel with built-in KUnit tests, you can

run the kernel and display the test results with the ``exec`` argument:

.. code-block:: bash

	./tools/testing/kunit/kunit.py exec

The ``run`` command which is discussed above is equivalent to running all three

of these in sequence.

All of these commands accept a number of optional command-line arguments. The

``--help`` flag will give a complete list of these, or keep reading this page

for a guide to some of the more useful ones.

Parsing Test Results

====================

KUnit tests output their results in TAP (Test Anything Protocol) format.

kunit_tool will, when running tests, parse this output and print a summary

which is much more pleasant to read. If you wish to look at the raw test

results in TAP format, you can pass the ``--raw_output`` argument.

.. code-block:: bash

	./tools/testing/kunit/kunit.py run --raw_output

The raw output from test runs may contain other, non-KUnit kernel log

lines. You can see just KUnit output with ``--raw_output=kunit``:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run --raw_output=kunit

If you have KUnit results in their raw TAP format, you can parse them and print

the human-readable summary with the ``parse`` command for kunit_tool. This

accepts a filename for an argument, or will read from standard input.

.. code-block:: bash

	# Reading from a file

	./tools/testing/kunit/kunit.py parse /var/log/dmesg

	# Reading from stdin

	dmesg | ./tools/testing/kunit/kunit.py parse

This is very useful if you wish to run tests in a configuration not supported

by kunit_tool (such as on real hardware, or an unsupported architecture).

Filtering Tests

===============

It's possible to run only a subset of the tests built into a kernel by passing

a filter to the ``exec`` or ``run`` commands. For example, if you only wanted

to run KUnit resource tests, you could use:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run 'kunit-resource*'

This uses the standard glob format for wildcards.

Running Tests on QEMU

=====================

kunit_tool supports running tests on QEMU as well as via UML (as mentioned

elsewhere). The default way of running tests on QEMU requires two flags:

``--arch``

	Selects a collection of configs (Kconfig as well as QEMU configs

	options, etc) that allow KUnit tests to be run on the specified

	architecture in a minimal way; this is usually not much slower than

	using UML. The architecture argument is the same as the name of the

	option passed to the ``ARCH`` variable used by Kbuild. Not all

	architectures are currently supported by this flag, but can be handled

	by the ``--qemu_config`` discussed later. If ``um`` is passed (or this

	this flag is ignored) the tests will run via UML. Non-UML architectures,

	e.g. i386, x86_64, arm, um, etc. Non-UML run on QEMU.

``--cross_compile``

	Specifies the use of a toolchain by Kbuild. The argument passed here is

	the same passed to the ``CROSS_COMPILE`` variable used by Kbuild. As a

	reminder this will be the prefix for the toolchain binaries such as gcc

	for example ``sparc64-linux-gnu-`` if you have the sparc toolchain

	installed on your system, or

	``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/microblaze-linux-``

	if you have downloaded the microblaze toolchain from the 0-day website

	to a directory in your home directory called ``toolchains``.

In many cases it is likely that you may want to run an architecture which is

not supported by the ``--arch`` flag, or you may want to just run KUnit tests

on QEMU using a non-default configuration. For this use case, you can write

your own QemuConfig. These QemuConfigs are written in Python. They must have an

import line ``from ..qemu_config import QemuArchParams`` at the top of the file

and the file must contain a variable called ``QEMU_ARCH`` that has an instance

of ``QemuArchParams`` assigned to it. An example can be seen in

``tools/testing/kunit/qemu_configs/x86_64.py``.

Once you have a QemuConfig you can pass it into kunit_tool using the

``--qemu_config`` flag; when used this flag replaces the ``--arch`` flag. If we

were to do this with the ``x86_64.py`` example from above, the invocation would

look something like this:

.. code-block:: bash

	./tools/testing/kunit/kunit.py run \

		--timeout=60 \

		--jobs=12 \

		--qemu_config=./tools/testing/kunit/qemu_configs/x86_64.py

Other Useful Options

====================

kunit_tool has a number of other command-line arguments which can be useful

when adapting it to fit your environment or needs.

Some of the more useful ones are:

``--help``

	Lists all of the available options. Note that different commands

	(``config``, ``build``, ``run``, etc) will have different supported

	options. Place ``--help`` before the command to list common options,

	and after the command for options specific to that command.

``--build_dir``

	Specifies the build directory that kunit_tool will use. This is where

	the .kunitconfig file is located, as well as where the .config and

	compiled kernel will be placed. Defaults to ``.kunit``.

``--make_options``

	Specifies additional options to pass to ``make`` when compiling a

	kernel (with the ``build`` or ``run`` commands). For example, to enable

	compiler warnings, you can pass ``--make_options W=1``.

``--alltests``

        Builds a UML kernel with all config options enabled using ``make

        allyesconfig``. This allows you to run as many tests as is possible,

        but is very slow and prone to breakage as new options are added or

        modified. In most cases, enabling all tests which have satisfied

        dependencies by adding ``CONFIG_KUNIT_ALL_TESTS=1`` to your

        .kunitconfig is preferable.

There are several other options (and new ones are often added), so do check

``--help`` if you're looking for something not mentioned here.