Unittest of Python

Assert -- 内置类UNIT TEST API

https://realpython.com/python-testing/

You can write both integration tests and unit tests in Python. To write a unit test for the built-in function sum(), you would check the output of sum() against a known output.

For example, here’s how you check that the sum() of the numbers (1, 2, 3) equals 6:

>>> assert sum([1, 2, 3]) == 6, "Should be 6"

This will not output anything on the REPL because the values are correct.

If the result from sum() is incorrect, this will fail with an AssertionError and the message "Should be 6". Try an assertion statement again with the wrong values to see an AssertionError:

>>> assert sum([1, 1, 1]) == 6, "Should be 6"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError: Should be 6

In the REPL, you are seeing the raised AssertionError because the result of sum() does not match 6.

https://www.cnblogs.com/shangren/p/8038935.html

那么我们什么时候应该使用断言呢？如果没有特别的目的，断言应该用于如下情况：

• 防御性的编程
• 运行时对程序逻辑的检测
• 合约性检查（比如前置条件，后置条件）
• 程序中的常量
• 检查文档

这里是我建议不使用断言的情况：

*不要用于测试用户提供的数据，或者那些需要在所有情况下需要改变检查的地方

*不要用于检查你认为在通常使用中可能失败的地方。断言用于非常特别的失败条件。你的用户绝不看到一个AssertionError，如果看到了，那就是个必须修复的缺陷。

*特别地不要因为断言只是比一个明确的测试加一个触发异常矮小而使用它。断言不是懒惰的代码编写者的捷径。

*不要将断言用于公共函数库输入参数的检查，因为你不能控制调用者，并且不能保证它不破坏函数的合约。

*不要将断言用于你期望修改的任何错误。换句话，你没有任何理由在产品代码捕获一个AssertionError异常。

*不要太多使用断言，它们使代码变得晦涩难懂。

Test runner Comparsion

pytest 简单易用， 且兼容 unittest。是现在最流行的 python单元测试框架。

https://github.com/renzon/pytest-vs-unittest

Comparison between pytest and unittes test frameworks

##Comparison Table

Feature	Pytest	Unittest	Winner
Installation	Third Party	Built in	Unittest
Basic Infra	Can be only a function	Inheritance	Pytest
Basic Assertion	Builtin assert	TestCase instance methods	Pytest
Flat is better than nested	Function (1 level)	Method (2 level)	Pytest
Can run each other test	Can run unittest tests	Can't pytest test	Pytest
Test Result on console	Error Highlight, code snippet	Only line error, no highlight	Pytest
Multi param test	Yes, parametrize, keep flat	Yes, sub-test, increase nesting	Pytest
Test setup	fixture: module, session, function	Template Method: setup, tearDown	Pytest
Name Refactoring	poor, because of name conventions	rich, regular object orientation	Unittest
Running Failed Tests	built in (--lf, --ff)	your own =,(	Pytest
Marks	built in	your own =,(	Pytest

https://knapsackpro.com/testing_frameworks/difference_between/pytest/vs/unittest

	pytest https://docs.pytest.org/en/latest/	unittest https://docs.Python.org/3/library/unittest.html
Programming language	Python	Python
Category	Unit Testing	Unit Testing
General info	Pytest is the TDD 'all in one' testing framework for Python Pytest is a powerful Python testing framework that can test all and levels of software. It is considered by many to be the best testing framework in Python with many projects on the internet having switched to it from other frameworks, including Mozilla and Dropbox. This is due to its many powerful features such as ‘assert‘ rewriting, a third-party plugin model and a powerful yet simple fixture model.	unittest is a unit testing framework for Python The unittest test framework is Python’s xUnit style framework. It is a standard module that is bundled with Python and supports the automation and aggregation of tests and common setup and shutdown code for them.
xUnit Set of frameworks originating from SUnit (Smalltalk's testing framework). They share similar structure and functionality.	No	Yes unittest is a xUnit style frameworkfor Python, it was previously called PyUnit.
Client-side Allows testing code execution on the client, such as a web browser	Yes pytest can test any part of the stack including front-end components	Yes Front-end functionality and behaviour can be tested by unittest.
Server-side Allows testing the bahovior of a server-side code	Yes pytest is powerful enough to test database and server components and functionality	Yes Since the webserver funtionalities have their own features and each feature has its own functions, we can write tests with unittest to test each function
Fixtures Allows defining a fixed, specific states of data (fixtures) that are test-local. This ensures specific environment for a single test	Yes Pytest has a powerful yet simple fixture model that is unmatched in any other testing framework.	Yes By use of the 'setUp()' function which is called to prepare the test fixture
Group fixtures Allows defining a fixed, specific states of data for a group of tests (group-fixtures). This ensures specific environment for a given group of tests.	Yes Pytest's powerful fixture model allows grouping of fixtures	Yes unittest allows you to group your initialization code into a setUp function and clean up code in a tearDown function
Generators Supports data generators for tests. Data generators generate input data for test. The test is then run for each input data produced in this way.	Yes pytest has a hook function called pytest_generate_tests hook which is called when collecting a test function and one can use it to generate data	Yes unittest contains generator methods in the module 'unittest.TestCase'
Licence Licence type governing the use and redistribution of the software	MIT License	MIT License
Mocks Mocks are objects that simulate the behavior of real objects. Using mocks allows testing some part of the code in isolation (with other parts mocked when needed)	Yes By either using unittest.mock or using pytest-mock a thin wrapper that provides mock functionality for pytest	Yes Mocks are available from the library unittest.mock which allows you to replace parts of your system under test with mock objects
Grouping Allows organizing tests in groups	Yes Tests can be grouped with pytest by use of markers which are applied to various tests and one can run tests with the marker applied	Yes One can build suites either manually or use test discovery to build the suite automatically by scanning a directory
Other Other useful information about the testing framework

pytest

https://docs.pytest.org/en/reorganize-docs/index.html

pytest is a framework that makes building simple and scalable tests easy. Tests are expressive and readable—no boilerplate code required. Get started in minutes with a small unit test or complex functional test for your application or library.

pytest: helps you write better programs

a mature full-featured Python testing tool

• runs on Posix/Windows, Python 2.6-3.5, PyPy and (possibly still) Jython-2.5.1
• free and open source software, distributed under the terms of the MIT license
• well tested with more than a thousand tests against itself
• strict backward compatibility policy for safe pytest upgrades
• comprehensive online and PDF documentation
• many third party plugins and builtin helpers,
• used in many small and large projects and organisations
• comes with many tested examples

 Demo code

https://github.com/renzon/pytest-vs-unittest/blob/master/test/test_pytest/test_basic.py

def test_basic_stuff():
    assert True


def test_list():
    lst = [1, 3, 5, 6]
    assert [1, 3, 4] == lst


def test_string():
    assert 'unitest' == 'unittest'

目录布置

集中式和分散式同时支持。

https://docs.pytest.org/en/reorganize-docs/new-docs/user/directory_structure.html

pytest supports two common test layouts:

• putting tests into an extra directory outside your actual application code, useful if you have many functional tests or for other reasons want to keep tests separate from actual application code (often a good idea):

  setup.py   # your setuptools Python package metadata
  src/
      __init__.py
      appmodule.py
      module1/
          part1.py
          part2.py
      ...
  tests/
      test_appmodule.py
      module1/
          test_part1.py
          test_part2.py
      ...
  

• inlining test directories into your application package, useful if you have direct relation between (unit-)test and application modules and want to distribute your tests along with your application:

  setup.py   # your setuptools Python package metadata
  src/
      __init__.py
      appmodule.py
      tests/
          test_appmodule.py
      module1/
          part1.py
          part2.py
          tests/
              test_part1.py
              test_part2.py
      ...
  

Typically you can run tests by pointing to test directories or modules:

pytest tests/test_appmodule.py      # for external test dirs
pytest src/tests/test_appmodule.py  # for inlined test dirs
pytest src                          # run tests in all below test directories
pytest                              # run all tests below current dir
...

You shouldn’t have __init__.py files in your test directories.

fixture -- setup/teardown

https://docs.pytest.org/en/reorganize-docs/fixture.html#fixtures

The purpose of test fixtures is to provide a fixed baseline upon which tests can reliably and repeatedly execute. pytest fixtures offer dramatic improvements over the classic xUnit style of setup/teardown functions:

• fixtures have explicit names and are activated by declaring their use from test functions, modules, classes or whole projects.
• fixtures are implemented in a modular manner, as each fixture name triggers a fixture function which can itself use other fixtures.
• fixture management scales from simple unit to complex functional testing, allowing to parametrize fixtures and tests according to configuration and component options, or to re-use fixtures across class, module or whole test session scopes.

# content of ./test_smtpsimple.py
import pytest

@pytest.fixture
def smtp():
    import smtplib
    return smtplib.SMTP("smtp.gmail.com")

def test_ehlo(smtp):
    response, msg = smtp.ehlo()
    assert response == 250
    assert 0 # for demo purposes

Parametrizing

https://docs.pytest.org/en/reorganize-docs/parametrize.html#parametrize

pytest supports test parametrization in several well-integrated ways:

• pytest.fixture() allows to define parametrization at the level of fixture functions.

• @pytest.mark.parametrize allows to define parametrization at the function or class level, provides multiple argument/fixture sets for a particular test function or class.
• pytest_generate_tests enables implementing your own custom dynamic parametrization scheme or extensions.

# content of test_expectation.py
import pytest
@pytest.mark.parametrize("test_input,expected", [
    ("3+5", 8),
    ("2+4", 6),
    ("6*9", 42),
])
def test_eval(test_input, expected):
    assert eval(test_input) == expected