zoukankan      html  css  js  c++  java
  • coverlet简介

    https://github.com/tonerdo/coverlet/tree/master/Documentation

    https://github.com/tonerdo/coverlet/blob/master/Documentation/MSBuildIntegration.md

    In this mode, Coverlet doesn't require any additional setup other than including the NuGet package in the unit test project. It integrates with the dotnet test infrastructure built into the .NET Core CLI and when enabled, will automatically generate coverage results after tests are run.

    If a property takes multiple comma-separated values please note that you will have to add escaped quotes around the string like this: /p:Exclude="[coverlet.*]*,[*]Coverlet.Core*"/p:Include="[coverlet.*]*,[*]Coverlet.Core*", or /p:CoverletOutputFormat="json,opencover".

    Code Coverage

    Enabling code coverage is as simple as setting the CollectCoverage property to true

    dotnet test /p:CollectCoverage=true

    After the above command is run, a coverage.json file containing the results will be generated in the root directory of the test project. A summary of the results will also be displayed in the terminal.

    Coverage Output

    Coverlet can generate coverage results in multiple formats, which is specified using the CoverletOutputFormat property. For example, the following command emits coverage results in the opencover format:

    dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover

    Supported Formats:

    • json (default)
    • lcov
    • opencover
    • cobertura
    • teamcity

    You can specify multiple output formats by separating them with a comma (,).

    The output of the coverage result can be specified using the CoverletOutput property.

    dotnet test /p:CollectCoverage=true /p:CoverletOutput='./result.json'

    To specify a directory where all results will be written to (especially if using multiple formats), end the value with a /.

    dotnet test /p:CollectCoverage=true /p:CoverletOutput='./results/'

    TeamCity Output

    Coverlet can output basic code coverage statistics using TeamCity service messages.

    dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=teamcity

    The currently supported TeamCity statistics are:

    TeamCity Statistic KeyDescription
    CodeCoverageL Line-level code coverage
    CodeCoverageB Branch-level code coverage
    CodeCoverageM Method-level code coverage
    CodeCoverageAbsLTotal The total number of lines
    CodeCoverageAbsLCovered The number of covered lines
    CodeCoverageAbsBTotal The total number of branches
    CodeCoverageAbsBCovered The number of covered branches
    CodeCoverageAbsMTotal The total number of methods
    CodeCoverageAbsMCovered The number of covered methods


    Merging Results

    With Coverlet you can combine the output of multiple coverage runs into a single result.

    dotnet test /p:CollectCoverage=true /p:MergeWith='/path/to/result.json'

    The value given to /p:MergeWith must be a path to Coverlet's own json result format. The results in result.json will be read, and added to the new results written to by Coverlet. Check the sample.

    Threshold

    Coverlet allows you to specify a coverage threshold below which it fails the build. This allows you to enforce a minimum coverage percent on all changes to your project.

    dotnet test /p:CollectCoverage=true /p:Threshold=80

    The above command will automatically fail the build if the line, branch or method coverage of any of the instrumented modules falls below 80%. You can specify what type of coverage to apply the threshold value to using the ThresholdType property. For example to apply the threshold check to only line coverage:

    dotnet test /p:CollectCoverage=true /p:Threshold=80 /p:ThresholdType=line

    You can specify multiple values for ThresholdType by separating them with commas. Valid values include linebranch and method.

    By default, Coverlet will validate the threshold value against the coverage result of each module. The /p:ThresholdStat option allows you to change this behaviour and can have any of the following values:

    • Minimum (Default): Ensures the coverage result of each module isn't less than the threshold
    • Total: Ensures the total combined coverage result of all modules isn't less than the threshold
    • Average: Ensures the average coverage result of all modules isn't less than the threshold

    The following command will compare the threshold value with the overall total coverage of all modules:

    dotnet test /p:CollectCoverage=true /p:Threshold=80 /p:ThresholdType=line /p:ThresholdStat=total

    Excluding From Coverage

    Attributes

    You can ignore a method an entire class or assembly from code coverage by creating and applying the ExcludeFromCodeCoverage attribute present in the System.Diagnostics.CodeAnalysis namespace.

    You can also ignore additional attributes by using the ExcludeByAttribute property (short name or full name supported):

    dotnet test /p:CollectCoverage=true /p:ExcludeByAttribute="Obsolete,GeneratedCodeAttribute,CompilerGeneratedAttribute"

    Source Files

    You can also ignore specific source files from code coverage using the ExcludeByFile property

    • Use single or multiple paths (separate by comma)
    • Use absolute or relative paths (relative to the project directory)
    • Use file path or directory path with globbing (e.g dir1/*.cs)
    dotnet test /p:CollectCoverage=true /p:ExcludeByFile="../dir1/class1.cs,../dir2/*.cs,../dir3/**/*.cs"

    Filters

    Coverlet gives the ability to have fine grained control over what gets excluded using "filter expressions".

    Syntax: /p:Exclude=[Assembly-Filter]Type-Filter

    Wildcards

    • * => matches zero or more characters
    • ? => the prefixed character is optional

    Examples

    • /p:Exclude="[*]*" => Excludes all types in all assemblies (nothing is instrumented)
    • /p:Exclude="[coverlet.*]Coverlet.Core.Coverage" => Excludes the Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)
    • /p:Exclude="[*]Coverlet.Core.Instrumentation.*" => Excludes all types belonging to Coverlet.Core.Instrumentation namespace in any assembly
    • /p:Exclude="[coverlet.*.tests?]*" => Excludes all types in any assembly starting with coverlet. and ending with .test or .tests (the ? makes the s optional)
    • /p:Exclude="[coverlet.*]*,[*]Coverlet.Core*" => Excludes assemblies matching coverlet.* and excludes all types belonging to the Coverlet.Core namespace in any assembly
    dotnet test /p:CollectCoverage=true /p:Exclude="[coverlet.*]Coverlet.Core.Coverage"

    Coverlet goes a step in the other direction by also letting you explicitly set what can be included using the Include property.

    Examples

    • /p:Include="[*]*" => Includes all types in all assemblies (everything is instrumented)
    • /p:Include="[coverlet.*]Coverlet.Core.Coverage" => Includes the Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)
    • /p:Include="[coverlet.*.tests?]*" => Includes all types in any assembly starting with coverlet. and ending with .test or .tests (the ? makes the s optional)

    Both Exclude and Include properties can be used together but Exclude takes precedence. You can specify multiple filter expressions by separting them with a comma (,).

    You can also include coverage of the test assembly itself by setting /p:IncludeTestAssembly to true.

    Note for Powershell / VSTS users

    To exclude or include multiple assemblies when using Powershell scripts or creating a .yaml file for a VSTS build %2c should be used as a separator. Msbuild will translate this symbol to ,.

    /p:Exclude="[*]*Examples?%2c[*]*Startup"

    VSTS builds do not require double quotes to be unescaped:

    dotnet test --configuration $(buildConfiguration) --no-build /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ /p:Exclude="[MyAppName.DebugHost]*%2c[MyAppNamet.WebHost]*%2c[MyAppName.App]*"
    

    Note for Linux users

    Note for Linux users

    There is an issue with MSBuild on Linux affects the ability to escape quotes while specifying multiple comma-separated values. Linux MSBuild automatically translates  to / in properties, tasks, etc. before using them, which means if you specified /p:CoverletOutputFormat="json,opencover" in an MSBuild script, it will be converted to /p:CoverletOutputFormat=/"json,opencover/" before execution. This yields an error similar to the following:

    MSBUILD : error MSB1006: Property is not valid. [/home/vsts/work/1/s/default.proj]
      Switch: opencover/
    

    You'll see this if directly consuming Linux MSBuild or if using the Azure DevOps MSBuild task on a Linux agent.

    The workaround is to use the .NET Core dotnet msbuild command instead of using MSBuild directly. The issue is not present in dotnet msbuild and the script will run with correctly escaped quotes.

    SourceLink

    Coverlet supports SourceLink custom debug information contained in PDBs. When you specify the --use-source-link flag in the global tool or /p:UseSourceLink=true property in the MSBuild command, Coverlet will generate results that contain the URL to the source files in your source control instead of absolute file paths.

    https://github.com/tonerdo/coverlet/blob/master/Documentation/GlobalTool.md

    Coverlet as a Global Tool

    To see a list of options, run:

    coverlet --help

    The current options are (output of coverlet --help):

    Cross platform .NET Core code coverage tool 1.0.0.0
    
    Usage: coverlet [arguments] [options]
    
    Arguments:
      <ASSEMBLY>  Path to the test assembly.
    
    Options:
      -h|--help                Show help information
      -v|--version             Show version information
      -t|--target              Path to the test runner application.
      -a|--targetargs          Arguments to be passed to the test runner.
      -o|--output              Output of the generated coverage report
      -v|--verbosity           Sets the verbosity level of the command. Allowed values are quiet, minimal, normal, detailed.
      -f|--format              Format of the generated coverage report[multiple value].
      --threshold              Exits with error if the coverage % is below value.
      --threshold-type         Coverage type to apply the threshold to[multiple value].
      --threshold-stat         Coverage statistic used to enforce the threshold value.
      --exclude                Filter expressions to exclude specific modules and types[multiple value].
      --include                Filter expressions to include specific modules and types[multiple value].
      --include-directory      Include directories containing additional assemblies to be instrumented[multiple value].
      --exclude-by-file        Glob patterns specifying source files to exclude[multiple value].
      --exclude-by-attribute   Attributes to exclude from code coverage[multiple value].
      --include-test-assembly  Specifies whether to report code coverage of the test assembly.
      --single-hit             Specifies whether to limit code coverage hit reporting to a single hit for each location.
      --merge-with             Path to existing coverage result to merge.
      --use-source-link        Specifies whether to use SourceLink URIs in place of file system paths.

    NB. For a [multiple value] options you have to specify values multiple times i.e.

    --exclude-by-attribute 'Obsolete' --exclude-by-attribute'GeneratedCode' --exclude-by-attribute 'CompilerGenerated'
    

    For --merge-with check the sample.

    Code Coverage

    The coverlet tool is invoked by specifying the path to the assembly that contains the unit tests. You also need to specify the test runner and the arguments to pass to the test runner using the --target and --targetargs options respectively. The invocation of the test runner with the supplied arguments must not involve a recompilation of the unit test assembly or no coverage data will be generated.

    The following example shows how to use the familiar dotnet test toolchain:

    coverlet /path/to/test-assembly.dll --target "dotnet" --targetargs "test /path/to/test-project --no-build"

    After the above command is run, a coverage.json file containing the results will be generated in the directory the coverlet command was run. A summary of the results will also be displayed in the terminal.

    Note: The --no-build flag is specified so that the /path/to/test-assembly.dll isn't rebuilt

    Coverage Output

    Coverlet can generate coverage results in multiple formats, which is specified using the --format or -f options. For example, the following command emits coverage results in the opencover format instead of json:

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --format opencover

    Supported Formats:

    • json (default)
    • lcov
    • opencover
    • cobertura
    • teamcity

    The --format option can be specified multiple times to output multiple formats in a single run:

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --format opencover --format lcov

    By default, Coverlet will output the coverage results file(s) in the current working directory. The --output or -o options can be used to override this behaviour.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --output "/custom/path/result.json"

    The above command will write the results to the supplied path, if no file extension is specified it'll use the standard extension of the selected output format. To specify a directory instead, simply append a / to the end of the value.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --output "/custom/directory/" -f json -f lcov

    TeamCity Output

    Coverlet can output basic code coverage statistics using TeamCity service messages.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --output teamcity

    The currently supported TeamCity statistics are:

    TeamCity Statistic KeyDescription
    CodeCoverageL Line-level code coverage
    CodeCoverageB Branch-level code coverage
    CodeCoverageM Method-level code coverage
    CodeCoverageAbsLTotal The total number of lines
    CodeCoverageAbsLCovered The number of covered lines
    CodeCoverageAbsBTotal The total number of branches
    CodeCoverageAbsBCovered The number of covered branches
    CodeCoverageAbsMTotal The total number of methods
    CodeCoverageAbsMCovered The number of covered methods

    Merging Results

    With Coverlet you can combine the output of multiple coverage runs into a single result.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --merge-with "/path/to/result.json" --format opencover

    The value given to --merge-with must be a path to Coverlet's own json result format.

    Threshold

    Coverlet allows you to specify a coverage threshold below which it returns a non-zero exit code. This allows you to enforce a minimum coverage percent on all changes to your project.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --threshold 80

    The above command will automatically fail the build if the line, branch or method coverage of any of the instrumented modules falls below 80%. You can specify what type of coverage to apply the threshold value to using the --threshold-type option. For example to apply the threshold check to only line coverage:

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --threshold 80 --threshold-type line

    You can specify the --threshold-type option multiple times. Valid values include linebranch and method.

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --threshold 80 --threshold-type line --threshold-type method

    By default, Coverlet will validate the threshold value against the coverage result of each module. The --threshold-stat option allows you to change this behaviour and can have any of the following values:

    • Minimum (Default): Ensures the coverage result of each module isn't less than the threshold
    • Total: Ensures the total combined coverage result of all modules isn't less than the threshold
    • Average: Ensures the average coverage result of all modules isn't less than the threshold

    The following command will compare the threshold value with the overall total coverage of all modules:

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --threshold 80 --threshold-type line --threshold-stat total

    Excluding From Coverage

    Attributes

    You can ignore a method or an entire class from code coverage by creating and applying the ExcludeFromCodeCoverage attribute present in the System.Diagnostics.CodeAnalysis namespace.

    You can also ignore additional attributes by using the ExcludeByAttribute property (short name or full name supported):

    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --exclude-by-attribute 'Obsolete' --exclude-by-attribute'GeneratedCode' --exclude-by-attribute 'CompilerGenerated'

    Source Files

    You can also ignore specific source files from code coverage using the --exclude-by-file option

    • Can be specified multiple times
    • Use absolute or relative paths (relative to the project directory)
    • Use file path or directory path with globbing (e.g dir1/*.cs)
    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --exclude-by-file "../dir1/class1.cs"

    Filters

    Coverlet gives the ability to have fine grained control over what gets excluded using "filter expressions".

    Syntax: --exclude '[Assembly-Filter]Type-Filter'

    Wildcards

    • * => matches zero or more characters
    • ? => the prefixed character is optional

    Examples

    • --exclude "[*]*" => Excludes all types in all assemblies (nothing is instrumented)
    • --exclude "[coverlet.*]Coverlet.Core.Coverage" => Excludes the Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)
    • --exclude "[*]Coverlet.Core.Instrumentation.*" => Excludes all types belonging to Coverlet.Core.Instrumentation namespace in any assembly
    • --exclude "[coverlet.*.tests?]*" => Excludes all types in any assembly starting with coverlet. and ending with .test or .tests (the ? makes the s optional)
    • --exclude "[coverlet.*]*" --exclude "[*]Coverlet.Core*" => Excludes assemblies matching coverlet.* and excludes all types belonging to the Coverlet.Core namespace in any assembly
    coverlet <ASSEMBLY> --target <TARGET> --targetargs <TARGETARGS> --exclude "[coverlet.*]Coverlet.Core.Coverage"

    Coverlet goes a step in the other direction by also letting you explicitly set what can be included using the --include option.

    Examples

    • --include "[*]*" => Includes all types in all assemblies (everything is instrumented)
    • --include "[coverlet.*]Coverlet.Core.Coverage" => Includes the Coverage class in the Coverlet.Core namespace belonging to any assembly that matches coverlet.* (e.g coverlet.core)
    • --include "[coverlet.*.tests?]*" => Includes all types in any assembly starting with coverlet. and ending with .test or .tests (the ? makes the s optional)

    Both --exclude and --include options can be used together but --exclude takes precedence. You can specify the --exclude and --include options multiple times to allow for multiple filter expressions.

    You can also include coverage of the test assembly itself by specifying the --include-test-assembly flag.

    Exit Codes

    Coverlet outputs specific exit codes to better support build automation systems for determining failures and take action on it.

    0 - Success.
    1 - If any test fails.
    2 - Coverage percentage is below threshold.
    3 - Test fails and also coverage percentage is below threshold.
    101 - General exception occured during coverlet process.
    102 - Missing options or invalid arguments for coverlet process.

    https://github.com/tonerdo/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md

    Consume nightly build

    Consume nightly build

    You can check metadata of nightly build packages here:

    Msbuild https://www.myget.org/feed/coverlet-dev/package/nuget/coverlet.msbuild
    VSTest collector https://www.myget.org/feed/coverlet-dev/package/nuget/coverlet.collector
    .Net tools https://www.myget.org/feed/coverlet-dev/package/nuget/coverlet.console

    To consume nightly build create a NuGet.Config on your root solution directory and add following content

    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <packageSources>
        <clear />
        <!-- Coverlet nightly build feed -->
        <add key="coverletNightly" value="https://www.myget.org/F/coverlet-dev/api/v3/index.json" /> 
        <!-- Defaul nuget feed -->
        <add key="nuget" value="https://api.nuget.org/v3/index.json" /> 
        <!-- Add all other needed feed -->
      </packageSources>
    </configuration>

    Install packages

    You can install nightly package using visual studio

    File

    Nuget(PM console)

    PM> Install-Package coverlet.msbuild -Version 2.6.25-g6209239d69 -Source https://www.myget.org/F/coverlet-dev/api/v3/index.json
    

    .NET CLI

     dotnet add package coverlet.msbuild --version 2.6.25-g6209239d69 --source https://www.myget.org/F/coverlet-dev/api/v3/index.json
    

    .csproj

    <PackageReference Include="coverlet.msbuild" Version="2.6.25-g6209239d69" />
    

    How to verify version

    You can understand which version you're using comparing nightly build release date and repo commits.
    For instance if we want to consume last msbuild nightly build:

    As you can see we build at 00.00 UTC and build takes some seconds, so it's possible that release date won't be the same of commit repo.

  • 相关阅读:
    Django集成Markdown编辑器【附源码】
    Django+JWT实现Token认证
    Docker环境的持续部署优化实践
    2018-行远自迩,登高自卑
    SVN Hooks的介绍及使用
    Django开发密码管理表实例【附源码】
    Django+Echarts画图实例
    Django使用Signals监测model字段变化发送通知
    运维效率之数据迁移自动化
    Python之路(第三十四篇) 网络编程:验证客户端合法性
  • 原文地址:https://www.cnblogs.com/panpanwelcome/p/12672669.html
Copyright © 2011-2022 走看看