TestNG源代码分析：依赖管理的实现

TestNG源代码分析：依赖管理的实现

2018-03-19

1 背景

当case之间有依赖关系，有依赖关系的case，它们的执行顺序是有限制的。TestNG提供了依赖管理功能

2 基础理论

这个执行顺序可以用拓扑排序算法实现。

只要是有向无环图就能被拓扑排序，拓扑排序维基典型实现算法：

L ← Empty list that will contain the sorted elements
S ← Set of all nodes with no incoming edges
while S is non-empty do
    remove a node n from S
    insert n into L
    foreach node m with an edge e from n to m do
        remove edge e from thegraph
        if m has no other incoming edges then
            insert m into S
if graph has edges then
    return error (graph has at least one cycle)
else 
    return L (a topologically sorted order)

TestNG依赖管理功能

• dependsOnMethods

@Test  
public void serverStartedOk(){}  
@Test(dependsOnMethods={ "serverStartedOk" })  
public void method1() {}  

• dependsOnGroups

@Test(groups = { "init" })  
public void serverStartedOk(){}  
@Test(groups = { "init" })  
public void initEnvironment(){}  
@Test(dependsOnGroups = { "init.*})  
public void method1() {} 

3 TestNG依赖管理源代码

如何找到依赖原理源代码？

通过报错，即即故意制造一个循环依赖，然后看stack trace：

错误代码如下：

    @Test(dependsOnMethods={"MethodA"})
    public void MethodA(){}

Stack Trace

at org.testng.internal.Graph.topologicalSort(Graph.java:143)
at org.testng.internal.MethodHelper.topologicalSort(MethodHelper.java:231)

org.testng.TestNGException: 
The following methods have cyclic dependencies:
Sample.MethodA()[pri:0, instance:Demo.Sample@50c87b21]

    at org.testng.internal.Graph.topologicalSort(Graph.java:143)
    at org.testng.internal.MethodHelper.topologicalSort(MethodHelper.java:231)
    at org.testng.internal.MethodHelper.sortMethods(MethodHelper.java:287)
    at org.testng.internal.MethodHelper.collectAndOrderMethods(MethodHelper.java:60)
    at org.testng.TestRunner.initMethods(TestRunner.java:464)
    at org.testng.TestRunner.init(TestRunner.java:247)
    at org.testng.TestRunner.init(TestRunner.java:217)
    at org.testng.TestRunner.<init>(TestRunner.java:169)
    at org.testng.remote.support.RemoteTestNG6_9_10$1.newTestRunner(RemoteTestNG6_9_10.java:28)
    at org.testng.remote.support.RemoteTestNG6_9_10$DelegatingTestRunnerFactory.newTestRunner(RemoteTestNG6_9_10.java:61)
    at org.testng.SuiteRunner$ProxyTestRunnerFactory.newTestRunner(SuiteRunner.java:594)
    at org.testng.SuiteRunner.init(SuiteRunner.java:168)
    at org.testng.SuiteRunner.<init>(SuiteRunner.java:117)
    at org.testng.TestNG.createSuiteRunner(TestNG.java:1339)
    at org.testng.TestNG.createSuiteRunners(TestNG.java:1326)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1180)
    at org.testng.TestNG.runSuites(TestNG.java:1104)
    at org.testng.TestNG.run(TestNG.java:1076)
    at org.testng.remote.AbstractRemoteTestNG.run(AbstractRemoteTestNG.java:126)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:152)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:57)

4 源代码分析

//MethodHelper.topologicalSort
private static Graph<ITestNGMethod> topologicalSort(ITestNGMethod[] methods, List<ITestNGMethod> sequentialList, List<ITestNGMethod> parallelList)
//Graph.topologicalSort
public void topologicalSort()

注意：

• methods变量包括所有的要跑的用例和用例依赖的用例（依赖用例依赖的用例也包括）
• 依赖管理代码在Graph类中实现，MethodHelper.topologicalSort方法会调用Graph类中的方法

Graph类有三个变量

  //m_nodes:放的是所有的节点的引用
  private Map<T, Node<T>> m_nodes = Maps.newLinkedHashMap();
  //m_independentNodes:所有独立节点的引用，这样的节点（用例）可并发运行
  private List<T> m_strictlySortedNodes = null;
  //m_strictlySortedNodes:经过严格排序的非独立节点
  private Map<T, Node<T>> m_independentNodes = null;

Graph.topologicalSort方法实现依赖节点的拓扑排序

//m_nodes:放的是所有的节点(用例)的引用
//m_independentNodes:所有独立节点的引用
//m_strictlySortedNodes:经过严格排序的非独立节点
public void topologicalSort() {
    ppp("================ SORTING");
    //m_strictlySortedNodes集合: 最后的结果放到这个集合中
    m_strictlySortedNodes=Lists.newArrayList(); 
    initializeIndependentNodes();
    
    //nodes2集合: 非独立的节点
    List<Node<T>>nodes2 =Lists.newArrayList();
    
    //1 创建非独立节点集合，即存在依赖关系的方法的集合
    for (Node<T> n :getNodes()) {  //getNodes()返回m_nodes
        if (!isIndependent(n.getObject())){// 判断该节点是否独立，如果不独立的话，添加到nodes2中
            ppp("ADDING FOR SORT: " +n.getObject());
            nodes2.add(n.clone()); //使用的是clone方法来进行对象的复制，一般不推荐使用clone方法，参见Effective Java Item 11
        }
        else {
            ppp("SKIPPING INDEPENDENT NODE" + n);
        }
    }
    
    //2 将非独立的节点集合排序，为了让属于同类中的方法在集合中的位置近一些，从而在调用的顺序上能够相邻一些
    Collections.sort(nodes2);
    
    //3 进行拓扑排序，如果发现有循环依赖，马上抛出异常
    while (!nodes2.isEmpty()) {
        Node<T> node =findNodeWithNoPredecessors(nodes2); // 从nodes2集合中找到没有前驱节点的节点
        if (null == node) {   // 如果没有找到节点，那么创建一个Tarjan对象来得到一个cycle
            List<T> cycle =newTarjan<T>(this,nodes2.get(0).getObject()).getCycle(); // 这里实现了Tarjan算法，用来得到环的路径信息
            StringBuffer sb = new StringBuffer();  //在非并发环境中应该尽量使用StringBuilder
            sb.append("The following methodshave cyclic dependencies:
");
            for (T m : cycle) {
                sb.append(m).append("
");
            }
            throw newTestNGException(sb.toString());
        }
        else {   //如果找到了，将这个没有任何前驱节点的节点放到结果结合中，然后从nodes2集合中删除该节点
            m_strictlySortedNodes.add(node.getObject());
            removeFromNodes(nodes2, node);
        }
    }
    ppp("===============DONESORTING");
    if (m_verbose) {
        dumpSortedNodes();
    }
}

调用方法

private void initializeIndependentNodes() {
    if (null == m_independentNodes) {
        List<Node<T>> list = Lists.newArrayList(m_nodes.values());
        Collections.sort(list);
        m_independentNodes = Maps.newLinkedHashMap();
        for (Node<T> node : list) {
            m_independentNodes.put(node.getObject(), node);
        }
    }
}

private Collection<Node<T>> getNodes() {
    return m_nodes.values();
}

MethodHelper.topologicalSort方法，调用Graph方法区分独立节点和依赖节点，并调用toplogicalSort进行依赖节点拓扑排序

private static Graph<ITestNGMethod> topologicalSort(ITestNGMethod[] methods, List<ITestNGMethod> sequentialList, List<ITestNGMethod> parallelList) {
  Graph<ITestNGMethod> result = new Graph<>();
  if (methods.length == 0) {
    return result;  //如果传入的methods数组长度为0，直接返回了空的graph
  }
  
  // 区分出要顺序执行的用例（依赖用例）和可并行的用例（独立用例）
  for (ITestNGMethod m : methods) {
    result.addNode(m); //对于每个方法instance，添加到graph中    
    
    List<ITestNGMethod> predecessors = Lists.newArrayList(); //获得该方法依赖的方法
    String[] methodsDependedUpon = m.getMethodsDependedUpon();
    String[] groupsDependedUpon = m.getGroupsDependedUpon();
    if (methodsDependedUpon.length > 0) {
      ITestNGMethod[] methodsNamed =
        MethodHelper.findDependedUponMethods(m, methods);
      for (ITestNGMethod pred : methodsNamed) {
        predecessors.add(pred);
      }
    }
    if (groupsDependedUpon.length > 0) {
      for (String group : groupsDependedUpon) {
        ITestNGMethod[] methodsThatBelongToGroup =
          MethodGroupsHelper.findMethodsThatBelongToGroup(m, methods, group);
        for (ITestNGMethod pred : methodsThatBelongToGroup) {
          predecessors.add(pred);
        }
      }
    }

    for (ITestNGMethod predecessor : predecessor s) {
      result.addPredecessor(m, predecessor); //将依赖方法加入graph中
    }
  }

  result.topologicalSort();
  sequentialList.addAll(result.getStrictlySortedNodes());
  parallelList.addAll(result.getIndependentNodes());

  return result;
}

调用方法

public void addNode(T tm) {
    ppp("ADDING NODE " + tm + " " + tm.hashCode());
    m_nodes.put(tm, new Node<>(tm));
    // Initially, all the nodes are put in the independent list as well
}

public void addPredecessor(T tm, T predecessor) {
    Node<T> node = findNode(tm);
    if (null == node) {
      throw new TestNGException("Non-existing node: " + tm);
    }
    else {
      node.addPredecessor(predecessor);
      addNeighbor(tm, predecessor);
      // Remove these two nodes from the independent list
      initializeIndependentNodes();
      m_independentNodes.remove(predecessor);
      m_independentNodes.remove(tm);
      ppp("  REMOVED " + predecessor + " FROM INDEPENDENT OBJECTS");
    }
}

public Set<T> getIndependentNodes() {
    return m_independentNodes.keySet();
}

public List<T> getStrictlySortedNodes() {
    return m_strictlySortedNodes;
}

参考

[1] TestNG源代码分析 --- 依赖管理的实现（一）

[2] TestNG源代码分析 --- 依赖管理的实现（二）