TestNG源码解析3 非并发下的执行流程

TestNG源码解析3 非并发下的执行流程

本文介绍

非并发下的执行流程，首先将XmlSuite交给SuiteRunnerWorker去执行测试用例，而该woker是调用suiteRunner.run()方法【5】，然后触发ISuiteListener监听器【6】，进而在触发所有用例的@BeforeSuite以及结束后的@AfterSuite方法【7】,在SuiteRunner中循环调用TestRunner的run方法【8】,将对应的XmlTest所有的测试用例转换为graph节点，然后对每个节点交给对应的IWorker，调用worker.run()去执行【10】，先执行@BeforeClass 然后执行测试方法，最后执行@AfterClass【11】,最终交给TestInvoke去通过反射执行测试用例方法【13】

• 大概的流程

ISuiteListener.start();
@BeforeSuite
单个XmlTest-@BeforeClass
单个XmlTest-@BeforeGroup
单个XmlTest-@BeforeMethod
测试用例方法
单个XmlTest-@AfterMethod
单个XmlTest-@AfterGroup
单个XmlTest-@AfterClass
@AfterSuite
ISuiteListener.finish();

入口代码

public void run() {
    // 初始化xml数据，监听器
    initializeEverything();
    //检测测试用例是否合法
    sanityCheck();
    // 执行监听器start
    runExecutionListeners(true /* start */);
    runSuiteAlterationListeners();
    // 开始执行时间
    m_start = System.currentTimeMillis();
    // 这里就开始初始化测试对象并进行执行
    List<ISuite> suiteRunners = runSuites();
    // 执行结束时间
    m_end = System.currentTimeMillis();
    // 生成报告
    if (null != suiteRunners) {
      generateReports(suiteRunners);
    }
    // 监听器结束 finish
    runExecutionListeners(false /* finish */);
    exitCode = this.exitCodeListener.getStatus();
    if (exitCodeListener.noTestsFound()) {
      if (TestRunner.getVerbose() > 1) {
        System.err.println("[TestNG] No tests found. Nothing was run");
        usage();
      }
    }
    m_instance = null;
    m_jCommander = null;
  }

1. TestNG#initializeEverything()

public void initializeEverything() {
    if (m_isInitialized) {
      return;
    }
    // 解析xml里的XmlSuite（从xml或者命令行传来的路径里）
    initializeSuitesAndJarFile();
    // 设置类加载器之类的
    initializeConfiguration();
    // 开启监听器
    initializeDefaultListeners();
    initializeCommandLineSuites();
    initializeCommandLineSuitesParams();
    initializeCommandLineSuitesGroups();

    m_isInitialized = true;
  }

2. TestNG#runSuites()

protected List<ISuite> runSuites() {
    return runSuitesLocally();
}

3. TestNG#runSuitesLocally()

分析非并发执行的测试用例执行过程，具体执行过程见方法4

public List<ISuite> runSuitesLocally() {
    // 没有测试用例需要执行 直接返回
    if (m_suites.isEmpty()) {
      error("No test suite found. Nothing to run");
      usage();
      return Collections.emptyList();
    }
    // 定义一个SuiteRunnerMap
    SuiteRunnerMap suiteRunnerMap = new SuiteRunnerMap();

    if (m_suites.get(0).getVerbose() >= 2) {
      Version.displayBanner();
    }

    // First initialize the suite runners to ensure there are no configuration issues.
    // Create a map with XmlSuite as key and corresponding SuiteRunner as value
    // 将测试类进行实例化， 将测试类实例装入testRunners,有几个testCase就装入几个testRunner
    // 一个site节点对应一个suiteRunner，注册一些监听器
    for (XmlSuite xmlSuite : m_suites) {
      createSuiteRunners(suiteRunnerMap, xmlSuite);
    }
    // Run suites
    // 非并发的用例集都在这里执行
    if (m_suiteThreadPoolSize == 1 && !m_randomizeSuites) {
      // Single threaded and not randomized: run the suites in order
      for (XmlSuite xmlSuite : m_suites) {
          // 用例执行
        runSuitesSequentially(
            xmlSuite, suiteRunnerMap, getVerbose(xmlSuite), getDefaultSuiteName());
      }
      //
      // Generate the suites report
      //
      return Lists.newArrayList(suiteRunnerMap.values());
    }
    return Lists.newArrayList(suiteRunnerMap.values());
  }

4. TestNG#runSuitesSequentially()

将测试集合生成对应的SuiteRunnerWorker，用worker去执行用例(方法5)

private void runSuitesSequentially(
      XmlSuite xmlSuite, SuiteRunnerMap suiteRunnerMap, int verbose, String defaultSuiteName) {
    // 循环递归子元素
    for (XmlSuite childSuite : xmlSuite.getChildSuites()) {
      runSuitesSequentially(childSuite, suiteRunnerMap, verbose, defaultSuiteName);
    }
    // 生成Suite执行Worker
    SuiteRunnerWorker srw =
        new SuiteRunnerWorker(
            suiteRunnerMap.get(xmlSuite), suiteRunnerMap, verbose, defaultSuiteName);
    srw.run();
  }

5. SuiteRunnerWorker#runSuite()

这里就是执行的是 suiteRunner的run方法（方法6）

private void runSuite(SuiteRunnerMap suiteRunnerMap /* OUT */, XmlSuite xmlSuite) {
    if (m_verbose > 0) {
      String allFiles =
          "  "
              + (xmlSuite.getFileName() != null ? xmlSuite.getFileName() : m_defaultSuiteName)
              + '\n';
      Utils.log("TestNG", 0, "Running:\n" + allFiles);
    }

    SuiteRunner suiteRunner = (SuiteRunner) suiteRunnerMap.get(xmlSuite);
    // 先执行继承了ISuiteListener的监听器
    suiteRunner.run();
  }

6. SuiteRunner#run()

执行ISuiteListener的监听器onStart() 测试方法(具体看方法7) onFinsh()方法

public void run() {
    // 执行所有 ISuiteListener的start方法
    invokeListeners(true /* start */);
    try {
      privateRun();
    } finally {
      // 执行所有 ISuiteListener的stop方法
      invokeListeners(false /* stop */);
 	}
  }

7. SuiteRunner#privateRun()

循环所有的TestRunner(测试用例)，统计所有的@BeforeSuite，@AfterSuite放入LinkedMap中

先依次执行所有的用例中的@BeforeSuite，再执行runSequentially()[见方法8]，最后依次执行所有的用例中的@AfterSuite

private void privateRun() {
    // Map for unicity, Linked for guaranteed order
    Map<Method, ITestNGMethod> beforeSuiteMethods = new LinkedHashMap<>();
    Map<Method, ITestNGMethod> afterSuiteMethods = new LinkedHashMap<>();
    IInvoker invoker = null;
    // Get the invoker and find all the suite level methods
    // 将suite的前置和后置方法加入局部Map中
    for (TestRunner tr : testRunners) {
      // TODO: Code smell.  Invoker should belong to SuiteRunner, not TestRunner
      // -- cbeust
      invoker = tr.getInvoker();

      for (ITestNGMethod m : tr.getBeforeSuiteMethods()) {
        beforeSuiteMethods.put(m.getConstructorOrMethod().getMethod(), m);
      }

      for (ITestNGMethod m : tr.getAfterSuiteMethods()) {
        afterSuiteMethods.put(m.getConstructorOrMethod().getMethod(), m);
      }
    }
    // Invoke beforeSuite methods (the invoker can be null
    // if the suite we are currently running only contains
    // a <file-suite> tag and no real tests)
    // 执行@BeforeSuite 执行
    if (invoker != null) {
      if (!beforeSuiteMethods.values().isEmpty()) {
        ConfigMethodArguments arguments = new Builder()
            .usingConfigMethodsAs(beforeSuiteMethods.values())
            .forSuite(xmlSuite)
            .usingParameters(xmlSuite.getParameters())
            .build();
        invoker.getConfigInvoker().invokeConfigurations(arguments);
      }
      // Run all the test runners
      //
      boolean testsInParallel = XmlSuite.ParallelMode.TESTS.equals(xmlSuite.getParallel());
      if (RuntimeBehavior.strictParallelism()) {
        testsInParallel = !XmlSuite.ParallelMode.NONE.equals(xmlSuite.getParallel());
      }
      if (testsInParallel) {
        runInParallelTestMode();
      } else {
          // 执行测试用例
        runSequentially();
      }
      // Invoke afterSuite methods
      // 执行@AfterSuite
      if (!afterSuiteMethods.values().isEmpty()) {
        ConfigMethodArguments arguments = new Builder()
            .usingConfigMethodsAs(afterSuiteMethods.values())
            .forSuite(xmlSuite)
            .usingParameters(xmlSuite.getAllParameters())
            .build();
        invoker.getConfigInvoker().invokeConfigurations(arguments);
      }
    }
  }

8. SuiteRunner#runSequentially()

将测试集中的所有testRunner执行runTest(tr)方法，见方法9

 private void runSequentially() {
    for (TestRunner tr : testRunners) {
      runTest(tr);
    }
  }

9. SuiteRunner#runTest()

testRunner自己调用run方法即可 具体看方法10

 private void runTest(TestRunner tr) {
    visualisers.forEach(tr::addListener);
    tr.run();
     // 获取执行结果
    ISuiteResult sr = new SuiteResult(xmlSuite, tr);
    synchronized (suiteResults) {
      suiteResults.put(tr.getName(), sr);
    }
  }

10. TestRunner#privateRun()

将所有的测试用例转换为graph节点，然后对每个节点交给对应的IWorker，调用worker.run()去执行

private void privateRun(XmlTest xmlTest) {
    boolean parallel = xmlTest.getParallel().isParallel();// 是否并行
    // parallel
    int threadCount = parallel ? xmlTest.getThreadCount() : 1;
    // Make sure we create a graph based on the intercepted methods, otherwise an interceptor
    // removing methods would cause the graph never to terminate (because it would expect
    // termination from methods that never get invoked).
    ITestNGMethod[] interceptedOrder = intercept(getAllTestMethods());
    AtomicReference<IDynamicGraph<ITestNGMethod>> reference = new AtomicReference<>();
    TimeUtils.computeAndShowTime("DynamicGraphHelper.createDynamicGraph()",
        () -> {
          IDynamicGraph<ITestNGMethod> ref = DynamicGraphHelper
              .createDynamicGraph(interceptedOrder, getCurrentXmlTest());
          reference.set(ref);
        }
    );
    IDynamicGraph<ITestNGMethod> graph = reference.get();

    graph.setVisualisers(this.visualisers);
    // In some cases, additional sorting is needed to make sure tests run in the appropriate order.
    // If the user specified a method interceptor, or if we have any methods that have a non-default
    // priority on them, we need to sort.
    boolean needPrioritySort = sortOnPriority(interceptedOrder);
    Comparator<ITestNGMethod> methodComparator = newComparator(needPrioritySort);
    List<ITestNGMethod> freeNodes = graph.getFreeNodes();

    if (graph.getNodeCount() > 0 && freeNodes.isEmpty()) {
      throw new TestNGException("No free nodes found in:" + graph);
    }

    while (!freeNodes.isEmpty()) {
      if (needPrioritySort) {
        freeNodes.sort(methodComparator);
        // Since this is sequential, let's run one at a time and fetch/sort freeNodes after each method.
        // Future task: To optimize this, we can only update freeNodes after running a test that another test is dependent upon.
        freeNodes = freeNodes.subList(0, 1);
      }
      // 每个测试用例都会有一个worker，除了并行的外
      List<IWorker<ITestNGMethod>> workers = createWorkers(freeNodes);
      for (IWorker<ITestNGMethod> worker : workers) {
        worker.run();
      }
      //createWorkers(freeNodes).forEach(Runnable::run);
      graph.setStatus(freeNodes, IDynamicGraph.Status.FINISHED);
      freeNodes = graph.getFreeNodes();
    }
  }

11. TestMethodWorker#run()

先执行@BeforeClass 然后执行测试方法，最后执行@AfterClass

invokeTestMethods里就是交给TestInvoker.java去处理的具体看方法12，13

 public void run() {
    for (IMethodInstance testMthdInst : m_methodInstances) {
      ITestNGMethod testMethod = testMthdInst.getMethod();
      // 执行@BeforeClass
      if (canInvokeBeforeClassMethods()) {
        synchronized (testMethod.getInstance()) {
          invokeBeforeClassMethods(testMethod.getTestClass(), testMthdInst);
        }
      }

      // Invoke test method
      try {
        // 执行@BeforeGroup @BeforeMethod  当前方法 @AfterMethod @AfterGroup
        invokeTestMethods(testMethod, testMthdInst.getInstance());
      } finally {
        // 执行@AfterClass
        invokeAfterClassMethods(testMethod.getTestClass(), testMthdInst);
      }
    }
  }

12. TestMethodWorker#invokeTestMethods()

交给testInvoker去执行测试用例 具体见方法13

 protected void invokeTestMethods(ITestNGMethod tm, Object instance) {
    // Potential bug here:  we look up the method index of tm among all
    // the test methods (not very efficient) but if this method appears
    // several times and these methods are run in parallel, the results
    // are unpredictable...  Need to think about this more (and make it
    // more efficient)
    List<ITestResult> testResults =
        m_testInvoker.invokeTestMethods(tm, m_groupMethods, instance, m_testContext);

    if (testResults != null) {
      m_testResults.addAll(testResults);
    }
  }

13. TestInvoker#invokeTestMethods()

执行具体测试用例的顺序 @BeforeGroup @BeforeMethod 当前方法 @AfterMethod @AfterGroup

然后是交给 TestInvoker#MethodInvocationAgent.class去执行，最终还是在TestInvoker中的方法去执行，原理是反射.调用MethodInvocationHelper.invokeMethod()

protected static Object invokeMethod(Method thisMethod, Object instance, Object[] parameters)
   throws InvocationTargetException, IllegalAccessException {
thisMethod = clazz.getDeclaredMethod(thisMethod.getName(), thisMethod.getParameterTypes());
}

public List<ITestResult> invokeTestMethods(ITestNGMethod testMethod,
      ConfigurationGroupMethods groupMethods,
      Object instance,
      ITestContext context) {
    Map<String, String> parameters = testMethod.findMethodParameters(context.getCurrentXmlTest());
    // By the time this testMethod to be invoked,
    // all dependencies should be already run or we need to skip this method,
    // so invocation count should not affect dependencies check
    String okToProceed = checkDependencies(testMethod);
    // For invocationCount > 1 and threadPoolSize > 1 run this method in its own pool thread.
    if (testMethod.getInvocationCount() > 1 && testMethod.getThreadPoolSize() > 1) {
      return invokePooledTestMethods(testMethod, parameters, groupMethods, context);
    }

    long timeOutInvocationCount = testMethod.getInvocationTimeOut();
    // FIXME: Is this correct?
    boolean onlyOne = testMethod.getThreadPoolSize() > 1 || timeOutInvocationCount > 0;


    ITestClass testClass = testMethod.getTestClass();
    // @BeforeMethod
    ITestNGMethod[] beforeMethods =
        TestNgMethodUtils.filterBeforeTestMethods(testClass, CAN_RUN_FROM_CLASS);
      // @AfterMethod
    ITestNGMethod[] afterMethods =
        TestNgMethodUtils.filterAfterTestMethods(testClass, CAN_RUN_FROM_CLASS);
    // 执行次数 就是invocationCount参数
    int invocationCount = onlyOne ? 1 : testMethod.getInvocationCount();

    TestMethodArguments arguments = new TestMethodArguments.Builder()
        .usingInstance(instance)
        .forTestMethod(testMethod)
        .withParameters(parameters)
        .forTestClass(testClass)
        .usingBeforeMethods(beforeMethods)
        .usingAfterMethods(afterMethods)
        .usingGroupMethods(groupMethods)
        .build();
    MethodInvocationAgent agent = new MethodInvocationAgent(arguments, this, context);
    while (invocationCount-- > 0) {
        // 执行测试用例
      invocationCount = agent.invoke(invocationCount);
    }

    return agent.getResult();
  }

注意

1. 对于测试类的静态变量，由于执行测试用例时，类都是同一个Object，所以静态变量只有一份；

   class Test {
        private static int a = 10;
   
       @Test
       private void test1 () {
           a++;
           System.out.println("test1:"+a);
       }
   
       @Test
       private void test2 () {
           a++;
           System.out.println("test2:"+a);
       }
   }
   

    <test name="Test_test" timeout="600000">
       <classes>
         <class name="XXX.Test">
           <methods>
             <include name="test1"/>
           </methods>
         </class>
         <class name="XXX.Test">
           <methods>
             <include name="test2"/>
           </methods>
         </class>
       </classes>
     </test>
   <!-- 执行结果 -->
   <!-- test1:11 -->
   <!-- test2:12 -->
   

    <test name="Test_test1" timeout="600000">
       <classes>
         <class name="XXX.Test">
           <methods>
             <include name="test1"/>
           </methods>
         </class>
       </classes>
     </test>
     <test name="Test_test2" timeout="600000">
       <classes>
         <class name="XXX.Test">
           <methods>
             <include name="test2"/>
           </methods>
         </class>
       </classes>
     </test>
   <!-- 执行结果 -->
   <!-- test1:11 -->
   <!-- test2:12 -->