TestNG源码解析4 单个用例(Method级)并发下的执行流程

TestNG源码解析4 单个用例(Method级)并发下的执行流程

本文介绍

@Test(threadPoolSize = 3, invocationCount=10)

单个测试用例并发下的执行流程，他和非并发的用例执行类似，前12个方法均一样，从第13个方法开始进行单用例并发逻辑。首先将XmlSuite交给SuiteRunnerWorker去执行测试用例，而该woker是调用suiteRunner.run()方法【5】，然后触发ISuiteListener监听器【6】，进而在触发所有用例的@BeforeSuite以及结束后的@AfterSuite方法【7】,在SuiteRunner中循环调用TestRunner的run方法【8】,将对应的XmlTest所有的测试用例转换为graph节点，然后对每个节点交给对应的IWorker，调用worker.run()去执行【10】，先执行@BeforeClass 然后执行测试方法，最后执行@AfterClass【11】。

从方法【13】开始，单用例并发就执行【14】,他会将这条用例clone出并发次数个,但对应实例对象都是同一个（相当于并发下调用同一个对象的某个测试方法N次），然后对每个用例的执行次数和线程并发数设置为1，加入workers集合中，然后先执行@BeforeGroup 在执行并发，后执行@AfterGroup【15】。在并发执行时，实际上是内部创建了一个线程池，然后循环调用workers集合中的run()方法，而这个run()方法其实就是方法【11】,进而执行到方法【13】的时候，由于线程数和并发数都是1所以直接走非并发执行逻辑了，就完成了单用例并发流程。

• 大概的流程

ISuiteListener.start();
@BeforeSuite
单个XmlTest-@BeforeClass
单个XmlTest-@BeforeGroup

// 并发
单个XmlTest-@BeforeGroup
单个XmlTest-@BeforeMethod
测试用例方法
单个XmlTest-@AfterMethod
单个XmlTest-@AfterGroup
// 并发
单个XmlTest-@BeforeGroup
单个XmlTest-@BeforeMethod
测试用例方法
单个XmlTest-@AfterMethod
单个XmlTest-@AfterGroup
// 并发...
    
单个XmlTest-@AfterGroup
单个XmlTest-@AfterClass
@AfterSuite
ISuiteListener.finish();

常见的几种并发

这里的描述较为通俗，比如线程池有2个线程，但有10个方法需要在这个线程池中执行，那么肯定有多个方法在同一个线程中，但我在下面描述的不同线程中运行就是为了突出区分！！！

tests级别的并发

<suite name="SuiteA" parallel="tests" thread-count="2">
    <test>线程A执行</test>
    <test>线程B执行</test>
</suite>

同一test标签下的class用例在同一个线程中执行，不同test标签在不同线程中执行

classes级别的并发

<suite name="SuiteA" parallel="classes" thread-count="3">
    <test>
        <classes>
            <class>
            	线程A
            </class>
            <class>
                线程B
            </class>
        </classes>
    </test>
    <test>
        <classes>
            <class>
            	线程C
            </class>
        </classes>
    </test>
</suite>

同一class标签下的所有method用例在同一线程运行，不同class标签之间在不同的线程运行

method级别的并发

<suite name="SuiteA" parallel="methods" thread-count="3">
    <test>
        <classes>
            <class>
            	<methods>
                	<include>线程A</include>
                </methods>
                <methods>
                	<include>线程B</include>
                </methods>
            </class>
            <class>
                <methods>
                	<include>线程C</include>
                </methods>
            </class>
        </classes>
    </test>
</suite>

每一个Method，也就是每一个被@Test标记的用例都单独在一个线程中，本文分析的就是该类型的执行流程。

<suite name="SuiteA" parallel="instances" >
    <test name="test">
        <classes>
            <class name="ObjectFactory"/>
        </classes>
    </test>
</suite>

每个实例的所有方法都运行在同一个线程内，不同实例运行在不同线程中。

入口代码

public void run() {
    // 初始化xml数据，监听器
    initializeEverything();
    //检测测试用例是否合法
    sanityCheck();
    // 执行监听器start
    runExecutionListeners(true /* start */);
    runSuiteAlterationListeners();
    // 开始执行时间
    m_start = System.currentTimeMillis();
    // 这里就开始初始化测试对象并进行执行
    List<ISuite> suiteRunners = runSuites();
    // 执行结束时间
    m_end = System.currentTimeMillis();
    // 生成报告
    if (null != suiteRunners) {
      generateReports(suiteRunners);
    }
    // 监听器结束 finish
    runExecutionListeners(false /* finish */);
    exitCode = this.exitCodeListener.getStatus();
    if (exitCodeListener.noTestsFound()) {
      if (TestRunner.getVerbose() > 1) {
        System.err.println("[TestNG] No tests found. Nothing was run");
        usage();
      }
    }
    m_instance = null;
    m_jCommander = null;
  }

1. TestNG#initializeEverything()

public void initializeEverything() {
    if (m_isInitialized) {
      return;
    }
    // 解析xml里的XmlSuite（从xml或者命令行传来的路径里）
    initializeSuitesAndJarFile();
    // 设置类加载器之类的
    initializeConfiguration();
    // 开启监听器
    initializeDefaultListeners();
    initializeCommandLineSuites();
    initializeCommandLineSuitesParams();
    initializeCommandLineSuitesGroups();

    m_isInitialized = true;
  }

2. TestNG#runSuites()

protected List<ISuite> runSuites() {
    return runSuitesLocally();
}

3. TestNG#runSuitesLocally()

分析非并发执行的测试用例执行过程，具体执行过程见方法4

public List<ISuite> runSuitesLocally() {
    // 没有测试用例需要执行 直接返回
    if (m_suites.isEmpty()) {
      error("No test suite found. Nothing to run");
      usage();
      return Collections.emptyList();
    }
    // 定义一个SuiteRunnerMap
    SuiteRunnerMap suiteRunnerMap = new SuiteRunnerMap();

    if (m_suites.get(0).getVerbose() >= 2) {
      Version.displayBanner();
    }

    // First initialize the suite runners to ensure there are no configuration issues.
    // Create a map with XmlSuite as key and corresponding SuiteRunner as value
    // 将测试类进行实例化， 将测试类实例装入testRunners,有几个testCase就装入几个testRunner
    // 一个site节点对应一个suiteRunner，注册一些监听器
    for (XmlSuite xmlSuite : m_suites) {
      createSuiteRunners(suiteRunnerMap, xmlSuite);
    }
    // Run suites
    // 非并发的用例集都在这里执行
    if (m_suiteThreadPoolSize == 1 && !m_randomizeSuites) {
      // Single threaded and not randomized: run the suites in order
      for (XmlSuite xmlSuite : m_suites) {
          // 用例执行
        runSuitesSequentially(
            xmlSuite, suiteRunnerMap, getVerbose(xmlSuite), getDefaultSuiteName());
      }
      //
      // Generate the suites report
      //
      return Lists.newArrayList(suiteRunnerMap.values());
    }
    return Lists.newArrayList(suiteRunnerMap.values());
  }

4. TestNG#runSuitesSequentially()

将测试集合生成对应的SuiteRunnerWorker，用worker去执行用例(方法5)

private void runSuitesSequentially(
      XmlSuite xmlSuite, SuiteRunnerMap suiteRunnerMap, int verbose, String defaultSuiteName) {
    // 循环递归子元素
    for (XmlSuite childSuite : xmlSuite.getChildSuites()) {
      runSuitesSequentially(childSuite, suiteRunnerMap, verbose, defaultSuiteName);
    }
    // 生成Suite执行Worker
    SuiteRunnerWorker srw =
        new SuiteRunnerWorker(
            suiteRunnerMap.get(xmlSuite), suiteRunnerMap, verbose, defaultSuiteName);
    srw.run();
  }

5. SuiteRunnerWorker#runSuite()

这里就是执行的是 suiteRunner的run方法（方法6）

private void runSuite(SuiteRunnerMap suiteRunnerMap /* OUT */, XmlSuite xmlSuite) {
    if (m_verbose > 0) {
      String allFiles =
          "  "
              + (xmlSuite.getFileName() != null ? xmlSuite.getFileName() : m_defaultSuiteName)
              + '\n';
      Utils.log("TestNG", 0, "Running:\n" + allFiles);
    }

    SuiteRunner suiteRunner = (SuiteRunner) suiteRunnerMap.get(xmlSuite);
    // 先执行继承了ISuiteListener的监听器
    suiteRunner.run();
  }

6. SuiteRunner#run()

执行ISuiteListener的监听器onStart() 测试方法(具体看方法7) onFinsh()方法

public void run() {
    // 执行所有 ISuiteListener的start方法
    invokeListeners(true /* start */);
    try {
      privateRun();
    } finally {
      // 执行所有 ISuiteListener的stop方法
      invokeListeners(false /* stop */);
 	}
  }

7. SuiteRunner#privateRun()

循环所有的TestRunner(测试用例)，统计所有的@BeforeSuite，@AfterSuite放入LinkedMap中

先依次执行所有的用例中的@BeforeSuite，再执行runSequentially()[见方法8]，最后依次执行所有的用例中的@AfterSuite

private void privateRun() {
    // Map for unicity, Linked for guaranteed order
    Map<Method, ITestNGMethod> beforeSuiteMethods = new LinkedHashMap<>();
    Map<Method, ITestNGMethod> afterSuiteMethods = new LinkedHashMap<>();
    IInvoker invoker = null;
    // Get the invoker and find all the suite level methods
    // 将suite的前置和后置方法加入局部Map中
    for (TestRunner tr : testRunners) {
      // TODO: Code smell.  Invoker should belong to SuiteRunner, not TestRunner
      // -- cbeust
      invoker = tr.getInvoker();

      for (ITestNGMethod m : tr.getBeforeSuiteMethods()) {
        beforeSuiteMethods.put(m.getConstructorOrMethod().getMethod(), m);
      }

      for (ITestNGMethod m : tr.getAfterSuiteMethods()) {
        afterSuiteMethods.put(m.getConstructorOrMethod().getMethod(), m);
      }
    }
    // Invoke beforeSuite methods (the invoker can be null
    // if the suite we are currently running only contains
    // a <file-suite> tag and no real tests)
    // 执行@BeforeSuite 执行
    if (invoker != null) {
      if (!beforeSuiteMethods.values().isEmpty()) {
        ConfigMethodArguments arguments = new Builder()
            .usingConfigMethodsAs(beforeSuiteMethods.values())
            .forSuite(xmlSuite)
            .usingParameters(xmlSuite.getParameters())
            .build();
        invoker.getConfigInvoker().invokeConfigurations(arguments);
      }
      // Run all the test runners
      //
      boolean testsInParallel = XmlSuite.ParallelMode.TESTS.equals(xmlSuite.getParallel());
      if (RuntimeBehavior.strictParallelism()) {
        testsInParallel = !XmlSuite.ParallelMode.NONE.equals(xmlSuite.getParallel());
      }
      if (testsInParallel) {
        runInParallelTestMode();
      } else {
          // 执行测试用例
        runSequentially();
      }
      // Invoke afterSuite methods
      // 执行@AfterSuite
      if (!afterSuiteMethods.values().isEmpty()) {
        ConfigMethodArguments arguments = new Builder()
            .usingConfigMethodsAs(afterSuiteMethods.values())
            .forSuite(xmlSuite)
            .usingParameters(xmlSuite.getAllParameters())
            .build();
        invoker.getConfigInvoker().invokeConfigurations(arguments);
      }
    }
  }

8. SuiteRunner#runSequentially()

将测试集中的所有testRunner执行runTest(tr)方法，见方法9

 private void runSequentially() {
    for (TestRunner tr : testRunners) {
      runTest(tr);
    }
  }

9. SuiteRunner#runTest()

testRunner自己调用run方法即可 具体看方法10

 private void runTest(TestRunner tr) {
    visualisers.forEach(tr::addListener);
    tr.run();
     // 获取执行结果
    ISuiteResult sr = new SuiteResult(xmlSuite, tr);
    synchronized (suiteResults) {
      suiteResults.put(tr.getName(), sr);
    }
  }

10. TestRunner#privateRun()

将所有的测试用例转换为graph节点，然后对每个节点交给对应的IWorker，调用worker.run()去执行

private void privateRun(XmlTest xmlTest) {
    boolean parallel = xmlTest.getParallel().isParallel();// 是否并行
    // parallel
    int threadCount = parallel ? xmlTest.getThreadCount() : 1;
    // Make sure we create a graph based on the intercepted methods, otherwise an interceptor
    // removing methods would cause the graph never to terminate (because it would expect
    // termination from methods that never get invoked).
    ITestNGMethod[] interceptedOrder = intercept(getAllTestMethods());
    AtomicReference<IDynamicGraph<ITestNGMethod>> reference = new AtomicReference<>();
    TimeUtils.computeAndShowTime("DynamicGraphHelper.createDynamicGraph()",
        () -> {
          IDynamicGraph<ITestNGMethod> ref = DynamicGraphHelper
              .createDynamicGraph(interceptedOrder, getCurrentXmlTest());
          reference.set(ref);
        }
    );
    IDynamicGraph<ITestNGMethod> graph = reference.get();

    graph.setVisualisers(this.visualisers);
    // In some cases, additional sorting is needed to make sure tests run in the appropriate order.
    // If the user specified a method interceptor, or if we have any methods that have a non-default
    // priority on them, we need to sort.
    boolean needPrioritySort = sortOnPriority(interceptedOrder);
    Comparator<ITestNGMethod> methodComparator = newComparator(needPrioritySort);
    List<ITestNGMethod> freeNodes = graph.getFreeNodes();

    if (graph.getNodeCount() > 0 && freeNodes.isEmpty()) {
      throw new TestNGException("No free nodes found in:" + graph);
    }

    while (!freeNodes.isEmpty()) {
      if (needPrioritySort) {
        freeNodes.sort(methodComparator);
        // Since this is sequential, let's run one at a time and fetch/sort freeNodes after each method.
        // Future task: To optimize this, we can only update freeNodes after running a test that another test is dependent upon.
        freeNodes = freeNodes.subList(0, 1);
      }
      // 每个测试用例都会有一个worker，除了并行的外
      List<IWorker<ITestNGMethod>> workers = createWorkers(freeNodes);
      for (IWorker<ITestNGMethod> worker : workers) {
        worker.run();
      }
      //createWorkers(freeNodes).forEach(Runnable::run);
      graph.setStatus(freeNodes, IDynamicGraph.Status.FINISHED);
      freeNodes = graph.getFreeNodes();
    }
  }

11. TestMethodWorker#run()

先执行@BeforeClass 然后执行测试方法，最后执行@AfterClass

invokeTestMethods里就是交给TestInvoker.java去处理的具体看方法12，13

 public void run() {
    for (IMethodInstance testMthdInst : m_methodInstances) {
      ITestNGMethod testMethod = testMthdInst.getMethod();
      // 执行@BeforeClass
      if (canInvokeBeforeClassMethods()) {
        synchronized (testMethod.getInstance()) {
          invokeBeforeClassMethods(testMethod.getTestClass(), testMthdInst);
        }
      }

      // Invoke test method
      try {
        // 执行@BeforeGroup @BeforeMethod  当前方法 @AfterMethod @AfterGroup
        invokeTestMethods(testMethod, testMthdInst.getInstance());
      } finally {
        // 执行@AfterClass
        invokeAfterClassMethods(testMethod.getTestClass(), testMthdInst);
      }
    }
  }

12. TestMethodWorker#invokeTestMethods()

交给testInvoker去执行测试用例 具体见方法13

 protected void invokeTestMethods(ITestNGMethod tm, Object instance) {
    // Potential bug here:  we look up the method index of tm among all
    // the test methods (not very efficient) but if this method appears
    // several times and these methods are run in parallel, the results
    // are unpredictable...  Need to think about this more (and make it
    // more efficient)
    List<ITestResult> testResults =
        m_testInvoker.invokeTestMethods(tm, m_groupMethods, instance, m_testContext);

    if (testResults != null) {
      m_testResults.addAll(testResults);
    }
  }

13. TestInvoker#invokeTestMethods()

这里和非并发执行用例分开了，并发执行的用例需要保证InvocationCount>1且ThreadPoolSize>1 具体见14

@Test(threadPoolSize = 3, invocationCount=10)

public List<ITestResult> invokeTestMethods(ITestNGMethod testMethod,
      ConfigurationGroupMethods groupMethods,
      Object instance,
      ITestContext context) {
       // For invocationCount > 1 and threadPoolSize > 1 run this method in its own pool thread.
    // 若在方法上标注了threadPoolSize = 3, invocationCount=10,则表示当前方法需要并发，走这里
    if (testMethod.getInvocationCount() > 1 && testMethod.getThreadPoolSize() > 1) {
      return invokePooledTestMethods(testMethod, parameters, groupMethods, context);
    }
    // 非并发执行下面代码（此处省略）[单用例并发其实后续还会走这里，可以见方法16]
  }

14 .TestInvoker#invokePooledTestMethods()

这里根据重复执行次数，clone出对应个数的ITestNGMethod加入workers集合中，其中，clone的时候，是将invocationCount和ThreadPoolSize设置为1（原因是重新执行方法11的时候不会走并发，具体见方法16的解释 ）然后传递给runWorkers()去执行，原理其实就是内部创建线程池去调用，具体见方法15

private List<ITestResult> invokePooledTestMethods(
      ITestNGMethod testMethod,
      Map<String, String> parameters,
      ConfigurationGroupMethods groupMethods,
      ITestContext testContext) {
    //
    // Create the workers
    //
    List<IWorker<ITestNGMethod>> workers = Lists.newArrayList();

    // Create one worker per invocationCount
    // 根据重复执行次数InvocationCount值，clone出相应个数的ITestNGMethod并加入workers集合中
    for (int i = 0; i < testMethod.getInvocationCount(); i++) {
      // we use clones for reporting purposes
      ITestNGMethod clonedMethod = testMethod.clone();
        // 这里将执行次数和线程数量设置为1，原因见方法16
      clonedMethod.setInvocationCount(1);
      clonedMethod.setThreadPoolSize(1);

      MethodInstance mi = new MethodInstance(clonedMethod);
      workers.add(new SingleTestMethodWorker(this, invoker, mi, parameters, testContext, m_classListeners));
    }
    // 内部创建线程池执行
    return runWorkers(
        testMethod, workers, testMethod.getThreadPoolSize(), groupMethods, parameters);
  }

15. TestInvoker#runWorkers()

这里在并发执行前 先执行@BeforeGroup 然后并发执行workers（见方法16），再执行@AfterGroup

private List<ITestResult> runWorkers(
      ITestNGMethod testMethod,
      List<IWorker<ITestNGMethod>> workers,
      int threadPoolSize,
      ConfigurationGroupMethods groupMethods,
      Map<String, String> parameters) {
    // Invoke @BeforeGroups on the original method (reduce thread contention,
    // and also solve thread confinement)
    // 执行@beforeGroup
    ITestClass testClass = testMethod.getTestClass();
    Object[] instances = testClass.getInstances(true);
    for (Object instance : instances) {
      GroupConfigMethodArguments arguments = new GroupConfigMethodArguments.Builder()
          .forTestMethod(testMethod)
          .withGroupConfigMethods(groupMethods)
          .withParameters(parameters)
          .forInstance(instance)
          .build();
      invoker.invokeBeforeGroupsConfigurations(arguments);
    }

    long maxTimeOut = workers.parallelStream()
        .map(IWorker::getTimeOut)
        .max(Long::compare)
        .orElse(-1L);

    // 并发执行workers
    ThreadUtil.execute("methods", workers, threadPoolSize, maxTimeOut);

    //
    // Collect all the TestResults 汇总测试结果
    //
    List<ITestResult> result = workers.parallelStream()
        .filter(tmw -> tmw instanceof TestMethodWorker)
        .flatMap(tmw -> ((TestMethodWorker) tmw).getTestResults().stream())
        .collect(Collectors.toList());

    // 执行@afterGroup
    for (Object instance : instances) {
      GroupConfigMethodArguments arguments = new GroupConfigMethodArguments.Builder()
          .forTestMethod(testMethod)
          .withGroupConfigMethods(groupMethods)
          .withParameters(parameters)
          .forInstance(instance)
          .build();
      invoker.invokeAfterGroupsConfigurations(arguments);
    }

    return result;
  }

16. ThreadUtil#execute()

这里就是内部创建线程池，然后将workers交给线程池去处理，workers就强转成Runnable

那这里的run()方法实际上会重新走方法11，由于方法14重新clone了ITestNGMethod，并将执行次数和线程数改为1，所以在走方法13的时候不会再走并发处，而走非并发执行的逻辑。后面就跟非并发执行的逻辑是一样的了

  public static void execute(
      String name,
      List<? extends Runnable> tasks,
      int threadPoolSize,
      long timeout) {
    // 创建线程池
    ExecutorService pooledExecutor =
        new ThreadPoolExecutor(
            threadPoolSize,
            threadPoolSize,
            timeout,
            TimeUnit.MILLISECONDS,
            new LinkedBlockingQueue<>(),
            new TestNGThreadFactory(name));

    // 此处的tasks就是传入的workers，留意这个run方法
    List<Callable<Object>> callables = Lists.newArrayList();
    for (final Runnable task : tasks) {
      callables.add(
          () -> {
            task.run();
            return null;
          });
    }
    try {
      // 开始通过线程池去执行
      if (timeout != 0) {
        pooledExecutor.invokeAll(callables, timeout, TimeUnit.MILLISECONDS);
      } else {
        pooledExecutor.invokeAll(callables);
      }
    } catch (InterruptedException handled) {
      Logger.getLogger(ThreadUtil.class).error(handled.getMessage(), handled);
      Thread.currentThread().interrupt();
    } finally {
      pooledExecutor.shutdown();
    }
  }