这是谷歌如何测试系列文章的第一篇。
Tuesday, January 25, 2011 9:08 AM
By James Whittaker
在所有我被问及的问题中，最多的就是关于谷歌是如何测试的。尽管在博客中【google testing blog】中有过零碎的解释说明，但还是需要更多的系统阐述。虽然谷歌的技术路线在执行的过程中不断地进化，但公司的测试策略却从来没有变化过。谷歌现在是一家拥有搜索、应用、广告、移动、操作系统等产品的公司，我们在这些涉及到的产品领域里发挥着非常有意义的作用。当我们涉及到一些新的领域或者在旧领域里快速成长的时候，必须要求我们的测试也在同步的扩张和改进。在这个系列文章中提及的测试技术，多数是我们当前正在使用的，还有一些是希望以后在不久的将来可以用到。
首先先介绍一下组织结构，这一部分也可能会让你感到惊奇。其实在谷歌没有真正的测试部门，测试依托在各个产品领域部门里，我们称之为“工程生产力”【原文， Engineering Productivity】。工程生产力部门拥有数量不等的水平或者垂直的工程学科，测试是其中的大头。简单地说，工程生产力部门由以下几部分构成：
1. 一个工具产品团队【a product team】，负责内部和外部开源的促进生产力的工具开发与维护，这些工具会被公司范围内的各种工程师使用。这些工具包括代码分析工具、IDE、测试用例管理系统、自动化测试工具、Build系统、源码管理系统、代码审核调度系统、缺陷管理系统等等。 这些工具的都是为了提高工程师效率的，并且这些工具在策略上的目标多数是为了防止问题的发生，而不是发现问题本身。
2. 一个服务团队【a services team】，给产品部门【注：这里的产品部门团队是和工程生产力部门平级的，例如Search、Gamil、Chrome等产品部门】提供一些专业的建议，包括一系列工具、文档、测试、发布管理、培训等方面，这些专家建议涵盖可靠性、安全、国际化等，甚至包括产品团队面对的功能问题。所有的其他产品领域也都会得到这样的建议指导。
3. 嵌入式的工程师【Embedded engineers】，在需要的时候被产品部门高效地“借”去使用，这些工程师有些会和产品部门的团队坐在一起工作数年，另外一些当他们被需要的时候会被借调到其他的产品团队。谷歌鼓励所有他们的工程师更换产品团队，用以保持团队忙绿且不断有新面孔，并如实地不带有任何偏见与政治。测试人员也是这样，但是可以有节奏地选择更换产品团队的频率。我的下属里，有测试人员已经在Chrome团队工作了好几年，也有一些待了一年半后就换到了其他团队。对于测试经理来说，必须在团队的产品经验和新鲜度上做出很好的平衡。
所以这意味着测试同学向工程生产力部门的经理汇报，但是他们会把自己看成产品部门团队的一员，像搜索、邮箱、和Chrome部门。从组织架构上看，测试都是两个团队的一部分。测试和产品团队坐在一起，参与计划，一起吃饭，共享奖金，享受像全职的产品团队成员一样的待遇。这种单独的组织汇报关系的好处是可以给测试人员之间提供良好的共享信息的讨论机会，好的测试思路可以很容易的在工程生产力部门内部蔓延，无论公司内的哪条产品线，都可以很快地使用这些最好的测试技术。
测试人员的这种项目分离和汇报组织结构也有它的缺点，目前来看，最大的问题是测试人员被看做外部资源。产品部门团队不能对测试人员有太多的依赖，他们自己必须要合理地控制产品质量。是的，没错，在谷歌，是产品部门团队对产品质量负责，而不是测试人员。每个产品部门的开发人员都需要做测试工作，测试人员的任务是为产品部门团队搭建自动化测试基础设施和流程，测试人员让开发可以自给自足地、独立地做完成测试工作。
在这种模式下，我比较喜欢的是，开发和测试将有相同的地位。在质量方面，开发和测试成为了真正的伙伴，最大的质量重担交给本应属于的开发人员，开发的职责就是正确地实现产品功能。另外这样可以保持多对一的开发测试比率，开发人员在数量上远超测试人员，并且测试工作做的越多，开发测试比率就会越大。产品部门团队也会对这样的高开发测试比率而感到骄傲。
好，现在好像大家都是好朋友了，对吧？ 相信你已经看到这种模式的一个问题，开发人员不能很好地驱动缺陷【Bug】的运转，开发不会测试。难道我要否认这一点么？不管怎样，我都不会否认，特别是去年在GTAC talk 【GTAC 2010: Turning Quality on its Head，link http://www.youtube.com/watch?v=cqwXUTjcabs】上做了一个开发和测试对抗的游戏后。（友情提示：测试赢了游戏）【这里感觉翻译的不好，原文是： No amount of corporate kool-aid could get me to deny it, especially coming off my GTAC talk last year where I pretty much made a game of developer vs. tester (spoiler alert: the tester wins).】
在谷歌，解决这个问题的办法是将角色再细分，我们通过设立不同的测试角色来解决这两种不同的测试问题。在下一篇文章里，我将详细阐述这些测试角色和谷歌是怎样将测试问题分成两部分来分别解决的。
公直 2012/3/8
英文原文
http://googletesting.blogspot.com/2011/01/how-google-tests-software.html
Tuesday, January 25, 2011 9:08 AM
By James Whittaker
This is the first in a series of posts on this topic.
The one question I get more than any other is “How does Google test?” It’s been explained in bits and pieces on this blog but the explanation is due an update. The Google testing strategy has never changed but the tactical ways we execute it has evolved as the company has evolved. We’re now a search, apps, ads, mobile, operating system, and so on and so forth company. Each of these Focus Areas (as we call them) have to do things that make sense for their problem domain. As we add new FAs and grow the existing ones, our testing has to expand and improve. What I am documenting in this series of posts is a combination of what we are doing today and the direction we are trending toward in the foreseeable future.
Let’s begin with organizational structure and it’s one that might surprise you. There isn’t an actual testing organization at Google. Test exists within a Focus Area called Engineering Productivity. Eng Prod owns any number of horizontal and vertical engineering disciplines, Test is the biggest. In a nutshell, Eng Prod is made of:
1. A product team that produces internal and open source productivity tools that are consumed by all walks of engineers across the company. We build and maintain code analyzers, IDEs, test case management systems, automated testing tools, build systems, source control systems, code review schedulers, bug databases… The idea is to make the tools that make engineers more productive. Tools are a very large part of the strategic goal of prevention over detection.
2. A services team that provides expertise to Google product teams on a wide array of topics including tools, documentation, testing, release management, training and so forth. Our expertise covers reliability, security, internationalization, etc., as well as product-specific functional issues that Google product teams might face. Every other FA has access to Eng Prod expertise.
3. Embedded engineers that are effectively loaned out to Google product teams on an as-needed basis. Some of these engineers might sit with the same product teams for years, others cycle through teams wherever they are needed most. Google encourages all its engineers to change product teams often to stay fresh, engaged and objective. Testers are no different but the cadence of changing teams is left to the individual. I have testers on Chrome that have been there for several years and others who join for 18 months and cycle off. Keeping a healthy balance between product knowledge and fresh eyes is something a test manager has to pay close attention to.
So this means that testers report to Eng Prod managers but identify themselves with a product team, like Search, Gmail or Chrome. Organizationally they are part of both teams. They sit with the product teams, participate in their planning, go to lunch with them, share in ship bonuses and get treated like full members of the team. The benefit of the separate reporting structure is that it provides a forum for testers to share information. Good testing ideas migrate easily within Eng Prod giving all testers, no matter their product ties, access to the best technology within the company.
This separation of project and reporting structures has its challenges. By far the biggest is that testers are an external resource. Product teams can’t place too big a bet on them and must keep their quality house in order. Yes, that’s right: at Google it’s the product teams that own quality, not testers. Every developer is expected to do their own testing. The job of the tester is to make sure they have the automation infrastructure and enabling processes that support this self reliance. Testers enable developers to test.
What I like about this strategy is that it puts developers and testers on equal footing. It makes us true partners in quality and puts the biggest quality burden where it belongs: on the developers who are responsible for getting the product right. Another side effect is that it allows us a many-to-one dev-to-test ratio. Developers outnumber testers. The better they are at testing the more they outnumber us. Product teams should be proud of a high ratio!
Ok, now we’re all friends here right? You see the hole in this strategy I am sure. It’s big enough to drive a bug through. Developers can’t test! Well, who am I to deny that? No amount of corporate kool-aid could get me to deny it, especially coming off my GTAC talk last year where I pretty much made a game of developer vs. tester (spoiler alert: the tester wins).
Google’s answer is to split the role. We solve this problem by having two types of testing roles at Google to solve two very different testing problems. In my next post, I’ll talk about these roles and how we split the testing problem into two parts.
Wednesday, February 09, 2011 6:36 PM
By James Whittaker
为了实现”谁的屁股谁自己擦”这句名言所说的那样，在传统的软件开发人员的之上，有必要增加了几个角色，特别是需要工程技术方面的特殊角色，这种角色可以让开发更高效低做测试。在谷歌，这样角色的职责是让其他人工作的更有效率，这样的工程师通常会把自己当做测试人员，但他们真正的使命是提高生产力/生产率。他们的存在是为了让开发人员效率提升，特别是在质量方面的提升，因为产品质量是生产率中最重要的一部分。这里是这些角色的总结：
【注，“you build it, you break it”， you build it ,you break it , you fix it, 原意指在Build Lab的人永远不会去修复build break的问题，只有开发人员自己才能修复。这里的意思是开发人员自己要对自己写的代码负责，比专职的测试人员更适合做测试工作。这里意译为”谁拉的shi，谁的屁股谁自己擦”】
软件开发工程师【SWE，Software Engineer】， 就是传统的开发人员。软件工程师实现一些功能代码并把最终产品提供给用户使用，他们创建设计文档、设计数据结构和总体的架构搭建，他们大多数时间都在写代码和评审代码。同时，他们也会写很多的测试代码，包括测试驱动设计，单元测试，并参与后面的文章会讲到的小、中、大型测试的创建工作。软件工程师需要对他们自己写的代码、修复缺陷的代码、改进的代码，只要是他们接触过的代码的质量负责。
软件测试开发工程师【SET or Software Engineer in Test】，和软件开发工程师一样是开发工程师，主要负责软件的可测试性。他们参与设计评审，近距离地关注代码质量和风险，对代码做重构为了系统有更好的可测试性，同时他们负责写单元测试框架和自动化测试的框架。在代码级别上他们和软件开发工程师是合作伙伴，但如果和增加新功能或提升性能相比较，他们更关心产品的质量和测试覆盖率的提升。
软件测试工程师【Test Engineer】，和软件测试开发工程师【SET】恰恰相反，他得主要工作是做测试而不是开发。许多谷歌的软件测试工程师会花很多的时间在写测试代码上，包括自动化脚本、使用场景的代码、甚至模拟最终用户的操作方面的代码。他们对软件开发工程师和软件测试开发工程师的测试工作做一些组织安排，解释测试结果、驱动测试的执行，特别是在项目即将发布的后期将起到非常重要的作用。软件测试工程师既是产品专家也是质量顾问更是风险分析师。
从质量的角度来看，软件开发工程师对功能开发和质量负有全责。同时，他们还负责容错设计、故障恢复、TDD、单元测试、和在软件测试开发工程师的帮助下写测试代码，这些测试代码会验证开发的功能。
软件测试开发工程师是提供测试支持的开发人员。他们提供一种能够将新添加的代码通过模拟其依赖的方式做功能验证的技术框架，并应用在代码提交之前的提交队列管理之中【注，这样可以在代码check in 的时候保证新代码的功能完备】。可以这样说，软件测试开发工程师就是为了让软件工程师可以测试他们的功能代码，所有真正的测试都是软件开发工程师完成的，软件测试开发工程师是保证这些功能有很好的可测试性，这样可以让软件开发工程师很积极地参与到测试用例代码的编写中去。
现在所有的一切很清楚了，软件测试开发工程师就是服务人员，他们的主要职责就让开发人员很方便简单的做测试并保证模块级别的产品质量。读者可能已经意识到一个问题，在这样的研发流程下，使用软件的最终用户会怎样？
在谷歌，软件测试工程师的职责就是最终用户级别的测试。如果软件开发工程师和软件测试开发工程师很好地做了模块级别的功能测试，下一个工作就是看许多功能集成和数据的组合是否能够满足最终用户的使用需求。软件测试工程师在这里就是扮演一个双重确认开发工程师测试工作的角色，任何明显的缺陷都会说明之前一轮的开发自测不够或比较草率，如果没有出现这种情况之后，软件测试工程师会将注意力转移到普通用户的使用场景测试上，保证性能、安全、国际化等方面没有问题。软件测试工程师们需要做大量的测试工作，并在测试工程师、测试合同工、吃狗粮的尝鲜者、beta测试用户、早期最终用户之间做很多沟通交流的工作，他们会和基础设计、功能复杂度、避免错误的方法等方面遇到问题的人做确认。一旦软件测试工程师开始介入，总是会没完没了。
好了，现在大家对这些角色应该有比较好的理解了，接下来我会在他们之间是怎么分工合作做更详尽的剖析。下次再聊。。。谢谢您的关注。
公直
2012/3/11
英文原文，
How Google Tests Software – Part Two
Wednesday, February 09, 2011 6:36 PM
http://googletesting.blogspot.com/2011/02/how-google-tests-software-part-two.html
By James Whittaker
In order for the “you build it, you break it” motto to be real, there are roles beyond the traditional developer that are necessary. Specifically, engineering roles that enable developers to do testing efficiently and effectively have to exist. At Google we have created roles in which some engineers are responsible for making others more productive. These engineers often identify themselves as testers but their actual mission is one of productivity. They exist to make developers more productive and quality is a large part of that productivity. Here’s a summary of those roles:
The SWE or Software Engineer is the traditional developer role. SWEs write functional code that ships to users. They create design documentation, design data structures and overall architecture and spend the vast majority of their time writing and reviewing code. SWEs write a lot of test code including test driven design, unit tests and, as we explain in future posts, participate in the construction of small, medium and large tests. SWEs own quality for everything they touch whether they wrote it, fixed it or modified it.
The SET or Software Engineer in Test is also a developer role except their focus is on testability. They review designs and look closely at code quality and risk. They refactor code to make it more testable. SETs write unit testing frameworks and automation. They are a partner in the SWE code base but are more concerned with increasing quality and test coverage than adding new features or increasing performance.
The TE or Test Engineer is the exact reverse of the SET. It is a a role that puts testing first and development second. Many Google TEs spend a good deal of their time writing code in the form of automation scripts and code that drives usage scenarios and even mimics a user. They also organize the testing work of SWEs and SETs, interpret test results and drive test execution, particular in the late stages of a project as the push toward release intensifies. TEs are product experts, quality advisers and analyzers of risk.
From a quality standpoint, SWEs own features and the quality of those features in isolation. They are responsible for fault tolerant designs, failure recovery, TDD, unit tests and in working with the SET to write tests that exercise the code for their feature.
SETs are developers that provide testing features. A framework that can isolate newly developed code by simulating its dependencies with stubs, mocks and fakes and submit queues for managing code check-ins. In other words, SETs write code that allows SWEs to test their features. Much of the actual testing is performed by the SWEs, SETs are there to ensure that features are testable and that the SWEs are actively involved in writing test cases.
Clearly SETs primary focus is on the developer. Individual feature quality is the target and enabling developers to easily test the code they write is the primary focus of the SET. This development focus leaves one large hole which I am sure is already evident to the reader: what about the user?
User focused testing is the job of the Google TE. Assuming that the SWEs and SETs performed module and feature level testing adequately, the next task is to understand how well this collection of executable code and data works together to satisfy the needs of the user. TEs act as a double-check on the diligence of the developers. Any obvious bugs are an indication that early cycle developer testing was inadequate or sloppy. When such bugs are rare, TEs can turn to their primary task of ensuring that the software runs common user scenarios, is performant and secure, is internationalized and so forth. TEs perform a lot of testing and test coordination tasks among TEs, contract testers, crowd sourced testers, dog fooders, beta users, early adopters. They communicate among all parties the risks inherent in the basic design, feature complexity and failure avoidance methods. Once TEs get engaged, there is no end to their mission.
Ok, now that the roles are better understood, I’ll dig into more details on how we choreograph the work items among them. Until next time…thanks for your interest.
156
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
