测试工作总结-2005~2008

在某公司做测试三年多的总结，主要做黑盒手工测试，产品是windows单机程序。

 

Criteria of best test plan:
1.       Clear – Action & Expected Results should be as clear as possible.          
2.       Good Structure – Test cases should be categorized properly, different functionalities are organized in different parts, so it’s easy to execute and maintain.            
3.       Depth – All the functionalities for the feature should be tested thoroughly.       
4.       Coverage – Test plan should cover all points in spec, and also related parts with other features.                 
5.       Schedule – Test plan should meet Code Complete and Feature Complete milestones.                  
6.       Good Review Feedback    

       
Here are some items I collect in order to get a test design with high quality. We do different things in different phases.
1.       Spec reading phase
Understand spec:
  Read MRD will help us to know customers’ requirements and understand why we need this feature in current release.
  Communicate with Dev. Especially when you don’t understand some technical issues in the spec. Get some ideas about the feature development status and technology. Clarify your understanding. If there are differences, ask PD.
  Think if there are related features and their interactions. Try to find if some functions support Undo/Redo. Read related features spec. Try to use current application if you are not familiar with the existing features.
  Ask PD if there are any questions.
  Learn related domain knowledge and technical/computer knowledge if necessary. Ask for help when difficult.
  Read Dev’s Development documents if necessary.
尽早开始学习相关知识，不管是产品还是测试相关的知识，因为到了测试后期根本就没有时间去学
Think how to test:
  If test environment is available? If not, ask support or search.
  Test data: how and where to get test data? If we need prepare for it by ourselves, practice the application when you have time. Sometimes we can get test data from others.
  Get some ideas from Dev or people who have test this feature before. Maybe they know the feature’s weakness which we need more tests.

2.       Writing test design
  Define Test Plan Peer/Team Review process. We can refer to “ACRD_AEC_QA Best Practice.docx”.
  Try to use all kinds of test data in your feature when you don’t know if the functions will be affected by data types. Now we have Test Data column in test design. We can use one or more types for one testing point if the cases have not special requirement for test data.
  Add undo/redo and file workflow(save, close and reopen) related to your feature in the test design. It’s impossible to test all features Undo/Redo and save functions in one test design. And there are many problems here according to our previous experiences.
  Add more interaction cases, consider test cases from multi-aspect.
  Apply some test methods. Here is the test strategy from chapter 4 of book <the Art of Testing>:
1)                   If the specification contains combinations of input conditions, start with cause-effect graphing.
2)                   In any event, use boundary-value analysis. Remember that this is an analysis of input and output boundaries. The boundary-value analysis yields a set of supplemental test conditions.
3)                   Identify the valid and invalid equivalence classes for the input and output, and supplement the test cases identified above if necessary.
4)                   Use the error-guessing technique to add additional test cases.
5)                   Examine the program’s logic with regard to the set of test cases. Use the decision-coverage, condition-coverage, decision/condition-coverage, or multiple-condition-coverage criterion (the last being the most complete). If the coverage criterion has not been met by the test cases identified in the prior four steps, and if meeting the criterion is not impossible (i.e., certain combinations of conditions may be impossible to create because of the nature of the program), add sufficient test cases to cause the criterion to be satisfied.
   写test design：怎样的test design会比较好使用，方便记录测试结果？如何才能够使test design尽可能全面？
Test design应该包括ID,priority, summary, steps,expectedresult,pass/fail/block,test data, comment.还可以加上测试的先决条件或者说环境准备，后面的case中就没有必要再写这些步骤了。
设置优先级有助于在时间紧迫的情况下执行回归测试时挑选测试用例。
Case的描述应该尽可能简单清楚，最好走case时不用思考就可以直接用，以提高执行效率，比如输入长字符，直接把字符串写到case中；再比如需要用什么测试文件都提前准备好。如果有些操作其他人甚至自己可能不熟悉，那么就把过程写的详细些（测用命令行打开文件或程序）；反之，对于那些项目组成员都应该清楚的步骤就不必写的太详细，除非测得就是这个feature本身。
一个Case可以有很多步骤，其中有些步骤需要验证的，期望结果中就可以用对应的步骤号。
考虑问题应该尽可能全面（测试环境，测试数据，其他相关feature，save, undo-redo）， 同时要避免不同feature的case重复（重要的基本功能的case可以重复），同事间的协商和review非常重要
3.       Testing phase
  Update test design timely according to testing result or defects.
1)       Add more detailed cases. For example, when you find a defect which is related with test file type, then add more test cases and do more tests using different files. If the case is to validate the quantity is correct. And you find a defect that it relates with takeoff type (auto <> manual, linear/area/count/volume). Also add more test cases and do more tests according to takeoff type.
2)       Add valuable defects to test design.
  Update test design timely (including figures) when there are ECO or other changes.
  Read Dev’s subdoc for your feature and related feature. Sometimes you will know which parts will be affected by this code changes.
  Communicate with Dev. You can know design changes timely and then have time to confirm this and update test design. Sometimes we get feature weakness from them and then can test more about this.
 走case时经常是走第一遍能发现很多bug，后面几轮发现的就很少，这时可以用交换测试人员，执行case时改变一些测试数据测试环境等(改变case参数，执行顺序，执行方法（button，menu。。)，挑选测试用例优先级高的先走。
执行case和验证bug都应该仔细观察程序反应，验证bug要警惕修复了一个bug却导致原有的相关功能出错。
及时的更新和补充测试用例很重要，特别是测试时发现的有价值bug一定要加上。
系统测试的用例可以单独写，侧重各feature的交互，不同的场景user-case。
 怎样避免报告重复的bug：是别人feature的，不确定时先咨询该feature负责人；每天可以花点时间浏览一下别的feature重要的bug；利用bug管理系统的查询功能。
  测试方法：这方面还有很多东西可以学。
1）如何尽可能覆盖更多类型的文件？测试中偶然发现某些类型的文件会有问题，如何才能确认其它类型的文件是否也有同样的问题？
2）有什么方法可以比较全面的测试一类问题，比如在测试my view时发现在一页中点击某种类型的my view，切换到另一页，在返回的时候，模型变了。那么是不是其它类型的my view也会有类似的问题，另外是三维之间的切换还是二维三维之间的切换。似乎不可能进行全面的测试。
3）如何重现很多步骤后出现的bug，特别是项目后期，基本功能稳定后，经常会做很多操作都没有问题，突然crash。这时记不起做过哪些操作，很难重现？有没有什么好办法针对这种情况？
4）怎样挑选回归测试用例，或者说怎样给测试用例分级，具体标准是什么。
 发现bug要及早上报，不管是不是自己feature的。有些问题开发可能没有考虑到，而不是还没有完成。Bug等级设置合适，尽量提供详尽信息
4. 测试管理
最好每个阶段开始就有明确的指示，该做什么，怎么做，做到什么程度，日程安排。
对成员可能不熟悉的内容，尽早安排培训或其他方式的学习，而且最好选择效率比较高的方式，比如glob testing, 兼容性测试。如果大家都不熟悉，是否可以指定一个人学习到一定程度给大家培训。是否可以从别的地方获取相关资源。
测试环境，数据要在测试开始之前准备好。
在相关文档读得差不多的时候，组织成员开会，各自介绍自己的feature，以便大家熟悉产品。
测试任务分工明确，对一些基本功能，如undo-redo, save, print应提醒组员测试自己模块的这部分功能。对交叉功能更应该讨论清楚，避免大家都没有测到，或者两个人甚至更多测试同一内容。
安排测试用例的review，最好包括开发人员。
及时沟通。如果发现在某一基本功能上经常出问题，可以共享信息，让别的feature也测试这个部分，比如这次的undo-redo, switch section, print
测试计划，测试用例等的文档管理：设置版本号v*.*

5. Test plan:
1) Test goals and release criteria
2) Planned test areas
3) Test methodology
4) Test schedule