大话测试数据（二）：概念测试数据的获取

测试数据在整个测试过程中扮演着极为重要的角色，但是它却像个没有星象的演员，明明至少是男二号，但总是被观众忽略。在测试过程中，我们往往在测试计划阶段就忽略了测试数据，在起先没有给测试数据的设计、准备留出足够的时间，投入足够的精力，到了测试执行阶段追悔莫及。
只有吃过大亏的测试人员，才会在下一个测试开始的初期就认真的对待它。笔者也算是吃过亏的人。因此在现在经手的测试工作中，总会提着测试数据这根弦。恰巧有同学问到这方面的问题，就分享一下个人的经验总结，与大家一起探讨。

在大话测试数据（一）文章中，我提到，获取数据的第一步是获取概念上数据。这一步看起来简单，其实不是那么容易。获取概念数据和获取需求的过程是交织在一起的，事实上，它们其实是一个事儿，因为数据是需求中最重要的组成部分。
需求工程是个大话题，目前有很多种流派和实践方式来来搞定需求，但它们的思想都比较一致，那就是：不断的由粗到精的迭代（如下图）。关于需求这里不再展开，如果大家有兴趣的话，推荐两本我觉得还不错的书：德国人写的《需求工程，基础原理和技术》和国人写的《软件需求最佳实践》，大家读后结合工作实践会很有收获。

由上述文字可知，（测试）数据获取也是一个迭代的过程。实际上在项目早期，我们就能获得概念数据。概念数据是什么呢？用大白话说就是：这种数据叫什么，大体啥样子，是干嘛用的。举个例子：
如果你的项目是一个信用卡项目，项目有一个功能就是，每月给用户发送“电子对账单”。对于80后，甚至90后的你，一秒钟你就知道这个“电子对账单”大概将会是个什么东西了。“不就是一封电子邮件里放一个网页，里边告诉用户：尊敬的某某先生/小姐！您本月消费了几笔，每笔多少钱，都是哪一天花的。最重要的是，您在本月X日前必须把钱还了。“这样你就建立了对“电子对账单”这种测试数据的概念，也就是说得到了“电子对账单”这种概念的测试数据。
Pretty easy？事实没有那么简单的。事情的本质是：你有一个超级聪明的大脑，能瞬间把你的经验综合起来对需要识别的东西作一判断，并给出一个大致的评估。但如果你大脑没有相关的知识，你就没有那么幸运了。不信，请读一下下图中的文字：
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-tImn1psk-1650603395176)(https://ceshiren.com/uploads/default/original/3X/3/c/3c2702956a1f2b7335ac2bbf031d06893fbc28b3.jpeg)]
脂多糖是神马？膜蛋白复合体是神马？神马是beta链？桶壁是神马？？？？这特么的都是神马？如果你没有一些生物学知识、高中生物又不幸光睡觉了的话，这段选自《环球科学》的文字不会让你觉得比读日文简单。
因此识别概念上的测试数据，你脑子里还得有点儿货才行，这些货是：“技术层面的知识”，“业务层面的知识（领域知识）”，“对于产品本身的认识”，还有“你的常识”。这四点的总结是从测试大师 James Bach 的课程中获取的，你可以尝试获取他关于快速软件测试的 PDF 文件。
你说了，没有这些知识怎么办？答案特别简单，“学啊”！。勤学勤问勤练勤观察，入行几年后，如果不是特别懒惰，前三项都会提高到一个不错的高度。这些都变成了你的价值。经过一段时间爬坡，你就可以很快的获取概念测试数据了。
你说了，废话，我也知道要学，但有没有更具体点儿的？干货，有么？要能咯掉牙的！
好吧，可以参考下面的干货资料（英文版，也正好练习下英文），你就当它是个 checklist，按图索骥吧：关于测试数据的获取（不仅仅是概念测试数据的获取），测试思路的获取，甚至是需求的获取，你一定会有收获。
We recommend collecting test ideas continuously from a variety of information sources.
Consider the following, and think about values, risks, opportunities; find shortcuts to cover what is important.
1.Capabilities.
The first and obvious test ideas deal with what the product is supposed to do. A good start is requirements, examples and other specifications, or a function list created from actual software. But also try to identify implicit requirements, things users expect that haven’t been documented. Be alert to unwanted capabilities.
2.Failure Modes.
Software can fail in many ways, so ask “what if” questions to formulate test ideas that expose the handling of internal/external, anticipated/unexpected, (un)intentional, realistic/provoked failures. Challenge the fault tolerance of the system; any object or component can break.
3.Quality Characteristics.
Quality characteristics are always important for the project to be successful, although the OK zone can be easy to reach, or difficult and critical. Our definition includes capability, reliability, usability, charisma, security, performance, IT-bility, compatibility, supportability, testability, maintainability, portability, and a plethora of sub-categories.
Many of these can be used as ongoing test ideas in the back of your head, executed for free, but ready to identify violations.
4.Usage Scenarios.
Users want to accomplish or experience something with software, so create tests that in a variety of ways simulate sequences of product behavior, rather than features in isolation.
The more credible usage patterns you know of, the more realistic tests you can perform. Eccentric soap opera tests broaden coverage.
5.Creative Ideas.
All products are unique, and require some test ideas not seen before. Try lateral thinking techniques (e.g. Edward De Bono’s Six Thinking Hats, provocative operators, the opposite, random stimulation, Google Goggles) to come up with creative test ideas.
Metaphors and analogies is a good way to get you started in new directions.
6.Models.
A state model helps identify test ideas around states, transitions and paths. A system anatomy map shows what can be tested, and can highlight interactions. Create a custom model using structures like SFDPOT from Heuristic Test Strategy Model.
A visual model is easier to communicate, and the modeling activity usually brings understanding and new ideas.
7.Data.
By identifying intentional and unintentional data (there is always noise), you have a good start for a bunch of test ideas. Follow easy and tricky data through the application, be inside and outside boundaries, use CRUD (Create, Read, Update, Delete), exploit dependencies, and look at the data in many places.
8.Surroundings.
Environment compatibility (hardware, OS, applications, configurations, languages) is an important testing problem, but also investigate nearby activities to your product. By understanding the big system you can get credible test ideas that are far-fetched when looking at functionality in isolation.
9.White Box.
By putting a tester’s destructive mindset on developers’ perspective of architecture, design and code, you can challenge assumptions, and also find mistakes. Pay special attention to decisions and paths you might not understand from a black-box perspective. Code coverage is not worthless, it can be used to find things not yet tested.
10.Public collections.
Take advantage of generic or specific lists of bugs, coding errors, or testing ideas. As you are building your own checklist suitable for your situation, try these:
• Appendix A of Testing Computer Software (Kaner, Falk, and Nguyen)
• Boris Beizer Taxonomy (Otto Vinter)
• Shopping Cart Taxonomy (Giri Vijayaraghavan)
• Testing Heuristics Cheat Sheet (Elisabeth Hendrickson)
• You Are Not Done Yet (Michael Hunter)
Learn some testing tricks or techniques from books, blogs, conferences; search for test design heuristics, or invent the best ones for you.
11.Internal Collections.
Use or create lists of things that often are important in your context, some call these quality/test patterns.
12.Business Objectives.
What are the top objectives for the company (and department break-downs) ? Are there any requirements that contradict those objectives? Do you know the big picture, the product vision and value drivers?
13.Information Objectives.
Explicit and implicit purposes of the testing are important guides to your effort. If you don’t have them, you can create quality objectives that steer test ideas for any feature.
14.Product Image.
The behavior and characteristics the product is intended to display might be explicit or implicit, inside the walls and minds of the people producing or consuming the software.
You will be able to write compelling problem reports if you know and can show threats to the product’s image, e.g. by pointing to a violation of marketing material.
15.Product Fears.
Things that stakeholders are really worried about are much stronger than risks, they don’t need prioritization, they need testing. Typical hard-to-verify, but useful-for-testing fears are: loss of image, wrong decisions, damage, people won’t like the software. Different people have different fears; find out which matters most.
16.Project Risks.
Some of the difficult things in a project can be addressed by testing. You want to know about which functionality developers are having trouble with, and you will adjust your schedule depending on risks that need mitigation first.
17.Rumors.
There are usually lots of talk about quality and problems floating around. Some hurt the product and the organization. Use the rumors themselves as test ideas. It is your mission to kill them or prove them right.
18.Product History.
Old problems are likely to appear in new shapes. Search your bug/support system or create an error catalogue, remember critical failures and root cause analysis. Use old versions of your software as inspiration and oracle.
19.Test Artifacts.
Not only your own test ideas, logs and results can be used for sub-sequent tests, also try out test results from other projects, Beta testing reports, usability evaluations, 3rd party test results etc. What questions do you want to be able to answer in future
status reports?
20.Debt.
The idea that a debt is constantly growing because of all the shortcuts being made. This could be project debt, managerial debt, technical debt, software debt, testing debt or whatever you wish to call it. If the team keep track on what is on the debt list, you can map a set of test ideas against those items.
21.Business Knowledge.
If you know the purpose of the software, and the context it operates in, you can understand if it will provide vale to customers. If you can’t acquire this knowledge, co-operate with someone who knows the needs, logic and environment.
22.Field Information.
Besides knowledge about customer failures, their environments, needs and feelings, you can take the time to understand your customers both in error and success mode. Interview pre-sales, sales, marketing, consultants, support people, or even better: work there for a while.
23.Users.
Think about different types of users (people you know, personas), different needs, different feelings, and different situations.
Find out what they like and dislike, what they do next to your software. Setup a scene in the test lab where you assign the testers to role play different users, what test ideas are triggered from that? Best is of course unfiltered information directly from users, in their context.
Remember that two similar users might think very differently about the same area.
24.Conversations.
The informal information you get from people may contain things that are more important than what’s included in specifications. Many people can help you with your test design, some are better judges of importance, what can you gain from MIP:s(Mention In Passing)?
If developers know you can find interesting stuff, they might give you insider information about dubious parts of the software. A set of questions to a developer might be an innocent “what do you think we should test?” or “what part of your code would you have liked to do better?”
25.Actual Software.
By interacting with the software, you will get a lot of ideas about what is error-prone, connected, interesting. If you can eat your own dog food (euphemism: sip your own champagne), you are in much better position to understand what is important about the software. If “Quality is value to some person”, it is pretty good if that person is “me”.
26.Technologies.
By knowing the inner workings of the technology your software operates in, you can see problematic areas and things that tend to go wrong; understand possibilities and security aspects; which parameters to change, and when. You can do the right variations, and have technical discussions with developers.
27.Standards.
Dig up relevant business standards, laws and regulations. Read and understand user interface standards, security compliance, policies. There are articles out there that describe how you can break something even if it adheres to the standards, can you include their test ideas in yours?
28.References.
Reference material of various kinds is a good source for oracles and testing inspiration, e.g. an atlas for a geographic product. General knowledge of all types can be handy, and Wikipedia can be enough to get a quick understanding of a statistical method.
29.Competitors.
By looking at different solutions to similar problems you can get straightforward test ideas, but also a feeling of which characteristics end users might focus on. There might be in-house solutions (e.g. Excel sheets) to be inspired by, and often there exists analogue solutions for similar purposes. Can you gain any insightful test ideas from your competitors support, FAQ or other material?
30.Tools.
If something can be done very fast, it is a good idea to try it. Tools are not only the means to an end, they can also be used as the starting point for exploration.
31.Context Analysis.
What else in the current situation should affect the things you test, and how? Do you know about the market forces and project drivers? Is there anything that has changed that should lead to new ways of testing? What is tested by others?
32.Legal aspects.
Do you need to consider contracts, penalties or other legal obligations? What would cost the company the most in a legal issue? Do you have lawyer that can give you hints on what must be avoided?
33.Many Deliverables.
There are many things to test: the executable, the installation kit, programming interfaces, extensions, code & comments, file properties, Help, other documentation, Release Notes, readme:s, marketing, training material, demos etc. All of these also contain information you can use as inspiration.
34.YOU!
Your experience, knowledge, skills, feelings, subjectivity, and familiarity with problems. What do you want to test?
btw，在接下来的文章中，我将会着重讲解如何获取细化的测试数据。

活动推荐 | 大咖公开课 Live

霍格沃兹测试学院每周都会组织测试大咖公开课，分享来自 Google、BAT、TMD 等顶级互联网公司的软件测试和质量保障实战经验。近期分享公开课主题：

• 《Appium 自动化测试框架剖析》
• • 《专项测试之移动端性能测试进阶实战》
• 公开课报名方式：扫码加霍格沃兹测试学院小助手微信，回复“公开课”入群。

BAT、TMD 测试开发名企内推

近 100 家互联网名企测试团队都在霍格沃兹测试学院长期招募中高级测试开发人才（技能水平对标阿里 P6-P7，挑战年薪50W-100W）。有意向跳槽进入 BAT、TMD 等大厂工作的同学，请关注霍格沃兹测试学院中高级测试开发工程师「名企定向培养计划」。

可扫上图小助手微信二维码，回复“名企定向”入群咨询。点击阅读原文，了解更多详情。
原文链接

获取更多技术文章分享

1. 最浅显的道理：说白了测试用例的执行工作主要是做一些输入操作，然后观察输出。测试数据就是输入的内容，没有测试数据，你咋执行用例？
2. 2. 测试数据是测试设计的重要组成部分，测试用例的有效性严重依赖测试数据的选取或者设计，要记住测试的本质是抽样，样品的选取其实是一门深奥的科学，有学过统计学的同学会深切明白这个道理。
3. 3. 没有把测试数据这一块儿理顺，良好的自动化测试简直是空谈。试想，测试自动化采取的最普遍默模式就是“录制-回放”模式，如果搞不定数据，回放基本上会失败，自动化验证自然也就无法有效完成了。
4. 4. 测试数据能够启发测试设计。做测试多的同学都会有过选取一组测试用例后来了感觉发现自己思如泉涌的经历。
5. 5. 如果是已上线系统，或者生命周期较长的系统，从生产系统上 log 下来的数据可以很好的指导测试。（通过一些统计可以帮助识别那些业务重要，为能够制定正确的测试策略提供重要信息；对数据做 pattern 分析的话可以用于补充测试场景、用例，同样十分有益；这些数据还可以在测试中进行复用）。
6. 6. 其它种种好处 …
7. 我们可以从多个维度对测试数据进行分类，下面讲一下我的分类方式：
8. 1. 从测试数据的生命周期角度看可以将测试数据分为：稳定和数据、可消耗的数据和混合类型数据。
9. 稳定的数据：在一轮/多轮测试执行过程中几乎不会发生变化的数据，如常见的电商系统中的一些基础数据–城市，邮政编码，一些商品的属性，如衣服尺寸码等。
   10.可消耗数据：测试执行完某个步骤后，数据发生不可逆改变，或者发生逆转操作需要耗费大量精力的改变。这类数据的例子有：商品的库存，票务系统里的票，要被运维程序删除的过期数据，网络数据包等等。
   混合类型数据：某些数据是复合型数据，如 XML 结构或者 Json 结构的某些数据，一条数据中的一部分是稳定的数据，另一部分是可消耗数据，这样的例子其实很常见，一般这样的数据会被当做可消耗数据来处理。
10. 从数据是否可构造的角度来看可以将测试数据分为：可直接构造数据和需要间接获取的数据。
    11.可直接构造的数据：常见系统的大部分数据都可以直接被构造，通过操作系统本身，或者通过调用某些接口（SQL 也算接口）插入数据，有时候甚至这些数据就是文本，直接准备好他们就行了。
    需要间接获取的数据：手工制造成本太高（理论上我们可以制造所有测试数据，但有时候就是成本太高），如某些以二进制存储的含有信息的数据（被序列化的数据），某些非文本数据，例如音频数据，视频数据，传感器上传过来的极为复杂的带有某些 pattern 的数据。
    举个很好玩的例子，见过“猎曲奇兵”这款软件么？偶然听到一首歌，打开猎曲奇兵，十秒钟左右它就能告诉你是哪一首歌。你基本上无法自己创造一条有效的测试数据，除非你是张学友或者Lady Gaga。
11. 从业务角度来看数据可以分为：合规数据、非合规数据、Fuzz数据。
    12.合规数据：望文生义，就是符合业务规则的数据，如能够通过校验的身份证号。
    非合规数据：显然就是不符合业务规则的数据，当你被要求输入注册邮箱的时候，你给出这样的输入：skytraveler@@163.com 显然不会合规。
    Fuzz 数据：Fuzz 数据主要是利用一些工具生成的乱七八糟的数据，主要用于系统稳定性测试和安全测试。这是一个大话题，有兴趣的话推荐看《模糊测试》这本好书。
12. 从测试数据来源来看，可以分为：生产 dump 数据，自己生成的数据。
    13.上面的分类其实并不是很准确，但是分类就是为了帮助更高效的解决问题。接下来我会讲解对于上面类型的数据我是如何来处理的。
    概念上的数据：也就是抽象的数据，例如，你的被测物是一款收银软件，你知道每笔交易结束后，需要生成一张“发票”，见过发票的你大概知道 它长什么样子（如下图），“发票”就是概念上的数据，但它并不能直接使用。
    
    被细化了的数据：也就是具体的数据项。拿到发票后，我们还要分析发票里到底有哪些数据，经过需求的获取及分析，我们知道发票包含：发票日期，发票代码，付款方，收款方，金额，防伪码，二维码等。同时我们也知道了这些数据的规约，例如，发票的日期格式是：yyyy年mm月dd日。
    真正能使用的数据：我们知道了数据的细节，就可以按照这些细节准备被测系统能够识别和接受的数据了。例如，上面所说的发票付款方，收款方，真正的操作可能会直接在 GUI 中输入，或者可以调用某些接口，或者可以直接插入数据库。这时候你要做的就是，让你的数据变得能用起来。
    从上面的解释可以得到测试数据从被识别，到能够被使用的大体步骤:
    
    事实上，实际工作中，测试数据的准备远远不是这么简单。很多时候上面的每一步骤的推动都是一个艰苦的过程。搞定它需要像柯南一样抽丝剥茧的能力和工匠的细致和耐心。在后续的文章中，将进一步讲述一些实际经验。
    

活动推荐 | 大咖公开课 Live

霍格沃兹测试学院每周都会组织测试大咖公开课，分享来自 Google、BAT、TMD 等顶级互联网公司的软件测试和质量保障实战经验。近期分享公开课主题：

• 《Appium 自动化测试框架剖析》
• • 《专项测试之移动端性能测试进阶实战》
• 公开课报名方式：扫码加霍格沃兹测试学院小助手微信，回复“公开课”入群。
  
  名企内推：
  近 100 家互联网名企测试团队都在霍格沃兹测试学院长期招募中高级测试开发人才（技能水平对标阿里 P6-P7）。有意向求职换工作的同学，请关注霍格沃兹测试学院中高级测试开发工程师「名企定向培养计划」。

可加小助手微信，回复“名企定向”入群咨询。点击阅读原文，了解更多详情。
原文链接

获取更多技术文章分享