1. INTRODUCTION
本文目的是给出一种 Evaluation and Test 成为 SEI CMM KPA 的方案。文档第一部分界定评估和测试内涵的范围。第二部分则解释为什么要开发这个独立的 KPA 。第三部分给出该 KPA 提案，包括概念、目标，执行承诺、执行能力、执行活动、度量和分析以及实施验证。本文的最后将考虑如何与其他 KPA 集成，这包括确定该 KPA 的级别，以及对现有 KPA 的重新调整。
The objective of this document is to present a proposal that Evaluation and Test become a Key Process Area (KPA) in the SEI Capability Maturity Model (CMM). The first section addresses the scope of what is meant by evaluation and test. The second section identifies the justifications for making this a separate KPA. The third section presents the proposed KPA definition including: definition, goals, commitment to perform, activities performed, measurements and analysis, and verifying implementation. The final section addresses integrating this KPA with the existing KPAs. This includes identifying which level to assign it to and some repackaging suggestions for existing KPAs.
关于“验证”和“确认”在 ISO9000 中有严格的定义。
2 验证：通过检查和提供客观证据，表明规定要求已经满足的认可。“验证”强调的是“规定规格要求”
2 确认：通过检查和提供客观证据，表明一些针对某一特定预期用途的要求已经满足的认可。“确认”强调的是“预期用途的要求”。
目的：
2 验证的目的是证实设计阶段输出是否确保设计阶段输入要求；
2 确认的目的是通过产品确认设计是否满足使用要求。
对象：
2 验证的对象是设计输出文件，计算书或样品等；
2 确认的对象是最终产品（样品）。
参与人员：
2 验证的参与人员通常是设计部门；
2 确认的参与人员必须包括使用者或能代表使用要求的人员。
时机：
2 验证的时机是设计适当阶段，一般是设计阶段输出形成结果时；
2 确认的时机是成功的设计验证后，一般针对最终产品，也可分阶段确认。
2. DEFINING EVALUATION AND TEST （评价与测试的定义）
(Verification and Validation)
Evaluation is the activity of verifying the various system specifications and models produced during the software development process. Testing is the machine based activity of executing and validating tests against the code. Most software organizations define evaluation and test very narrowly. They use it to refer to just the activities of executing physical test cases against the code. In fact, many companies do not even assign testers to a project until coding is well under way. They further narrow the scope of this activity to just function testing and maybe performance testing.
评价是对软件开发过程中产生的各种系统规格和模型进行的验证活动。测试则是一种基于机器的对代码执行测试，确认测试的活动。大部分组织对评价和测试的定义都相对狭义，一般是指对代码执行物理测试用例的活动。事实上，很多公司甚至直到编码已经开始时才指定 / 安排测试人员。更有甚者，他们将这一活动的范围仅仅限于功能测试，也许有时做一下性能测试。
This view is underscored （被强调） in the description of evaluation and test in the current CMM. It is part of the Software Product Engineering KPA. The activities in this KPA, activities 5, 6, and 7, only use code based testing for examples and only explicitly mention function testing. Other types of testing are euphemistically （婉转的 , 委婉说法的） referenced by the phrase “...ensure the software satisfies the software requirements”.
这种观点在目前的 CMM 有关 evaluation and test 的描述中被进一步强调，这就是 SPE ，软件产品工程 KPA 。在 SPE KPA 活动中，活动 5 、 6 、 7 ，仅仅用了基于代码的测试作为 examples ，只明确地提到了功能测试。其他类型的测试只是用一句非常含糊的话来指代： “….. 保证软件满足软件需求 ” 。
People who build skyscrapers, on the other hand, thoroughly integrate evaluation and test into the development process long before the first brick is laid. Evaluations are done via models to verify such things as stability, water pressure, lighting layouts, power requirements, etc. The software evaluation and test approach used by many organizations is equivalent to an architect waiting until a building is built before testing it and then only testing it to ensure that the plumbing and lighting work.
另一方面，建造摩天大厦的人们，则远在砌第一块砖之前将评价和测试集成到了开发过程之中。通过建模来验证稳定性、防水性、照明布置以及电源的需求等等从而实施评价。而目前，组织所使用软件评价和测试方法就像是设计师一直等到大楼已经建成才进行测试，而此时的测试只是能保证给水和照明可以工作而已。
The CMM further compounds （混合） the limited view of evaluation and test by making a particular evaluation technique, peer reviews, its own KPA. This implies that prior to the delivery of code the only evaluation going on is via peer reviews and that this is sufficient. The steps in the evaluation and test of something are: define the completion/success criteria, design cases to cover this criteria, build the cases, perform/execute the cases, verify the results, and verify that everything has been covered. Peer reviews provide a means of executing a paper based test. They do not inherently provide the success criteria nor do they provide any formal means for defining the cases, if any, to be used in the peer review. They are also fundamentally subjective. Therefore, the same misconceptions that lead a programmer to introduce a defect into the product may cause them to miss the defect in the peer review.
CMM 只是进一步将评价和测试的部分思想进行融合，用一个特殊的评价技术来代替，这个技术就是 CMM 中的一个 KPA ，同行评审。这 (CMM 设计者们的这种做法 ) 也意味着，在提交代码之前，唯一可干的评价就是同行评审，且已经足够了。事实上，对于一件事情的评价和测试的步骤包括： (1) 定义完成 / 成功准则； (2) 涉及覆盖这些准则的用例； (3) 执行用例； (4) 验证结果，验证所有的内容都已覆盖。同行评审只是提供了一个基于纸面的测试机制。它既不能从根本上提供成功准则，也不能提供任何正式的机制以支持用例定义以用于同行评审中。同行评审本质是主观的，因此，基于误解使程序员将缺陷引入产品，而到同行评审时，基于同样的误解，也使得人们无法发现这些 defect 。
A robust scope for evaluation and test must encompass every project deliverable at each phase in the development life cycle. It also address each desired characteristic of each deliverable. It must address each of the evaluation/testing steps. Let's look at two examples: evaluating requirements and evaluating a design.
评价和测试的一个相对坚固的内涵范围必须包括项目在开发周期每一个阶段的每一个交付产品。它也必须考虑每个交付产品的每一个预期特性。而且必须包括每一个评价 / 测试步骤。下面我们看两个例子：评价需求和对一个设计的评价。
A requirements document should be complete, consistent, correct, and unambiguous. One step is to validate the requirements against the project/product objectives (i.e., the statement of “why” the project is being done). This ensures that the right set of functions are being defined. Another evaluation is to walk use-case scenarios through the functional rules, preferably aided by screen prototypes if appropriate. A third evaluation is a peer review of the document by domain experts. A fourth is to do a formal ambiguity review by non-domain experts. (They cannot read into the document assumed functional knowledge. It helps ensure that the rules are defined explicitly, not implicitly.) A fifth evaluation is to translate the requirements into a Boolean graph. This identifies issues concerning the precedence relationships between the rules as well as missing cases. A sixth is a logical consistency check with the aid of CASE tools. A seventh is the review, by domain experts, of the test scripts derived from the requirements. This