The art of software testing. Myers (2nd edition) (2004) (The art of software testing. Myers (2nd edition) (2004).pdf), страница 32

For example, you may not fully understand the acceptabledata types and boundaries for the input values of an application if youstart coding first. So how can you write a unit test to perform boundary analysis without understanding the acceptable inputs? Can theapplication accept only numbers, only characters, or both? If youcreate the unit tests first, you must understand the specification. Thepractice of creating unit tests first is the shining point of the XPmethodology, as it forces you to understand the specification toresolve ambiguities before you begin coding.As mentioned in Chapter 5, you determine the unit’s scope.

Giventhat today’s popular programming languages such as Java, C#, andVisual Basic are mostly object oriented, modules are often classes oreven individual class methods. You may sometimes define a moduleas a group of classes or methods that represent some functionality.Only you, as the programmer, know the architecture of the application and how best to build the unit tests for it.Manually running unit tests, even for the smallest application, canbe a daunting task.

As the application grows, you may generate hundreds or thousands of unit tests. Therefore, you typically use an automated software testing suite to ease the burden of constantly runningunit tests. With these suites you script the tests and then run all or partof them. In addition, testing suites typically allow you to create reportsand classify the bugs that frequently occur in your application. Thisinformation may help you proactively eliminate bugs in the future.Interestingly enough, once you create and validate your unit tests,the “testing” code base becomes as valuable as the software applicationExtreme Testing185you are trying to create.

As a result, you should keep the tests in a coderepository for protection. In addition, you should ensure that adequate backups occur, as well as that the needed security is in place.Acceptance TestingAcceptance testing represents the second, and an equally important,type of XT that occurs in the XP methodology. The purpose ofacceptance testing is to determine whether the application meets otherrequirements such as functionality and usability. You and the customercreate the acceptance tests during the design/planning phases.Unlike the other forms of testing discussed thus far, customers, notyou or your programming partners, conduct the acceptance tests. Inthis manner, customers provide the unbiased verification that theapplication meets their needs.

Customers create the acceptance testsfrom user stories. The ratio of user stories to acceptance tests is usually one to many. That is, more than one acceptance test may beneeded for each user story.Acceptance tests in XT may or may not be automated. For example, an unautomated test is required when the customer must validatethat a user-input screen meets its specification with respect to colorand screen layout. An example of an automated test is when theapplication must calculate payroll values using data input via somedata source such as a flat file to simulate production values.With acceptance tests, the customer validates an expected resultfrom the application. A deviation from the expected result is considered a bug and is reported to the development team.

If customers discover several bugs, then they must prioritize them before passing thelist to your development group. After you correct the bugs, or afterany change, the customers rerun the acceptance tests. In this manner,the acceptance tests also become a form of regression testing.An important note is that a program can pass all unit tests but failthe acceptance tests. Why? Because a unit test validates whether aprogram unit meets some specification such as calculating payrolldeductions correctly, not some defined functionality or aesthetics.For a commercial application, the look and feel is a very important186The Art of Software Testingcomponent.

Understanding the specification, but not the functionality, generally creates this scenario.Extreme Testing AppliedIn this section we create a small Java application and employ JUnit, aJava-based open-source unit testing suite, to illustrate the concepts ofExtreme Testing. The example itself is trivial; however, the conceptsapply to most any programming situation.Our example is a command-line application that simply determineswhether an input value is a prime number.

For brevity, the sourcecode, check4Prime.java, and its test harness, check4PrimeTest.java,are listed in Appendix A. In this section we provide snippets from theapplication to illustrate the main points.The specification of this program is as follows:Develop a command-line application that accepts any positiveinteger, n, where 0 ⬉ n ⬉ 1,000, and determine whether it is aprime number. If n is a prime number, then the application shouldreturn a message stating it is a prime number. If n is not a primenumber, then the application should return a message stating it isnot a prime number.

If n is not a valid input, then the applicationshould display a help message.Following the XP methodology and the principles listed in Chapter 5, we begin the application by designing unit tests. With thisapplication, we can identify two discrete tasks: validating inputs anddetermining prime numbers.

We could use black-box and white-boxtesting approaches, boundary-value analysis, and the decisioncoverage criterion, respectively. However, the XT practice mandatesa hands-off black-box approach to eliminate any bias.Test-Case DesignWe begin designing test cases by identifying a testing approach. Inthis instance we will use boundary analysis to validate the inputsExtreme Testing187because this application can only accept positive integers within acertain range.

All other input values, including character datatypesand negative numbers, should raise an error and not be used. Ofcourse, you could certainly make the case that input validation couldfall under the decision-coverage criterion, as the application mustdecide whether the input is valid. The important concept is to identify, and commit, to a testing approach when designing your tests.With the testing approach identified, develop a list of test casesbased on possible inputs and expected outcome. Table 8.2 shows theTable 8.2Test Case Descriptions for check4Prime.javaCaseNumber1Inputn=3ExpectedOutputAffirm n is a primenumber.••2n = 1,000Affirm n is not aprime number.••3n=04•n = −1Affirm n is not aprime number.Print help message.5n = 1,001Print help message.•62 or moreinputsn = “a”Print help message.•Print help message.•Print help message.•78n is empty(blank)•CommentsTests for a valid primenumberTests input betweenboundariesTests input equal to upperboundsTests if n is an invalidprimeTests input equal to lowerboundsTests input below lowerboundsTests input greater thanthe upper boundsTests for correct numberof input valuesTests input is an integerand not a characterdatatypeTests if an input value issupplied188The Art of Software Testingeight test cases identified.

(Note: We are using a very simple exampleto illustrate the basics of Extreme Testing. In practice you would havea much more detailed program specification that may include itemssuch as user-interface requirements and output verbiage. As a result,the list of test cases would increase substantially.)Test case 1 from Table 8.2 combines two test scenarios.

It checkswhether the input is a valid prime and how the application behaveswith a valid input value. You may use any valid prime in this test.Appendix B provides a list of the prime numbers less than 1,000 thatcould be used.We also test two scenarios with test case 2: What happens when theinput value is equal to the upper bounds and when the input is not aprime number? This case could have been broken out into two unittests, but one goal of software testing in general is to minimize thenumber of test cases while still adequately checking for error conditions. Test case 3 checks the lower boundary of valid inputs as well astesting for invalid primes. The second part of the check is not neededbecause test case 2 handles this scenario. However, it is included byJUnit is a freely available open-source tool used to automate unittests of Java applications in Extreme Programming environments.The creators, Kent Beck and Erich Gamma, developed JUnit tosupport the significant unit testing that occurs in the Extreme Programming environment.

JUnit is very small, but very flexible andfeature rich. You can create individual tests or a suite of tests. Youcan automatically generate reports detailing the errors.Before using JUnit, or any testing suite, you must fully understand how to use it. JUnit is powerful but only after you master itsAPI. However, whether or not you adopt an XP methodology,JUnit is a useful tool to provide sanity checks for your own code.Check out www.junit.org for more information and to downloadthe test suite. In addition, there is a wealth of information on XPand XT at this Website.Extreme Testing189default because 0 is not a prime number.