PROJECT 3 – Automated Testing I. Project Introduction

For this assignment, you will write (and test!) software for a game that plays Yahtzee, a 5-dice game. The rules to this game can be found in Appendix A, along with a score sheet of the paper version in Appendix B.

You are not expected to write an entire game for your assignment! Rather, you are to write the software which fills in a virtual, one-game, one-player scoresheet during a game, and which tracks a player’s score as the play progresses. However, you do not need to handle giving the user three dice rolls per turn.

This means, given: • The face values of five dice • The category that the user wants to use

the program will correctly calculate the proper score to be added into that place on the scoresheet. This means that, essentially, the expected output is the changes to the scoresheet.

For example, if the dice were 6-6-3-3-3, then that would score: • 9 in the 3s category • 12 in the 6s category • 21 in the 3-of-a-kind category • 25 in the Full House category • 21 in the Chance category • and a 0 in all the other categories

II. Important! You MUST complete the project using the following guidance:

1. In the spirit of TDD and the V-Model, you should write your requirements and test cases before proceeding to write your software. You need not adhere strictly to TDD guidance, but your code should be written using your test plan to help write the code. This will be done iteratively and incrementally, with small bits of functionality being added with each iteration.

2. Remember, the start of a good test plan is well-written requirements – particularly, requirements written at the right level of abstraction. “The program will implement scoring for a Yahtzee game” is not a requirement specific enough for writing test cases. Neither is “The program will score 30 points for a small straight. (Those may be good introductory or supplemental sentences in a requirements document, but they are not necessarily testable requirements in and of themselves.)

III. Test Cases and Automated Testing

A test case for this project might look something like this:

Test Case 1: Testing Full House 
 Input: 6-3-6-3-3, Full House Expected Output: 25 points scored in Full House category Actual Output: Pass/Fail: [ ] Pass [ ] Fail

However, this presents a challenge for Alice: Is Alice supposed to play the game over and over again until she happens to be lucky enough to roll a 6-3-6-3-3?

Also, as the number of test cases grows, it becomes more time-consuming to fully test the program with each new iteration. For this reason, you will utilize automated testing for this project.

Your automated tests will run your test cases on your code. Each test case requires the values of the five dice as input, plus a selection of where on the scorecard this should be played in the game. The input and expected output are both coded into the testing harness, and the harness software will monitor whether or not the code being tested returns the correct output, informing the user of the results of the tests using output from the test harness.

IV. Wikipedia excerpt (Test Harness):

In software testing, a test harness or automated test framework is a collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs. It has two main parts: the test execution engine and the test script repository.

Test harnesses allow for the automation of tests. They can call functions with supplied parameters and print out and compare the results to the desired value. The test harness is a hook to the developed code, which can be tested using an automation framework.

These individual objectives may be fulfilled by unit test framework tools, stubs or drivers.

A test harness may provide some of the following benefits: • Increased productivity due to automation of the testing process. • Increased probability that regression testing will occur. • Repeatability of subsequent test runs. • Offline testing (e.g. at times that the office is not staffed, like overnight). • Access to conditions and/or use cases that are otherwise difficult to simulate

V. In the end, your program structure should look like this:

The testing harness will run several test cases, feeding them one at a time into your Yahtzee software, comparing the actual result from your Yahtzee software with the expected result specified in the testing harness.

VI. Test Plan Guidance

Ultimately, your test plan should test each category enough times to increase confidence that your Yahtzee software is working correctly. You are encouraged to use techniques discussed in class to accomplish this.

Eventually, your test cases should be grouped into sets of 13, so that a full game can be played over the course of 13 turns. This will allow you to test the subtotals, the upper section bonus, and the total for the game, as well as each individual scoresheet entry.

I would expect your final automated test plan to play at least 3 (if not 4) complete games, in order to strategically test various combinations of possible results.

VII. Project Scope

To summarize, in this project, you are to:

• Write requirements for the Yahtzee scoresheet software • Write test cases to ensure all the requirements are correctly met • Write software that returns a correct score for each entry on a Yahtzee score sheet • Incorporate your test cases into an automated test plan • Write a testing harness to run the automated test plan against your Yahtzee software

Do not get sidetracked doing error testing, such as trying to pass six dice parameters instead of five, or passing a 7 as one of the 6-sided dice values. There is enough to do in this project without testing for erroneous input. Focus instead on being thorough with the goals specified in the project itself.

VIII. Your FINAL turn-in (i.e., the project) should be organized as follows (i.e., in this order):

1. Full project requirements (these can be assembled over time as you work on different phases of the project).

2. A full paper-based test plan (assembled, individual test cases), including space for pass/fail results, written out as though Alice were going to run this test plan.

3. Output from the complete automated test plan being run on the final version of the code. 4. (Optional but encouraged) A short write-up on your assumptions, rationale, and/or lessons

learned. 5. Code for the Test Harness software. 6. Code for the Yahtzee software.

IX. Project Schedule

The final project is due on the Monday before Thanksgiving. Each Wednesday in the interim, a partial version of the project is turned-in, graded as a HW assignment.

HW3A – Write the requirements and the test cases for the Top Section of the Yahtzee scoring sheet. No code is required for HW3A.

HW3B – Complete the Yahtzee game code for the requirements specified in HW3A. Write the requirements and test cases 3-of-a-kind, 4-of-a-kind, Chance, and Yahtzee categories.

HW3C – Complete the Yahtzee game code for the requirements specified in HW3B. Write the requirements and test cases for the Full House, Small and Large Straight, and the Yahtzee Bonuses.

HW3D – Complete the Yahtzee game code for the requirements specified in HW3C. Assemble all test cases into sets of 13 so that they can be run as a full game. Be sure to specify the expected values for the Upper Section (both pre- and post-bonus), the Lower Section, and the Total for each game.

HW3E – Write the testing harness. Ideally, your harness gets its input by reading each test case from a file, so that the test plan can be easily changed yet still be automated without recompiling the test harness. HW3E should include the output from one complete game (13 test cases).

Final Project – Automate the full test plan (three or four games total) developed in the previous HW assignments. Run this and print the results, and incorporate it into your final project as specified in Section VIII.

Appendix A – Yahtzee Rules Objective of the Game

The game consists of 13 rounds. In each round, a player rolls the dice and then scores the roll in one of 13 categories. Players must score once in each category – which means that a player may have to settle for scoring zero in some categories before the end of the game. The score is determined by a different rule for each category; see the section on Scoring below.

The object of the game is to maximize the player’s total score. The game ends once all 13 categories have been scored.

Rolling the Dice

The game uses five dice, all of which are rolled at the start of each round. After all five dice have been rerolled, you can either score the current roll, or re-roll any (or all) of the five dice.
 
 The dice can be rolled for a total of up to three times per round: the initial roll (in which all five dice are rolled), plus two re-rolls of any or all dice. (Note: You do NOT implement this functionality for your project.)

After the three rolls are complete, the player must score the roll, at which point the process is repeated beginning with another roll of all five dice. This continues until all 13 categories have been filled, at which time the game is over.

Scoring

Once the rolling for each round is complete, the roll is scored in one of 13 categories. The categories are divided into two regions: Upper Scores and Lower Scores. (These names are a holdover from the original version of the game, which was scored with paper sheets; see Appendix B.) Once a category has been scored, it is closed out for the rest of the game; it cannot be changed. Each category has unique scoring rules as described below.

Upper Scores

In the upper scores, you total only the specified die face. So, for example, if the end roll was:

4 – 4 – 3 – 4 – 6

then score for the 4’s category would be 12 (4 + 4 + 4). This same roll would yield zero points if you scored it in the 1’s, 2’s, or 5’s category, but it would score three points if it was scored it in the 3’s category, or six points if it was scored it in the 6’s category. 
 When the game is over, a score of 63 or more in the Upper Scores area (which happens to equal the sum if 3 die faces per category are scored) earns an upper bonus of 35 points.

Lower Scores

In the lower scores, you score either a set amount (defined by the category), or zero if you don’t satisfy the category requirements, as follows:

3-of-a-Kind & 4-of-a-Kind

For 3-of-a-Kind to score more than zero, there must be at least three of the same value on three dice. If so, the score is the sum total of all five die faces. 4-of-a-Kind, is scored the same way, except at least four of the five dice must have the same value. So, for example, if a player rolled:

4 – 4 – 3 – 4 – 6

the player would score 21 points in the 3-of-a-Kind category, but zero points in 4-of-a-Kind.

Full House

Much like in poker, a Full House is a roll with both 3-of-a-Kind, plus a pair. A full house always scores 25 points (if the dice indeed show a full house), or zero points. For example:

4 – 4 – 3 – 4 – 6 (zero points for full house) 4 – 4 – 3 – 4 – 3 (25 points for full house)

Small Straight & Large Straight

Also similar to poker, a straight is a sequence of consecutive die values. On the Yahtzee scorecard, a Small Straight is 4 consecutive values and scores 30 points; a Large Straight is 5 consecutive values and scores 40 points. For example:

4 – 4 – 3 – 4 – 6 (zero points for small straight and zero points for large straight) 4 – 4 – 3 – 5 – 6 (30 points for small straight and zero points for large straight) 2 – 4 – 3 – 5 – 6 (30 points for small straight and 40 points for large straight)

Chance 
 Chance is a catch-all roll. The point total is simply the total of all five dice values. 
 Yahtzee

A Yahtzee is a 5-of-a-Kind (i.e. all the die faces are the same), and it scores 50 points. If a player rolls more than one Yahtzee in a single game, a 100-point bonus will be earned for each additional Yahtzee roll, provided that the player has already scored a 50 in the Yahtzee category. If the Yahtzee category has not been scored, or has been scored with a zero, the 100-point bonus is not awarded.

Jokers

A 5-of-a-kind can also be used as a joker in the lower scores section, provided the following criteria are both satisfied:

1. The Yahtzee category has already been filled with a 0 or 50. 2. The corresponding category in the upper scores section has been filled.

For example, if you have rolled:

4 – 4 – 4 – 4 – 4 
 
 then the 4’s category and the Yahtzee category must be filled in order to use this roll as a joker.

A joker allows you to use the Yahtzee to get full credit in any lower scores category. For example:

4 – 4 – 4 – 4 – 4

could score 40 points in the Large Straight category, provided the aforementioned criteria are met.

Note: Joker functionality is NOT required if you are enrolled in 3110. However, if you are a graduate student taking 5110, you must incorporate the joker functionality into your program, and this functionality must be tested as part of your test script.

Appendix B – Yahtzee Score Sheet

Appendix C – Sample Work

Sample Requirement:

When scoring the 3-of-a-kind category, the program shall ensure at least three dice have the same value. If so, the program shall add all 5 dice and assign that sum as the score for 3-of-a-kind. If not, the program shall assign a zero for 3-of-a-kind.

Sample Preliminary Test Cases:

Check to ensure 3-of-a-Kind works properly in all the following scenarios:

Dice show 3-of-a-kind with 2 other numbers (e.g., 4-4-4-1-3; sum would be 4+4+4+1+3 = 16) 
 Dice show 4-of-a-kind with 1 other number (e.g., 4-4-4-4-3; sum would be 4+4+4+4+3 = 19) Dice show two pair but not 3-of-a-kind (e.g., 4-4-5-5-3; total pts awarded would be 0)

Sample Cases Written in Tabular Form:

Test Case Testing Dice Select Exp. Result Act. Result P/F 1a 3-of-a-kind 4-4-4-1-3 3-of-a-kind 16 1b 3-of-a-kind 4-4-3-4-4 3-of-a-kind 19 
 1c 3-of-a-kind 4-4-5-5-3 3-of-a-kind 0

Note: The three test cases shown above could be used in three different games in the automated test.

For this case, the automated test file could have a line of text that reads something like:

1A, 7, 4-4-4-1-3, 16

and the test harness might print out something like the following:

Now running test 1A…

Dice values are: 4 4 4 1 3 Player selects 3-of-a-kind

Sending input to Yahtzee program…

Result returned: 16 points Expected result: 16

Results match – test PASSES