diff options
author | Haidong Ji | 2022-04-15 15:51:30 -0500 |
---|---|---|
committer | Haidong Ji | 2022-04-15 15:51:30 -0500 |
commit | 442a49ad5a48d417345959b903ae6a6d32d55759 (patch) | |
tree | c7127bb497e5e439018b1915e0136eec2c9cb124 /09_testing2 |
Excellent fundamentals and displine training, many tools and techniques
exercises: gdb, emacs, valgrind, git
Diffstat (limited to '09_testing2')
-rw-r--r-- | 09_testing2/README | 64 | ||||
-rw-r--r-- | 09_testing2/grade.txt | 300 | ||||
-rwxr-xr-x | 09_testing2/run_all.sh | 33 | ||||
-rw-r--r-- | 09_testing2/tests.txt | 22 |
4 files changed, 419 insertions, 0 deletions
diff --git a/09_testing2/README b/09_testing2/README new file mode 100644 index 0000000..8081bf0 --- /dev/null +++ b/09_testing2/README @@ -0,0 +1,64 @@ +For this assignment, you will continue working on your blackbox testing +skills. However, this time, you will test a slightly more complex program +with more broken implementations. + +You can find the correct implementation of the program at + +/usr/local/l2p/match5/correct-match5 + +This program determines who wins a very simple "card game". This "game" +is played with 5 "cards" per player and 2 players. A "card" in this +game is an ASCII character (so 'x' 'a' '!' '7' '$' etc are all valid cards). +Each player has exactly 5 cards, and the player with the most matching +cards wins. That is, 5 of a kind beats 4 of a kind, which bests 3 of a kind, +which beats a pair, which beats having no matching cards. If both +players have the same number of matching cards (e.g., both have 3 of a kind), +the hand is a tie. There are no tie breakers (so one player has +3 of a kind and the other has 3 of a kind + a pair, the pair does not matter). +The values of the cards don't matter (so three 'a's and three 'b's both tie). + +If you run the correct implementation, you can see the correct behavior: + +As some examples: + +$ /usr/local/l2p/match5/correct-match5 aaaaa bbbbb +Both hands tie: each has five of a kind + +$ /usr/local/l2p/match5/correct-match5 aaaaa bbbbc +Hand 1's five of a kind beats Hand 2's four of a kind + +$ /usr/local/l2p/match5/correct-match5 aaabb bbbbc +Hand 1's three of a kind loses to Hand 2's four of a kind + +If you look in /usr/local/l2p/match5, you will see that there are 296 broken +implementations. Note that you do NOT need to come up with 296 test cases---you +may have one test case which shows that several of these implementations +are broken. + +Furthermore, to help make this testing managable for you, we have provided +run_all.sh, which will read test cases from a file you create called tests.txt, +and run them against all broken implementations. In particular, each line +of tests.txt should be the command line arguments to one invocation of the +program under test. So if you wanted to test the program with the three +examples shown above, you would write the following in tests.txt + +aaaaa bbbbb +aaaaa bbbbc +aaabb bbbbc + +and then each broken program would be run three times, and have its behavior +compared to the correct implementation. If ANY test case in tests.txt identifies +a problem with a particular broken implementation, your test suite has +succeeded for that particular implementation. You need to develop +your test suite so that it identifies the problems with all broken implementations. + + +Lastly, I'll note that each of these broken implementations arises from +making small changes in the correct implementation which could represent +reasonable mistakes in programming (this is called "mutation testing" by +the way). None of these require finding +an obscure "magic bullet" input---they can all be found with a reasonble, +comprehensive test suite. + +You will submit tests.txt for this assignment. + diff --git a/09_testing2/grade.txt b/09_testing2/grade.txt new file mode 100644 index 0000000..41bfbf6 --- /dev/null +++ b/09_testing2/grade.txt @@ -0,0 +1,300 @@ +Grading at Mon 29 Nov 2021 02:07:09 AM UTC +Your test cases identified the problem with match5-000 +Your test cases identified the problem with match5-001 +Your test cases identified the problem with match5-002 +Your test cases identified the problem with match5-003 +Your test cases identified the problem with match5-004 +Your test cases identified the problem with match5-005 +Your test cases identified the problem with match5-006 +Your test cases identified the problem with match5-007 +Your test cases identified the problem with match5-008 +Your test cases identified the problem with match5-009 +Your test cases identified the problem with match5-010 +Your test cases identified the problem with match5-011 +Your test cases identified the problem with match5-012 +Your test cases identified the problem with match5-013 +Your test cases identified the problem with match5-014 +Your test cases identified the problem with match5-015 +Your test cases identified the problem with match5-016 +Your test cases identified the problem with match5-017 +Your test cases identified the problem with match5-018 +Your test cases identified the problem with match5-019 +Your test cases identified the problem with match5-020 +Your test cases identified the problem with match5-021 +Your test cases identified the problem with match5-022 +Your test cases identified the problem with match5-023 +Your test cases identified the problem with match5-024 +Your test cases identified the problem with match5-025 +Your test cases identified the problem with match5-026 +Your test cases identified the problem with match5-027 +Your test cases identified the problem with match5-028 +Your test cases identified the problem with match5-029 +Your test cases identified the problem with match5-030 +Your test cases identified the problem with match5-031 +Your test cases identified the problem with match5-032 +Your test cases identified the problem with match5-033 +Your test cases identified the problem with match5-034 +Your test cases identified the problem with match5-035 +Your test cases identified the problem with match5-036 +Your test cases identified the problem with match5-037 +Your test cases identified the problem with match5-038 +Your test cases identified the problem with match5-039 +Your test cases identified the problem with match5-040 +Your test cases identified the problem with match5-041 +Your test cases identified the problem with match5-042 +Your test cases identified the problem with match5-043 +Your test cases identified the problem with match5-044 +Your test cases identified the problem with match5-045 +Your test cases identified the problem with match5-046 +Your test cases identified the problem with match5-047 +Your test cases identified the problem with match5-048 +Your test cases identified the problem with match5-049 +Your test cases identified the problem with match5-050 +Your test cases identified the problem with match5-051 +Your test cases identified the problem with match5-052 +Your test cases identified the problem with match5-053 +Your test cases identified the problem with match5-054 +Your test cases identified the problem with match5-055 +Your test cases identified the problem with match5-056 +Your test cases identified the problem with match5-057 +Your test cases identified the problem with match5-058 +Your test cases identified the problem with match5-059 +Your test cases identified the problem with match5-060 +Your test cases identified the problem with match5-061 +Your test cases identified the problem with match5-062 +Your test cases identified the problem with match5-063 +Your test cases identified the problem with match5-064 +Your test cases identified the problem with match5-065 +Your test cases identified the problem with match5-066 +Your test cases identified the problem with match5-067 +Your test cases identified the problem with match5-068 +Your test cases identified the problem with match5-069 +Your test cases identified the problem with match5-070 +Your test cases identified the problem with match5-071 +Your test cases identified the problem with match5-072 +Your test cases identified the problem with match5-073 +Your test cases identified the problem with match5-074 +Your test cases identified the problem with match5-075 +Your test cases identified the problem with match5-076 +Your test cases identified the problem with match5-077 +Your test cases identified the problem with match5-078 +Your test cases identified the problem with match5-079 +Your test cases identified the problem with match5-080 +Your test cases identified the problem with match5-081 +Your test cases identified the problem with match5-082 +Your test cases identified the problem with match5-083 +Your test cases identified the problem with match5-084 +Your test cases identified the problem with match5-085 +Your test cases identified the problem with match5-086 +Your test cases identified the problem with match5-087 +Your test cases identified the problem with match5-088 +Your test cases identified the problem with match5-089 +Your test cases identified the problem with match5-090 +Your test cases identified the problem with match5-091 +Your test cases identified the problem with match5-092 +Your test cases identified the problem with match5-093 +Your test cases identified the problem with match5-094 +Your test cases identified the problem with match5-095 +Your test cases identified the problem with match5-096 +Your test cases identified the problem with match5-097 +Your test cases identified the problem with match5-098 +Your test cases identified the problem with match5-099 +Your test cases identified the problem with match5-100 +Your test cases identified the problem with match5-101 +Your test cases identified the problem with match5-102 +Your test cases identified the problem with match5-103 +Your test cases identified the problem with match5-104 +Your test cases identified the problem with match5-105 +Your test cases identified the problem with match5-106 +Your test cases identified the problem with match5-107 +Your test cases identified the problem with match5-108 +Your test cases identified the problem with match5-109 +Your test cases identified the problem with match5-110 +Your test cases identified the problem with match5-111 +Your test cases identified the problem with match5-112 +Your test cases identified the problem with match5-113 +Your test cases identified the problem with match5-114 +Your test cases identified the problem with match5-115 +Your test cases identified the problem with match5-116 +Your test cases identified the problem with match5-117 +Your test cases identified the problem with match5-118 +Your test cases identified the problem with match5-119 +Your test cases identified the problem with match5-120 +Your test cases identified the problem with match5-121 +Your test cases identified the problem with match5-122 +Your test cases identified the problem with match5-123 +Your test cases identified the problem with match5-124 +Your test cases identified the problem with match5-125 +Your test cases identified the problem with match5-126 +Your test cases identified the problem with match5-127 +Your test cases identified the problem with match5-128 +Your test cases identified the problem with match5-129 +Your test cases identified the problem with match5-130 +Your test cases identified the problem with match5-131 +Your test cases identified the problem with match5-132 +Your test cases identified the problem with match5-133 +Your test cases identified the problem with match5-134 +Your test cases identified the problem with match5-135 +Your test cases identified the problem with match5-136 +Your test cases identified the problem with match5-137 +Your test cases identified the problem with match5-138 +Your test cases identified the problem with match5-139 +Your test cases identified the problem with match5-140 +Your test cases identified the problem with match5-141 +Your test cases identified the problem with match5-142 +Your test cases identified the problem with match5-143 +Your test cases identified the problem with match5-144 +Your test cases identified the problem with match5-145 +Your test cases identified the problem with match5-146 +Your test cases identified the problem with match5-147 +Your test cases identified the problem with match5-148 +Your test cases identified the problem with match5-149 +Your test cases identified the problem with match5-150 +Your test cases identified the problem with match5-151 +Your test cases identified the problem with match5-152 +Your test cases identified the problem with match5-153 +Your test cases identified the problem with match5-154 +Your test cases identified the problem with match5-155 +Your test cases identified the problem with match5-156 +Your test cases identified the problem with match5-157 +Your test cases identified the problem with match5-158 +Your test cases identified the problem with match5-159 +Your test cases identified the problem with match5-160 +Your test cases identified the problem with match5-161 +Your test cases identified the problem with match5-162 +Your test cases identified the problem with match5-163 +Your test cases identified the problem with match5-164 +Your test cases identified the problem with match5-165 +Your test cases identified the problem with match5-166 +Your test cases identified the problem with match5-167 +Your test cases identified the problem with match5-168 +Your test cases identified the problem with match5-169 +Your test cases identified the problem with match5-170 +Your test cases identified the problem with match5-171 +Your test cases identified the problem with match5-172 +Your test cases identified the problem with match5-173 +Your test cases identified the problem with match5-174 +Your test cases identified the problem with match5-175 +Your test cases identified the problem with match5-176 +Your test cases identified the problem with match5-177 +Your test cases identified the problem with match5-178 +Your test cases identified the problem with match5-179 +Your test cases identified the problem with match5-180 +Your test cases identified the problem with match5-181 +Your test cases identified the problem with match5-182 +Your test cases identified the problem with match5-183 +Your test cases identified the problem with match5-184 +Your test cases identified the problem with match5-185 +Your test cases identified the problem with match5-186 +Your test cases identified the problem with match5-187 +Your test cases identified the problem with match5-188 +Your test cases identified the problem with match5-189 +Your test cases identified the problem with match5-190 +Your test cases identified the problem with match5-191 +Your test cases identified the problem with match5-192 +Your test cases identified the problem with match5-193 +Your test cases identified the problem with match5-194 +Your test cases identified the problem with match5-195 +Your test cases identified the problem with match5-196 +Your test cases identified the problem with match5-197 +Your test cases identified the problem with match5-198 +Your test cases identified the problem with match5-199 +Your test cases identified the problem with match5-200 +Your test cases identified the problem with match5-201 +Your test cases identified the problem with match5-202 +Your test cases identified the problem with match5-203 +Your test cases identified the problem with match5-204 +Your test cases identified the problem with match5-205 +Your test cases identified the problem with match5-206 +Your test cases identified the problem with match5-207 +Your test cases identified the problem with match5-208 +Your test cases identified the problem with match5-209 +Your test cases identified the problem with match5-210 +Your test cases identified the problem with match5-211 +Your test cases identified the problem with match5-212 +Your test cases identified the problem with match5-213 +Your test cases identified the problem with match5-214 +Your test cases identified the problem with match5-215 +Your test cases identified the problem with match5-216 +Your test cases identified the problem with match5-217 +Your test cases identified the problem with match5-218 +Your test cases identified the problem with match5-219 +Your test cases identified the problem with match5-220 +Your test cases identified the problem with match5-221 +Your test cases identified the problem with match5-222 +Your test cases identified the problem with match5-223 +Your test cases identified the problem with match5-224 +Your test cases identified the problem with match5-225 +Your test cases identified the problem with match5-226 +Your test cases identified the problem with match5-227 +Your test cases identified the problem with match5-228 +Your test cases identified the problem with match5-229 +Your test cases identified the problem with match5-230 +Your test cases identified the problem with match5-231 +Your test cases identified the problem with match5-232 +Your test cases identified the problem with match5-233 +Your test cases identified the problem with match5-234 +Your test cases identified the problem with match5-235 +Your test cases identified the problem with match5-236 +Your test cases identified the problem with match5-237 +Your test cases identified the problem with match5-238 +Your test cases identified the problem with match5-239 +Your test cases identified the problem with match5-240 +Your test cases identified the problem with match5-241 +Your test cases identified the problem with match5-242 +Your test cases identified the problem with match5-243 +Your test cases identified the problem with match5-244 +Your test cases identified the problem with match5-245 +Your test cases identified the problem with match5-246 +Your test cases identified the problem with match5-247 +Your test cases identified the problem with match5-248 +Your test cases identified the problem with match5-249 +Your test cases identified the problem with match5-250 +Your test cases identified the problem with match5-251 +Your test cases identified the problem with match5-252 +Your test cases identified the problem with match5-253 +Your test cases identified the problem with match5-254 +Your test cases identified the problem with match5-255 +Your test cases identified the problem with match5-256 +Your test cases identified the problem with match5-257 +Your test cases identified the problem with match5-258 +Your test cases identified the problem with match5-259 +Your test cases identified the problem with match5-260 +Your test cases identified the problem with match5-261 +Your test cases identified the problem with match5-262 +Your test cases identified the problem with match5-263 +Your test cases identified the problem with match5-264 +Your test cases identified the problem with match5-265 +Your test cases identified the problem with match5-266 +Your test cases identified the problem with match5-267 +Your test cases identified the problem with match5-268 +Your test cases identified the problem with match5-269 +Your test cases identified the problem with match5-270 +Your test cases identified the problem with match5-271 +Your test cases identified the problem with match5-272 +Your test cases identified the problem with match5-273 +Your test cases identified the problem with match5-274 +Your test cases identified the problem with match5-275 +Your test cases identified the problem with match5-276 +Your test cases identified the problem with match5-277 +Your test cases identified the problem with match5-278 +Your test cases identified the problem with match5-279 +Your test cases identified the problem with match5-280 +Your test cases identified the problem with match5-281 +Your test cases identified the problem with match5-282 +Your test cases identified the problem with match5-283 +Your test cases identified the problem with match5-284 +Your test cases identified the problem with match5-285 +Your test cases identified the problem with match5-286 +Your test cases identified the problem with match5-287 +Your test cases identified the problem with match5-288 +Your test cases identified the problem with match5-289 +Your test cases identified the problem with match5-290 +Your test cases identified the problem with match5-291 +Your test cases identified the problem with match5-292 +Your test cases identified the problem with match5-293 +Your test cases identified the problem with match5-294 +Your test cases identified the problem with match5-295 +Your test cases identified the problem with match5-296 + +Overall Grade: PASSED diff --git a/09_testing2/run_all.sh b/09_testing2/run_all.sh new file mode 100755 index 0000000..e62c149 --- /dev/null +++ b/09_testing2/run_all.sh @@ -0,0 +1,33 @@ +#!/bin/bash +run_test(){ + prog="$1" + testfile="$2" + IFS=$'\n' + for line in `cat $testfile` + do + IFS=" " correct=`/usr/local/l2p/match5/correct-match5 $line 2>&1` + IFS=" " broken=`$prog $line 2>&1` + if [ "$broken" != "$correct" ] + then + return 0 + fi + done + return 1 +} + +found=0 +notfound=0 +for i in /usr/local/l2p/match5/match5-* +do + run_test $i tests.txt + x="$?" + if [ "$x" != "0" ] + then + echo "Your test cases did not identify the problem with `basename $i`" + let notfound=${notfound}+1 + else + let found=${found}+1 + fi +done +echo "Test cases identified $found problems" +echo "Test cases failed to identify $notfound problems" diff --git a/09_testing2/tests.txt b/09_testing2/tests.txt new file mode 100644 index 0000000..ca82980 --- /dev/null +++ b/09_testing2/tests.txt @@ -0,0 +1,22 @@ +aaaaa bbbbb +aaaaa bbbbc +aaabb bbbbc +aaaaa aaaaa +aaaa bbbbb +aaaaaa bbbbb +aaaaa bbbb +aaaaa bbbbbb +%%&*( 01234 +aaaaa + +&^%$# *&^%$ +aabbc lmddd +----- ~~~~~ +||||| &&&&& +""""" ''''' +NULL 00000 +aaaab bbbbc ddddd +bbbba bbbba +abbbb abbbb +aabbb acbbb +bbbcc bbbca |