summaryrefslogtreecommitdiff
path: root/09_testing2
diff options
context:
space:
mode:
authorHaidong Ji2022-04-15 15:51:30 -0500
committerHaidong Ji2022-04-15 15:51:30 -0500
commit442a49ad5a48d417345959b903ae6a6d32d55759 (patch)
treec7127bb497e5e439018b1915e0136eec2c9cb124 /09_testing2
Great C programming funHEADmaster
Excellent fundamentals and displine training, many tools and techniques exercises: gdb, emacs, valgrind, git
Diffstat (limited to '09_testing2')
-rw-r--r--09_testing2/README64
-rw-r--r--09_testing2/grade.txt300
-rwxr-xr-x09_testing2/run_all.sh33
-rw-r--r--09_testing2/tests.txt22
4 files changed, 419 insertions, 0 deletions
diff --git a/09_testing2/README b/09_testing2/README
new file mode 100644
index 0000000..8081bf0
--- /dev/null
+++ b/09_testing2/README
@@ -0,0 +1,64 @@
+For this assignment, you will continue working on your blackbox testing
+skills. However, this time, you will test a slightly more complex program
+with more broken implementations.
+
+You can find the correct implementation of the program at
+
+/usr/local/l2p/match5/correct-match5
+
+This program determines who wins a very simple "card game". This "game"
+is played with 5 "cards" per player and 2 players. A "card" in this
+game is an ASCII character (so 'x' 'a' '!' '7' '$' etc are all valid cards).
+Each player has exactly 5 cards, and the player with the most matching
+cards wins. That is, 5 of a kind beats 4 of a kind, which bests 3 of a kind,
+which beats a pair, which beats having no matching cards. If both
+players have the same number of matching cards (e.g., both have 3 of a kind),
+the hand is a tie. There are no tie breakers (so one player has
+3 of a kind and the other has 3 of a kind + a pair, the pair does not matter).
+The values of the cards don't matter (so three 'a's and three 'b's both tie).
+
+If you run the correct implementation, you can see the correct behavior:
+
+As some examples:
+
+$ /usr/local/l2p/match5/correct-match5 aaaaa bbbbb
+Both hands tie: each has five of a kind
+
+$ /usr/local/l2p/match5/correct-match5 aaaaa bbbbc
+Hand 1's five of a kind beats Hand 2's four of a kind
+
+$ /usr/local/l2p/match5/correct-match5 aaabb bbbbc
+Hand 1's three of a kind loses to Hand 2's four of a kind
+
+If you look in /usr/local/l2p/match5, you will see that there are 296 broken
+implementations. Note that you do NOT need to come up with 296 test cases---you
+may have one test case which shows that several of these implementations
+are broken.
+
+Furthermore, to help make this testing managable for you, we have provided
+run_all.sh, which will read test cases from a file you create called tests.txt,
+and run them against all broken implementations. In particular, each line
+of tests.txt should be the command line arguments to one invocation of the
+program under test. So if you wanted to test the program with the three
+examples shown above, you would write the following in tests.txt
+
+aaaaa bbbbb
+aaaaa bbbbc
+aaabb bbbbc
+
+and then each broken program would be run three times, and have its behavior
+compared to the correct implementation. If ANY test case in tests.txt identifies
+a problem with a particular broken implementation, your test suite has
+succeeded for that particular implementation. You need to develop
+your test suite so that it identifies the problems with all broken implementations.
+
+
+Lastly, I'll note that each of these broken implementations arises from
+making small changes in the correct implementation which could represent
+reasonable mistakes in programming (this is called "mutation testing" by
+the way). None of these require finding
+an obscure "magic bullet" input---they can all be found with a reasonble,
+comprehensive test suite.
+
+You will submit tests.txt for this assignment.
+
diff --git a/09_testing2/grade.txt b/09_testing2/grade.txt
new file mode 100644
index 0000000..41bfbf6
--- /dev/null
+++ b/09_testing2/grade.txt
@@ -0,0 +1,300 @@
+Grading at Mon 29 Nov 2021 02:07:09 AM UTC
+Your test cases identified the problem with match5-000
+Your test cases identified the problem with match5-001
+Your test cases identified the problem with match5-002
+Your test cases identified the problem with match5-003
+Your test cases identified the problem with match5-004
+Your test cases identified the problem with match5-005
+Your test cases identified the problem with match5-006
+Your test cases identified the problem with match5-007
+Your test cases identified the problem with match5-008
+Your test cases identified the problem with match5-009
+Your test cases identified the problem with match5-010
+Your test cases identified the problem with match5-011
+Your test cases identified the problem with match5-012
+Your test cases identified the problem with match5-013
+Your test cases identified the problem with match5-014
+Your test cases identified the problem with match5-015
+Your test cases identified the problem with match5-016
+Your test cases identified the problem with match5-017
+Your test cases identified the problem with match5-018
+Your test cases identified the problem with match5-019
+Your test cases identified the problem with match5-020
+Your test cases identified the problem with match5-021
+Your test cases identified the problem with match5-022
+Your test cases identified the problem with match5-023
+Your test cases identified the problem with match5-024
+Your test cases identified the problem with match5-025
+Your test cases identified the problem with match5-026
+Your test cases identified the problem with match5-027
+Your test cases identified the problem with match5-028
+Your test cases identified the problem with match5-029
+Your test cases identified the problem with match5-030
+Your test cases identified the problem with match5-031
+Your test cases identified the problem with match5-032
+Your test cases identified the problem with match5-033
+Your test cases identified the problem with match5-034
+Your test cases identified the problem with match5-035
+Your test cases identified the problem with match5-036
+Your test cases identified the problem with match5-037
+Your test cases identified the problem with match5-038
+Your test cases identified the problem with match5-039
+Your test cases identified the problem with match5-040
+Your test cases identified the problem with match5-041
+Your test cases identified the problem with match5-042
+Your test cases identified the problem with match5-043
+Your test cases identified the problem with match5-044
+Your test cases identified the problem with match5-045
+Your test cases identified the problem with match5-046
+Your test cases identified the problem with match5-047
+Your test cases identified the problem with match5-048
+Your test cases identified the problem with match5-049
+Your test cases identified the problem with match5-050
+Your test cases identified the problem with match5-051
+Your test cases identified the problem with match5-052
+Your test cases identified the problem with match5-053
+Your test cases identified the problem with match5-054
+Your test cases identified the problem with match5-055
+Your test cases identified the problem with match5-056
+Your test cases identified the problem with match5-057
+Your test cases identified the problem with match5-058
+Your test cases identified the problem with match5-059
+Your test cases identified the problem with match5-060
+Your test cases identified the problem with match5-061
+Your test cases identified the problem with match5-062
+Your test cases identified the problem with match5-063
+Your test cases identified the problem with match5-064
+Your test cases identified the problem with match5-065
+Your test cases identified the problem with match5-066
+Your test cases identified the problem with match5-067
+Your test cases identified the problem with match5-068
+Your test cases identified the problem with match5-069
+Your test cases identified the problem with match5-070
+Your test cases identified the problem with match5-071
+Your test cases identified the problem with match5-072
+Your test cases identified the problem with match5-073
+Your test cases identified the problem with match5-074
+Your test cases identified the problem with match5-075
+Your test cases identified the problem with match5-076
+Your test cases identified the problem with match5-077
+Your test cases identified the problem with match5-078
+Your test cases identified the problem with match5-079
+Your test cases identified the problem with match5-080
+Your test cases identified the problem with match5-081
+Your test cases identified the problem with match5-082
+Your test cases identified the problem with match5-083
+Your test cases identified the problem with match5-084
+Your test cases identified the problem with match5-085
+Your test cases identified the problem with match5-086
+Your test cases identified the problem with match5-087
+Your test cases identified the problem with match5-088
+Your test cases identified the problem with match5-089
+Your test cases identified the problem with match5-090
+Your test cases identified the problem with match5-091
+Your test cases identified the problem with match5-092
+Your test cases identified the problem with match5-093
+Your test cases identified the problem with match5-094
+Your test cases identified the problem with match5-095
+Your test cases identified the problem with match5-096
+Your test cases identified the problem with match5-097
+Your test cases identified the problem with match5-098
+Your test cases identified the problem with match5-099
+Your test cases identified the problem with match5-100
+Your test cases identified the problem with match5-101
+Your test cases identified the problem with match5-102
+Your test cases identified the problem with match5-103
+Your test cases identified the problem with match5-104
+Your test cases identified the problem with match5-105
+Your test cases identified the problem with match5-106
+Your test cases identified the problem with match5-107
+Your test cases identified the problem with match5-108
+Your test cases identified the problem with match5-109
+Your test cases identified the problem with match5-110
+Your test cases identified the problem with match5-111
+Your test cases identified the problem with match5-112
+Your test cases identified the problem with match5-113
+Your test cases identified the problem with match5-114
+Your test cases identified the problem with match5-115
+Your test cases identified the problem with match5-116
+Your test cases identified the problem with match5-117
+Your test cases identified the problem with match5-118
+Your test cases identified the problem with match5-119
+Your test cases identified the problem with match5-120
+Your test cases identified the problem with match5-121
+Your test cases identified the problem with match5-122
+Your test cases identified the problem with match5-123
+Your test cases identified the problem with match5-124
+Your test cases identified the problem with match5-125
+Your test cases identified the problem with match5-126
+Your test cases identified the problem with match5-127
+Your test cases identified the problem with match5-128
+Your test cases identified the problem with match5-129
+Your test cases identified the problem with match5-130
+Your test cases identified the problem with match5-131
+Your test cases identified the problem with match5-132
+Your test cases identified the problem with match5-133
+Your test cases identified the problem with match5-134
+Your test cases identified the problem with match5-135
+Your test cases identified the problem with match5-136
+Your test cases identified the problem with match5-137
+Your test cases identified the problem with match5-138
+Your test cases identified the problem with match5-139
+Your test cases identified the problem with match5-140
+Your test cases identified the problem with match5-141
+Your test cases identified the problem with match5-142
+Your test cases identified the problem with match5-143
+Your test cases identified the problem with match5-144
+Your test cases identified the problem with match5-145
+Your test cases identified the problem with match5-146
+Your test cases identified the problem with match5-147
+Your test cases identified the problem with match5-148
+Your test cases identified the problem with match5-149
+Your test cases identified the problem with match5-150
+Your test cases identified the problem with match5-151
+Your test cases identified the problem with match5-152
+Your test cases identified the problem with match5-153
+Your test cases identified the problem with match5-154
+Your test cases identified the problem with match5-155
+Your test cases identified the problem with match5-156
+Your test cases identified the problem with match5-157
+Your test cases identified the problem with match5-158
+Your test cases identified the problem with match5-159
+Your test cases identified the problem with match5-160
+Your test cases identified the problem with match5-161
+Your test cases identified the problem with match5-162
+Your test cases identified the problem with match5-163
+Your test cases identified the problem with match5-164
+Your test cases identified the problem with match5-165
+Your test cases identified the problem with match5-166
+Your test cases identified the problem with match5-167
+Your test cases identified the problem with match5-168
+Your test cases identified the problem with match5-169
+Your test cases identified the problem with match5-170
+Your test cases identified the problem with match5-171
+Your test cases identified the problem with match5-172
+Your test cases identified the problem with match5-173
+Your test cases identified the problem with match5-174
+Your test cases identified the problem with match5-175
+Your test cases identified the problem with match5-176
+Your test cases identified the problem with match5-177
+Your test cases identified the problem with match5-178
+Your test cases identified the problem with match5-179
+Your test cases identified the problem with match5-180
+Your test cases identified the problem with match5-181
+Your test cases identified the problem with match5-182
+Your test cases identified the problem with match5-183
+Your test cases identified the problem with match5-184
+Your test cases identified the problem with match5-185
+Your test cases identified the problem with match5-186
+Your test cases identified the problem with match5-187
+Your test cases identified the problem with match5-188
+Your test cases identified the problem with match5-189
+Your test cases identified the problem with match5-190
+Your test cases identified the problem with match5-191
+Your test cases identified the problem with match5-192
+Your test cases identified the problem with match5-193
+Your test cases identified the problem with match5-194
+Your test cases identified the problem with match5-195
+Your test cases identified the problem with match5-196
+Your test cases identified the problem with match5-197
+Your test cases identified the problem with match5-198
+Your test cases identified the problem with match5-199
+Your test cases identified the problem with match5-200
+Your test cases identified the problem with match5-201
+Your test cases identified the problem with match5-202
+Your test cases identified the problem with match5-203
+Your test cases identified the problem with match5-204
+Your test cases identified the problem with match5-205
+Your test cases identified the problem with match5-206
+Your test cases identified the problem with match5-207
+Your test cases identified the problem with match5-208
+Your test cases identified the problem with match5-209
+Your test cases identified the problem with match5-210
+Your test cases identified the problem with match5-211
+Your test cases identified the problem with match5-212
+Your test cases identified the problem with match5-213
+Your test cases identified the problem with match5-214
+Your test cases identified the problem with match5-215
+Your test cases identified the problem with match5-216
+Your test cases identified the problem with match5-217
+Your test cases identified the problem with match5-218
+Your test cases identified the problem with match5-219
+Your test cases identified the problem with match5-220
+Your test cases identified the problem with match5-221
+Your test cases identified the problem with match5-222
+Your test cases identified the problem with match5-223
+Your test cases identified the problem with match5-224
+Your test cases identified the problem with match5-225
+Your test cases identified the problem with match5-226
+Your test cases identified the problem with match5-227
+Your test cases identified the problem with match5-228
+Your test cases identified the problem with match5-229
+Your test cases identified the problem with match5-230
+Your test cases identified the problem with match5-231
+Your test cases identified the problem with match5-232
+Your test cases identified the problem with match5-233
+Your test cases identified the problem with match5-234
+Your test cases identified the problem with match5-235
+Your test cases identified the problem with match5-236
+Your test cases identified the problem with match5-237
+Your test cases identified the problem with match5-238
+Your test cases identified the problem with match5-239
+Your test cases identified the problem with match5-240
+Your test cases identified the problem with match5-241
+Your test cases identified the problem with match5-242
+Your test cases identified the problem with match5-243
+Your test cases identified the problem with match5-244
+Your test cases identified the problem with match5-245
+Your test cases identified the problem with match5-246
+Your test cases identified the problem with match5-247
+Your test cases identified the problem with match5-248
+Your test cases identified the problem with match5-249
+Your test cases identified the problem with match5-250
+Your test cases identified the problem with match5-251
+Your test cases identified the problem with match5-252
+Your test cases identified the problem with match5-253
+Your test cases identified the problem with match5-254
+Your test cases identified the problem with match5-255
+Your test cases identified the problem with match5-256
+Your test cases identified the problem with match5-257
+Your test cases identified the problem with match5-258
+Your test cases identified the problem with match5-259
+Your test cases identified the problem with match5-260
+Your test cases identified the problem with match5-261
+Your test cases identified the problem with match5-262
+Your test cases identified the problem with match5-263
+Your test cases identified the problem with match5-264
+Your test cases identified the problem with match5-265
+Your test cases identified the problem with match5-266
+Your test cases identified the problem with match5-267
+Your test cases identified the problem with match5-268
+Your test cases identified the problem with match5-269
+Your test cases identified the problem with match5-270
+Your test cases identified the problem with match5-271
+Your test cases identified the problem with match5-272
+Your test cases identified the problem with match5-273
+Your test cases identified the problem with match5-274
+Your test cases identified the problem with match5-275
+Your test cases identified the problem with match5-276
+Your test cases identified the problem with match5-277
+Your test cases identified the problem with match5-278
+Your test cases identified the problem with match5-279
+Your test cases identified the problem with match5-280
+Your test cases identified the problem with match5-281
+Your test cases identified the problem with match5-282
+Your test cases identified the problem with match5-283
+Your test cases identified the problem with match5-284
+Your test cases identified the problem with match5-285
+Your test cases identified the problem with match5-286
+Your test cases identified the problem with match5-287
+Your test cases identified the problem with match5-288
+Your test cases identified the problem with match5-289
+Your test cases identified the problem with match5-290
+Your test cases identified the problem with match5-291
+Your test cases identified the problem with match5-292
+Your test cases identified the problem with match5-293
+Your test cases identified the problem with match5-294
+Your test cases identified the problem with match5-295
+Your test cases identified the problem with match5-296
+
+Overall Grade: PASSED
diff --git a/09_testing2/run_all.sh b/09_testing2/run_all.sh
new file mode 100755
index 0000000..e62c149
--- /dev/null
+++ b/09_testing2/run_all.sh
@@ -0,0 +1,33 @@
+#!/bin/bash
+run_test(){
+ prog="$1"
+ testfile="$2"
+ IFS=$'\n'
+ for line in `cat $testfile`
+ do
+ IFS=" " correct=`/usr/local/l2p/match5/correct-match5 $line 2>&1`
+ IFS=" " broken=`$prog $line 2>&1`
+ if [ "$broken" != "$correct" ]
+ then
+ return 0
+ fi
+ done
+ return 1
+}
+
+found=0
+notfound=0
+for i in /usr/local/l2p/match5/match5-*
+do
+ run_test $i tests.txt
+ x="$?"
+ if [ "$x" != "0" ]
+ then
+ echo "Your test cases did not identify the problem with `basename $i`"
+ let notfound=${notfound}+1
+ else
+ let found=${found}+1
+ fi
+done
+echo "Test cases identified $found problems"
+echo "Test cases failed to identify $notfound problems"
diff --git a/09_testing2/tests.txt b/09_testing2/tests.txt
new file mode 100644
index 0000000..ca82980
--- /dev/null
+++ b/09_testing2/tests.txt
@@ -0,0 +1,22 @@
+aaaaa bbbbb
+aaaaa bbbbc
+aaabb bbbbc
+aaaaa aaaaa
+aaaa bbbbb
+aaaaaa bbbbb
+aaaaa bbbb
+aaaaa bbbbbb
+%%&*( 01234
+aaaaa
+
+&^%$# *&^%$
+aabbc lmddd
+----- ~~~~~
+||||| &&&&&
+""""" '''''
+NULL 00000
+aaaab bbbbc ddddd
+bbbba bbbba
+abbbb abbbb
+aabbb acbbb
+bbbcc bbbca