Prior graphing research has demonstrated that clinical interviews and free-response instruments produce very different results than multiple-choice instruments, indicating potential validity problems when using multiple-choice instruments to assess graphing skills (Berg & Smith in "Science Education," 78(6), 527-554, 1994). Extending this inquiry, we studied whether empirically derived, participant-generated graphs used as choices on the multiple-choice graphing instrument produced results that corresponded to participants' responses on free-response instruments. The 5-8 choices on the multiple-choice instrument came from graphs drawn by 770 participants from prior research on graphing (Berg, 1989; Berg & Phillips in "Journal of Research in Science Teaching," 31(4), 323-344, 1994; Berg & Smith in "Science Education," 78(6), 527-554, 1994). Statistical analysis of the 736 7th-12th grade participants indicate that the empirically derived multiple-choice format still produced significantly more "picture-of-the-event" responses than did the free-response format for all three graphing questions. For two of the questions, participants who drew graphs on the free-response instruments produced significantly more correct responses than those who answered multiple-choice items. In addition, participants having "low classroom performance" were affected more significantly and negatively by the multiple-choice format than participants having "medium" or "high classroom performance." In some cases, prior research indicating the prevalence of "picture-of-the-event" and graphing treatment effects may be spurious results, a product of the multiple-choice item format and not a valid measure of graphing abilities. We also examined how including a picture of the scenario on the instrument versus only a written description affected responses and whether asking participants to add marker points to their constructed or chosen graph would overcome the short-circuited thinking that multiple-choice items seem to produce.