How will I know if RAP works?

            To evaluate the effectiveness of using the RAP method several types of data will be used.  Benchmarking data was collected from teachers prior to the introduction of RAP.  It will also be collected at the half-way mark and at the conclusion of the study.  The “Kentucky Extended-Response Scoring Guide” (image at the bottom of this post) that is used to score extended-response questions on the KPREP was/will be used to score these samples (Kentucky Department of Education, 2012).  The benchmark assessment documented the accuracy of student work when responding to an extended-response question without having an “attack” strategy to guide their answer.  After the RAP method is presented, student work will be evaluated using a rubric that assigns a number from 0 to 4 for each part of RAP based on evidence of each part of RAP as well as accuracy with the question (see Appendix B).  Student samples will be collected to test quality of work throughout the process.  Scores will be documented in a data journal to help me easily see the overall progress of the students each week (Mertler, 2012; Pappas-Tucker, 2011).  Three times throughout the study, teachers will be asked to grade extended-response questions using the RAP rubric to provide reliability for the study (Mertler, 2012; Pappas-Tucker, 2011).

            As mentioned in the previous post, students from different focus groups will participate in interviews to determine student confidence with writing and feelings about testing prior to the implementation of the RAP method.  The focus groups will be kept to 4-6 students.  Although Marshall & Rossman (2006) stated that participants in focus groups are typically “unfamiliar” with each other, the focus groups in this study will consist of the students’ peers due to the nature of the project.  Marshall and Rossman explained that “the interviewer creates a supportive environment, asking focused questions to encourage discussion and the expression of differing opinions and points of view” (p.114).  I believe that having students in focus groups with their peers will create a more comfortable situation for many of the participants.  At the conclusion of the study, the same students will be interviewed again to examine the impact of the RAP method for students when approaching a test that requires a written response.  Interviews were chosen instead of a survey for several reasons.  One reason is that answers can be clarified when needed (Mertler, 2012, Pappas and Tucker-Raymond, 2011; Rossman & Rallis, 2003, p.180).  In addition, interviewing children provides insight to the students’ opinions about writing and testing instead of adult speculation about what testing and writing is like (Marshall & Rossman, 2006, p. 114).  The goal of the interview process will be to have a “conversation with a purpose” (Kahn and Cannell, 1957, p. 149 as cited in Marshall & Rossman, 2006, p. 101).  Rossman and Rallis (2003) listed several reasons that interviews are beneficial for collecting data including, but not limited to: “understand[ing] individual perspectives, generat[ing] rich, descriptive data, and gather[ing] insight into participants’ thinking” (p.180).  I will use the “interview guide approach” to talk about how students feel about writing and testing (Rossman & Rallis, 2003, p. 181).  I will present open-ended questions to the students and have them respond in their own manner to allow for independent discussion about these topics if students choose to present ideas that have not been planned for, but that relate to the study (Marshall & Rossman, 2006; Rossman & Rallis, 2003).  The guiding questions that will be used are below.

Questions to be asked prior to the implementation of the RAP method:

1a. How do you feel about answering extended response questions?

1b. Why do you feel that way?

2. What do you think is the hardest part of writing?

3a. When you are asked to complete a test how do you feel inside? 

3b.  Can you explain why you feel this way?

4. What do you think would make you feel more confident about testing?

Questions to be asked at the conclusion of the project:

1a. How do you feel about answering extended response questions after learning the RAP method?

1b. Why do you feel that way?

2. Have your feelings toward testing changed since learning how to approach extended response questions using RAP?

 

Resources

Kentucky Department of Education. (2012). Kentucky extended response questions scoring guide. Retrieved from http://education.ky.gov/aa/distsupp/pages/k-prep.aspx.

Marshall, C. & Rossman, G.B. (2006). Designing qualitative research, (4^th ed.). Thousand    Oaks: Sage Publications.

Mertler, C. A. (2012). Action research: Improving schools and empowering educator, (3^rd ed.). Los Angeles: Sage Publications.

Pappas, C. C. & Tucker-Raymond, E. (2011). Becoming a teacher researcher in literacy and learning: Strategies and tools for the inquiry process. New York: Routledge

Rossman, G.B. & Rallis, S.F. (2003). Learning in the field: An introduction to qualitative    research. Thousand Oaks: Sage Publications.