How will I know if RAP works?

            To evaluate the effectiveness of using the RAP method several types of data will be used.  Benchmarking data was collected from teachers prior to the introduction of RAP.  It will also be collected at the half-way mark and at the conclusion of the study.  The “Kentucky Extended-Response Scoring Guide” (image at the bottom of this post) that is used to score extended-response questions on the KPREP was/will be used to score these samples (Kentucky Department of Education, 2012).  The benchmark assessment documented the accuracy of student work when responding to an extended-response question without having an “attack” strategy to guide their answer.  After the RAP method is presented, student work will be evaluated using a rubric that assigns a number from 0 to 4 for each part of RAP based on evidence of each part of RAP as well as accuracy with the question (see Appendix B).  Student samples will be collected to test quality of work throughout the process.  Scores will be documented in a data journal to help me easily see the overall progress of the students each week (Mertler, 2012; Pappas-Tucker, 2011).  Three times throughout the study, teachers will be asked to grade extended-response questions using the RAP rubric to provide reliability for the study (Mertler, 2012; Pappas-Tucker, 2011).

            As mentioned in the previous post, students from different focus groups will participate in interviews to determine student confidence with writing and feelings about testing prior to the implementation of the RAP method.  The focus groups will be kept to 4-6 students.  Although Marshall & Rossman (2006) stated that participants in focus groups are typically “unfamiliar” with each other, the focus groups in this study will consist of the students’ peers due to the nature of the project.  Marshall and Rossman explained that “the interviewer creates a supportive environment, asking focused questions to encourage discussion and the expression of differing opinions and points of view” (p.114).  I believe that having students in focus groups with their peers will create a more comfortable situation for many of the participants.  At the conclusion of the study, the same students will be interviewed again to examine the impact of the RAP method for students when approaching a test that requires a written response.  Interviews were chosen instead of a survey for several reasons.  One reason is that answers can be clarified when needed (Mertler, 2012, Pappas and Tucker-Raymond, 2011; Rossman & Rallis, 2003, p.180).  In addition, interviewing children provides insight to the students’ opinions about writing and testing instead of adult speculation about what testing and writing is like (Marshall & Rossman, 2006, p. 114).  The goal of the interview process will be to have a “conversation with a purpose” (Kahn and Cannell, 1957, p. 149 as cited in Marshall & Rossman, 2006, p. 101).  Rossman and Rallis (2003) listed several reasons that interviews are beneficial for collecting data including, but not limited to: “understand[ing] individual perspectives, generat[ing] rich, descriptive data, and gather[ing] insight into participants’ thinking” (p.180).  I will use the “interview guide approach” to talk about how students feel about writing and testing (Rossman & Rallis, 2003, p. 181).  I will present open-ended questions to the students and have them respond in their own manner to allow for independent discussion about these topics if students choose to present ideas that have not been planned for, but that relate to the study (Marshall & Rossman, 2006; Rossman & Rallis, 2003).  The guiding questions that will be used are below.

Questions to be asked prior to the implementation of the RAP method:

1a. How do you feel about answering extended response questions?

1b. Why do you feel that way?

2. What do you think is the hardest part of writing?

3a. When you are asked to complete a test how do you feel inside? 

3b.  Can you explain why you feel this way?

4. What do you think would make you feel more confident about testing?

Questions to be asked at the conclusion of the project:

1a. How do you feel about answering extended response questions after learning the RAP method?

1b. Why do you feel that way?

2. Have your feelings toward testing changed since learning how to approach extended response questions using RAP?

 

Resources

Kentucky Department of Education. (2012). Kentucky extended response questions scoring guide. Retrieved from http://education.ky.gov/aa/distsupp/pages/k-prep.aspx.

Marshall, C. & Rossman, G.B. (2006). Designing qualitative research, (4^th ed.). Thousand    Oaks: Sage Publications.

Mertler, C. A. (2012). Action research: Improving schools and empowering educator, (3^rd ed.). Los Angeles: Sage Publications.

Pappas, C. C. & Tucker-Raymond, E. (2011). Becoming a teacher researcher in literacy and learning: Strategies and tools for the inquiry process. New York: Routledge

Rossman, G.B. & Rallis, S.F. (2003). Learning in the field: An introduction to qualitative    research. Thousand Oaks: Sage Publications.

 

Finally a little progress...

We finally made it back to school Monday!  Life feels a little chaotic and I am doing my best to pull it all together.  As far as research is concerned, I am still waiting for a few letters of consent to be returned.  As I was talking to students about this project they were so excited to hear that they get to create their own psuedonyms for the project.  It was really funny to hear their ideas.  So far I have heard, James Bond and Jason Bourne.  I assured them that if I ever use their name it will only be the first, but they can have fun pretending to be those people if they like.   
    I plan to pull students for interviews within focus groups this week.  My biggest struggle is finding time to pull them when I have free time and they aren't needed in the classroom.  The teachers involved are supportive, but they have work to complete too.  I'm sure I'll figure it out.  My goal for the survey is to hear about their feelings toward writing as well as testing.  I love sitting down with a small group and just listening to their thoughts and ideas.  It is amazing what they will open up and share when they are comfortable with each other and you.  My goal is to make the environment relaxed and help them to understand that this is a safe zone so they feel free to be honest (Marshall & Rossman, 2006).  Look for more about my data collection goals in the following post.


Resource:

Rossman, G.B. & Rallis, S.F. (2003). Learning in the field: An introduction to qualitative research.   Thousand Oaks: Sage Publications.

Current Status?

As of today, research is running slow due to a crazy number of snow days.  I have collected some writing samples for each of the participants.  Those samples are from extended-response questions that were answered without using RAP.  I am still waiting on letters of consent to be returned so I can establish my full group.  I will be conducting a few interviews whenever we get back to school to get a good understanding of how students feel about writing in general and their feelings toward testing.  I look forward to those interviews.  I love hearing what students think because I think sometimes we assume too many things and I like to hear their honesty. 

What is being researched?

Over the past few years, Buffalo Elementary (psuedonym) teachers have identified a disconnect in the writing instruction across all grade levels.  Teachers have expressed concern and frustration with the quality of student writing and a need to create a school-wide writing program that can be used across the building.  In order to meet this need, our Instructional Practices Committee has been working to find writing strategies that can be taught to students in kindergarten and built upon each year so that students in fifth grade can use them as well.  We easily decided on a program to use for simple sentence writing that can be turned into paragraphs and finally full essays.  Teachers saw good progress in student writing and that program seems to be working well. 

Unfortunately, our 3rd, 4th, and 5th grade teachers have determined that we still need a strategy that could be used to help students when answering an extended-response question.  Teachers expressed frustration with student work and determined that students didn't have the skills necessary to earn full credit on these types of questions.  Our biggest concern with this is that extended-response questions are a big part of the standarized tests that are given at the end of the year.  We know are teachers are teaching the content and skills needed for students to understand the questions, but their delivery of the answer is not clear enough to earn full credit.  After much research and discussion our teachers decided to use RAP (read/restate the question, answer all parts, and prove your answer) method.  This research project will center around the effectiveness of the RAP method.  More on who will participate will come shortly.

What exactly is the RAP method?

After examining several attack strategies, our teachers decided the RAP method would be the best approach to use school-wide to help students learn how to respond to an extended-response question. The acronym RAP stands for simple steps to help students include all information necessary to completely answer any extended-response question.

            R- Read/Restate- Students should read all parts of the question.  While reading, students should underline the question, box any key words, and circle any important numbers.  Then they will restate the question using the key words.
           

           A- Answer All Parts- Students should fully answer all parts of the question based   on the key words and numbers identified in “R”.

            P- Prove - Students should prove their answer using information from the passage 

and/or by connecting to the text with a personal experience.

This simple framework can be applied to any type of extended-response question. It is an easy mnemonic that helps students remember the best way to approach the question and demonstrate full understanding of the content without leaving out important information. The steps under “R” require students to underline the question and identify key words so that they are focusing on what task(s) they must complete. By circling important numbers students will know what “all” parts of the question are and what must be done to receive full credit for their answer. “Proving” their answer demonstrates full understanding of the concept and requires students to find a way to relate with the text which deepens their understanding.

Who and How?

In order to examine the effects of using the RAP method as an attack strategy for responding to an extended-response question, I turned to the students who earned a novice on the Kentucky Performance Rating for Educational Progress (K-PREP) test during the 2012-2013 school year. Novice is the lowest rating a student could have earned on this type of test. Scores increased to apprentice, proficient, and finally distinguished based on the quality and accuracy of the work by the student. The fifth graders from last year have been promoted to middle school so the current fourth and fifth graders that received a novice during third and fourth grade created the group. For lack of a better term, all participants will be referred to as novice students. These students were chosen in order to have a manageable project and because there are a variety of student abilities in the classroom within this group and their different levels will allow for a larger view of the effects of the RAP method.

Within this group of novice students, every student was identified in one or more gap groups. Sixty percent of the novice students were identified as special education students and have an individualized education plan in speech, reading, writing, and/or mathematics. Two participants were of Asian ethnicity, but were not considered to be part of the gap group for ethnicity. All other students were white/Caucasian. 48.5 % of the school received free or reduced lunch. Sixteen of twenty-four students were males with just eight females.

The RAP method will be taught within the students’ regular reading class. Extended- response questions will be created based on the Common Core Standards being taught that week. Teachers will guide students on how to apply RAP to an extended-response question and data will be collected based on student response questions on reading tests using the RAP rubric.  Interviews with the participants will be conducted in another classroom in either small group or individual settings.  (More on data collection to come in a later blog.)

Welcome!

Welcome!  Join me as I journey through the confusing and stressful, but I'm sure sometimes exciting world of research.  Over the next few months, I will be sharing with you the highs and lows of an apprentice researcher.  I'm sure the experience may not always be pretty, but I look forward to the lessons I will gain as I persevere through this project.