WWC Intervention Report U S DEPARTMENT OF ies ed gov

Wwc Intervention Report U S Department Of Ies Ed Gov-Free PDF

  • Date:11 Sep 2020
  • Views:3
  • Downloads:0
  • Pages:10
  • Size:764.77 KB

Share Pdf : Wwc Intervention Report U S Department Of Ies Ed Gov

Download and Preview : Wwc Intervention Report U S Department Of Ies Ed Gov


Report CopyRight/DMCA Form For : Wwc Intervention Report U S Department Of Ies Ed Gov


Transcription:

Additional program Developer and contact Math can be used as a standalone curriculum or as a supple. information Odyssey Math was developed and is distributed by CompassLearn ment to other mathematics curricula to provide an alternative. ing Inc Address 203 Colorado Street Austin TX 78701 Email instructional model differentiated instruction or gain feedback on. info compasslearning com Web http www compasslearning com particular concepts students are struggling with Specifically the. Telephone 800 232 9556 management tool allows teachers to customize instruction for indi. vidual students and or for classroom lesson plans The program s. Scope of use reporting system automatically prescribes individualized learning. CompassLearning was founded in 1969 and develops education paths with activities that are tailored for each student based on his. software The web based Odyssey Math product was launched in or her test scores Teachers can also generate reports detailing. 2003 as a follow up product to the original server based Odyssey student achievement and areas that need further review Odyssey. product now termed Odyssey Classic that was delivered under Math s assessment tool provides a means to build and customize. the Josten s brand and then by CompassLearning Odyssey Classic assessments based on key focus areas individual learning styles. is no longer sold or supported by CompassLearning 6 According or local state and national standards Teachers can use the bank. to the developers more than 11 million students in 20 000 schools of test items or they can build their own items and item banks for. nationwide use Odyssey Math and other Odyssey programs any grade level and any subject. Teaching Cost, Odyssey Math offers a full mathematics curriculum for grades K 8 Odyssey Math pricing depends on the implementation. that combines both online and offline tools and activities Odyssey. Research Fourteen studies reviewed by the WWC investigated the effects the Odyssey Math treatment group Teachers were asked to. of Odyssey Math One study DiLeo 2007 is a randomized use Odyssey Math at least 90 minutes per week the developer. controlled trial that meets WWC evidence standards with recommended amount as a supplement to their core mathematics. reservations The remaining 13 studies do not meet either WWC curriculum Houghton Mifflin The study focused primarily on. eligibility screens or evidence standards the relationship between Odyssey usage in the treatment group. DiLeo 2007 analyzed data from a randomized controlled and student achievement but the author also examined outcome. trial RCT conducted in a single school district in Pennsylvania differences of the treatment and control groups The original. Teachers and students were assigned to classrooms by princi randomly assigned sample included 13 classrooms 7 treatment. pals with efforts made to balance classrooms by student ability and 6 control in five schools A single magnet school with two. Teachers were then randomly assigned to use either Odyssey comparison classrooms was dropped from the analysis because. Language Arts or Odyssey Math in their classroom The group the school s demographic composition was significantly different. using Odyssey Language Arts served as the control group to from the other schools in the sample Therefore the final analysis. 6 This review focuses only on studies of the newer web based Odyssey Math product launched in 2003. WWC Intervention Report Odyssey Math August 2009 2. Research continued sample included 11 classrooms 7 treatment and 4 control in Extent of evidence. 4 schools with a total of 207 fifth grade students 7 Despite the The WWC categorizes the extent of evidence in each domain. severe differential attrition of treatment and control classrooms as small or medium to large see the WWC Procedures and. post attrition treatment and control groups had only small Standards Handbook Appendix G The extent of evidence. differences on baseline test scores that were not statistically takes into account the number of studies and the total sample. significant but that did require the author control for the baseline size across the studies that meet WWC evidence standards with. pretest according to the Elementary Math protocol 8 Because or without reservations 9. of the differential attrition the WWC rated this RCT as meeting The WWC considers the extent of evidence for Odyssey Math. evidence standards with reservations to be small for math achievement. Effectiveness Findings substantively important that is an effect size of at least 0 25. The WWC review of interventions for Elementary School Math Therefore based on this one study Odyssey Math appears to. addresses student outcomes in math achievement The findings have potentially positive effects on mathematics achievement. below present the author s estimates and WWC calculated. estimates of the size and the statistical significance of the effects Rating of effectiveness. of Odyssey Math on students 10 The WWC rates the effects of an intervention in a given outcome. DiLeo 2007 reported a statistically significant positive effect domain as positive potentially positive mixed no discernible. on mathematics achievement as measured by the Pennsylvania effects potentially negative or negative The rating of effectiveness. System of School Assessment PSSA between the students takes into account four factors the quality of the research design. randomly assigned to use Odyssey Math the treatment group the statistical significance of the findings the size of the difference. and Odyssey Reading the control group In WWC calculations between participants in the intervention and the comparison condi. this effect was not statistically significant However the WWC tions and the consistency in findings across studies see the WWC. determined the effects were large enough to be considered Procedures and Standards Handbook Appendix E. 7 An additional school with 73 students was included in the author s analysis but is not included in this report because teachers in that school were not. randomly assigned to use either the Language Arts or Math software as described below and could have used both and therefore were classified as a. separate group by the author The author provided the WWC with sample sizes and standard deviations details about the random assignment process. and information about attrition not presented in the published 2007 study which was a dissertation. 8 The author presented third grade Pennsylvania System of School Assessment PSSA scores to establish baseline equivalence Fourth grade students. did not take the PSSA in 2005 so third grade was the most recent year available. 9 The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating focusing on the. number and size of studies Additional factors associated with a related concept external validity such as the students demographics and the types. of settings in which studies took place are not taken into account for the categorization Information about how the extent of evidence rating was. determined for Odyssey Math is in Appendix A5, 10 The level of statistical significance was reported by the study authors or when necessary calculated by the WWC to correct for clustering within class. rooms or schools and for multiple comparisons For an explanation see the WWC Tutorial on Mismatch For the formulas the WWC used to calculate. the statistical significance see WWC Procedures and Standards Handbook Appendix C for clustering and WWC Procedures and Standards Handbook. Appendix D for multiple comparisons In the case of DiLeo 2007 a correction for clustering was needed so the significance levels may differ from. those reported in the original study, WWC Intervention Report Odyssey Math August 2009 3. The WWC found Improvement index Based on the one study of Odyssey Math that meets evidence. Odyssey Math to have The WWC computes an improvement index for each individual standards with reservations the improvement index for math. potentially positive finding In addition within each outcome domain the WWC achievement is 17 percentile points. effects for math computes an average improvement index for each study and an. achievement average improvement index across studies see WWC Procedures Summary. and Standards Handbook Appendix F The improvement index The WWC reviewed 14 studies of Odyssey Math One of these. represents the difference between the percentile rank of the aver studies meets WWC evidence standards with reservations. age student in the intervention condition versus the percentile rank The remaining 13 studies do not meet either WWC evidence. of the average student in the comparison condition Unlike the standards or eligibility screens Based on the one study the. rating of effectiveness the improvement index is based entirely WWC found potentially positive effects in math achievement. on the size of the effect regardless of the statistical significance The conclusions presented in this report may change as new. of the effect the study design or the analyses The improvement research emerges. index can take on values between 50 and 50 with positive. numbers denoting results favorable to the intervention group. References Meets WWC evidence standards with reservations Clariana R 2007 Odyssey school effectiveness report Pem. DiLeo J 2007 A study of a specific language arts and math berton township school district Austin TX CompassLearning. ematics software program Is there a correlation between Inc The study is ineligible for review because it does not use. usage levels and achievement Unpublished doctoral disser a comparison group. tation Indiana University of Pennsylvania Indiana PA CompassLearning Inc 2005 Odyssey school effectiveness. report Maple Leaf intermediate school Austin TX Author. Studies that fall outside the Elementary School Math The study does not meet WWC evidence standards because. protocol or do not meet WWC evidence standards the intervention and comparison groups are not shown to be. Brandt W C Hutchison C 2005 Romulus community equivalent at baseline. schools 2002 2005 Summary research and evaluation report CompassLearning Inc 2006 CompassLearning Odyssey. San Diego CA CompassLearning Inc The study is ineligible school effectiveness report Lillie Burney elementary school. for review because it does not use a comparison group Austin TX Author The study is ineligible for review because it. Additional source does not use a comparison group, Brandt W C Hutchison C Learning Point Associates CompassLearning Inc 2006 Odyssey school effectiveness. 2006 Romulus community schools comprehensive school report Boone county school district Austin TX Author The. reform evaluation Naperville IL Learning Point Associates study does not meet WWC evidence standards because. Clariana R 2005 CompassLearning classrooms of the future An the measures of effect cannot be attributed solely to the. interim report of one to one wireless laptop use in upper elemen intervention there was only one unit of analysis in one or. tary mathematics Let them solo Austin TX CompassLearning both conditions. Inc The study is ineligible for review because it does not use a CompassLearning Inc 2007 Impact of CompassLearning. sample within the age or grade range specified in the protocol Odyssey Reading Language Arts Mathematics on NWEA. WWC Intervention Report Odyssey Math August 2009 4. References continued RIT scores and Lexile range Akron elementary school Austin Milhollin C C 2005 Integration of an integrated learning. TX Author The study is ineligible for review because it does system does it make a difference An observational research. not use a comparison group study Unpublished doctoral dissertation Valdosta State. CompassLearning Inc 2007 Odyssey school effectiveness University Valdosta GA The study is ineligible for review. report Washoe county school district Austin TX Author because it does not use a comparison group. The study does not meet WWC evidence standards because Webb D R 2008 The effects of the Toyota production system. the intervention and comparison groups are not shown to be on student academic performance Unpublished doctoral dis. equivalent at baseline sertation Duquesne University Pittsburgh PA The study is. CompassLearning Inc 2007 Summer school pilot program for ineligible for review because it does not examine the effective. third grade intervention Tulsa independent school district ness of an intervention. Austin TX Author The study is ineligible for review because it Wright W 2006 Catching up in math The case of newly. does not use a comparison group arrived Cambodian students in a Texas intermediate school. CompassLearning Inc 2008 Careful planning commitment Texas Association for Bilingual Education Journal 9 1 1 22. yield outstanding results for Carroll county school Austin TX The study is ineligible for review because it does not use a. Author The study is ineligible for review because it is not a comparison group. primary analysis of the effectiveness of an intervention. WWC Intervention Report Odyssey Math August 2009 5. Appendix A1 Study Characteristics DiLeo 2007,Characteristic Description.
Study citation DiLeo J 2007 A study of a specific language arts and mathematics software program Is there a correlation between usage levels and achievement Unpublished doctoral. dissertation Indiana University of Pennsylvania Indiana PA. Participants1 In five out of ten schools within a single school district fifth grade teachers were randomly assigned to use CompassLearning s Odyssey Language Arts or Odyssey Math in their. WWC Intervention Report Odyssey Math August 2009 1 WWC Intervention Report U S DEPARTMENT OF EDUCATION What Works Clearinghouse Elementary School Math August 2009 Odyssey Math Program Description 1 Odyssey Math published by CompassLearning is a web based K 8 mathematics curriculum and assessment tool designed to

Related Books