ABSTRACT Title of Dissertation: CLASSROOM ASSESSMENT IN U.S. HIGH SCHOOL BAND PROGRAMS: METHODS, PURPOSES, AND INFLUENCES Phillip Matthew Kancianic, Doctor of Philosophy, 2006 Dissertation Directed by: Assistant Professor Michael P. Hewitt School of Music Professor Marie F. McCarthy School of Music The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. MENC: The National Association for Music Education provided a membership list from which 2,000 U.S. high school band directors were selected by simple random sampling. Participants received a postcard via mail inviting them to complete an online survey. Non-respondents received a second postcard two weeks later and a paper version of the survey four weeks later. The independent variables included 11 personal and 11 school characteristics. The dependent variables included 23 assessment methods, 19 purposes of assessment, and 23 factors that influence the use of classroom assessment. The overall survey return rate was 39.75% (N = 795); the usable response was 31.7% (N = 634). Descriptive statistics illustrated the respondents? use of classroom assessment methods, the level of importance they attributed to purposes of assessment, and the level of influence they attributed to factors that affect assessment. Pearson product-moment correlations and multiple analyses of variance were performed on the data to test 22 null hypotheses. Excepting the MANOVAs (? = .05), the experimentwise alpha was set at .01 to reduce the risk of Type I error. Classroom assessments focused primarily on the evaluation of student performance skills. Lack of time was viewed as a major impediment to assessment. Teachers were more influenced by internal factors (e.g., philosophy of education and class goals) than by external factors (e.g., school requirements and local, state, or national standards). Music colleagues were influential among less-experienced teachers and those who had district-wide assessment training. Three prevalent issues emerged from the results: teacher autonomy, the role of assessment training, and teacher workload. Recommendations included investigating the relationship between teacher autonomy and classroom assessment, examining and improving current assessment training for pre-service and in-service teachers, and developing efficient assessment strategies that have a minimal impact on teacher workload. It was also recommended that the many non-statistically significant findings be examined by future researchers. CLASSROOM ASSESSMENT IN U.S. HIGH SCHOOL BAND PROGRAMS: METHODS, PURPOSES, AND INFLUENCES By Phillip Matthew Kancianic Dissertation submitted to the Faculty of the Graduate School of the University of Maryland, College Park, in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2006 Advisory Committee: Assistant Professor Michael P. Hewitt, Co-Chair Professor Marie F. McCarthy, Co-Chair Professor Robert J. Mislevy Assistant Professor Philip E. Silvey Associate Professor Wayne H. Slater ? Copyright by Phillip Matthew Kancianic 2006 PREFACE Through my experiences as a middle and high school band director in Ohio and Maryland, I found it quite challenging to assess student progress in ensemble classes. Never one to turn down an opportunity that would allow students to reap the benefits of performing in public, my schedule quickly filled with concerts, festivals, parades, and competitions. Thus, the goal of every rehearsal was made clear: to prepare students to perform at their highest possible level of musicianship in the least amount of time. But with this goal and the resulting focus on a limited repertoire came the guilt of not exposing students to the variety of musical experiences currently available to them: composition, improvisation, music from other cultures, small ensemble literature, music in other artistic genres, and computer technology, to name a few. I had been fortunate to have a high school experience that included all of these, and desperate to find the instructional time that would allow me to share the same with my students. However, the immediate challenge was to drive toward the goal of performance, and this required knowing where my students were, musically speaking. Attempts at tape- recorded tests, in-class evaluation, small ensemble performances, and critical self- listening/reflection were met with only partial satisfaction and success by both teacher and students. The struggle to find a set of classroom assessments that are efficient, valid, and reliable for assessing high school band students continues. Using my experience as a backdrop, while setting my personal issues aside, this study takes a ?snapshot? of the status of classroom assessment in U.S. high school band programs. ii ACKNOWLEDGEMENTS First and foremost, I would like to thank all the high school band directors who invested their time and effort in completing the survey. This research would have been impossible without you. I am also grateful to MENC: The National Association for Music Education for allowing free access to a portion of their membership roster for this study. The committee co-chairs deserve many thanks for their assistance with my studies at the University of Maryland. Dr. Michael Hewitt has been an intelligent and supportive advisor whose guidance helped solidify the foundations of my dissertation. Dr. Marie McCarthy has been a constant source of inspiration whose comments helped bring clarity and depth to my writing. Dr. Philip Silvey, Dr. Wayne Slater, and Dr. Robert Mislevy each brought distinctive perspectives to the project and were supportive throughout. Any errors remaining in the document are of my own doing. I would like to thank my close friends Richard Scerbo and Dr. Susan Slingland for their moral support and listening ears. The camaraderie of my present and past colleagues in the degree program?especially Dr. Regina Carlow, Janet Jay, Craig Resta, and the Beyond 690 cohort?has helped make these five years memorable and enjoyable. I am most thankful to my family and especially to my wife, Jennifer. I remain humbled by her unending support and love, and hope that all the favors can be returned in due time. A special thank-you to my parents, Eugene and Margaret Kancianic, who have always supported my musical and educational aspirations while reminding me to keep a sense of humor throughout life. Finally, thanks to Chester, Dale, and Annie for their feline companionship during my early, late, and long hours of writing. iii TABLE OF CONTENTS PREFACE........................................................................................................................... ii ACKNOWLEDGEMENTS...............................................................................................iii LIST OF TABLES............................................................................................................ vii LIST OF FIGURES ........................................................................................................... ix CHAPTER 1: INTRODUCTION....................................................................................... 1 Need for the Study .................................................................................................. 2 Purpose of the Study ............................................................................................... 4 Research Questions................................................................................................. 4 Independent Variables ............................................................................................ 5 Personal characteristics............................................................................... 5 School characteristics.................................................................................. 6 Dependent Variables............................................................................................... 8 Use of classroom assessment methods ....................................................... 8 Purposes of classroom assessment.............................................................. 8 Factors that influence the use of classroom assessment ............................. 8 Null Hypotheses...................................................................................................... 9 Personal characteristics............................................................................... 9 School characteristics................................................................................ 10 Definitions............................................................................................................. 12 Assumptions.......................................................................................................... 14 Limitations ............................................................................................................ 14 Summary and Overview ....................................................................................... 16 CHAPTER 2: LITERATURE REVIEW.......................................................................... 17 Assessment History............................................................................................... 18 Early 20 century. th ..................................................................................... 18 Mid 20 century. th ....................................................................................... 20 Late 20 century to the present. th ................................................................ 22 Classroom Assessment.......................................................................................... 24 Classroom Assessment in Music Education ......................................................... 28 Summary............................................................................................................... 41 CHAPTER 3: METHOD .................................................................................................. 44 Research Method .................................................................................................. 44 Sample Selection................................................................................................... 45 Minimization of Survey Error............................................................................... 47 Coverage error .......................................................................................... 47 iv Sampling error .......................................................................................... 49 Measurement error.................................................................................... 49 Non-response error.................................................................................... 50 Independent Variable Development ..................................................................... 51 Personal characteristics............................................................................. 51 School characteristics................................................................................ 52 Dependent Variable Development........................................................................ 53 Use of classroom assessment methods ..................................................... 53 Purposes of classroom assessment............................................................ 54 Factors that influence the use of classroom assessment ........................... 56 Pilot Study Procedures.......................................................................................... 57 Main Study Procedures......................................................................................... 58 Data Analysis........................................................................................................ 59 Data screening........................................................................................... 60 Missing values .......................................................................................... 61 Alpha levels .............................................................................................. 63 Multivariate analysis of variance.............................................................. 63 Correlation ................................................................................................ 65 Summary............................................................................................................... 66 CHAPTER 4: RESULTS.................................................................................................. 68 Demographic Information..................................................................................... 68 Descriptive Statistics: Research Questions 1-3..................................................... 73 Use of classroom assessment methods ..................................................... 73 Purposes of classroom assessment............................................................ 75 Factors that influence the use of classroom assessment ........................... 76 Inferential Statistics: Research Questions 4-5 ...................................................... 79 Personal characteristics............................................................................. 79 School characteristics................................................................................ 91 Summary............................................................................................................... 97 CHAPTER 5: DISCUSSION AND CONCLUSIONS..................................................... 98 Discussion............................................................................................................. 98 Use of classroom assessment methods ..................................................... 98 Purposes of classroom assessment.......................................................... 101 Factors that influence the use of classroom assessment ......................... 104 Prevalent Issues, Implications, and Recommendations ...................................... 108 Teacher autonomy................................................................................... 108 Assessment training ................................................................................ 110 Teacher workload.................................................................................... 111 Non-statistically Significant Findings................................................................. 114 Conclusions......................................................................................................... 116 APPENDIX A: STATISTICAL DESIGN OF THE STUDY ........................................ 118 v APPENDIX B: ONLINE SURVEY INSTRUMENT .................................................... 119 APPENDIX C: FIRST MAILING.................................................................................. 137 APPENDIX D: SECOND MAILING ............................................................................ 138 APPENDIX E: THIRD MAILING................................................................................. 139 APPENDIX F: PILOT SURVEY FEEDBACK............................................................. 145 APPENDIX G: QUALITATIVE RESPONSES: METHODS ....................................... 147 APPENDIX H: QUALITATIVE RESPONSES: PURPOSES....................................... 158 APPENDIX I: QUALITATIVE RESPONSES: INFLUENCES.................................... 162 APPENDIX J: INTERCORRELATIONS BETWEEN SELECTED VARIABLES ..... 168 REFERENCES ............................................................................................................... 169 vi LIST OF TABLES Table 1: Percentage of Paper and Online Responses and their Usability..........................60 Table 2: Percentage of Female and Male Respondents from each MENC Division.........68 Table 3: Percentage of Respondents with Various Teaching Licenses and their Highest Educational Degree Earned ......................................................................69 Table 4: Percentage of Students in each Level of Socioeconomic Status and their School?s Geographic Setting..................................................................................70 Table 5: Percentage of Respondents who had Received Specific Types of Assessment Training..............................................................................................70 Table 6: All Other Personal and School Characteristics Measured by the Survey Instrument ..............................................................................................................71 Table 7: Percentage of Female and Male Respondents and their Chosen Response Mode ......................................................................................................................72 Table 8: Annual Use of Classroom Assessment Methods Among High School Band Directors .......................................................................................................74 Table 9: Level of Importance Attributed by High School Band Directors to Specified Purposes of Classroom Assessment.......................................................76 Table 10: Level of Influence Attributed by High School Band Directors to Specified Factors that Influence the Use of Classroom Assessment Methods..................................................................................................................78 Table 11: Statistically Significant Differences between Females and Males on Selected Dependent Variables (H ) 1 .......................................................................80 Table 12: Statistically Significant Correlations between the Respondent?s Age and Selected Dependent Variables (H ) 2 .......................................................................81 Table 13: Statistically Significant Correlations between the Number of Years Teaching Any Subject or Grade and Selected Dependent Variables (H ) 4 .............82 Table 14: Statistically Significant Correlations between the Number of Years Teaching High School Band and Selected Dependent Variables (H ) 5 ..................83 Table 15: Statistically Significant Correlations between the Number of Years Teaching at their Current School and Selected Dependent Variables (H ) 6 ...........84 vii Table 16: Statistically Significant Correlations between the Number of Other (Non-High School Band) Classes Currently Teaching and Selected Dependent Variables (H ) 7 ......................................................................................85 Table 17: Statistically Significant Correlations between the Number of High School Band Classes Currently Teaching and Selected Dependent Variables (H ) 8 ........................................................................................................86 Table 18: Statistically Significant Differences between Teachers with Assessment Training and Teachers without Assessment Training on Selected Dependent Variables (H ) 11 .....................................................................................88 Table 19: Statistically Significant Differences between Teachers with District- wide Training in Assessment and Teachers without District-wide Training in Assessment on Selected Dependent Variables (H ) 11 .........................................89 Table 20: Statistically Significant Differences between Teachers with Graduate Coursework Training in Assessment and Teachers without Graduate Coursework Training in Assessment on Selected Dependent Variables (H ) 11 .......................................................................................................................90 Table 21: Statistically Significant Differences between Teachers with Workshop Training in Assessment and Teachers without Workshop Training in Assessment on Selected Dependent Variables (H ) 11 .............................................91 Table 22: Statistically Significant Differences between Public and Non-Public Schools on Selected Dependent Variables (H ) 15 ...................................................93 Table 23: Statistically Significant Correlations between Total School Enrollment and Selected Dependent Variables (H ) 17 ...............................................................94 Table 24: Statistically Significant Correlations between Total Class Enrollment and Selected Dependent Variables (H ) 19 ...............................................................95 Table 25: Statistically Significant Correlations between the Number of Extra Rehearsals Held Outside School Hours and Selected Dependent Variables (H ) 22 .......................................................................................................................96 viii LIST OF FIGURES Figure 1: Relationships among Variables Examined in the Study. .................................... 5 ix CHAPTER 1 INTRODUCTION In Music Matters: A New Philosophy of Music Education, David Elliott (1995) stated, ?Achieving the aims of music education depends on assessment. The primary function of assessment in music education is not to determine grades but to provide accurate feedback to students about the quality of their growing musicianship? (p. 264). Teachers? classroom assessments are rarely examined by educational researchers and infrequently debated by the public. Assessment discourse and research typically focus on large-scale testing programs such as the National Assessment of Educational Progress, the Scholastic Assessment Tests, and statewide tests such as Maryland?s High School Assessments. Classroom assessments have more potential to impact students than most large- scale standardized tests. Ordinarily, teachers design these small-scale assessments for use in their classrooms; therefore, they are customized to meet the needs of particular students at a particular time. Colwell (2003) wrote, ?The responsibility for assessment is on the teacher in the classroom who uses the data to improve learning? (p. 17). The continuous cycle of planning, instructing, assessing, and reflecting drives high-quality instruction. Stiggins (2005) said that although there are many important decision-makers in education, ?decisions made at the classroom level contribute the most to student success? (p. 21). Many researchers (Brookhart, 1994; Frary, Cross, & Weber, 1993; Hill, 1999; Lehman, 1998; McClung, 1996; McCoy, 1991; McMillan, 2001; McMillan, Myran, & Workman, 2002; Music Educators National Conference, 1998; Simanton, 2000) have 1 studied the grading systems that teachers use to evaluate student work. The current study expands upon that research by examining the procedures teachers use to assess student learning, apart from the process of grading. This study used a survey research method to investigate classroom assessment practices among randomly selected U.S. high school band directors who were members of MENC: The National Association for Music Education (MENC). Need for the Study If music is to achieve and maintain a status of great importance to K-12 education, student progress must be measured (Brandt, 1987; Zerull, 1990). English and math are considered important, as evidenced by ?high stakes assessments? that exist in those disciplines (Colwell, 2002). Statewide assessments in music are currently under development in several states, including Maryland (High School Assessments) and New York (High School Arts Assessment). Further reflecting the importance of high stakes testing, ?a new pay-for-performance program for Florida?s teachers will tie raises and bonuses directly to pupils? standardized-test scores? (Whoriskey, 2006, p. A1), ?even for subjects such as music and art? (p. A14). Some music educators are beginning to endorse standardized testing. Florida state arts curriculum specialist June Hinckley stated, ?What we want to do is track the fact that students learn things in music. Unfortunately, too many times we look at music as an activity? (quoted in Reed, 2005, ? 13). Others are skeptical of the impact of standardized tests on classroom instruction and student learning, and some question whether the arts are capable of being assessed (Brandt, 1987; Colwell, 2000; Eisner, 2002; Zerull, 1990). 2 As a complement to standardized assessment, classroom assessment is a useful and necessary tool for student evaluation. Classroom assessment is defined as ?the collection, evaluation, and use of information to help teachers make better decisions? (McMillan, 2004, p. 8) regarding classroom instruction. Teachers develop and use classroom assessments for a variety of purposes, such as determining student needs, improving class instruction, and communicating results to others. Assessing student learning informs the instructional process by evaluating the strengths and weaknesses of individual students. Classroom assessment informs lesson planning and plots the course trajectory toward achieving instructional goals. The present investigation of high school band directors will help determine the type and frequency of classroom assessments currently used in U.S. high school band programs. The results of this study may be helpful toward improving instructional methods, raising student achievement, informing music teacher educators, and ultimately, elevating the status of school music programs. Colwell (2002) wrote that music educators have exhibited a ?past lack of serious interest in assessment? (p. 1146) and cautioned, ?teachers cannot continue to randomly add and subtract experiences and objectives? (p. 1155) from their lessons. Colwell (2003) also wrote that ?music educators accept the general principle of assessment but remain ignorant of the detailed actions required for a reasonably valid assessment? (p. 16). This perceived lack of concern may stem from a dearth of knowledge about educational measurement and statistics. Teachers? assessment knowledge is typically derived from their experiences as students, from their colleagues, and from in-service professional development, but not from their undergraduate education (Frary, Cross, & Weber, 1993). 3 Lehman (2000) predicted that by the year 2020, music educators would need to have a broad knowledge of assessment procedures and materials, and know how to analyze assessment results accurately in order to benefit student learning. He proposed that a collaboration of music industry representatives, professional organizations, and music educators should be involved in the creation of materials for assessment. He suggested that improvements are needed in the assessment training of current and future instrumental music educators. The present research may help determine the needs for classroom assessment training by examining the current uses of classroom assessment among high school band directors. Purpose of the Study The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. Research Questions The following research questions were examined: 1. How many times per school year do high school band directors use specific classroom assessment methods? 2. What level of importance do high school band directors attribute to specific purposes of classroom assessment? 3. What level of influence do high school band directors attribute to specific factors that affect the use of classroom assessment? 4 4. What are the relationships among the characteristics of high school band directors, the purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment? 5. What are the relationships among the characteristics of high school band directors? school settings, the purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment? Figure 1 illustrates the relationships among variables that were examined in the present study. INDEPENDENT VARIABLES DEPENDENT VARIABLES Personal characteristics (11) School characteristics Uses of classroom assessment methods (23) Purposes of classroom assessment (19) Factors that affect the use of classroom assessment (23) Figure 1. Relationships among Variables Examined in the Study. Note. Parenthetical numbers indicate the number of survey items that were related to each variable. Independent Variables Personal characteristics. The independent variables related to the personal characteristics of high school band directors (see research question 4) were as follows: 1. Gender (i.e., female or male). 2. Age. 5 3. Employment status: part time (i.e., less than 20 hours per week) or full time (i.e., 20 hours or more per week). 4. Number of years teaching any subject or grade level (including the current year). 5. Number of years teaching high school band (including the current year). 6. Number of years teaching at the school where currently teaching (including the current year). 7. Number of high school band classes currently teaching (not including extra-curricular classes). 8. Number of other classes currently teaching (not including extra-curricular classes). 9. Highest college or university degree received (i.e., high school diploma, associate degree, bachelor degree, master degree, doctoral degree, or post- doctoral work). 10. State music teaching license or certificate status (i.e., regular license or certificate to teach music, license or certificate to teach music with a validity period of two years or less, license or certificate to teach another subject, or none). 11. Types of assessment training received (i.e., departmental training, school- wide training, district-wide training, undergraduate coursework, graduate coursework, professional conference or workshop, self-study, or none). School characteristics. The independent variables related to characteristics of high school band directors? school settings (see research question 5) were as follows: 6 1. The state where currently teaching. 2. The MENC division where currently teaching [i.e., eastern (CT, DC, MD, NH, NY, RI, DE, ME, MA, NJ, PA, VT); north central (IL, IN, MN, ND, SD, IA, MI, NE, OH, WI); northwest (AK, ID, MT, OR, WA, WY); southern (AL, GA, LA, NC, TN, WV, FL, KY, MS, SC, VA); southwestern (AR, KS, NM, TX, CO, MO, OK); or western (AZ, CA, HI, NV, UT)]. 3. Geographic setting (i.e., urban/inner city, suburban, small town, or rural/remote). 4. School type (i.e., public or non-public). 5. Socioeconomic status of the majority of students in the school (i.e., low, medium, or high). 6. Number of students enrolled in the school. 7. The presence of other music specialists in the classroom. 8. Number of students enrolled in the class. 9. Average number of days per week that the class meets (i.e., 1, 1?, 2, 2?, 3, 3?, 4, 4?, or 5). 10. Average length of each class, in minutes. 11. Average number of times per school year that the class meets outside of regular school hours as a full ensemble. 7 Dependent Variables The three dependent variable clusters consisted of the use of classroom assessment methods, purposes of classroom assessment, and factors that influence the use of classroom assessment (see research questions 1-3). Use of classroom assessment methods. Few studies have investigated classroom assessment in high school band programs; however, many practitioner articles have been written about the use of specific classroom assessments in music. Classroom assessment is defined as ?the collection, evaluation, and use of information to help teachers make better decisions? (McMillan, 2004, p. 8) regarding classroom instruction. One aim of the current research was to measure the frequency and types of classroom assessments currently used by high school band directors. Purposes of classroom assessment. The purposes of classroom assessment answer the question, ?Why are you doing the assessment?? (McMillan, 2004, p. 9). Farrell (1997) stated that ?educators use assessment to set standards, create instructional directions, motivate performance, provide diagnostic feedback, assess or evaluate progress, and communicate that progress to others? (p. 1). Boyle and Radocy (1987) gave the following five purposes for assessment: ?(a) accountability, (b) instructional effectiveness, (c) teacher effectiveness, (d) policy making and management, and (e) research and project evaluation? (p. 14). This study investigated the level of importance that high school band directors attribute to purposes of classroom assessment such as these. Factors that influence the use of classroom assessment. The factors that influence the use of classroom assessment are defined as ?pressures that need to be considered? 8 (McMillan, 2004, p. 10) when planning for assessment. McMillan found many internal and external influences on teachers? use of classroom assessment. Investigating the factors that influence the use of classroom assessment may help one understand teachers? use of classroom assessment and their purposes for classroom assessment. Null Hypotheses Twenty-two null hypotheses were tested to answer research questions 4 and 5 (see Appendix A). These were created by combining each independent variable related to personal characteristics and school characteristics with the dependent variables, which are abbreviated herein as ?methods, purposes, and influences?. Personal characteristics. The null hypotheses related to the characteristics of high school band directors (see research question 4) were as follows: H 1. There are no statistically significant relationships among the gender of high school band directors and their classroom assessment methods, purposes, and influences. H 2. There are no statistically significant relationships among the age of high school band directors and their classroom assessment methods, purposes, and influences. H 3. There are no statistically significant relationships among the employment status of high school band directors and their classroom assessment methods, purposes, and influences. H 4. There are no statistically significant relationships among the number of years that high school band directors have taught any subject or grade and their classroom assessment methods, purposes, and influences. 9 H 5. There are no statistically significant relationships among the number of years that high school band directors have taught high school band and their classroom assessment methods, purposes, and influences. H 6. There are no statistically significant relationships among the number of years high school band directors have taught at their current school and their classroom assessment methods, purposes, and influences. H 7. There are no statistically significant relationships among the number of other classes taught by high school band directors and their classroom assessment methods, purposes, and influences. H 8. There are no statistically significant relationships among the number of high school band classes taught by high school band directors and their classroom assessment methods, purposes, and influences. H 9. There are no statistically significant relationships among the highest college or university degree received by high school band directors and their classroom assessment methods, purposes, and influences. H 10. There are no statistically significant relationships among the state teaching license or certificate status of high school band directors and their classroom assessment methods, purposes, and influences. H 11. There are no statistically significant relationships among the types of classroom assessment training received by high school band directors and their classroom assessment methods, purposes, and influences. School characteristics. The null hypotheses related to characteristics of high school band directors? school settings (see research question 5) were as follows: 10 H 12. There are no statistically significant relationships among the state in which a school is located and high school band directors? classroom assessment methods, purposes, and influences. H 13. There are no statistically significant relationships among the MENC division in which a school is located and high school band directors? classroom assessment methods, purposes, and influences. H 14. There are no statistically significant relationships among the geographic setting of a school and high school band directors? classroom assessment methods, purposes, and influences. H 15. There are no statistically significant relationships among the school type and high school band directors? classroom assessment methods, purposes, and influences. H 16. There are no statistically significant relationships among the socioeconomic status of the majority of students and high school band directors? classroom assessment methods, purposes, and influences. H 17. There are no statistically significant relationships among total school enrollment and high school band directors? classroom assessment methods, purposes, and influences. H 18. There are no statistically significant relationships among the number of music specialists present in the classroom and high school band directors? classroom assessment methods, purposes, and influences. 11 H 19. There are no statistically significant relationships among total class enrollment and high school band directors? classroom assessment methods, purposes, and influences. H 20. There are no statistically significant relationships among the average number of days per week that the class meets and high school band directors? classroom assessment methods, purposes, and influences. H 21. There are no statistically significant relationships among the average number of minutes per class meeting and high school band directors? classroom assessment methods, purposes, and influences. H 22. There are no statistically significant relationships among the average number of class meetings per year held outside regular school hours and high school band directors? classroom assessment methods, purposes, and influences. Definitions Due to the evolution of assessment over the last century, terms such as assessment, measurement, and evaluation are often used erroneously. The following definitions will serve to clarify the terms used in the current study. Measurement is the process of ?obtaining a numerical description of the degree to which an individual possesses a particular characteristic? (Linn & Miller, 2005, p. 26) and evaluation is the process of interpreting assessment results to yield useful information (McMillan, 2004). Assessment methods refer to ?any of a variety of procedures used to obtain information about student performance? (Linn & Miller, 2005, p. 26). Classroom 12 assessment is ?the collection, evaluation, and use of information to help teachers make better decisions? (McMillan, 2004, p. 8) regarding classroom instruction. American authors differentiate between formative and summative uses of classroom assessment, while British authors refer to all classroom assessments as formative (Brookhart, 2004). Formative assessment is an ongoing part of the instructional process and provides constructive feedback to teachers and students about the level of student performance. Formative assessment can occur when teachers provide verbal feedback to their students. Summative assessment ?takes place at the end of a unit of study? and ?document[s] student performance after instruction is completed? (McMillan, 2004, pp. 105-106). Summative assessment can occur through band concerts, festivals, and competitions. Non-music subjects employ summative assessment through written tests, essays, projects, and oral reports. Alternative assessment is a broad category that includes any non-traditional (e.g., pencil-and-paper test) assessment method (McMillan, 2004). For example, performance assessment requires an observation of specific behaviors or outcomes and a judgment of the appropriateness of the response (Oosterhof, 2003). Performance assessment is reportedly the most common type of assessment used in music classes (Music Educators National Conference, 1998; Robinson, 1995). ?The performing arts have a history of using performance assessments, such as public competitions and auditions which replicate the real world of the arts, and use audience inclusion and live demonstrations? (Goranson, 2002, p. 190). 13 Assumptions This study operated under the following assumptions, which are common to most surveys: (a) surveys are used ?to describe the attitudes, opinions, behaviors, or characteristics of the population? (Creswell, 2002, p. 396), (b) members of the selected sample have ?similar characteristics to the target population? (p. 401), and (c) the responses of participants accurately represent the characteristics being measured (Groves et al., 2004). Limitations Survey research is influenced by numerous design, implementation, and response factors that contribute measurement error to the results. Much of this error can be minimized with careful development of surveys. This study was limited by the effects of the following types of survey error (Creswell, 2002; Groves et al., 2004): (a) coverage error: using an insufficient sampling frame (e.g., an incomplete list of potential survey respondents), (b) sampling error: selecting a sample that does not represent the population (e.g., an inappropriate list of potential survey respondents), (c) measurement error: faults attributed to ?the interviewer, the respondent, the survey questionnaire, and the mode of communication? (Groves, 1989, p. 295), and (d) non-response error: the consequence of surveys that are not returned. Chapter 3 discusses how the current survey instrument was designed and implemented in order to maximize data accuracy in spite of these limitations. The sample selected for this study consisted solely of MENC members. This limitation was necessary because a large database of high school band directors was not readily available elsewhere. Previous nationwide studies in music assessment have used 14 MENC membership lists for their sample (e.g., Simanton, 2000), while statewide studies have used lists available from statewide associations (e.g., Hill, 1999). The results of this study may only be generalized to U.S. high school band directors who are members of MENC, and who may differ systematically from those who are not members of MENC. For example, MENC members typically attend state and national conventions, receive two music education journals, and are otherwise active in the profession. Future research could study those teachers who are not MENC members in order to draw comparisons among the groups. Both online and paper surveys were used in an effort to increase the usable response rate. Several statistically significant response mode effects are presented in Chapter 4. Since the paper version was not offered as a response mode until the third mailing, it was impossible to determine whether these differences were due solely to the survey response mode. Other factors, such as respondent motivation or internet availability, could have influenced teachers? decisions to participate in the study. Future researchers should take necessary precautions when implementing a mixed-mode survey, and should understand both its benefits and costs to overall survey error. Participants were asked to respond to each applicable question with regard to their largest size band class. Recent surveys have asked band directors to consider their class with the most 11 th grade students (Simanton, 2000) or have left this detail unspecified (Hanzlik, 2001; McCreary, 2000). Examining band classes with the largest number of students permitted the analysis of class environments in which assessment may have been a more difficult issue. However, this request did not guarantee that all class sizes would be equally large. Classes in the current study ranged from 3 to 240 students; 37.38% had 15 less than 50 students, 51.11% had 50 to 100 students, and 11.51% had greater than 100 students. Summary and Overview The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. This chapter introduced the need for the present study and presented its purpose, research questions, and null hypotheses. Chapter 2 (Literature Review) presents a description and synthesis of the sources consulted in areas identified as central to the present study: assessment history, classroom assessment, and classroom assessment in music education. Chapter 3 (Method) defines the survey research method utilized in this study, details the development and implementation of the survey instrument, and describes the preliminary analysis of the survey data. Chapter 4 (Results) reports the survey data and presents the study?s findings. Chapter 5 (Discussion and Conclusions) discusses the results of the current study, presents three prevalent issues, draws implications, and makes recommendations for future research and practice. 16 CHAPTER 2 LITERATURE REVIEW A search of the extant literature base was conducted in areas related to assessment and music education. The current chapter presents a description and synthesis of the sources consulted in areas identified as central to the present study: assessment history, classroom assessment, and classroom assessment in music education. A brief history of educational assessment frames the current study in a context of evolution, and contrasts contemporary views of assessment with those of the early and mid 20 th century. Recent studies in classroom assessment?both in general education and music education?were reviewed as they related to the purpose of the present investigation. Studies pertaining to the topics of program assessment, teacher assessment, and measurement were consulted, but were deemed tangential to the present literature review. The literature was searched using a variety of traditional and online methods. The University of Maryland Libraries subscribes to an extensive array of databases, several of which were indispensable to the current research. Materials were also borrowed from other institutions through the University of Maryland Interlibrary Loan Office. In addition to printed sources such as handbooks, textbooks, and journals, many online sources were consulted. The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. 17 Assessment History In recent years, assessment has become a widely discussed and debated topic in educational journals, professional conferences, and the mainstream media. The current interest in assessment was generated, in part, by the standards and accountability movements of the 1990s. This brief history of educational assessment through the past century helps position the status of classroom assessment in music education. For more information regarding the history of educational evaluation and assessment, the following sources may be consulted: Linn and Miller (2005), Madaus and Kellaghan (1992), Meisenheimer (1996), Stiggins (1991a), Thorndike (2005), and Trice (2000). In addition, Boyle (1992), Boyle and Radocy (1987), Colwell (1970, 1991, 2002), Cutietta (1992), Lehman (1992), Leonhard (1958), Taebel (1992), and Webster (1992) discussed the history of assessment relative to music education. Early 20 century. th The modern era of educational assessment commenced with the Binet-Simon test of 1905, which was created by Alfred Binet, of France (Thorndike, 2005). The Binet-Simon test measured mental ability and identified children with ?severe learning problems? (Trice, 2000, p. 13). In 1913, Lewis Terman translated it from French into a revised American version, known as the Stanford-Binet test. Current IQ tests continue to reflect the work of Binet and Terman. In the early 20 th century, E. L. Thorndike was working at the Teachers College of Columbia University on issues related to ability testing. With the help of his students, he created measurement scales for reading and arithmetic, and developed theories for educational testing and measurement (Thorndike, 2005). He wrote that educational measurement is possible because ?whatever exists exists in some amount? (quoted in 18 Mark, 1992, p. 48). His work was of great interest to educators because?unlike that of his contemporary, Sigmund Freud?it had immediate implications for teaching and learning (Eisner, 1998). When America entered World War I in 1917, tests were needed to determine who was fit for officers? training school, and to enable the U.S. Army to grow very quickly. The Army Alpha test, the first widely distributed multiple-choice test, determined intelligence by measuring verbal ability. Upon discovering that many of the military recruits were functionally illiterate, the Army produced a second test. The Army Beta test used mazes and puzzles to measure intelligence and required no specific language skills (Thorndike, 2005). The format and structure of the original Army Alpha test had a lasting impact on educational assessment, as evidenced by the continued popularity of multiple- choice tests. The success of the 1913 Stanford-Binet test led to the creation of many new achievement and aptitude tests from the 1920s through the 1950s, such as the competing Wechsler-Bellevue Intelligence Scale (Thorndike, 2005). Some early 20 th -century tests are still used today (albeit in revised forms), such as the Iowa Tests of Basic Skills, the Stanford-9, and the Scholastic Assessment Tests (Trice, 2000). In the 1930s, a period of great criticism emerged as a reaction to test publishers who sacrificed quality for quantity?and profitability. The outcry for more reliable and valid tests made it clear to the measurement community that improvements were needed. Oscar Buros, a professor of educational and psychological measurement at Rutgers University, recognized that the testing profession needed to monitor itself in order to ensure high-quality standards. In 1935, he created the Mental Measurements Yearbook, 19 which published commercially available tests and peer reviews (Thorndike, 2005). This resource currently contains information concerning almost 4,000 tests. America?s 1941 entry into World War II required the creation of many new batteries of tests, mirroring earlier times preceding the First World War. Louis Leon Thurstone?s refinement of factor analysis procedures enabled tests to categorize individuals across several dimensions, instead of on a single criterion (Thorndike, 2005). The success of the factor analysis method resulted in fewer military dropouts and the creation of several taxonomies of human behavior. In particular, psychologist Benjamin Bloom?s Taxonomy of Educational Objectives (Bloom, Hastings, & Madaus, 1971) had a lasting impact on education. McMillan (2003) wrote, ?For decades Bloom?s taxonomy has dominated assessment and educational psychology textbook chapters on student objectives? (p. 39). Mid 20 century. th By the middle of the twentieth century, educational and psychological test publishing had become a ?big business? (Thorndike, 2005, p. 5). Many new standardized tests were published, such as the American College Test and the General Aptitude Test Battery, and a new industry centered on educational testing and measurement. In 1947, Henry Chauncey founded the Educational Testing Service, which continues to provide tests and other services to the education community. The great proliferation of test publishing did have its detractors, however. In the 1960s, a second period of criticism materialized as the civil rights movement grew. In certain areas of the country, test results indicated that the ability levels of minorities, especially women and African-Americans, were lower than the ability levels of white men. Tests were viewed by some as ?biased tools of White male oppression? (Thorndike, 20 2005, p. 6). Since that time, tests have been scrutinized for instances of biased content. In 1985, the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education (1999) developed the Standards for Educational and Psychological Testing. Most recently, the Joint Committee on Testing Practices (2004) revised its Code of Fair Testing Practices in Education. The U.S. federal government invested $1.3 billion in public education with the Elementary and Secondary Education Act of 1965 (Mark & Gary, 1999). The most recent renewal of this legislation is the No Child Left Behind Act (NCLB) of 2001. With this influx of funding came heightened expectations for student learning and, subsequently, new testing programs at local and state levels. NCLB requires school wide accountability for student learning; schools that fail to demonstrate adequate yearly progress are in jeopardy of losing certain federal funding. For example, NCLB requires that high school seniors in 2007-2008 must achieve a minimum passing score on their respective statewide tests in order to graduate (Thorndike, 2005). A technological revolution began in the 1960s with the invention and development of the computer. This new technological device allowed data to be gathered from more students with greater efficiency and less cost. Consequently, assessment and measurement techniques became removed from the purview of classroom teachers. The message sent by this divide was that teachers teach and assessors assess (Stiggins, 1991a). Some teachers essentially abdicated responsibility for creating classroom assessments and began to rely on published tests. As teachers? knowledge of measurement and statistics dwindled, teaching to the test became a common slogan. 21 This divide also affected the measurement and evaluation community in a profound and lasting way. Some traditional measurement specialists believed that they held the keys to assessment and teachers should be charged only with instruction. Younger contemporaries challenged that view by recognizing the essential and critical role of the teacher in the instruction-assessment process. The technological divide continues to affect classroom assessment as more user-friendly software becomes available to teachers and students. Late 20 century to the present. th In the 1980s, an era of professional reflection on educational assessment began. A Nation at Risk: The Imperative for Educational Reform (National Commission on Excellence in Education, 1983) reported many shortcomings in American education, including steady declines in standardized test scores and growing rates of functional illiteracy. All 50 states responded with a plan for educational reform (Linn & Miller, 2005). Education professionals began to understand that ?the majority of the educational outcomes?value[d] for students cannot be translated into objective paper and pencil test items? (Stiggins, 1991a, p. 267). They soon recognized that tests measuring only simple memorization of facts were not useful to students because, in part, ?the amount of available knowledge? [was] doubling at least every three years? (p. 267). Another result of this reflection on educational assessment was the creation of content standards in all academic disciplines. Those who created standards documents sent a new message to test publishers: ?Here are our achievement targets. Make your assessment processes fit these targets!? (Stiggins, 1991a, p. 268). The Consortium of National Arts Education Associations (1994) developed content and achievement 22 standards for dance, music, theatre, and the visual arts. The national content standards for music education consist of the following: 1. Singing, alone and with others, a varied repertoire of music. 2. Performing on instruments, alone and with others, a varied repertoire of music. 3. Improvising melodies, variations, and accompaniments. 4. Composing and arranging music within specified guidelines. 5. Reading and notating music. 6. Listening to, analyzing, and describing music. 7. Evaluating music and music performances. 8. Understanding relationships between music, the other arts, and disciplines outside the arts. 9. Understanding music in relation to history and culture. (Consortium of National Arts Education Associations, 1994, pp. 59-63) In the years following, most states developed content standards for the arts, some of which were based on the national content standards for music education. Because the profession had identified a common set of learning targets vis-?-vis the national standards, teachers began to focus their efforts on classroom assessment. By moving away from paper-and-pencil tests, students would now be required ?to demonstrate performance of certain skills or to create products that meet certain standards of quality? (Stiggins, 2005, p. 140). While performance assessments were new to other academic areas, they had commonly been used in high school band programs. 23 Recent advances in technology and the decreasing costs of personal computers have influenced educational assessment. For example, computer adaptive tests ask questions based on previous answers, and handle information more efficiently and reliably than traditional paper-and-pencil tests (Linn & Miller, 2005). At least three commercial software packages are currently available to assess instrumental music performance: Accent on Interactivity, Smart Music, and the Interactive Pyware Assessment System. It appears that computer technology has the potential to develop and maintain a useful role in music teaching, learning, and assessment. Classroom Assessment Gullickson (1985) surveyed teachers in public and private schools (N = 336) in a Midwestern state to identify their classroom assessment methods and to determine the differences among school levels and academic disciplines. Teachers were asked to rate the extent to which they used 11 assessment methods on a scale ranging from 0 to 3. For the purpose of analysis, the methods were collapsed into three groups: tests, non-test activities, and citizenship. Gullickson found statistically significant differences for all two-way interactions. Teachers? use of quizzes and standardized tests decreased with grade level while teacher- made objective tests and essay tests increased with grade level. There were few differences found among academic subjects at each level. A heavy emphasis on published tests was reported among high school teachers (n = 121). Gullickson, who focused on teacher education in assessment, concluded that ?if measurement instruction is to improve teacher practices, the instruction must deal directly with the practices teachers use? (p. 100). 24 Linn (1990) suggested that teachers should know how to use a wide range of classroom assessment techniques. He advised that the quality of student learning would improve if the quality of classroom assessments improved. This depends on training teachers how to create valid, reliable, and useful classroom assessments. In 1990, The American Federation of Teachers, the National Council on Measurement in Education, and the National Education Association released the following teacher competencies for classroom assessment: 1. Teachers should be skilled in choosing assessment methods appropriate for instructional decisions. 2. Teachers should be skilled in developing assessment methods appropriate for instructional decisions. 3. The teacher should be skilled in administering, scoring, and interpreting the results of both externally produced and teacher-produced assessment methods. 4. Teachers should be skilled in using assessment results when making decisions about individual students, planning teaching, developing curriculum, and school improvement. 5. Teachers should be skilled in developing valid pupil grading procedures which use pupil assessments. 6. Teachers should be skilled in communicating assessment results to students, parents, other lay audiences, and other educators. 25 7. Teachers should be skilled in recognizing unethical, illegal, and otherwise inappropriate assessment methods and uses of assessment information. (Linn & Miller, 2005, pp. 519-522) Shepard (2000) wrote that assessment ?should be moved into the middle of the teaching and learning process instead of being postponed as only the end-point of instruction? (p. 10). Her theory makes connections between classroom assessment, a reformed vision of curriculum, and cognitive and constructivist learning theories. She wrote that although instructional methods have improved in recent years, assessment methods have yet to evolve; only by making connections across the educational community can progress occur. Shepard concluded that ?our goal should be to find ways to fend off the negative effects of externally imposed tests and to develop instead classroom assessment practices that can be trusted to help students take the next steps in learning? (p. 12). Stiggins (2001) presented three barriers that can prevent teachers from implementing assessments. The first barrier involves teachers? emotions about their past assessment experiences. As former students, teachers have a variety of positive and negative memories about assessment that influence their current practice. Teachers who have had negative experiences with assessment as a student may avoid assessing their students; alternatively, they may try to reconcile their emotions by learning how to be better assessors. The second barrier is the lack of instructional time for assessment. Teachers can be so overwhelmed with instructional responsibilities that assessment occurs infrequently. They need assessment methods that require minimal time and yield high quality information about student learning. The third barrier is the lack of 26 assessment expertise among teachers. Teacher education programs rarely include more than basic topics in educational measurement and assessment. McMillan (2001) surveyed 6 th - through 12 th - grade Virginia teachers (N = 1,483) in the areas of science, social studies, mathematics, and English. The purpose of his study was to investigate classroom assessment and grading practices, and to determine the differences among grade levels, academic subjects, and student ability levels. Participants used a 6-point scale to communicate the extent to which they used various grading criteria. McMillan found that teachers preferred to use teacher-designed assessments, performance quizzes, objective assessments, essay questions, and performance assessments. The study also found that teachers used assessment primarily to measure understanding, application, and reasoning. McMillan concluded that ?Most secondary teachers use a multitude of factors in grading students?.Academic achievement is clearly the most important component in grading students? (p. 28). McMillan (2004) suggested that there are a number of internal and external factors to be considered when planning for classroom assessment. Internal factors include the knowledge, beliefs, expectations, and values of a teacher. External factors include state testing, district policies, and parents. The combination of these factors creates tensions that must be resolved by the teacher when planning for classroom assessment. Wiggins and McTighe (2005) advised using a backward design process, which involves planning assessments before planning instruction. The first step is to identify the specific knowledge and skills that students should learn. The second step involves choosing assessment methods and defining the criteria for acceptable evidence. In the last 27 step, instructional methods, sequencing, and resource materials are chosen. The backward design process helps ensure that teachers remain focused on the goals of instruction as they plan and implement their lessons. Classroom Assessment in Music Education Few authors have studied classroom assessment in high school band programs. In 1965, Gutsch acknowledged the ?lack of adequate techniques for evaluating instrumental music performance within the public school setting? (1965, p. 21). Most of the current classroom assessment literature in music education consists of practitioner articles that outline ?how to? assess student learning. Such writing reveals that music teachers remain concerned about classroom assessment issues. This review focuses mainly on peer- reviewed research articles of importance to the current study. Mark and Gary (1999) discussed the development of one of the first major ?assessments? designed specifically for instrumental music: the school band (and orchestra) contest. Hundreds of men who were trained by the military to be band musicians and conductors returned from World War I and became school band directors. The popularity of professional touring ensembles, such as the Sousa and Gilmore bands, prompted school bands to grow and ultimately compete with one another. In 1923, the first organized ?tournament? was in Chicago and featured 26 high school and 4 elementary bands. School band contests ultimately resulted in ?standardization of band instrumentation, the practice of publishing full band scores, increased emphasis on instrumental music in teacher-training programs, and a phenomenal growth in enrollment for school bands? (p. 273). National contests ceased during World War II and never resumed; however, band festivals and competitions continue to thrive today. 28 Boyle and Radocy (1987) wrote a landmark text that focused on measurement and evaluation in music education. The authors defined the forms and functions of educational assessment, discussed contemporary issues, and described the development of various types of classroom, teacher, and program measures. The text focused on summative forms of assessment and described several published music tests. Formative classroom assessment methods and the uses of assessment information were not discussed; these became prevalent issues in the 1990s. Wolf (1987) wrote that the arts share several qualities that make them unique: learning in the arts occurs slowly over lengths of time, individuality is encouraged as much as the development of skills, and students are assessors of their own and others? artistic quality. Teaching, learning, and assessment in the arts are often more dependent upon process than product. Wolf discussed projects such as Arts PROPEL and Harvard Project Zero, which studied assessment in the arts by putting specific strategies into practice (e.g., portfolios, projects, and reflective interviews). The results demonstrated that students and teachers could be taught to be critical interpreters of assessment information. In 1991, Colwell wrote a seminal chapter that reflected on the changes in educational assessment since the mid 20 th century. He focused on music program assessment for the purpose of accountability, and wrote, ?To operate schools is expensive, and the public appears reluctant to invest more heavily in education without valid data on past progress and evaluation of the prospects for the future? (1991, p. 247). Colwell criticized music teachers for their disinterest in developing standards and for their failure to use results of the National Assessment of Educational Progress ?to 29 discover whether or not their efforts were correctly focused? (p. 255). However, this situation seemed to improve in the mid 1990s following the publication of the National Standards for Arts Education (Consortium of National Arts Education Associations, 1994). McCoy (1991) stated that ?scant information has been gathered to document the actual content of music courses as they are currently being taught and the methods that teachers use to evaluate student learning? (pp. 181-182). This study compared the grading practices of band directors (N = 57) and choral directors (N = 44) in 98 randomly selected Illinois high schools, with their principals? (N = 40) recommendations for grading. The survey consisted of 25 criteria used for determining grades. Teachers marked the criteria they used to determine student grades and principals marked the criteria they felt were appropriate for grading. McCoy found that principals preferred grading students based on objective measures of student achievement, such as basic performance technique. Band and choral directors based student grades primarily on non-musical criteria, such as concert attendance and behavior. McCoy suggested that principals were reportedly more aware than teachers were about the issues associated with grading on attendance or behavior. Teachers who relied on non-musical criteria for assessment may have been ?teaching as they were taught? (p. 189) rather than on sound principles of classroom assessment. McClung?s (1996) survey used a convenience sample of high school choral directors (N = 81) and students (N = 615) who participated in three 1995 Georgia All- State Choruses. Their respective school principals (N = 117) were surveyed by mail. The 30 purpose of the study was to ?investigate and describe learning assessment and grading practices in the high school choral performance classroom? (p. v). McClung found that teachers preferred to determine grades mainly through class participation and performance tests. Paper-and-pencil tests were generally associated with academic areas other than chorus. Students, teachers, and principals reported that grades should reflect an ?objective measurement of student progress toward achieving the desired behavior? (p. 168). McClung concluded that teachers should develop skills for assessing individual and group learning. Blocher, Greenwood, and Shellahamer (1997) studied the amount of time that middle school (N = 12) and high school band directors (N = 9) exhibited specific behaviors during rehearsals. Participants were purposefully selected from all Florida secondary schools. Each participant videotaped two to three rehearsals within one week. The videotapes were analyzed by two trained observers (university instrumental music educators) using a Continuous Response Digital Interface (CRDI), which consists of a dial manipulated through a 256-degree arc. The arc was divided into seven segments, two of which corresponded to nonverbal and verbal feedback. While viewing the videotape, the dial was rotated to the segment that indicated the observed behavior. A computer then calculated the amount of time the dial was positioned in each segment to determine the length of time that the teacher was engaged in each specific behavior. Among high school band directors, verbal feedback occurred 3% of the time during rehearsals and nonverbal feedback occurred 0.63% of the time. In a 19 minute 20 second rehearsal segment, high school band directors gave 96 seconds of feedback, on average. Blocher, Greenwood, and Shellahamer concluded that teachers may have been 31 unaware of the positive benefits of student feedback, they may have had insufficient teacher training, or they may have lacked high-quality teacher role models. Goolsby (1997) compared specific rehearsal behaviors of student teachers (N = 10), novice teachers (N = 10), and expert teachers (N = 10) in middle- and high- school bands. Each teacher was videotaped during three rehearsals; the author and two research assistants constructed a form used to analyze the videotapes. Among the rehearsal variables studied were unspecific positive feedback and specific positive feedback. Goolsby found that student teachers and novice teachers gave unspecific positive feedback to the band students as compared with expert teachers, who were more likely to give specific positive feedback. The second part of Goolsby?s study investigated the verbal communication of upper-level pre-service teachers (N = 11) over three semesters. More feedback was given by students in the fall semester of their senior year than in the fall semester of their junior year. Goolsby also found that expert teachers (i.e., those teaching over eight years) gave more efficient feedback than both novice teachers and student teachers. Over one-third of the student teacher teaching segments included no feedback or instruction. Goolsby recommended that teachers should receive instruction in proper questioning techniques to ?assist students in learning more about music? (p. 38). Assessment in band classes is unique due to music?s temporal nature. Saunders and Holahan (1997) stated, ?Instrumental music performances are an unfolding of aural events across time? (p. 259). To complicate matters of assessment, there can be over 50 students playing over a dozen different types of instruments in a high school band class. Lehman (1998) also identified large class sizes as a major barrier to classroom 32 assessment in music. Few other academic subjects, if any, can claim this level of instructional complexity in the classroom. Goolsby (1999) suggested that classroom assessment usually saves time in the end, as opposed to the conventional wisdom that assessment uses valuable instructional time. He recommended several practical tools for instrumental music assessment, including checklists, worksheets, audiotapes, and self-evaluation. Hill (1999) surveyed high school band directors (N = 93), students (N = 327), and school administrators (N = 38). The band directors who participated were attendees at a meeting of the Mississippi Bandmaster?s Association. The purpose of the study was to ?compare responses of administrators, faculty, and students regarding the types of assessments and grading procedures utilized in the high school instrumental performance classroom in Mississippi? (p. 4). Hill found that high school band directors preferred to assess students based on musical criteria; however, non-musical criteria such as attendance, attitude, and participation were viewed as important sources of information. Teachers, students, and school administrators reported that paper-and-pencil tests, portfolios, and sight-reading were suitable for individual assessment, but paper-and-pencil tests were most associated with other classes. Hill investigated attitudes toward assessment methods, and recommended that a study of assessment methods used in instrumental music classes would be valuable. Simanton (2000) surveyed U.S. high school band directors (N = 202) to determine their current assessment and grading practices, local satisfaction with their assessment practices, and variations in their assessment practices based on regional, school, and 33 teacher variables. The study used a national sample of public schools stratified by each division of MENC: The National Association for Music Education (MENC). Simanton?s first mailing included a paper survey; his second mailing invited respondents to complete the survey online. Of the respondents, 13.37% completed the online version of the survey; no statistically significant differences were found for response mode. The survey also collected demographic information related to the respondent?s level of education, school characteristics, and band program. Participants were instructed to respond with regard to their band with the most 11 th grade students. Simanton found that 18% of high school band directors did not assess individual student performance at all. Almost 70% of band directors assessed students during class time and 65.8% assessed students outside of class time. Some teachers reported using a tape recorder (33.2%) or video camera (6.7%) for individual assessment, while 6.6% used a computer for non-performance assessment. Many band directors (63%) reported using paper-and-pencil methods of assessment, including quizzes and journals. Most teachers (76%) were satisfied with their assessment methods; however, most teachers (89.5%) stated that they would do more classroom assessment if they had more class time. With regard to demographics, Simanton found that class size, MENC division, and teacher education had statistically significant relationships with several assessment variables. Band directors of small bands and those with more teaching experience reported using a tape recorder more often. Time allotted for assessment was almost three times greater in the north central MENC division than in the western, south western, and north western divisions. Simanton suggested that more teachers in the north central 34 division might teach private lessons during the school day than in other regions, using a portion of that time for student assessment. In 2001, The National Board for Professional Teaching Standards developed guidelines specific to music teacher competencies in assessment. They recommended that successful music teachers should be able to: 1. Create a variety of assessment tasks and materials for assessing student learning. 2. Plan assessments before planning instruction. 3. Present assessments at appropriate times in the instructional sequence. 4. Ensure that students understand what they are expected to know and be able to do. 5. Ensure that students understand how they will be assessed, upon what criteria they will be judged, and how this information will help them to improve. 6. Use a variety of meaningful student self-assessment techniques. (National Board for Professional Teaching Standards, 2001, p. 16) Hanzlik (2001) surveyed high school band directors (N = 154) in Iowa. The purpose of this study was ?to determine the types and frequency of assessment?and examine the effects of selected teacher background variables on teachers? attitudes toward assessment? (p. 9). The response rate for the survey was 77%; all respondents were members of the Iowa High School Music Association. Hanzlik found that high school band directors assessed performance skills more frequently than cognitive or affective skills. ?The assessment practices used by band 35 directors 80% of the time were playing band music and scales and rudiments; sight- reading music; teacher observation and playing etudes? (p. 125). Teachers at small schools had more positive attitudes toward assessment than teachers at large schools. Hanzlik stated that this may have been due to the practice of hiring band directors who are recent college graduates to teach at small schools; they may be more enthusiastic, motivated, and knowledgable about classroom assessment. Hanzlik?s study showed that instrumental music teachers with 10 to 25 years of teaching experience had negative attitudes toward assessment when compared to music teachers who had other amounts of teaching experience. Hanzlik hypothesized that the less-experienced teachers may be more enthusiastic about implementing new methods in their classes and the more experienced teachers (who tend to work in larger schools) may have assistants who help with their administrative work, thereby leaving more instructional time for assessment. McCreary?s (2001) research utilized a small sample of instrumental music teachers (N = 10) selected systematically from among 97 secondary schools listed in the Hawaii Music Educators Association 2000-2001 Directory. The students (N = 467) of the teachers selected were also participants in the study. Her purpose was to ?collect and examine the methods and procedures currently used in evaluating secondary school instrumental (band and orchestra) students? (p. 4). McCreary reported that teachers used ?traditional methods of paper and pencil tests, playing tests, practice time, and attendance and/or attitude to evaluate their students? (p. 103) and did not use alternative methods of assessment. None of the 36 respondents thought portfolios were suitable for measuring musical progress. McCreary suggested that teachers might lack knowledge regarding alternative assessment methods. Kotora (2001) compared the assessment methods used by high school choral teachers (N = 246) in Ohio with the assessment methods taught by teachers of choral methods classes (N = 20) in Ohio colleges and universities. Surveys were sent to all high school choral teachers who were members of the Ohio Music Education Association (N = 608), and to all Ohio college choral methods teachers listed in the 1996-1997 College Music Society?s Directory of Music Faculties in Colleges and Universities, U.S. and Canada (N = 38). Kotora found that high school choral directors relied heavily on non-musical criteria, such as student attitude, participation, and attendance for assessment purposes. Responses indicated that some teachers were ?torn between preparing their students to perform and trying to document individual student achievement? (p. 180), that assessing individuals in large classes can create discipline problems, and that the lack of class time is a major barrier to assessment. The study showed that standards set by the school district, the state, or MENC, had almost no influence on assessments used by teachers. Kotora concluded that teachers? assessment decisions are based mainly on personal choice, and more instruction on classroom assessment methods should be offered at professional conferences and workshops. Bauer and Berg (2001) surveyed high school instrumental teachers (N = 120) in Indiana, Illinois, and Ohio to investigate the influences on their implementation of classroom assessments. The survey used a 5-point Likert-type scale to measure the 37 degree of influence that 17 factors had on their planning for instruction, implementation of learning activities, and assessment of student learning. The factors most often considered when assessing student learning were the teacher?s experience, influence from colleagues, and professional development activities. The factors least often considered were parent expectations, the student teaching supervisor, and undergraduate courses. Bauer and Berg concluded that ?undergraduate music and music education courses, and the university supervisor during student teaching exert little influence on instrumental music teacher planning, teaching, and assessment? (p. 64). Teachers viewed their former ensemble conductors, studio teachers, and cooperating teachers as major influences, suggesting, ?These individuals must [italics added] be good models for planning, teaching, and assessment? (p. 65). Burrack (2002) described portfolio assessment based on his experience with Arts PROPEL in the Carroll Community Schools of Iowa. Instrumental music students first made audio recordings of their individual performance of scales and excerpts. Next, they listened to their recordings with a trained instructor, who guided the listening experience. Then the students individually assessed their own performance on rhythm, pitch, tone, technique, and musicianship. Finally, their written comments and recorded performances became part of a music portfolio that was maintained throughout their schooling. This assessment event was planned for one time in fifth and sixth grades, twice in seventh and eighth grades, and quarterly in high school. Tracy (2002) investigated ?the issues which impact assessment of individual choral students in group settings? (p. 2). The participants (N = 183) were high school choral teachers randomly selected from the MENC southern division, using statewide 38 stratification. The author found that ?a teacher?s belief in the importance of assessment exercises the greatest influence over practice? (p. 147). Beliefs were found to be a greater influence on assessment practice than variables such as the student-teacher ratio, instructional time, and training in assessment. Tracy called this a ?red flag to those working in teacher preparation programs? (p. 151) and suggested that philosophical aspects of assessment should be addressed in undergraduate methods courses. Tracy also investigated the specific types of assessments used in high school choral classes. The methods used most often were observation in rehearsals, attendance records, and individual or small group performances. Conway and Jeffers (2004) researched parent, student, and teacher perceptions of assessments that occurred in beginning instrumental music classes. Parents and students in this study collaborated with the teacher to create classroom assessment tools and procedures based on Edwin Gordon?s music learning theory. The authors found that parents wanted specific feedback from teachers, including letter grades, which would allow them to compare their child?s progress to that of other students. They concluded, ?As educational policy dictates a move in the profession towards consistent documentation of student achievement in music courses we must continue to examine appropriate ways of assessing students and reporting that assessment? (p. 23). Morrison, Montemayor, and Wiltshire (2004) examined the effects of a recorded ensemble performance model on secondary band students? self-evaluations, achievement, and attitude. They found that the use of a recorded model ?may have allowed students to maintain a more consistent or objective perspective according to which they measured their progress? (p. 126). Students in the model condition wrote more comments about 39 their performances, suggesting that the model may have introduced a wider variety of performance concepts than they were previously aware of. The authors suggested that future researchers should address specific teaching techniques and implementation of recorded models in the classroom. Hewitt and Smith (2004) investigated the relationships among teachers? level of experience, their primary instrument, and their evaluation of trumpet players. Contrary to the findings of previous studies, they did not detect any relationships between experience and evaluation. Likewise, no relationships were found between the primary instrument played by the teacher and their evaluations. These findings suggested that teachers? evaluations were not dependent upon their instrumental experience or their teaching experience. The authors suggested that further research should be undertaken to determine the ?types and numbers of experiences that lead to this level of assessment skill? (p. 324). Self-evaluation can be a useful form of classroom assessment because it actively involves students in the evaluation process (Farrell, 1997; Goolsby, 1999). Hewitt (2005) found that middle- and high-school band students (N = 51) tended to overrate their own performances as compared to evaluations made by trained professionals. He suggested that tonguing, slurring, accents, and appropriate note lengths were more difficult for students to evaluate because they did not focus on these areas when they performed. Hewitt concluded that students? self-evaluations were generally inaccurate, and suggested training students in self-assessment techniques in order to develop their musical independence. 40 Summary Most of the literature reviewed focused on grading criteria and classroom assessment methods. Gullickson (1985), Hill (1999), Kotora (2001), McCoy (1991), and McClung (1996) used survey instruments to explore the methods that high school band and choral directors used to grade their students. Most studies found that grading on non- musical criteria, such as participation and behavior, was a common practice among teachers. Principals and school administrators, however, believed that student grades should be based on objective measures of musical performance (Hill, 1999; McCoy, 1991). The present study extends the work of authors (Hanzlik, 2001; Kotora, 2001; McCreary, 2001; Simanton, 2000; Tracy, 2002) who previously investigated the classroom assessment methods used by high school band, orchestra, and choral directors. Simanton found that nearly one-fifth of teachers did not use any assessment methods while Hanzlik found that the amount of assessment varied with school size and the years of teaching experience. Most studies have concluded that high school music teachers use traditional methods of music assessment (e.g., performing in class), rather than contemporary methods (e.g., videotaping). Feedback given to students based on their musical performance is a commonly studied assessment behavior. Blocher, Greenwood, and Shellahamer (1997) and Goolsby (1997) investigated the types of feedback given to middle- and high-school band students in rehearsal settings. They found that although specific verbal feedback is rarely offered, the amount of feedback increases among the more experienced teachers. Goolsby stated that over one-third of the student teachers offered no feedback at all. 41 Bauer and Berg (2001), McMillan (2001, 2004), and Tracy (2002) conducted research regarding the purposes teachers have for assessment and the factors that influence their use of classroom assessment. McMillan (2004) suggested that numerous internal and external factors make it difficult for teachers to decide how and when to assess student learning. Tracy reported that the philosophical beliefs of teachers were more influential on classroom practice than external factors, such as the amount of class time or training in assessment. Bauer and Berg found that studio teachers, ensemble directors, and cooperating teachers had a greater influence on teachers? assessment practices than their undergraduate coursework. Some authors have written about the uniqueness of assessment in the arts and the barriers that can prevent high quality assessments from occurring. Wolf (1987) stated that assessment in the arts is often about process, which can occur over extended periods. Saunders and Holahan (1997) remarked that assessment is difficult because there are so many events that occur aurally in an instrumental rehearsal. Stiggins (2001) presented three common barriers to classroom assessment: experiences with assessment, lack of instructional time for assessment, and lack of assessment expertise. Several authors have recommended ways to help reduce these assessment barriers. Wiggins and McTighe (2005) presented a backward design process that moves assessment planning to the front end of instructional planning. Shepard (2000) wrote that assessment methods are still young in their evolution, and that classroom assessment should be more central to the teaching and learning process. Goolsby (1999) stated that assessment is usually a time-saver in that it gives crucial information for future planning; he recommended several useful and efficient methods for instrumental music assessment. 42 Burrack (2002) focused on a description of how portfolio assessment has worked in instrumental music programs. Other authors have focused their efforts on teacher education in assessment (American Federation of Teachers, National Council on Measurement in Education, and National Education Association, 1990; National Board for Professional Teaching Standards, 2001). 43 CHAPTER 3 METHOD The present chapter defines the survey research method used in this study, details the development and implementation of the survey instrument, and describes the preliminary analysis of survey data. Survey questions are cross-referenced using abbreviations (e.g., Q 1 ) that refer to question numbers on the paper version of the survey (see Appendix E). The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. Research Method Survey research is an efficient method for gathering data from a large population, and ?is a common and valuable approach to determine status? (Abeles, 1992, p. 231). ?Surveys help describe the trends in a population or describe the relationship among variables or compare groups? (Creswell, 2002, p. 421). Mitchell and Jolley (1996) advised, ?Once you use a descriptive design to find out what happens, you can use an experimental design to try to find out why it happens? (p. 395). Few nationwide studies (Music Educators National Conference, 1998; Simanton, 2000) have investigated the status of classroom assessment in high school band programs. The current study investigated the current state of classroom assessment in high school band programs and its results could be explored further through experimental research. As an alternative to traditional paper surveys, internet surveys (see Appendix B) can be an effective way to study large populations. Online data collection is often more 44 accurate, cost effective, and time efficient than mail surveys because no data entry is required on the part of the researcher. While there are currently many websites that publish online surveys, http://www.zoomerang.com/ was selected because of its low student price and its available features. For example, the present survey used these features: mandatory questions, automated question skipping, and randomization of response alternatives. Other survey providers were also evaluated, such as http://www.surveymonkey.com/ and http://www.inquisite.com/, but were deemed expensive and more feature-laden than necessary for the purpose of this research. Sample Selection MENC: The National Association for Music Education (MENC) provided a list of names and addresses of high school band directors (N = 12,111). List members had identified themselves as high school band directors at the time of their MENC membership application or renewal. MENC has a nationwide membership of over 120,000 music educators who teach in public and non-public schools. While not accounting for non-MENC members, the list appeared to contain a substantial portion of the target population; there are approximately 18,732 public high schools in the U.S. (National Center for Education Statistics, 2005). The Instrumentalist Magazine and School Band and Orchestra Magazine were also contacted, but their representatives were unable to provide mailing lists. The sample size for the present study was determined through a pilot test and by using guidelines set forth in the survey research literature. Alreck and Settle (2004) wrote that a sample larger than 10% of the target population is rarely necessary, because as 45 sample size increases, sampling error decreases. One method used to determine sample size is the formula ( ) ( )( ) () 067,1 03.0 ?1?96.1 2 2 ? ? =n which assumes a 95% confidence interval, a 3% margin of error, and a 50% probability of the trait of interest occurring in a randomly selected individual (Lohr, 1999). The result is a sample size estimate for any population over 10,000 people; therefore, many large- scale surveys use a sample size of 1,100. Fowler (2002) proposed a sample size table, with results similar to Lohr (1999). Creswell (2002) suggested using a sample size of 350 for survey research, Alreck and Settle (2004) advised that a sample size of 200-1,000 is suitable for any large population, and Sudman (1976) recommended using at least 1,000 participants for a national survey. Recent surveys of high school band directors have reported usable response rates ranging from approximately 30% to 80%. A study similar to the current research reported a 34.2% usable return rate (Simanton, 2000); a survey of high school choral directors had a 43% usable return (Kotora, 2001). The current pilot study, discussed later in this chapter, had a 30% usable response rate. Using the pilot study finding and the survey research literature as guides, a sample size of 2,000 was selected for the main study. This was deemed satisfactory because a 30% return would yield 600 responses, which is within the range of suggestions previously mentioned. Simple random sampling was used to select participants from the MENC list; this was accomplished using the rand function in Microsoft Office Excel 2003 SP2. A simple random sample represents the target population more accurately than a sample chosen using other sampling procedures (Alreck & Settle, 2004) and gives each member of the 46 sampling frame an equal probability of selection. Bias that occurs naturally in the target population is therefore distributed among all members of the sample (Creswell, 2002). Minimization of Survey Error Survey research is influenced by numerous design, implementation, and response factors that introduce undesirable error. This section discusses how the following types of error were minimized in the current study: coverage, sampling, measurement, and non- response (Creswell, 2002; Groves et al., 2004). Coverage error. Coverage error results from using an insufficient sampling frame, such as a mailing list that does not include the entire target population; this is mitigated by utilizing the most complete mailing list available. For the purpose of this study, the MENC list was deemed suitable because it represented teachers in U.S. public and non- public schools. All participants were members (as of October 5, 2005) of the largest professional organization of music teachers in the country, which indicates that they received professional literature distributed by MENC (i.e., Music Educators Journal and Teaching Music). They may also have attended state, divisional, and national professional conferences. Therefore, this sample may have been more aware of current issues in classroom assessment than high school band directors who were not MENC members. Accordingly, the results of this study need not generalize to U.S. high school band directors who are not MENC members. Another condition associated with coverage error is the selection of participants who are not eligible to complete the survey. This may be due to people on the MENC list who were no longer teaching high school band or who were on the list erroneously. The first two questions on the survey helped prevent coverage error: 47 Q 1. Please enter the personal identification number (PIN) found on your survey invitation. Q 2. Are you a high school band director? The first question enabled responses to be tracked using a personal identification number (PIN) ranging from 1 to 999999. These were randomly assigned to each participant using the rand function in Microsoft Office Excel 2003 SP2. Although no personal information was stored with the data file, it could still be determined who had responded by referencing the original mailing list. These questions helped ensure that only valid invitees completed the survey. Heerwegh and Loosveldt (2002) found that requiring participants to enter a PIN had a positive effect on the quality and depth of the responses they provided, when compared to those whose PINs were entered automatically. The second question was answered either ?yes? or ?no?. A ?no? response would send the respondent to the final page of the survey and thank them for their participation. If they had made an error by answering ?no?, they were free to begin the survey again, and their error would be discovered during data screening. Coverage error can also be induced by members of the sample that are not reachable, that are not capable of responding, or that refuse to respond (Lohr, 1999). The first issue was controlled by mailing invitation postcards (see Appendix C) to the addresses provided by MENC. All participants should have been capable of responding at their school, if not at their home; Parsad and Jones (2005) found that almost all U.S. high schools have internet availability. Alreck and Settle (2004) suggested that those who refuse to respond to surveys usually do so within mere seconds of the initial contact. 48 To reduce error due to refusal, a reminder postcard (see Appendix D) was mailed two weeks later and a third mailing occurred four weeks later (see Appendix E). At the time of the third mailing, the response rate was only 20.1%. To ensure that the return rate met the goal of 30%, the third set of postcards was replaced by paper surveys. This provided a concerted effort to encourage all participants to respond. Schonlau, Asch, and Du (2003) reported that even among populations that were presumably computer savvy, more people preferred to respond to paper surveys than online surveys. Umbach (2004) stated that ?mixed-mode administration may be the answer to low response rates? (p. 35). Sampling error. Sampling error results from selecting a sample that is not characteristic of the target population. A simple random sample was chosen from the MENC list; therefore, all members of the target population had an equal probability of selection. A relatively large sample was selected, which also minimized the effects of sampling error. Measurement error. Measurement errors are attributed to ?the interviewer, the respondent, the survey questionnaire, and the mode of communication? (Groves, 1989, p. 295). Errors result when participants misunderstand the question presented to them, or when they censor their own responses. The questions on this survey were made as clear as possible?and pilot tested?because appropriate and unambiguous questions can motivate people to complete their surveys (Creswell, 2002). For example, the word assessment can be understood to mean student assessment, teacher assessment, or program assessment. In addition, the length of a question, the order of questions, and the order of response choices in a multiple-choice question can all produce error (Groves, 49 1989). The modes of communication considered for this survey included both mail and internet methods, and a combination of the two was determined most appropriate. Non-response error. Non-response error is the result of non-returned surveys. This is countered by using every effort to increase the response rate. Reminder postcards were mailed two weeks after the initial mailing to encourage all persons in the sample to respond. As previously discussed, a paper version of the survey was sent four weeks after the first mailing to target those people who preferred not to respond online. Specific features of the survey invitations were designed to increase response. Unlike envelopes, the postcard invitations were open and viewable immediately upon receipt. Every invitation included the University of Maryland logo (used with permission), which identified the sponsoring institution; this has been found to increase response rates (Groves, Cialdini, & Couper, 1992; Presser, Blair, & Triplett, 1992). The salutation was personalized with the recipient?s name and participants were advised that they had been purposefully selected to participate. The invitation pointed out that the survey was available online and would take about 10 minutes to complete. The results of the study were offered as an incentive to participate, and an email address was made available for questions or comments. Confidentiality was ensured and respondents were advised that they could end the survey at any time; this offering of personal freedom has been found to increase response rates (Biner, 1988). Finally, the respondents were thanked generously for their time and effort in completing the survey. These design features were implemented with the expectation of an increased response rate. 50 Independent Variable Development The 22 independent variables were categorized into two groups: 11 personal characteristics and 11 school characteristics (see Appendix A). This section describes the questions on the survey that measured each variable, beginning with a discussion of the independent variables related to personal characteristics of the high school band director. Personal characteristics. Gender (Q 3 ) and age (Q 4 ) are demographic variables commonly studied in the social sciences. Although there is no apparent evidence to suggest that these variables relate to the assessment practices of high school band directors, it would be important to note if relationships were indicated by the data. The survey asked for the year of birth, rather than age, in order to obtain an accurate and complete answer. The age of respondents was calculated during post-processing by subtracting their year of birth from 2006. Teachers with different amounts or types of teaching experience may vary in their assessment behaviors. Employment status (Q 6 ) was measured as either part time (less than 20 hours per week) or full time (20 hours or more per week). Five questions requested the respondents? number of years teaching high school band (Q 9 ), years teaching any subject or grade (Q 10 ), years teaching at their current school (Q 11 ), high school band classes they teach (Q 18 ), and other classes they teach (Q 19 ). The survey asked respondents to select their highest degree received (Q 7 ): high school diploma, associate?s degree, bachelor?s degree, master?s degree, doctoral degree, or post-doctoral work. The survey also asked whether the respondent is licensed to teach music, and whether their license is of a provisional or permanent nature (Q 8 ). Finally, the teacher was asked to select any or all of the following types of assessment training that 51 they had received (Q 15 ) from the following: departmental training, school-wide training, district-wide training, undergraduate coursework, graduate coursework, professional conference or workshop, self-study, or none. School characteristics. Variables related to school characteristics included the state (Q 5 ), MENC division (inferred from Q 5 ), and geographic setting (Q 13 ) of schools. The following response choices for geographic setting were determined after examining criteria used by the United States Census Bureau (2005) and the Washington State Department of Health (2005): urban/inner city, suburban, small town, or rural/remote. The survey contained three questions related to the school in which the respondent taught high school band. The first asked whether their school was public or non-public (Q 12 ), using two broad categories of schooling defined by the United States Department of Education (2005). The next item asked for the socioeconomic status? low, medium, or high?of the majority of students enrolled in the school. A third question (Q 20 ) requested the total school enrollment. The five remaining independent variable items pertained to the respondent?s largest high school band class. This permitted analysis of the largest size band classes, in which cases assessment may have been a more difficult issue. The first item asked whether they were the only music teacher present in their classroom (Q 25 ). Another asked for the class enrollment (Q 21 ). Two questions requested the length of each class in minutes (Q 23 ) and the number of days the class meets per week (Q 22 ). The final question determined how many rehearsals were held after school hours (annually) with this ensemble (Q 24 ). 52 Dependent Variable Development The three dependent variables were the use of classroom assessment methods (Q 26 -Q 27 ), purposes of classroom assessment (Q 28 -Q 29 ), and factors that influence the use of classroom assessment (Q 30 -Q 31 ). Respondents who taught more than one high school band class were asked to respond to each question with regard to their largest class. Again, this permitted analysis of the largest size band classes, in which cases assessment may have been a more difficult issue. Use of classroom assessment methods. A list of 23 classroom assessment methods used by high school band directors was generated from the music education literature, the pilot study, and suggestions from university faculty members, graduate students, and other music education colleagues. The survey question (Q 26 ) asked, ?In a typical marking period, how many times do you use the following student assessment methods in your largest high school band class?? Response options for the 23 methods ranged from 0 to 100+. The assessment methods included on the survey were the following: 1. Students play alone in front of the class. 2. Students play alone for the teacher only. 3. Students audiotape themselves playing alone. 4. Students videotape themselves playing alone. 5. Students play with others in a concert. 6. Students create portfolios of their work. 7. Students use computers to assess their learning. 8. Students play in a small ensemble. 53 9. Students have individual conferences with the teacher. 10. Students audition for chair placement. 11. Students audition for ensemble admission. 12. Students assess themselves. 13. Students assess other students. 14. Students complete written work in class. 15. Students complete written work at home. 16. Students complete a practice log/journal. 17. Students complete a published, standardized test. 18. Students notate music (dictation, composition, etc.). 19. Teacher plays an instrument for students to hear. 20. Teacher maintains an annotated log or journal. 21. Teacher uses a checklist to assess student learning. 22. Teacher uses a rubric to assess student learning. 23. Other music specialists or guest conductors assess students. Additionally, a qualitative response item (Q 27 ) allowed the respondent to enter any other classroom assessment methods that they used. This open-ended comment box provided an opportunity for the researcher to corroborate and expand upon the quantitative data measured by the survey. Purposes of classroom assessment. A list of 19 purposes of classroom assessment was developed, as before, from the music education literature, the pilot study, and suggestions from colleagues. The survey question (Q 28 ) read, ?How important are the following purposes of student assessment in your largest high school band class?? The 54 purposes appeared below a five-point Likert-type scale ranging from 1 (not at all important) to 5 (extremely important). The three intermediate points (i.e., 2, 3, and 4) were not labeled, which allowed the data to be analyzed as an interval scale; the five points of the scale were assumed equidistant by the participants (Alreck & Settle, 2004). In order to randomize any error due to order effect, the online survey randomized the 19 purposes for each participant. The assessment purposes included on the survey were the following: 1. To identify individual student needs. 2. To identify general class needs. 3. To determine level of musical preparedness for a public performance. 4. To set or maintain class standards. 5. To rank students according to individual performance level. 6. To meet local, state, or national standards. 7. To meet school or school district requirements. 8. To determine interim or report card grades. 9. To provide feedback to students. 10. To provide feedback to parents. 11. To provide feedback to school administrators. 12. To determine whether instruction has been successful. 13. To demonstrate accountability for student learning. 14. To help students prepare for a public performance. 15. To establish or maintain credibility for the music program. 16. To determine whether students are practicing at home. 55 17. To determine soloists for a specific piece of music. 18. To determine what concepts students are failing to understand. 19. To motivate students to practice their instruments. In addition, a qualitative response item (Q 29 ) allowed the respondent to enter any other purposes of classroom assessment they considered important. This open-ended comment box provided an opportunity for the researcher to corroborate and expand upon the quantitative data measured by the survey. Factors that influence the use of classroom assessment. A list of 23 factors that influence the use of classroom assessment was developed, as before, from the music education literature, the pilot study, and suggestions from colleagues. The survey (Q 30 ) asked, ?What level of influence do the following factors have on the assessment methods you use in your largest high school band class?? The factors appeared below a five-point Likert-type scale ranging from 1 (not at all influential) to 5 (extremely influential). Once again, the three intermediate points were not labeled, and the 23 factors were randomized for each participant. The assessment influences included on the survey were the following: 1. The amount of available class time. 2. The number of students enrolled in the class. 3. The expectations of your students. 4. The objectives or goals of your class. 5. The demands of your ensemble's performance schedule. 6. Requirements set by the school district. 7. The high school band curriculum. 56 8. Expectations of your students' parents. 9. Expectations of your school principal. 10. Expectations of other school administrators. 11. Your personal philosophy of education. 12. Professional development you have participated in. 13. Your experience with this specific class. 14. The teacher-student ratio in this class. 15. The available equipment (such as instruments or computers). 16. The expectations of your school district music supervisor. 17. Influence from your music colleagues. 18. Type of course scheduling (such as block or traditional). 19. Influence from a professional organization. 20. The available facilities. 21. The available funding. 22. Your graduate coursework. 23. Your undergraduate coursework. In addition, a qualitative response item (Q 31 ) allowed the respondent to enter any additional factors that they considered influential to their use of classroom assessment. This open-ended comment box provided an opportunity for the researcher to corroborate and expand upon the quantitative data measured by the survey. Pilot Study Procedures Locke, Spirduso, and Silverman (2000) advised that a pilot study has the potential to provide useful information relative to the design of the main study. The pilot study was 57 conceived as recommended by Jaeger (1988): ?Ideally, a pilot survey is the main survey in miniature? (p. 323). One hundred postcards inviting members of the pilot sample to participate in the online survey were mailed from Churchton, MD on November 4, 2005, via the United States Postal Service. Two weeks later, on November 14, 2005, a similar set of postcards was mailed to the 90 people who had not yet responded. Those who had already completed the survey were identifiable by their PIN entered online. At the time of the third mailing, the response rate was only 19%. To ensure that the return rate was as high as possible, the third set of postcards was replaced by paper surveys. The third mailing?an envelope consisting of a letter, a four-page paper survey, and a self-addressed stamped envelope?was mailed on November 23, 2005, to the remaining 81 non-respondents. Surveys were accepted online and via mail until December 9, 2005, at which time data collection ended and data analysis began; the final usable response rate was 30% (N = 30). The pilot study provided valuable feedback that was used to update the survey instrument (see Appendix F). These questions resulted in additional dependent variable items. In addition, a five-point Likert-type scale replaced the original six-point scale, as recommended by Thorndike (2005) and Alreck and Settle (2004). Meltzoff (1998) wrote that ?it appears that more is lost than is gained by dropping the midpoint? (p. 115). These modifications from the pilot survey were reflected in the main study?s online survey (see Appendix B) and paper survey (see Appendix E). Main Study Procedures Two thousand postcards inviting members of the sample to participate in the online survey were mailed from Churchton, MD on January 4, 2006, via the United 58 States Postal Service. Two weeks later, on January 18, 2006, a similar set of postcards was mailed to the 1,836 people who had not yet responded. Those who had already completed the survey were identifiable by their PIN entered online. At the time of the third mailing, the response rate was only 20.1%. To ensure that the return rate met the goal of 30%, the third set of postcards was replaced by paper surveys. The third mailing?a business envelope consisting of an introductory letter, a four-page paper survey, and a self-addressed stamped envelope?was sent on February 1, 2006, to the remaining 1,574 non-respondents. Surveys were accepted online and via mail until February 26, 2006, at which time data collection ended and data analysis began; the final usable response rate was 31.7% (N = 634). Throughout February 2006, the author and two research assistants used the online survey system to input 245 paper surveys; 167 of these were usable. A variable was added to the data file that identified the mode of data collection as either paper or electronic so that any differences due to the mode of data collection could be investigated. Data Analysis The final data file was downloaded from http://www.zoomerang.com/ on February 27, 2006. Of the 791 surveys received, 14 were incomplete, 143 surveys were deemed ineligible, and 634 were complete surveys used for the present analysis. Incomplete and ineligible surveys were discarded for the purpose of this study. In addition, four paper surveys were returned by mail with no PINs (all were deemed ineligible); therefore, the total response was 795 (39.75%) (see Table 1). Fifteen invitees were unable to be contacted by mail. 59 Table 1 Percentage of Paper and Online Responses and their Usability Response eligibility Paper (%) Online (%) Total (%) Usable 21.00 58.74 79.75 Unusable Ineligible 9.81 8.18 17.99 Incomplete 0.50 1.76 9.81 Total 31.32 68.68 100.00 Note. N = 795. Data screening. The first step in the data analysis was to screen the data, as recommended by Pallant (2005). Several inconsistencies and potential problems were apparent in the data file. From the standpoint of participant eligibility, there were four paper surveys and one online survey that did not contain PINs. These five cases were examined and determined to be valid by the consistency of their data; therefore, these were included in the analyses. There were two cases where PIN numbers were entered incorrectly. By cross-referencing the state in which the respondents taught (Q 5 ) with the original mailing list, it was found that the participants had transposed digits in their PINs. These two cases were included in the analyses. The question regarding total school enrollment (Q 20 ) was qualitative and some participants explained school enrollment as K-12 or 9-12. In these instances, the grade 9-12 enrollment was used because this study focuses on high school band programs. Other respondents gave a range of students (e.g., 450-500), in which cases the mean value replaced the original data. 60 It was necessary to recode the data associated with survey questions that included zero as a response option (Q 19 , Q 24 , and Q 26 ) because the online survey system coded all response options beginning with the number one. This was corrected by subtracting one from every response. A similar situation was found (Q 22 ) where the response options included fractional values (e.g., 1?). This was corrected by recoding the data to their appropriate values (e.g., 2 became 1.5). Several new variables were created and added to the data file during screening. These included age, MENC division, and total assessment frequency per year. The last variable was created because respondents answered the methods items (Q 26 ) with how many times they used each assessment method per marking period. This new variable allowed comparison between schools with various length marking periods (Q 17 ) and number of marking periods per year (Q 16 ). Missing values. For the questions regarding types of assessment training, it was necessary to replace all missing values with ?no? because only ?yes? values were captured by the survey (Q 15 ). This was due to the question design; each type of assessment training should have appeared with a yes-no option to differentiate between ?no? answers and missing values. Missing values were also problematic in the question (Q 26 ) regarding the number of times each assessment method was used per marking period. In these cases, it could have been that they: intended to answer zero, did not know, refused to answer, or skipped the question. Because the true values of these answers were unknown, they were not altered in the data file and remained as missing values throughout the analyses. 61 Dealing with missing values is an important issue in quantitative research methods, and perhaps most commonly found in survey research where respondents may refuse to answer. Given that this survey was self-administered (i.e., completed privately) and non-controversial, it was assumed that any missing values were errors of omission. Missing values in the data?other than those discussed above?were few and seemingly missing completely at random, where ?the missingness [was] unrelated to the unknown value of the question in case, and [was] unrelated to the values of other variables? (De Leeuw, 2001, p. 150). SPSS Missing Data Analysis software uses multiple imputation to determine missing values through an iterative algorithm that creates and averages several new and complete data sets (SPSS, n.d.). However, Allison (2000) wrote that multiple imputation is a new method that has not yet demonstrated its superiority over traditional methods. He found that listwise deletion (i.e., deletion of incomplete cases for all analyses) resulted in more accurate estimates of missing values than using multiple imputation; however, ?listwise deletion typically results in the loss of 20%-50% of the data? (Acock, 2005, p. 1015). Pairwise deletion (i.e., deletion of incomplete cases only for analyses that involve the missing values) was strongly recommended by several authors (Field, 2005; Pallant, 2005) and was used in the current study to control for missing values. Preliminary analyses involved running the frequencies, descriptives, and explore procedures in SPSS for Windows Graduate Student Version 13.0.1. Pallant?s (2005) suggestion to examine histograms, normal Q-Q plots, detrended normal q-q plots, and boxplots was followed in order to assess the normality of data and check for outliers. These procedures helped to determine whether the data met the assumptions for the two 62 main statistical tests to be performed: MANOVAs and Pearson product-moment correlations. Alpha levels. The probability of Type I error increases when multiple analyses are conducted on the same data set (Field, 2005; Pallant, 2005). To reduce this risk in the current study, the experimentwise alpha level was conservatively set at .01. This level was used for all statistical tests performed except for the MANOVAs. For these, the alpha level was set at .05 because the MANOVA controls for Type I error inflation by accounting for the relationships among the dependent variables being tested. Alpha was set at .001 for the associated Box?s M tests for equality of covariance matrices, as recommended by Field (2005) and Pallant (2005). Multivariate analysis of variance. Pallant (2005) discussed seven assumptions for MANOVA testing: sample size, normality, outliers, linearity, homogeneity of regression, multicollinearity and singularity, and homogeneity of variance-covariance matrices. While not all of these assumptions were met with equal strength, the large sample size of the study should have muted the effects of violations regarding the distribution of data. The Pillai-Bartlett trace test statistic was used because it ?is the most robust to violations of assumptions? (Field, 2005, p. 594). More cases must be present in each cell than the number of dependent variables; this was not a problem due to the large sample size. Even while using pairwise deletion to account for missing values, every MANOVA performed had over 550 valid cases. Pallant (2005) wrote, ?Having a larger sample can also help you ?get away with? violations of some of the other assumptions (e.g. normality)? (p. 249). 63 Deviations from normality were found by using the Kolmogorov-Smirnov test (Field, 2005), especially among variables where zero was a common answer (i.e., the number of times a particular method was used). Although this was to be expected, Pallant (2005) suggested that the MANOVA ?is reasonably robust to modest violations of normality? (p. 249). Other variables, such as age and number of years teaching, approximated normal distributions. Outliers and data linearity were assessed by viewing histograms and measuring Mahalanobis distances with SPSS. Several outlying data points were found among the data; these were apparently random and not due to any specific participants. Therefore, these cases were preserved for the purpose of analysis, given that ?MANOVA can tolerate a few outliers, particularly if their scores are not too extreme and you have a reasonable size data file? (Pallant, 2005, p. 250). Data linearity was only found among variables that were distributed normally. Homogeneity of regression was not a factor in this analysis because it ?is important only if you are intending to perform a stepdown analysis? (Pallant, 2005, p. 254). Multicollinearity and singularity were determined by correlating each set of dependent variables (i.e., methods, purposes, and influences). ?MANOVA works best when the dependent variables are only moderately correlated?.Correlations up around .8 or .9 are reason for concern? (Pallant, p. 255). Among the dependent variables, two method pairs and one influence pair were highly correlated: students use computers- students notate (r = .83, p = .00, n = 601), written class work-written homework (r = .82, p = .00, n = 597), and principal expectations-administrator expectations (r = .87, p = .00, 64 n = 622). In these cases, the first variable in each pair was removed prior to performing each MANOVA. Finally, the homogeneity of variance-covariance matrices was tested using Box?s M Test of Equality of Covariance Matrices. Pallant (2005) suggested that this assumption is violated when p ? .001. Field (2005) wrote that Box?s test is significant where p < .05, but cautioned that ?if group sizes are different, then robustness cannot be assumed (especially if Box?s test is significant at p < .001)? (p. 599). Therefore, where the statistical significance of Box?s test was p ? .001 in the current analysis, the results were not investigated. Correlation. Pallant (2005) discussed six assumptions for correlations: level of measurement, related pairs, independence of observation, normality, linearity, and homoscedasticity. While not all of these assumptions were met with equal strength, the large sample size of the study should have muted the effects of any violations. The level of measurement was deemed appropriate for performing Pearson product-moment correlations; all measures were on an interval scale. Each subject included in the analyses had a score on both the independent and dependent variables. Independence of observations was assumed because participants completed the survey without the influence of other participants. Deviations from normality were found by using the Kolmogorov-Smirnov test (Field, 2005), especially among variables where zero was a common answer (i.e., the number of times a particular method was used). However, most of the variables used in the correlations approximated normal distributions; linearity and homoscedasticity were only found among those variables. 65 Summary This chapter defined the survey research method used in this study, detailed the development and implementation of the survey instrument, and described the preliminary analysis of survey data. MENC provided a list of 12,111 high school band directors, from which 2,000 were selected by simple random sampling. A response rate of 30% was expected due to the results from a pilot study and suggestions in the survey research literature. The independent variables included 11 personal characteristics and 11 school characteristics. The dependent variables were the use of classroom assessment methods, purposes of classroom assessment, and factors that influence the use of classroom assessment. Each survey question was answered with regard to the largest high school band class taught by the participant. Survey research is influenced by numerous design, implementation, and response factors that introduce undesirable error. This chapter discussed how the following types of error were minimized in the current study: coverage, sampling, measurement, and non- response. A pilot study (N = 30) provided valuable feedback that was used to revise the survey instrument. Two thousand postcards were mailed inviting members of the sample to participate in the online survey; a reminder postcard was sent two weeks later. The third mailing consisted of an introductory letter, a four-page paper survey, and a self-addressed stamped envelope. Surveys were accepted online and via mail until February 26, 2006, at which time data collection ended and data analysis began; the final usable response rate was 31.7%. 66 The final data file was downloaded from http://www.zoomerang.com/ on February 27, 2006; 634 surveys were deemed valid for the present study. Upon data screening, several potential problems were identified and corrected. New variables were created and added to the data file where necessary. Preliminary analyses involved running descriptive statistics procedures in SPSS. This chapter also discussed the assumptions of inferential statistics used in the current research. 67 CHAPTER 4 RESULTS The current chapter describes the sample, reports the survey data, and presents the study?s findings. Research questions 1-3 generated the dependent variables and the descriptive statistics presented in the current chapter. In research questions 4 and 5, inferential statistics were used to investigate the relationships among the dependent variables (i.e., uses of, purposes of, and influences on the use of classroom assessment) and the independent variables (i.e., personal and school characteristics). Demographic Information The sample consisted of 484 men and 150 women; the median age was 43 years (M = 42.6, SD = 10.64, N = 634). Table 2 shows the percent of responses received from each geographic division of MENC: The National Association for Music Education (MENC). Table 2 Percentage of Female and Male Respondents from each MENC Division MENC division Female (%) Male (%) Total (%) Eastern 5.52 16.56 22.08 North central 9.46 20.82 30.28 Northwest 1.58 6.62 8.20 Southern 2.68 18.30 20.98 Southwestern 3.31 8.36 11.67 Western 1.10 5.68 6.78 Total 23.66 76.34 100.00 Note. N = 634. Totals may appear incorrect due to rounding. 68 Regarding employment, 1.91% were employed part-time and 98.09% were employed full-time; 91.3% taught in public schools and 8.7% taught in non-public schools. Among all respondents, 72.7% indicated that they were the only music specialist present in their classroom. Table 3 shows the percent of responses received from teachers with varying teaching licenses and educational degrees. Table 4 shows the percent of responses from each geographic setting and socioeconomic status. Table 3 Percentage of Respondents with Various Teaching Licenses and their Highest Educational Degree Earned Teaching license Highest degree earned Regular (%) Provisional (%) Other (%) None (%) Total (%) Bachelor 40.51 1.27 0.63 1.11 43.51 Master 51.74 0.95 0.00 0.95 53.64 Doctorate 2.53 0.00 0.00 0.00 2.53 Post-doctoral work 0.32 0.00 0.00 0.00 0.32 Total 95.10 2.22 0.63 2.06 100.00 Note. n = 632. Totals may appear incorrect due to rounding. 69 Table 4 Percentage of Students in each Level of Socioeconomic Status and their School?s Geographic Setting Socioeconomic status School geographic setting Low (%) Medium (%) High (%) Total (%) Urban or inner city 4.00 5.44 0.48 9.92 Suburban 1.28 24.00 9.76 35.04 Small town 7.68 23.84 1.60 33.12 Rural or remote 9.60 11.68 0.64 21.92 Total 22.56 64.96 12.48 100.00 Note. n = 625. Regarding assessment training, respondents were free to select all types that applied to their own training in assessment. Table 5 shows the percent of total respondents that had received each type of assessment training, in descending order. Table 5 Percentage of Respondents who had Received Specific Types of Assessment Training Note. N = 634. Assessment training Respondents (%) School-wide 60.10 Professional conference or workshop 58.52 District-wide 54.10 Undergraduate coursework 43.69 Self-study 41.17 Graduate coursework 41.01 Departmental 35.49 None 6.78 70 Twelve other characteristics of teachers and their schools were measured by the current survey in order to test the corresponding null hypotheses. Table 6 shows these results regarding personal and school characteristics. In addition to the mean and standard deviations, the mode for each item is given to aid interpretation. Table 6 All Other Personal and School Characteristics Measured by the Survey Instrument Characteristic Mode M SD n Personal Years teaching any subject or grade 3 17.15 10.23 626 Years teaching high school band 8 14.92 9.98 632 Years teaching at current school 1, 5 a 10.45 8.50 629 High school band classes currently teaching 1 2.27 1.52 631 Other classes currently teaching 3 3.26 2.50 631 School School enrollment 1,000 1,037.93 744.16 630 Class enrollment 60 63.14 37.29 634 Minutes per class meeting 50 58.03 17.78 630 Class meetings per week 5 4.41 1.00 633 Weeks per marking period 9 9.25 2.49 624 Marking periods per year 4 4.40 1.34 627 Extra class meetings per year 0 16.77 21.20 626 Note. a Both answers were reported in equal amounts. Table 7 shows the percent of surveys received by response mode and respondent gender. Men who responded to the survey were 1.85 times more likely than women were to utilize the online version of the survey instead of the paper version [? 2 (1, N = 634) = 8.81, p = .00]. 71 Respondents to the paper survey attributed more importance to the following purposes of classroom assessment: to identify general class needs [F (1, 593) = 22.28, p = .00, partial ? 2 = .04)], to determine level of musical preparedness for a public performance [F (1, 593) = 15.11, p = .00, partial ? 2 = .03)], to set or maintain class standards [F (1, 593) = 9.08, p = .00, partial ? 2 = .02)], to provide feedback to parents [F (1, 593) = 19.50, p = .00, partial ? 2 = .03)], to demonstrate accountability for student learning [F (1, 593) = 8.62, p = .00, partial ? 2 = .01)], and to establish or maintain credibility for the music program [F (1, 593) = 11.40, p = .00, partial ? 2 = .02)]. Respondents to the online version of the survey reported that their undergraduate [F (1, 553) = 7.37, p = .00, partial ? 2 = .01)] and graduate coursework [F (1, 553) = 12.73, p = .00, partial ? 2 = .02)] were greater influences on their classroom assessments than those who responded to the paper version. Respondents to the paper version of the survey reported that the available funding [F (1, 553) = 6.79, p = .01, partial ? 2 = .01)] was a greater influence on their classroom assessments than those who responded to the online version. No other statistically significant mode effects were detected. Table 7 Percentage of Female and Male Respondents and their Chosen Response Mode Response Mode Female (%) Male (%) Total (%) Online 64.00 76.65 73.66 Paper 36.00 23.35 26.34 Total 100.00 100.00 100.00 Note. N = 634. Totals may appear incorrect due to rounding. 72 Descriptive Statistics: Research Questions 1-3 Tables 8 through 10 report the means, standard deviations, and number of responses for each of the dependent variable items on the survey. This discussion focuses on the results obtained from these descriptive statistics in an effort to answer the first three research questions. Use of classroom assessment methods. Research question 1 asked, ?How many times per school year do high school band directors use specific classroom assessment methods?? The corresponding survey question (Q 26 ) asked, ?In a typical marking period, how many times do you use the following student assessment methods in your largest high school band class?? Response options for the 23 methods ranged from 0 to 100+. Table 8 shows these results in descending order of mean annual usage. A qualitative response item (Q 27 ) asked, ?What other student assessment methods do you use in your largest high school band class, if any?? This open-ended response option was offered by the survey so that classroom assessment methods used by the teacher?but not appearing on the survey?could be captured. Although qualitative responses were not specifically analyzed for the purpose of the present study, they are presented in Appendix G. The current research question was studied to provide descriptive statistics regarding the current use of classroom assessment among high school band directors. It also provided a portion of the dependent variables necessary to test the null hypotheses stated in Chapter 1. Further discussion regarding these results will be presented in Chapter 5. 73 Table 8 Annual Use of Classroom Assessment Methods Among High School Band Directors Dependent variable M SD n Method Teacher plays an instrument for students to hear 59.40 100.25 604 Teacher maintains an annotated log or journal 36.92 89.45 592 Teacher uses a checklist to assess student learning 29.10 73.34 600 Teacher uses a rubric to assess student learning 24.65 58.75 599 Students play in a small ensemble 21.81 55.13 616 Students play alone for the teacher only 21.25 59.10 611 Students assess themselves 20.02 66.93 602 Students play alone in front of the class 18.77 46.82 604 Students play with others in a concert 16.83 41.78 620 Students assess other students 15.46 57.09 594 Students complete written work in class 14.87 47.82 610 Students have individual conferences with the teacher 13.31 50.08 601 Students audition for chair placement 11.35 52.38 617 Students complete a practice log or journal 11.24 37.13 604 Other music specialists/guest conductors assess students 9.53 35.14 598 Students complete written work at home 8.39 45.11 603 Students notate music (dictation, composition, etc.) 8.01 33.90 606 Students audiotape themselves playing alone 6.72 37.26 612 Students audition for ensemble admission 5.75 38.13 606 Students use computers to assess their learning 4.37 30.18 603 Students create portfolios of their work 3.44 31.48 604 Students videotape themselves playing alone 0.75 5.67 603 Students complete a published, standardized test 0.57 2.82 604 74 Purposes of classroom assessment. Research question 2 asked, ?What level of importance do high school band directors attribute to specific purposes of classroom assessment?? The corresponding survey question (Q 28 ) asked, ?How important are the following purposes of student assessment in your largest high school band class?? Response choices for the 19 items appeared below a five-point Likert-type scale ranging from 1 (not at all important) to 5 (extremely important). Table 9 shows these results in descending order of mean importance. A qualitative response item (Q 29 ) asked, ?What other purposes of student assessment would you add to those above, if any?? This open-ended response option was offered by the survey so that purposes of classroom assessment deemed important to the teacher?but not appearing on the survey?could be captured. Although the qualitative responses were not specifically analyzed for the purpose of the present study, they are presented in Appendix H. The current research question was studied to provide descriptive statistics regarding the level of importance that high school band directors attributed to specific purposes of classroom assessment. It also provided a portion of the dependent variables necessary to test the null hypotheses stated in Chapter 1. Further discussion regarding these results will be presented in Chapter 5. 75 Table 9 Level of Importance Attributed by High School Band Directors to Specified Purposes of Classroom Assessment Dependent variable M SD n Purpose To help students prepare for a public performance 4.32 0.84 624 To provide feedback to students 4.23 0.91 626 To determine whether instruction has been successful 4.19 0.95 622 To identify individual student needs 4.19 0.93 624 To determine level of musical preparedness for a performance 4.17 0.95 626 To set or maintain class standards 4.13 1.01 628 To determine what concepts students are failing to understand 4.11 0.98 623 To demonstrate accountability for student learning 4.05 0.99 624 To motivate students to practice their instruments 4.00 1.00 627 To identify general class needs 3.97 1.00 627 To establish or maintain credibility for the music program 3.91 1.15 625 To determine interim or report-card grades 3.77 1.11 626 To provide feedback to parents 3.54 1.11 625 To determine whether students are practicing at home 3.36 1.13 627 To rank students according to individual performance level 3.18 1.24 626 To determine soloists for a specific piece of music 3.11 1.25 625 To meet local, state, or national standards 3.11 1.26 627 To meet school or school district requirements 2.98 1.35 627 To provide feedback to school administrators 2.88 1.27 624 Factors that influence the use of classroom assessment. Research question 3 asked, ?What level of influence do high school band directors attribute to specific factors that affect their use of classroom assessment?? The corresponding survey question (Q 30 ) 76 asked, ?What level of influence do the following factors have on the assessment methods you use in your largest high school band class?? Response choices for the 23 items appeared below a five-point Likert-type scale ranging from 1 (not at all important) to 5 (extremely important) Table 10 shows these results in descending order of mean influence. A qualitative response item (Q 31 ) asked, ?What other factors have an influence on the assessment methods you use in your largest high school band class, if any?? This open-ended response option was offered by the survey so that factors deemed influential on classroom assessment methods?but not appearing on the survey?could be captured. Although the qualitative responses were not specifically analyzed for the purpose of the present study, they are presented in Appendix I. The current research question was studied to provide descriptive statistics regarding the level of influence that high school band directors attributed to specific factors that affect their use of classroom assessment. It also provided a portion of the dependent variables necessary to test the null hypotheses stated in Chapter 1. Further discussion regarding these results will be presented in Chapter 5. 77 Table 10 Level of Influence Attributed by High School Band Directors to Specified Factors that Influence the Use of Classroom Assessment Methods Dependent variable M SD n Influence Your personal philosophy of education 4.45 0.78 626 The amount of available class time 4.32 1.02 628 The objectives or goals of your class 4.23 0.85 629 Your experience with this specific class 3.95 1.05 627 The demands of your ensemble's performance schedule 3.92 1.05 627 The number of students enrolled in the class 3.74 1.23 627 The teacher-student ratio in this class 3.62 1.27 624 The expectations of your students 3.61 1.10 626 The available equipment (such as instruments or computers) 3.45 1.40 622 Type of course scheduling (such as block or traditional) 3.43 1.44 629 The available facilities 3.33 1.40 625 The high school band curriculum 3.28 1.23 627 Professional development you have participated in 3.26 1.21 627 The available funding 2.92 1.47 623 Expectations of your students' parents 2.82 1.16 628 Your undergraduate coursework 2.77 1.27 619 Your graduate coursework 2.72 1.38 608 Expectations of your school principal 2.71 1.23 624 Influence from your music colleagues 2.71 1.23 626 Expectations of other school administrators 2.58 1.21 626 Influence from a professional organization 2.53 1.16 629 Requirements set by the school district 2.44 1.24 627 The expectations of your school district music supervisor 2.03 1.30 606 78 Inferential Statistics: Research Questions 4-5 As survey research that used both descriptive and inferential statistics, this study compared an array of variables related to characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. Accordingly, 1,430 comparisons were made across the independent and dependent variables. Tables 11 through 25 report the statistically significant relationships among the independent and dependent variable items on the survey. This discussion focuses on the statistically significant results obtained from these inferential statistics (i.e., MANOVAs and Pearson product-moment correlations) in an effort to answer the last two research questions. Personal characteristics. Research question 4 asked, ?What are the relationships among the characteristics of high school band directors, the purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment?? Null hypotheses 1 through 11 were tested to determine the answers to this research question. Results of the MANOVA [F (22, 532) = 2.56, p = .00, partial ? 2 = .10] rejected the null hypothesis (? < .05) that there are no statistically significant relationships among the gender of high school band directors and their assessment methods, purposes, and influences. Female participants had statistically significant (? < .01) higher mean scores than men on three dependent variables (see Table 11). 79 Table 11 Statistically Significant Differences between Females and Males on Selected Dependent Variables (H 1 ) Female a Male b Dependent variable M SD M SD p Partial ? 2 Influence Requirements set by the school district 2.79 1.26 2.32 1.23 .00 .03 Expectations of other school administrators 2.89 1.19 2.48 1.20 .00 .02 The available equipment 3.78 1.30 3.39 1.40 .01 .01 Note. a n = 126. b n = 429. Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the age of high school band directors and their assessment methods, purposes, and influences (see Table 12). Negative correlations were found among nine purposes and three influences. 80 Table 12 Statistically Significant Correlations between the Respondent?s Age and Selected Dependent Variables (H 2 ) Dependent variable r p n Purpose To demonstrate accountability for student learning -.20 .00 623 To provide feedback to students -.17 .00 625 To determine what concepts students are failing to understand -.17 .00 622 To identify general class needs -.15 .00 626 To determine whether students are practicing at home -.15 .00 626 To determine whether instruction has been successful -.14 .00 621 To establish or maintain credibility for the music program -.13 .00 624 To identify individual student needs -.12 .00 623 To meet local, state, or national standards -.12 .00 626 Influence Your undergraduate coursework -.19 .00 618 The objectives or goals of your class -.11 .01 628 Influence from your music colleagues -.11 .01 625 Pallant (2005) advised that for MANOVA testing, the minimum number of subjects in each cell must be greater than the number of dependent variables. Due to the small number of respondents who taught part time (n = 12), a MANOVA was not performed to detect relationships among the employment status of high school band directors and their assessment methods, purposes, and influences. Instead, independent t-tests were performed, and no statistical significance (? < .01) was found among any of the dependent variables. 81 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the number of years that high school band directors have taught any subject or grade and their assessment methods, purposes, and influences (see Table 13). Negative correlations were found among nine purposes and two influences. Table 13 Statistically Significant Correlations between the Number of Years Teaching Any Subject or Grade and Selected Dependent Variables (H 4 ) Dependent variable r p n Purpose To provide feedback to students -.20 .00 618 To demonstrate accountability for student learning -.20 .00 616 To identify general class needs -.16 .00 619 To determine what concepts students are failing to understand -.16 .00 615 To determine whether instruction has been successful -.15 .00 614 To establish or maintain credibility for the music program -.15 .00 617 To identify individual student needs -.14 .00 616 To meet local, state, or national standards -.12 .00 619 To determine whether students are practicing at home -.11 .00 619 Influence Your undergraduate coursework -.19 .00 611 The objectives or goals of your class -.11 .00 621 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the number of years that high school band directors have taught high school band and their assessment 82 methods, purposes, and influences (see Table 14). Negative correlations were found among eight purposes and one influence. Table 14 Statistically Significant Correlations between the Number of Years Teaching High School Band and Selected Dependent Variables (H 5 ) Dependent variable r p n Purpose To demonstrate accountability for student learning -.18 .00 622 To provide feedback to students -.17 .00 624 To identify general class needs -.15 .00 625 To identify individual student needs -.13 .00 622 To determine whether instruction has been successful -.13 .00 620 To determine what concepts students are failing to understand -.13 .00 621 To establish or maintain credibility for the music program -.12 .00 623 To determine whether students are practicing at home -.11 .01 625 Influence Your undergraduate coursework -.17 .00 617 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the number of years high school band directors have taught at their current school and their assessment methods, purposes, and influences (see Table 15). Negative correlations were found among four purposes and two influences. 83 Table 15 Statistically Significant Correlations between the Number of Years Teaching at their Current School and Selected Dependent Variables (H 6 ) Dependent variable r p n Purpose To provide feedback to students -.14 .00 621 To determine what concepts students are failing to understand -.12 .00 618 To identify general class needs -.11 .01 622 To determine whether instruction has been successful -.11 .01 617 Influence Your undergraduate coursework -.14 .00 614 The objectives or goals of your class -.10 .01 624 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the number of other classes taught by high school band directors and their assessment methods, purposes, and influences (see Table 16). Negative correlations were found among one purpose and two influences. 84 Table 16 Statistically Significant Correlations between the Number of Other (Non-High School Band) Classes Currently Teaching and Selected Dependent Variables (H 7 ) Dependent variable r p n Purpose To motivate students to practice their instruments -.12 .00 625 Influence The demands of your ensemble?s performance schedule -.11 .01 624 Your personal philosophy of education -.11 .01 623 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the number of high school band classes taught by high school band directors and their assessment methods, purposes, and influences (see Table 17). Negative correlations were found among one method, three purposes, and eight influences. 85 Table 17 Statistically Significant Correlations between the Number of High School Band Classes Currently Teaching and Selected Dependent Variables (H 8 ) Dependent variable r p n Method Students assess themselves .11 .01 599 Purpose To determine soloists for a specific piece of music .12 .00 622 To motivate students to practice their instruments .12 .00 624 To determine whether students are practicing at home .11 .01 624 Influence Your graduate coursework .15 .00 605 Your personal philosophy of education .14 .00 623 The demands of your ensemble?s performance schedule .13 .00 624 The objectives or goals of your class .12 .00 626 Your experience with this specific class .12 .00 624 Type of course scheduling (such as block or traditional) .11 .00 626 Professional development you have participated in .11 .01 624 The expectations of your school district music supervisor .11 .01 603 Results of the MANOVA [F (57, 1716) = 1.41, p = .02, partial ? 2 = .05] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the highest college or university degree received by high school band directors and their assessment methods, purposes, and influences. Although a relationship was initially indicated between the highest level of education and (the purpose) to determine level of musical preparedness for a public performance [F (3, 588) = 4.47, p = .00, partial 86 ? 2 = .02], the follow-up ANOVA [F (3, 621) = 3.73, p = .01, partial ? 2 = .02] failed to reach statistical significance (? < .01). Results of the MANOVA [F (63, 1506) = .64, p = .99, partial ? 2 = .03] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the state music teaching license or certificate status of high school band directors and their assessment methods, purposes, and influences. Eight separate MANOVAs tested the null hypothesis that there are no statistically significant relationships among the types of assessment training received by high school band directors and their assessment methods, purposes, and influences. The survey question?s (Q 15 ) eight dichotomous (i.e., yes-no) items necessitated individual analyses on each. Results of the MANOVA indicated statistical significance (? < .05) among four of the eight items: no assessment training [F (22, 532) = 2.00, p = .01, partial ? 2 = .08], district-wide training [F (22, 532) = 2.01, p = .00, partial ? 2 = .08], graduate coursework [F (22, 532) = 5.00, p = .00, partial ? 2 = .17], and professional conference or workshop [F (22, 532) = 5.00, p = .00, partial ? 2 = .10] (see Tables 18-21). 87 Table 18 Statistically Significant Differences between Teachers with Assessment Training and Teachers without Assessment Training on Selected Dependent Variables (H 11 ) Assessment training a None b Dependent variable M SD M SD p Partial ? 2 Influence Professional development you have participated in 3.33 1.19 2.49 1.28 .00 .03 The high school band curriculum 3.33 1.22 2.54 1.19 .00 .03 Your personal philosophy of education 4.49 0.74 3.95 1.08 .00 .03 Your graduate coursework 2.76 1.39 1.97 1.16 .00 .02 Your undergraduate coursework 2.80 1.27 2.08 1.18 .00 .02 The available funding 2.96 1.46 2.31 1.54 .01 .01 Requirements set by the school district 2.47 1.25 1.92 1.18 .00 .01 Influence from a professional organization 2.56 1.18 2.05 1.05 .01 .01 Your experience with this specific class c 3.99 1.01 3.54 1.41 .01 .01 Note. a n = 516. b n = 39. c Levene?s test [F (1, 553) = 14.00, p = .00]. 88 Table 19 Statistically Significant Differences between Teachers with District-wide Training in Assessment and Teachers without District-wide Training in Assessment on Selected Dependent Variables (H 11 ) District training a None b Dependent variable M SD M SD Partial ? 2 Influence Requirements set by the school district 2.65 1.27 2.16 1.18 .04 The high school band curriculum 3.46 1.19 3.04 1.25 .03 Expectations of your students? parents 2.97 1.16 2.68 1.14 .02 The expectations of your school district music supervisor c 2.17 1.37 1.83 1.19 .02 Influence from your music colleagues 2.83 1.27 2.53 1.19 .02 The available funding 3.12 1.44 2.68 1.49 .02 Note. p = .00. a n = 302. b n = 253. c Levene?s test [F (1, 553) = 16.05, p = .00]. 89 Table 20 Statistically Significant Differences between Teachers with Graduate Coursework Training in Assessment and Teachers without Graduate Coursework Training in Assessment on Selected Dependent Variables (H 11 ) Graduate training a None b Dependent variable M SD M SD Partial ? 2 Influence Your graduate coursework 3.22 1.33 2.33 1.31 .10 The number of students enrolled in the class 3.93 1.17 3.62 1.24 .02 The objectives or goals of your class 4.36 0.76 4.13 0.90 .02 Your personal philosophy of education c 4.58 0.67 4.36 0.85 .02 Professional development you have participated in 3.48 1.15 3.12 1.23 .02 Requirements set by the school district 2.59 1.28 2.31 1.22 .01 The high school band curriculum 3.44 1.23 3.14 1.23 .01 The teacher-student ratio in this class 3.80 1.22 3.52 1.28 .01 Influence from a professional organization 2.68 1.16 2.41 1.18 .01 Note. a n = 234. b n = 321. c Levene?s test [F (1, 553) = 14.19, p = .00]. 90 Table 21 Statistically Significant Differences between Teachers with Workshop Training in Assessment and Teachers without Workshop Training in Assessment on Selected Dependent Variables (H 11 ) Workshop training a None b Dependent variable M SD M SD Partial ? 2 Influence Professional development you have participated in 3.51 1.13 2.95 1.24 .05 The objectives or goals of your class 4.33 0.79 4.10 0.93 .02 Note. p = .00. a n = 322. b n = 233. To summarize the current research question, statistically significant relationships were found among certain characteristics of high school band directors and their classroom assessment methods, purposes, and influences. These included gender, age, years teaching any subject or grade, years teaching high school band, years teaching at current school, the number of classes currently teaching, the number of high school bands currently teaching, and certain types of assessment training received. Only one statistically significant finding was related to the use of classroom assessment methods (i.e., student self-assessment was related to the number of high school band classes currently taught). The remainder of the findings related to purposes of classroom assessment and factors that influence the use of classroom assessment. Further discussion regarding these results will be presented in Chapter 5. School characteristics. Research question 5 asked, ?What are the relationships among the characteristics of high school band directors? school settings, the purposes and 91 uses of classroom assessment methods, and factors that influence the use of classroom assessment?? Null hypotheses 12 through 22 were tested to determine the answers to this research question. Results of the MANOVA [F (1008, 9975) = 1.01, p = .42, partial ? 2 = .09] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the state in which a school is located and high school band directors? assessment methods, purposes, and influences. Results of the MANOVA [F (105, 2510) = 1.21, p = .08, partial ? 2 = .05] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the MENC division in which a school is located and high school band directors? assessment methods, purposes, and influences. Results of the MANOVA [F (63, 1500) = 1.30, p = .06, partial ? 2 = .05] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the geographic setting of a school and high school band directors? assessment methods, purposes, and influences. Results of the MANOVA [F (19, 571) = 2.13, p = .00, partial ? 2 = .07] rejected the null hypothesis (? < .05) that there are no statistically significant relationships among the school type and high school band directors? assessment methods, purposes, and influences. Participants who taught in public schools had statistically significant (? < .01) higher mean scores than those who taught in non-public schools on four dependent variables (see Table 22). 92 Table 22 Statistically Significant Differences between Public and Non-Public Schools on Selected Dependent Variables (H 15 ) Public a Non-public b Dependent variable M SD M SD Purpose To set or maintain class standards 4.18 0.97 3.78 1.14 To rank students according to individual performance level 3.23 1.22 2.75 1.43 To meet local, state, or national standards 3.15 1.24 2.65 1.32 To meet school or school district requirements 3.01 1.33 2.49 1.36 Note. p = .01. Partial ? 2 = .01. a n = 540. b n = 51. Results of the MANOVA [F (42, 998) = 1.15, p = .24, partial ? 2 = .05] retained the null hypothesis (? < .05) that there are no statistically significant relationships among the socioeconomic status of the majority of students and high school band directors? assessment methods, purposes, and influences. Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among total school enrollment and high school band directors? assessment methods, purposes, and influences (see Table 23). Negative correlations were found among one method, four purposes, and three influences. 93 Table 23 Statistically Significant Correlations between Total School Enrollment and Selected Dependent Variables (H 17 ) Dependent variable r p n Method Other music specialists or guest conductors assess students .12 .00 595 Purpose To set or maintain class standards .15 .00 624 To rank students according to individual performance level .13 .00 622 To demonstrate accountability for student learning .12 .00 620 To determine whether students are practicing at home .10 .01 623 Influence The expectations of your school district music supervisor .12 .00 602 Influence from your music colleagues .12 .00 622 Your personal philosophy of education .10 .01 622 Results of the MANOVA [F (22, 529) = 3.18, p = .00, partial ? 2 = .12] rejected the null hypothesis (? < .05) that there are no statistically significant relationships among the number of music specialists present in the classroom and high school band directors? assessment methods, purposes, and influences. Teachers who reported that they were the only music specialist present in their classroom (M = 2.56, SD = 1.23, n = 400) had a statistically significant lower mean score on the influence from your music colleagues item than those who reported that they were not the only music specialist present in their classroom (M = 3.06, SD = 1.19, n = 152). Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among total class 94 enrollment and high school band directors? assessment methods, purposes, and influences (see Table 24). Table 24 Statistically Significant Correlations between Total Class Enrollment and Selected Dependent Variables (H 19 ) Dependent variable r p n Method Students complete a practice log or journal -.12 .00 604 Students play with others in a concert .11 .01 620 Purpose To rank students according to individual performance level .18 .00 626 To determine soloists for a specific piece of music .13 .00 625 Influence The demands of your ensemble?s performance schedule .15 .00 627 The high school band curriculum .12 .00 627 The teacher-student ratio in this class .12 .00 624 Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the average number of days per week that the class meets and high school band directors? assessment methods, purposes, and influences. The influence of type of course scheduling (such as block or traditional) was statistically significant (r = -.12, p = .00, n = 628). Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there were no statistically significant relationships among the average number of minutes per class meeting and high school band directors? assessment methods, purposes, and influences. A relationship with the influence type of course 95 scheduling (such as block or traditional) was statistically significant (r = .16, p = .00, n = 626). Results of the Pearson product-moment correlation rejected the null hypothesis (? < .01) that there are no statistically significant relationships among the average number of class meetings per year held outside regular school hours and high school band directors? assessment methods, purposes, and influences (see Table 25). Table 25 Statistically Significant Correlations between the Number of Extra Rehearsals Held Outside School Hours and Selected Dependent Variables (H 22 ) Dependent variable r p Purpose To rank students according to individual performance level .14 .00 Influence The demands of your ensemble?s performance schedule .12 .00 Note. n = 619. To summarize the current research question, statistically significant relationships were found among high school band directors? school settings and their classroom assessment methods, purposes, and influences. These included the school type, school enrollment, other music teachers present in the classroom, class enrollment, the number of classes per week, the number of minutes per class, and the number of extra rehearsals held after school each year. Only three statistically significant findings were related to classroom assessment methods; the remainder of the findings related to classroom assessment purposes and influences on classroom assessment. Further discussion regarding these results will be presented in Chapter 5. 96 Summary Statistically significant relationships were found among the following independent variables: gender, age, years teaching, years teaching high school band, years teaching at current school, number of classes currently teaching, number of high school bands currently teaching, assessment training, school type, school enrollment, presence of other music teachers, class enrollment, classes per week, minutes per class, and extra rehearsals. Only four statistically significant findings were related to classroom assessment methods; the others were related to purposes of classroom assessment and factors that influence the use of classroom assessment. Further discussion regarding these results will be presented in Chapter 5. 97 CHAPTER 5 DISCUSSION AND CONCLUSIONS This chapter discusses the results of the current study in light of past research, presents three prevalent issues, draws implications, and makes recommendations for future research and practice. The purpose of this study was to investigate the relationships among characteristics of high school band directors and their school settings, purposes and uses of classroom assessment methods, and factors that influence the use of classroom assessment. Discussion This discussion is organized around the study?s three dependent variables: the use of classroom assessment methods, purposes of classroom assessment, and factors that influence the use of classroom assessment. Use of classroom assessment methods. The classroom assessment methods used most by high school band directors involved actions performed by the teacher: teacher plays an instrument, teacher maintains an annotated log, teacher uses a checklist, and teacher uses a rubric. Appendix G includes many qualitative survey responses that report teacher observation and student participation as major sources of assessment information. The item ?teacher plays an instrument for students to hear? may have been misunderstood by respondents because the survey did not specify how such activity would be presented as an assessment. However, it remains a widely used method with 84.7% of respondents reporting its use. These findings are consistent with the traditionally accepted belief that teachers are solely responsible for implementing 98 classroom assessment and students are relatively uninvolved in the process, other than their general participation in class activities. Graue (1993) advised, ?Learning and assessment activities should be more collaborative between teachers and students, allowing the possibility of working together to solve the learning puzzle? (p. 296). Brookhart (1997) wrote, ?A student who shares in the assessment process should perceive more control of and more responsibility for his or her learning, which should increase effort and achievement? (p. 178). Based on the current research, high school band directors do not appear to be adopting such social- constructivist perspectives of teaching, learning, and assessment. The classroom assessment methods used least included portfolios and methods involving technology such as audiotape, videotape, and computers. Goolsby (1999) and Burrack (2002) suggested that methods such as these can aid teachers with classroom assessment, but it seems that high school band directors use these methods infrequently. However, the use of computers (18.08%) has nearly tripled in the past six years with respect to Simanton?s (2000) findings (6.6%). This increase may be a result of lower computer prices, greater financial support for technology in the schools, and the growing availability of assessment software for music education. In the current study, less than 2% of the respondents reported using no classroom assessments. Simanton (2000) found that 18% of high school band directors did not assess individual student achievement. Given that these studies both involved national samples, the findings imply that more high school band directors are using classroom assessments now than in the past. Appendix G includes a few qualitative survey responses from teachers who do not assess their students at all. One teacher remarked, 99 ?None. I should probably use a few more [methods]. You have a lot of good ones on this survey? (Appendix G, # 82). The increased use of assessment over the past six years may be the result of heightened awareness and training among educators. Standards documents released in the 1990s provided teachers with instructional goals; classroom assessment is the logical tool to measure progress toward achievement of those goals. Teachers who taught more band classes reported doing more student self- assessment in their classes. Giving students ownership of assessment ?puts the onus for learning on the student? (Farrell, 1997, p. 21) and saves the teacher valuable time (Goolsby, 1999). The National Board for Professional Teaching Standards (2001) recommended that teachers ?use a variety of meaningful student self-assessment techniques? (p. 16). Wolf (1987) discussed projects, such as Arts PROPEL and Harvard Project Zero, in which student self-assessment played a vital role in the overall assessment process. Teachers in schools with large enrollments reported a more frequent use of guest conductors to assess student learning in their classes. Larger schools may have budgets that allow musical guests to be invited into the classroom more frequently. One example is inviting a guest conductor to provide formative feedback to the ensemble via a ?mock adjudication? prior to band festival. In addition, larger schools may be located in areas (e.g., cities) where guest clinicians are more readily available to assist with classroom assessment. Band directors with larger class enrollments held concerts more frequently and used practice logs less frequently. Larger bands may perform more often due to their high level of public visibility and status in the school and community. Reviewing practice logs 100 may be overly burdensome for teachers with large classes due to the sheer volume of written work. Appendix G includes many responses from teachers who used sectionals, small groups, recordings, and computer software to assess their students. These responses may reflect teachers? efforts to increase their classroom assessment efficiency. Given these findings, high school band directors seem to focus more on summative assessments (e.g., concerts) than formative assessments (e.g., practice logs). The use of formative assessment has been recommended because it helps guide students toward the intended goals of instruction (Pellegrino, Chudowsky, & Glaser, 2001). In summary, the classroom assessments used by high school band directors tend to focus on the evaluation of student performance skills. Students are not generally involved in the planning or execution of assessment. Teachers are currently doing more assessment and using computers more often than in the past. Those who teach more band classes use student self-assessment more often, and teachers in larger schools have guest musicians in their classrooms more frequently. Lastly, high school band directors with larger classes hold concerts more often and use practice logs less frequently. Purposes of classroom assessment. The purposes of classroom assessment considered most important by high school band directors were to help students prepare for a public performance and to provide feedback to students. Appendix H includes many qualitative survey responses that reflect the importance of student motivation, musical preparation, and goal setting. One teacher responded, ?I want assessment to always be a learning tool for my students, something they find helpful, not to be feared? (Appendix H, # 19). The overall emphasis on performance is warranted given that one of the main responsibilities of most high school band directors is to prepare their students for concerts 101 and other performances. Similarly, Hill (1999) and Hanzlik (2001) found that high school band directors tend to focus their assessment efforts on the measurement of students? musical performance skills. The purposes of classroom assessment considered least important by high school band directors centered on external factors such as school administrator feedback, school or district requirements, and local, state, or national standards. Teachers reported that using assessment to meet external criteria was unimportant. The level of influence such external factors had on classroom assessment was also reported as minimal. Results among four independent variables?age, years teaching any subject or level, years teaching high school band, and years teaching at current school?were similar due to the strong intercorrelations between these variables (see Appendix J). Many purposes of assessment were found to be less important among older and more experienced teachers, including the following: to identify general class needs, to provide feedback to students, to determine whether instruction has been successful, and to determine what concepts students are failing to understand. This indicates that classroom assessment, in general, may have less importance or a different purpose to those teachers. In addition, these findings indicate that teachers with less experience use classroom assessment to identify general class needs, to provide feedback to students, to determine whether their instruction had been successful, and to determine what concepts students were failing to understand. New teachers may feel compelled to demonstrate their accountability for student learning through assessment because they are trying to ?prove? their value as a teacher. Novices also may be more aware of these purposes for assessment due to their recent undergraduate education or other involvement in 102 professional development, conferences, or workshops. Goolsby (1997) found that novice teachers differed in the types of feedback they gave to students; the present study demonstrated that novice teachers also differ in their purposes for classroom assessment. Conway (2006) recommended that new teachers should get help with assessment (e.g., from their mentors) because they typically have had limited experiences with assessment in their undergraduate methods classes. Teachers who taught a greater number of high school bands reported using classroom assessment to motivate and monitor students? practice habits. They used assessment to motivate students to practice, to determine if students were practicing at home, and to determine soloists for specific pieces of music. Assessments, and particularly grades, have long been used as motivational devices in the classroom. Bray (2002) wrote that one of the three main purposes of assessment is to ?increase motivation and individual responsibility for learning? (p. 82). Public school teachers placed a greater importance on purposes related to standards and requirements set outside the classroom than non-public school teachers. These results suggest that the implementation of classroom assessment differs between public and non-public high school band directors. Public school teachers may be pressured to meet standardized requirements such as those evolving from school curricula, statewide tests, or national education programs. Non-public school teachers may have more academic freedom to teach and assess students using their own preferred methods; however, these results are tentative given that non-public school teachers represented only 8.68% of the respondents. 103 Ranking students was reportedly a very important purpose of classroom assessment among public school teachers in large schools, those with large classrooms, and those with more after-school rehearsals. One purpose of assessment is to sort individuals on one or more dimensions (Kohn, 1994). For teachers of large bands, the number of differentiations is high, and the differences can be negligible. Band students can be ranked for purposes such as chair placement, performance of solos, or grading. It appears that band directors of large high school programs use assessment primarily to rank their students according to individual performance level. In summary, the purposes of classroom assessment considered most important by high school band directors center on evaluating students? level of achievement and their readiness to perform. External factors such as school administrators, district requirements, and national standards are reportedly least important. Novice teachers may feel compelled to demonstrate their accountability for student learning through assessment, whereas more experienced teachers do not. Teachers who teach a greater number of high school bands use classroom assessment to motivate and monitor students? practice habits. Public school teachers place a greater importance on purposes related to standards and requirements set outside the classroom than non-public teachers. Finally, band directors of large high school programs use assessment primarily to rank their students according to individual performance level. Factors that influence the use of classroom assessment. The factors considered most influential on the use of classroom assessment centered on personal philosophy and class time. These findings suggest that high school band directors consider their personal philosophy of education to be a strong influence on their use of assessment. Appendix I 104 includes several qualitative survey responses that are related to personal philosophy. One teacher was reportedly influenced by ?What I believe to be important for the betterment of my students? (Appendix I, # 66). These findings parallel Tracy?s (2002) statement that ?a teacher?s belief in the importance of assessment exercises the greatest influence over practice? (p. 147). The current study also found that time was a major barrier to classroom assessment and this was reiterated by many qualitative survey responses included in Appendix I. One teacher wrote, State and federal requirements are forcing us to do a lot of busy work that is largely unnecessary. This tends to take time away from more important tasks such as actually teaching! All this assessment and accountability stuff is just a smoke screen for administrators at all levels to cover their butts. Just make sure good teachers are hired and let them teach - period. (Appendix I, # 39) External factors such as school administrators, district requirements, professional organizations, and colleagues were considered least influential on the classroom assessments used by high school band directors. These findings are consistent with previous findings on the least important purposes of classroom assessment. Kotora (2001) found that standards set by the local school district, the state, or MENC had almost no influence on the assessments used by teachers. Cizek, Fitzgerald, and Rachor (1995) wrote that K-12 teachers were ?generally unaware of their colleagues? [assessment] practices? (p. 175). The present results indicate that high school band directors are 105 influenced more by internal goals and objectives related to musical performance than by external requirements set by others. The personal philosophy of education held by teachers was reportedly a stronger influence among teachers who taught more band classes, those with some type of assessment training, and those who taught in larger schools. Teachers who had assessment training may have incorporated that knowledge into their own philosophy of education and used their philosophy to help make decisions regarding classroom assessment. Younger and less experienced teachers attributed more influence to class goals and objectives, their music colleagues, and undergraduate coursework than teachers with more experience. As with the findings regarding purposes of assessment, this suggests that novice teachers may feel compelled to demonstrate their accountability for student learning through assessment. Therefore, they look to the class goals and objectives, their colleagues, and their undergraduate education for assistance. High school band directors with more experience are less influenced by these factors. Music colleagues were reportedly more influential among younger teachers, those with district-wide assessment training, those in larger schools, and those with another music specialist present in their classroom. This suggests that team teaching could influence classroom assessment; perhaps one teacher was responsible for assessment while the other one taught. One person wrote, ?I use smartmusic and my assistants listen to each student during class time? (Appendix G, # 56). Teachers with district-wide training may have also interacted with their colleagues on topics related to assessment. Professional development that allows teachers 106 to share their ideas can be beneficial to their teaching and assessment practices. Cizek, Fitzgerald, and Rachor (1995) wrote, ?A lack of exposure to fundamentals of assessment also helps to explain the problem. Despite its seemingly obvious relevance to teachers? practice, it seems that teachers need help acquiring knowledge about sound assessment? (p. 162). Professional development or in-service days can provide the opportunity for teachers to learn evaluation techniques and discuss issues related to classroom assessment. ?In-service training sessions or other professional development activities are likely to be the only sources of information about educational measurement for secondary teachers of academic subjects other than advice from colleagues and their own experience as students? (Frary, Cross, & Weber, 1993, p. 29). The demands of the ensemble?s performance schedule was a greater influence among teachers who taught more high school band classes, those with larger classes, and those with more after school rehearsals. These results suggest that teachers who have larger bands and more band classes have performance schedules that are more demanding. McMillan (2004) discussed the tensions that exist between internal and external factors, and the contribution these make to decisions regarding classroom assessment. Performance demands are an external influence that may conflict with an internal influence, such as a personal philosophy of education. To illustrate, a teacher may believe that classroom assessment is important and essential but may limit their use of particular methods due to the demands of their performance schedule. In summary, the findings suggest that high school band directors consider their personal philosophy of education to be a strong influence on their classroom assessments, and time is a major barrier to classroom assessment. Teachers are more influenced by 107 internal goals focused on performance than by external goals set by others. Those who have had assessment training use that knowledge to make decisions regarding classroom assessment. Novice teachers look to class goals and objectives, their colleagues, and their undergraduate education for assistance with classroom assessment. Music colleagues are more influential among young teachers and those with district-wide assessment training. Lastly, the demands of the performance schedule is a greater influence among teachers who teach more high school band classes, those with larger classes, and those with more after school rehearsals. Prevalent Issues, Implications, and Recommendations Dependent variables that were statistically significant across several independent variables were identified and clustered by their similarities. For example, the most commonly found statistically significant variables were: ?to determine whether students are practicing at home? and the influence of ?the objectives or goals of your class?. The prevalence of statistical relationships such as these can be found by examining the tables in Chapter 4. From these commonalities, the following three issues were identified: teacher autonomy, the role of assessment training, and teacher workload. Teacher autonomy. Findings from the current study reflect the autonomy of high school band directors in matters of classroom assessment. Teachers reported that personal philosophy was the strongest influence on classroom assessment, class goals were more important than external requirements or standards, and students were relatively uninvolved in designing and implementing classroom assessment. While these characteristics were found among teachers in the current study, they are not necessarily unique to high school band directors. 108 Teacher autonomy?and isolation?are issues previously discussed in the classroom assessment literature. Cizek, Fitzgerald, and Rachor (1995) found that teachers ?who acknowledged that they were unsure about what their colleagues did vis-?-vis assessment and grading also indicated that they preferred it that way? (p. 175). High school band directors are often isolated by the content and context of their teaching position. Physically, band classrooms are usually located in distant areas of schools to keep sounds from transmitting into other classrooms. Band directors are often exempted from events that occur in other subjects, such as standardized testing programs, perhaps due to the perception that student assessment is unimportant in music classes. Isolation has also been studied as a potential problem among teachers that can lead to burnout. For example, Barlow (2005) recalled his early teaching experiences: The first year that I taught school, I walked into the school in September, picked up an English literature anthology, and went into the isolation chamber in room 207. I would have loved to have had the opportunity to meet regularly with other teachers teaching the same subject. (p. 64) This narrative may be indicative of feelings of autonomy or isolation held by high school band directors, which may be partially responsible for the low levels of influence attributed to factors external to the classroom setting. Isolation may be alleviated with the use of technology, if not by personal contact with other band directors. Schlichte, Yssel, and Merbler (2005) discussed novice teachers and their need for adequate mentoring to prevent attrition. ?Collegiality is recognized in the professional literature as one of the important variables in the successful first-year experience? (p. 36). Web sites and online discussion groups are also available for band directors to connect with each other (e.g., 109 http://www.menc.org/networks/band/openforum/wwwboard.htm). Future studies should research the impact of autonomy on classroom assessment among high school band directors, investigate the differences in the use of assessment between novice and expert teachers, and explore directors? use of technology to connect with colleagues. Assessment training. The role of assessment training is another key issue that emerged from the present study. Most teachers had received some sort of assessment training; however, only district-wide training, graduate coursework, and professional conferences influenced teachers? use of classroom assessment. This suggests that undergraduate coursework, self-study, departmental, and school-wide assessment training had little or no impact on teachers? assessments. Stiggins (1991a) declared, ?We have come face-to-face with the results of many decades of declining assessment literacy throughout the fabric of American education? (p. 269). The present study showed that many high school band directors used a limited range of available classroom assessment tools and focused mainly on performance skills. This may have been appropriate depending on the specific goals of their band program or curriculum. However, musical performance is only one possible goal of instrumental music education; others are outlined by standards documents such as the National Standards for Arts Education (Consortium of National Arts Education Associations, 1994), which ?help[s] ensure that the study of the arts is disciplined and well focused, and that that arts instruction has a point of reference for assessing its results? (pp. 9-10). Expanding the instructional palette of teachers would require them to learn additional? perhaps non-performance based?classroom assessment techniques. 110 Changes in the types of available classroom assessment training may be necessary, but will be slow to implement. Black and Wiliam (1998) studied nearly 600 articles related to classroom assessment, and found ?widespread evidence that fundamental change in education can be achieved only slowly?through programs of professional development that build on existing good practice? (p. 140). Meeting the goals of Vision 2020 will require improvements in the assessment training of current and future music educators (Lehman, 2000). Future research in classroom assessment should address the role of assessment training for pre-service and in-service high school band directors. In addition to documents that address assessment competencies for teachers (Linn & Miller, 2005; National Board for Professional Teaching Standards, 2001), several authors have offered specific recommendations for training teachers in measurement and assessment (Airasian, 1991; Brookhart, 1999; Schafer, 1991; Stiggins, 1991b). Undergraduate music education programs could help prepare teachers for high-quality classroom assessment by reviewing the Standards for Teacher Competence in Educational Assessment of Students (American Federation of Teachers, National Council on Measurement in Education, & National Education Association, 1990) and making appropriate changes to their curricula. Future investigations should evaluate the content of assessment training that high school band directors currently receive, use empirical research to determine what works, and follow up with appropriate recommendations for music teacher education. Teacher workload. The third issue relevant to the current study is teacher workload. Teachers in the current study cited the amount of class time and the demands 111 of their performance schedules as strong influences on their classroom assessment practices. In addition, Appendices H and I include over 20 qualitative survey responses that refer to the issues of time or workload. ?With only one teacher in a program that tries to meet the needs of approximately 30% of the students, I am doing as much student assessment as time will allow? (Appendix H, # 55). ?Work load. I teach elementary general music and jr hi [sic] school band and choir also. Between that and extra curricular things like drama and sports, assessment becomes minimal? (Appendix I, # 67). Teachers are typically barraged by responsibilities both inside and outside the classroom. For example, high school band directors commonly have after-school duties such as teaching marching band, jazz band, and private lessons. Apple (1988) called this high level of professional responsibility work intensification and was critical of its potential impact on teachers. Certain types of assessment, such as taped performances of students, require many hours of work outside of school time in order to evaluate fairly and accurately; however, technology can help increase the efficiency of classroom assessment. According to the present results, there was some indication that the use of computers has increased in recent years; the use of software for music assessment may be one way to lighten the workload for teachers. Software applications can provide musical accompaniments, training in music theory and history, and can model appropriate tone quality and intonation. Teachers who had another music specialist present in their classroom reported that this arrangement affected their use of assessment. Having multiple teachers is another way to mitigate the effects of work intensification. Work intensification appears to be a ubiquitous situation among high school band programs, with teachers responsible for more students, performances, and external 112 mandates than ever before. Andrews and Quinn (2004) wrote, ?Secondary teachers still are given unreasonable teaching assignments? (p. 78) and discussed the particular workload challenges of novice teachers, such as floating between schools, teaching out of field, and having too many different classes to teach (i.e., preparations). Music educators are often faced with many students due to the size of performing ensembles, and this adds considerably to their workload. Chiodo (2001), who taught about 700 K-4 music students each marking period, described ?assessment tools and strategies that would enable [her] to know the achievements of each of [her] students individually? (p. 17). Robinson (1995) presented a narrative regarding a high school band director who was clearly overworked: ?Phil?s schedule revolved around concerts, rehearsals, and extracurricular events, such as football games, parades, musicals, community festivals, and competitions? (p. 29). Through this fictional but realistic story, Robinson offered specific advice on how Phil could improve his classroom assessments. Scott (1998) presented a tiered evaluation system that allowed students to progress at their own individual rates. He recommended this method for large ensembles because it ?allows the teacher to evaluate students of varying levels of talent and achievement according to performance criteria representing varying levels of difficulty? (p. 17). High school band directors may need to reevaluate their performance schedules to ensure that adequate time for classroom assessment can be included to evaluate student learning. The present study asked participants to respond to each applicable question with regard to their largest size band class, which ranged from 3 to 240 students. One teacher remarked, ?I am afraid that all this information is skewed due to the fact that the largest student group is marching band and we do things very differently there than in the 113 cvoncert [sic] bands, especially freshman band? (Appendix G, # 46), which indicated that the size of the ensemble may have a strong influence on assessment practice. Future research in classroom assessment should address the issue of teacher workload and examine the impact of large class sizes on teaching, learning, and assessment in high school band programs. Non-statistically Significant Findings The vast majority of relationships examined in this study were found to be non- statistically significant. In particular, of the 506 relationships among the independent variables and assessment methods used, only four were statistically significant. This may have been due to measurement error on the part of the survey questions (e.g., respondent misunderstandings), an alpha level that was set too conservatively, or it may represent the true picture of classroom assessment. McMillan, Myran, and Workman (2002) found that ?an important characteristic of classroom assessment and grading practices is that they are highly individualized and may be unique from one teacher to another, even in the same school? (p. 212). Their conclusion that elementary school ?teachers use a ?hodgepodge? of factors when assessing and grading students? was consistent with previous studies (Brookhart, 1994; Cizek, Fitzgerald, & Rachor, 1995). The present research found the hodgepodge effect to be present in high school band programs, suggesting that these teachers may be satisfied or successful with their chosen array of classroom assessments. Likewise, Simanton (2000) found that 76% of high school band directors were ?satisfied with current assessment and grading practice? (p. 67) and unmotivated to change; such reported 114 satisfaction does not necessarily suggest a need for change as these teachers may be quite successful with their current assessments. There were no statistically significant relationships among the dependent variables and the following independent variables: highest degree received, type of teaching license, state, MENC region, geographic setting, or socioeconomic status. Again, this may have been due to measurement error on the part of the survey questions, an alpha level that was set too conservatively, or it may represent the true picture of classroom assessment. The lack of statistically significant findings may indicate that classroom assessment is extremely variable in these specific areas. The non-statistically significant findings should be explored by future researchers. While it is reasonable to expect that teachers use assessment methods that they feel will benefit student learning, the changes in practice that result from in-service assessment training should be examined. Do teachers change their methods or implementation of assessments due to in-service training or other professional development, or do they continue to assess as before? Is there any variation among teachers from different states due to statewide assessment programs or other factors? A targeted effort to investigate assessment practices in specific states or regions may help illuminate these questions. The first chapter discussed several limitations of the present study; future researchers should explore classroom assessment beyond these limitations. Band directors who are not MENC members should be studied, for example. The selection of survey mode (or modes) should be made carefully and examined with attention to the benefits and detriments of each design choice. The number of participants should be determined by pilot testing and with regard to potential financial limitations; large 115 surveys can quickly become expensive. Surveys should be proofread and tested among a diverse population, and suggestions for improvement should be solicited and discussed, where appropriate. The cognitive task load of the survey should be such that participants are motivated to continue through to the end. The current study featured a progress indicator (e.g., Page 1 of 5) that may have helped in this regard. An excellent resource for understanding survey design is Alreck and Settle (2004). Conclusions Chapter 1 opened with the following quote: ?Achieving the aims of music education depends on assessment. The primary function of assessment in music education is not to determine grades but to provide accurate feedback to students about the quality of their growing musicianship? (Elliott, 1995, p. 264). If these are the expectations for classroom assessment in music education, then the profession must implement high- quality assessment training for both pre-service and in-service teachers. Pre-service teachers should receive instruction on a variety of assessment methods, such as those presented in recent and forthcoming publications of MENC: The National Association for Music Education; these include Performance Standards for Music: Strategies and Benchmarks for Assessing Progress Toward the National Standards, Grades PreK-12 and Spotlight on Assessment in Music Education. Several texts cited in this study (Linn & Miller, 2005; McMillan, 2004; Stiggins, 2005; Wiggins & McTighe, 2005) would be appropriate for graduate study and teachers of in-service instruction. In addition to providing high-quality training in classroom assessment, the related issues of teacher autonomy and workload should be addressed by music education researchers and practitioners. High school band programs are typically unique: ensemble 116 classes tend to be large, there are numerous musical performances to prepare for, and there may be several extracurricular ensembles in the program. Given these constraints and the resulting lack of time for assessment, specific instructional strategies that maximize the efficiency of classroom assessment should be developed. These may involve the use of audio, video, or computer technology, or additional human resources. Any efforts to decrease the pressures faced by teachers in the classroom will likely have positive and lasting effects on teaching, learning, and assessment in high school band programs. 117 APPENDIX A STATISTICAL DESIGN OF THE STUDY Independent variable Survey question a Null Hypothesis b Statistical test Personal characteristic Gender Q 3 H 1 MANOVA Age Q 4 H 2 Correlation Full time or part time employment Q 6 H 3 MANOVA Years teaching any subject or grade Q 10 H 4 Correlation Years teaching high school band Q 9 H 5 Correlation Years teaching at current school Q 11 H 6 Correlation Number of other classes currently teaching Q 19 H 7 Correlation Number of band classes currently teaching Q 18 H 8 Correlation Highest degree received Q 7 H 9 MANOVA Music license or certificate status Q 8 H 10 MANOVA Assessment training Q 15 H 11 MANOVA School characteristic State Q 5 H 12 MANOVA MENC division - c H 13 MANOVA Geographic setting of school Q 13 H 14 MANOVA Public or non-public school type Q 12 H 15 MANOVA Socioeconomic status of students Q 14 H 16 MANOVA School enrollment Q 20 H 17 Correlation Only music specialist present in class Q 25 H 18 MANOVA Class enrollment Q 21 H 19 Correlation Class meetings per week Q 22 H 20 Correlation Minutes per class Q 23 H 21 Correlation Extra rehearsals per year Q 24 H 22 Correlation Note. a Denotes the question number on the paper survey (see Appendix E). b Corresponds with the null hypotheses presented in Chapter 1. c This value was inferred from Q 5 . 118 APPENDIX B ONLINE SURVEY INSTRUMENT The online survey consisted of eight Web pages, including an introduction and a conclusion. The five main data collection pages featured progress indicators to assist respondents in their completion of the survey. The online survey was launched at http://surv.mued.com/ on January 4, 2006, and concluded on February 27, 2006. There were 546 responses to the online survey, of which 467 were usable. 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 APPENDIX C FIRST MAILING On January 4, 2006, 2,000 postcards were mailed from the U.S. Post Office in Churchton, MD to invite all sample members to participate in the online survey. Approximately 247 usable surveys resulted from this mailing. The following is a sample postcard from this mailing. 137 APPENDIX D SECOND MAILING On January 18, 2006, 1,836 postcards were mailed from the U.S. Post Office in Churchton, MD to remind non-respondents of their invitation to participate in the online survey. Approximately 155 usable surveys resulted from this mailing. The following is a sample postcard from this mailing. 138 APPENDIX E THIRD MAILING On February 1, 2006, 1,574 letters were mailed from the U.S. Post Office in Churchton, MD to remind non-respondents of their invitations to participate in the online survey. Each envelope contained an introductory letter, a four-page paper survey, and a self-addressed stamped envelope. Approximately 232 usable surveys resulted from this mailing, including 167 paper responses and 65 online responses. The following is a sample letter and paper survey from this mailing. 139 140 141 142 143 144 APPENDIX F PILOT SURVEY FEEDBACK The following three questions appeared on the pilot survey only. Responses are unedited; they are arranged alphabetically and numbered for convenience. Approximately how many minutes did it take you to complete this survey? M = 10.27, SD = 3.85, N = 30. What did you like most about this survey? 1. clean and easy to read and answer 2. Drawing focus to the resources allocated to assessment 3. Easy and fast 4. easy to fill in 5. Easy to follow - hits on a lot of issues that are current for our school. 6. I think that the zoomerang format is a great way to give a survey to a large group of people. 7. It gave me some ideas on other ways to asses student development. 8. It helped me assess my own assessment techniques. Now I know one or two things I'd like to change. Thanks. 9. It helped provide an introspective look at my class during a most busy time of the year (when I would probably not take the time) 10. It provided a few additional assessment tools. 11. It hits on a lot of the things I face each day. 12. quick to fill out but accurate 13. Realistic eway of looking at time management and what I amn doing with assessment in my class. 145 14. There were interesting questions that I had not considered in my assessment practices. 15. Very interactive and easy to use. The range of choices met all of the possible answers I could give. Are there any ways in which you would improve this survey? 1. Depends on what you are trying to determine 2. I found that I had accidentally skipped some questions, because there was not enough contrast in color when the respose boxes were activated. 3. I would have used numbers 1-5 to rate/rank. With this, in my mind, a 3 would indicate average, and probably would have sped up completing the form. 4. Nope. Open and close ended questions were all there. That's what I look for in surveys. 5. Some questions could be more specific. Best of luck in your work. 146 APPENDIX G QUALITATIVE RESPONSES: METHODS The following survey responses were obtained for Q 27 : ?What other student assessment methods do you use in your largest high school band class, if any?? They are arranged alphabetically and numbered for convenience. Responses were edited in several cases only to preserve the respondent?s anonymity. 1. 1.students fill out feedback sheets (post it notes)on occasion.2.they give numeric feedback on certain questions by placing before/after feedback on charts. anonymity is important. 2. -9 week exam over written theory work -Every nine weeks students must play off 2+ scales 3. All 15 major keys - scale, arpeggio, thirds chromatic scale played individually Memorize Happy Birthday NYSSMA Major Organizations Evaluation Festival 4. Articulation, rhythm, fingering are all factors. 5. Attendance and Participation 6. Attendance at after required school practice 7. attendance, attendance at performances 8. audience acceptance of performance 9. audio-video recordings of ensemble, with only 8 players, each kid can hear themselves (or not) quite well) 10. casual conversation as a non-graded assessment 11. CD Listening projects 12. CD/MP3's and Video of concerts for self evaluation 13. challenges for chair - contests/festivals 14. challenging weekly 15. Class room behavior & Concert Attendence 147 16. classroom tests & projects, lesson tests every 5 weeks 17. Comments from festival judges are used to encourage and help the entire band nderstand weak areas. The band then listens to their performances and idnetifies the areas that the judges commmented on. 18. Comments of audience and administration 19. Common sense - Can you play this part? Why not? Correct it Now go practice it 20. competition in fall marching band, concert festivals spring 21. competitive marching band, district band contest, solo and ensemble contest 22. concert attendance pep band attendance 23. concert evaluations from watching video, large group adjudication form at festivals for other performing groups 24. concert video, student reflection, students are given a band arrangement and must prepare it for concert without the help of conductor/director, the in-class performance is used as an exam grade. 25. Constant teacher/student interaction and feedback during rehearsals. 26. Criterion referenced district devised test 27. Daily observation and dialog with students. 28. Daily observation of students' efforts to attain my expectations at listed in the band handbook and my own principles of perfromance which is both written and on the board in front of the room. 29. Daily Participation 30. Daily Participation Grades 31. daily participation, behavior, and attendance 32. Daily recollection 33. daily rehearsal grades 34. discussion of mood, color, visualization, association, etc. 35. district assessment 148 36. Each student receives at least 5 15-minute lessons each quarter, OR takes weekly private lessons with an expert on their instrument. I have included this as individual conferences and student plays alone for teacher only 37. Farnom Watkins Sight Reading test once per year. Students play individually for me at lunch or after school. 38. Formal and Informal observations 39. General daily observation 40. Grade is based on attendance! 41. group recording and group self-assessment 42. guest clinicians private lessons 43. home-made written tests, music playing tests, scale playing tests 44. I also give students the chance to challenge a chair if they think they are better than the placement I gave them. It proves to me that they care about their position in the band, and they also can practice harder. We are eventually going to introduce Smart music into the classroom as an evaluation tool in order to save time. 45. I also us an Independent progress report. I teach music in an alternative school. Many of the high school students I teach do not have prior music experience. As I work with them to devlop their skills I have them complete a form I created to help them assess their efforts. In order to move to the next level they must score an 85% based on the rubric guidelines I've presented to them. 46. I am afraid that all this information is skewed due to the fact that the largest student group is marching band and we do things very differently there than in the cvoncert bands, especially freshman band. 47. I assess their concert performances, as well as observations of rehearsal work. 48. I do try to have students prepare regional or All-State music for mid term 49. I don't. There is no time. Rehearsel for the next performance takes all available time. 50. I grade 5 students at random everyday on anything that we have previosly rehearsed. 149 51. I have to issue a grade each 9 weeks. Appropriate participation in rehearsals and performances (marching/concert/jazz/ensembles/pit/festivals/etc)is encouraged. However, being a small school makes for participation issues/limits. I expect each student to continually improve their musical skills (fingering, range, endurance) 52. I record rehearsals often and ask them to write what they can do better and what the group can do better. 53. I use a computer program called Music Student Evaluator that is an assessment recording program specifically designed by a band director for all music teachers (band, choir, orchestra). Its assessment is annonymous, so the students feel like they are being graded/evaluated fairly, and I get the job done in a third the time. Prints great results pages as well. 54. I use many of the above methods at different times in the year, or over the course of different years, or for different students (in other words not on a quarterly basis). For instance if there is a student who is significantly struggling then I will keep a log for that student. Or when we are preparing for solo and ensemble festivals we will do in class recitals where I use peer assessment. I also use small ensemble assessment every day in class, but with different people and almost never as a formal evaluation. 55. I use my ears! 56. I use smartmusic and my assistants listen to each student during class time. 57. I use some of the above mentioned assessment methods once per year, but not in a typical marking period, such as guest conductors, auditions for chairs, assesment alone with band director, and playing in a small ensemble. 58. In such a small rural school, the rehearsal time is spent almost entirely in preparation for competition. 59. Individual and group observation 60. Individual lesson grades - every 7 days 61. Individual Lessons 62. Individual playing test in class.Playing solos for local events (church,civic clubs) motivate students by going to live performances by top musicians. Guest personnel to do sectionals. 150 63. Individual playing tests which consists of pass-off's of current lit. we're planning to perform and pass-off's of exercises dependent upon the student's current grade level 9th, 10th, 11th, or 12th. 64. Informal assessment of playing and steady improvement. Playing test for teacher used if steady improvement not evident. 65. informal verbal assessment 66. Instead of recording on audio tape, recording on computer. 67. IPAS by PYWARE 68. It's not in class, but their concerts and band adjudications. 69. journals, performance, rubrics 70. Just participating! 71. large group adjudicated event, solo/ensemble adjudicated event 72. Largely I use the performance as the assessment. 73. lessons, taped performances (individual) 74. Listening to individual intonation and tonal concepts while tuning. 75. listening to performances and evaluate 76. Main thing is use of tapings (3 per 9 wks) 77. matching pitch tests 78. Memorization tests (marching band only) 79. My Ears 80. my ears 81. None. I do a lot of assessment in my elementary bands(5th and 6th grade band) 82. None. I should probably use a few more. You have a lot of good ones on this survey. 83. None---There is NO time for assessment. Much of the class grade is participation. 84. Notebook/sketch book 151 85. observation, recordings, competitions 86. Once each quarter every band student plays individually for me for a grade and for chair placement for the following quarter. It takes me 6 school days to do it (during their scheduled lesson times: during study halls & before/after school) I listed these playing tests as part of the students play alone for teacher - the other 3 in that category are lessons which are sometimes individual and sometimes group (2-6 at a time). 87. Our school is a private, college prep. school. Band is more of an activity than anything else. I have some very good players, but most just enjoy music as a break from academics. 88. our state standards BOE's (body of evidence) 89. Participation 90. participation, behavior 91. participation, preparedness, solo festival 92. participation/attendance 93. Partner teacher or other conductors in the area critique both the student products and my teaching and conducting style. Especially in Orchestra because I am not a string player. 94. Performance 95. Performance at concerts; accomplishment of parts. 96. Performance tests using the Robert C. Fussell book. 97. Performances, Competitions 98. Personal judgement of the student's playing ability and character 99. play scales for a grade in sections 100. Playing in a section (playing in a group above) is the best way that I use 101. playing test, written test 102. playing tests Smart Music 103. Playing tests, music theory workbooks w/quizzes after each unit 104. Points for attendance, and participation daily 152 105. practice records (tapes) 106. prepared private and group lessons( like instruments). Point system points awarded for all-state band, solo-ensemble festivals etc..... 107. Progress in lesson books. 108. Question and response. 109. Record band - Play back & assessment, Video tape class & assessment 110. Record full ensemble playing examples, after which students submit written response which includes rubric and personal comments 111. Record full rehearsals and listen back with class and alone. 112. record of participation in regular band and extra festivals; county, district, region, jazz, etc. 113. Recording band and listening together, assessing results. students work in sectionals and assess themselves as groups 114. Recording ensemble every other Friday 115. recording of rehearsal and performance with written assessments and discussion 116. Recording rehearsals for assessment. 117. Recording the ensemble and students listen and critique the recording and fix the problems 118. Regular unit tests and playing test of music that is being prepared 119. scales, rhythms, fundamentals ect 120. Section assessment of parts Individual assessment in lessons 121. sectional rehearsal 122. sectional time, leaving the podium to walk around & in the sections 123. Sectionals, during this time, section leaders give playing tests to their section and report to me. 124. sections play for a grade, students play alone in front of their section not the whole class 125. Semester tests 153 126. short exercises and scales based on simple rubric 10 - no mistakes, 9 - 1 mistake, 8 - 2or3 mistakes, etc. 127. small group lesson playing auditions 128. small group playing tests 129. small groups -- sectional testing -- ocassiaonally tapes --- challenges 130. small sectional pull-out lessons during the school day 131. smart music / save file / then emailed to me. 132. smart music assessment 133. Smart Music notation program 134. Smart Music software 135. Smart Music/Finale Performance Assessment 136. SmartMusic 137. Solo Competition practice on a volunteer basis/ 138. Solo festival, large ens. festival, All county & All State auditions 139. solo/ensemble festival, clincian evaluations 140. Standard musical progress requirements, etc. - technique, scales, etc. 141. standardized 1st semester and 2nd semester playing assessments in small groups 142. State adjudication of the full ensemble Guest clinicians and/or conductors 143. State rubric found online. 144. student involvement in rehearsal, students ability to perform parts correctly in rehearsal 145. Student Learning Expectations; Curricular Based Assessments; Mid-Year Exam; Final Exam 146. Student playing tests 147. student/director assessment of performances and rehearsals via video tape. this is done at least 12 times per marking period. 154 148. Students are assessed individually every four weeks using a progressive etude system. The students are tested on scales once per marking cycle. Twice per year, they must perform one etude with the Smartmusic Accompaniment System. Their final exam is the successful completion of the two concerts per semester (They must meet certain criterion). I am very interested in your work if you would be willing to share it. 149. student's are assessed on concert performances 150. Students are assessed while rehearsing in band. There are so few I can usually tell who's playing and who isn't. I also have sections play daily so I can tell if they'r getting it or not. 151. Students are required to prepare and perform a solo for the district music festival. Adjudicator's comments are usually quite enlightening for both the student and for me. 152. Students asses recordings of rehearsal and eprformances. 153. Students critique music performances of other groups at concerts/festivals. (Adjudication sheet, plus a page of written feedback.) 154. Students listen to recorded ensemble rehearsals and comment in a group setting. 155. Students listen to recordings of their own performance and evaluate using The RI standards and proficiencies. 156. Students must checkoff music performance to the Director or Section Leader. All section Leader must check off to the Director 157. Students perform concert music in quartets. 158. students perform for teacher in weekly small group lessons & are graded for accuracy of music 159. Students perform selected band music in small groups of 5-6 musicians of varied instruments (i.e.flute, clarinet, sax, trpt,trombone) with me conducting. Students are assessed on this performance. 160. Students play for me in small groups...pairs...stand partners,parts of sections.... small quizes on terminology...notation, section leaders move throughout section and assess players for remediation, lots of questions to individuals in class....on terminology, interpretation, fundamentals, etc. Ours is more informal on a daily basis...a combination of non-threatening methods. 161. Students play in small groups in front of class. 155 162. Students respond to recordings of their rehearsals or performances (written responses). 163. Students view soloist's music on an overhead transparency while they are performing. The students then write comments regarding the performance and give suggestions for improvement. 164. subjective 165. taped self-assessment with rubric, periodic sectional work, teacher- generated music diagnostic tapes, actual live performance, section performance, folder checks, scale checks, attendance 166. Teacher Created Written tests 167. Teacher feedback, in my situation my band is 7-12, teacher feedback/demonstration works best 168. Teacher obsevation, participation, attitude Horn care 169. Teacher records ensemble, students assess their group performance. 170. Teacher records entire class performing. 171. Teacher-written exams, Listening assessments, written performance evaluations 172. The class assesses itself on an irregular basis. 173. The students complete surveys and assess the over all programs, decisions of the director. I ask the booster club to do the same. This helps me write the curriculum and set the schedule based on the degree of their enthusiastic participation. I do not want my students to graduate and be glad they are out of band. I want them to leave wanting more. So far I have been very successful. We do not do too much(burn-out) and we do not do too little. 174. these questions aren't geared for the situation i'm in. i teach both middle and high school, and i'm sorry, but the questions just don't match the situation i'm in. 175. Twice per qtr. students play portions of assigned music for a performance grade. Sections play excerpts for assigned music grade (ex. clarinets will play a technical passage as a group) *I'm a big believer in individual music performance grades. Each student plays pass-off grades almost constantly! 176. Verbal questions regarding theory, timed note-naming quizzes, vocab test 156 177. Vermont Music Educators Association Rubric 178. Videotape and watch concert performances and critique them. 179. We are mostly portfolio based with self assessments and outside assessments 180. we play scales daily, rhythms from band music used in warm-ups, attendance at lessons during the day every other week 181. We record the full ensemble, then discuss. 182. written critiques of concerts, oral critiques of concerts 183. written rhythm quizzes, theory worksheets 184. written test based on journal, terminology sheet 157 APPENDIX H QUALITATIVE RESPONSES: PURPOSES The following survey responses were obtained for Q 29 : ?What other purposes of student assessment would you add to those above, if any?? They are arranged alphabetically and numbered for convenience. Responses were edited in several cases only to preserve the respondent?s anonymity. 1. motivation to participate in honor groups (i.e. region/district, AREA, All state) 2) to prepare students for college auditions 2. Accountability of teaching and standards as compared to real classes 3. as a method of demonstrating improvement and gaining new skills/technique 4. assist parents in getting students to practice 5. By listening to other student's performances in class, students will have a better understanding of what to practice and from discussion of what strengths and weaknesses are evident in those performances how to set priorities for themselves. 6. Constant assessment is important, and keeps kids practicing. 7. Establish intrinsic motivation as opposed to extrinsic 8. feedback from their peer groups. Many times students feel that peer approval is more important than many other types of assessment 9. For students to hear and understand what they need to do to improve, both individually and as an ensemble 10. Fosters achievement - success for the individual student - helps students understand there should be growth which can be assessed just like other classroom subject- but this can have an extremely positive effect on the entire membership 11. Giving students the opportunity (with a rubric) to think specifically about the many important aspects of playing a piece of music with excellence. 12. good list, also to pick liturature and determine if students are ready for it 158 13. Help diminish stage-fright--getting used to playing in front of peers. Isolate perfromance problems within sections. Provides another way to reinforce concepts rather than a routine rehearsal. 14. Honors band requirements 15. I just want to make sure that there will always be a need for music in the public school system here. 16. I rate each student, with consideration for individual capacity, at each lesson. 17. I teach life, i use music as a tool. if you work hard study and dedicate yourself to something, be respectfull, etc.... but music is the tool i use to teach kids about life. 18. I use assessments to help me plan what to teach next - whatever concepts are understood, and what aren't 19. I want assessment to always be a learning tool for my students, something they find helpful, not to be feared. 20. I want my students to be prepared to continue their music education with a sense of pride and confidence to meet the challanges of audition, and performance. No surprises! 21. It gives individual students goals -- some don't care at all about chair placement, but they want their videotaped scales to be perfect because they'll have to hear them. Varied assessments give students a chance to show what's important to them. 22. It really helps me to assess whether the material that I choose is appropriate to the level of the group, to give me direction in choosing new material, to get a feel for student interest in the material...ie...music that they enjoy is generally played better...even if it is more difficult. I can also get a sense of how students instruments are working with a quick individual playing session, as well as look for problems with technique. My assessment is more about playing the music than keeping score. 23. musical growth as a consumer 24. Our assessment comes by others at our performances only. Band at my school is a pass fail course. Course credibility comes from the success of our performance only. 25. Preparation for auditions 159 26. Self evaluation of concert performances in the form of written critiques Teacher designed written tests 27. self-discipline, self-confidence, self-esteem, work ethics 28. small ensemble placement 29. So that I can determine student expectations and teacher expectations 30. Strategies for performing under pressure and for coping with comparing themselves to others. 31. student assessment is not a part of program. 32. Student motivation - Student Reward for being in the ensemble 33. Students meeting 3 of 5 standards prescribed by Colo. Dept of Education. 34. the last one is especially important (motivate students to practice). No matter how much I stress the need to practice, it's when playing test time arrives that the practicing actually occurs & the band suddenly sounds better. For that reason I always schedule playing tests right before concerts. 35. To assess whether my students are becoming real musicians. 36. To build students discipline in learning proper techniques. To develop performance and ettiquete standards. 37. To motivate and build self-esteem 38. To determine if students enjoy making music that thrills audiences. 39. to determine what remediation is necessary, if any 40. to develop students' listening skills (aural ID) 41. to guage interest level in order to maintain and/or increase involvement 42. to help identify and illustrate performance standards 43. to help students develop a desire to do things correctly. No matter the content 44. To help students work together as group to accomplish tasks 45. To help with music selection by determining student needs 160 46. To instill the individual's worth in regards to the whole/finished product Gestalt/esprit des corps/always strive to do your best it's not my band-it's your (the student's)band the composer didn't write any unimportant parts (3rd trip is just as important as the 1st) 47. To motivate students to prepare to demonstrate competency on their insturment. 48. to promote self-satisfaction and pride in being a good player in a good band. 49. To provide students opportunities to perform so they can become accustomed to it, nerves, stage fright, etc. 50. To push the lower level, and raise the bar on the upper level students. 51. To see/hear what they are learning? 52. Too get better so as to have more fun as a motivator to keep it up. 53. We assess our performances as individuals, as a group, and I as the teacher. We do so for we do not want to give A 90% PERFORMANCE. We want to give a 100% performance 100% of the time. 54. Whether or not music is becoming a life skill for my students. 55. with only one teacher in a program that tries to meet the needs of approximately 30% of the students, I am doing as much student assessment as time will allow. 161 APPENDIX I QUALITATIVE RESPONSES: INFLUENCES The following survey responses were obtained for Q 31 : ?What other factors have an influence on the assessment methods you use in your largest high school band class, if any?? They are arranged alphabetically and numbered for convenience. Responses were edited in several cases only to preserve the respondent?s anonymity. 1. ...knowing their at-home practice habits are poor. I must keep them honest, or they simply won't perform up to standards. 2. 200 band members/3 bands for one teacher - time! We have no music supervisor. 3. amount of hours I am willing to put in on it outside of class 4. Assement of individual students is not teribly important. I do listen to students play individually but not for the sake of assesment. All students must perform at a 100% level. Lets say you have 100 students playing a song with 500 notes and each student plays at a 99% level thats 500 wrong notes. That would be an A in any other subject but is completly unacceptable for a musical performance. I work with the students till they reach 100%. you cant use a objective approach to an art form. Most administrators dont want to hear this but the most valuable type of education is subjective. I know this dosnt fit the CONTENT STANDARDS but this is one of the bigest problems with our education system today. 5. audio tapes seem to be the most effective, and also boost the general performance level of the ensembles. 6. Availability of adequate assessment tools that allow the student to grow without representing a threat. It is interesting to do this survey now in that I am attempting to devise a workable, fair, growth oriented assessment plan that will have student buy-in. 7. Bandwidth. Figuring out how to give kids useful assessment that does not suck up all the available time from the rest of the students or myself. 8. being consistent in the view of the student. I really take the time to adjudicate a form/text/method of evaluation before I implement it with the students. Kids today really need consistency from the adults in their lives. 162 9. Block Scheduling is a nightmare to having students schedule Band or have any kind of continuity to the course. This has had a negative impact on the musical improvement of my students. My schedule this year (band) - Marking Periods 1, 2, 6, 7 (out of 8). 10. Community perception of what band class is. Many do not see this as an academic class in which grades should be given.They love the performance aspect and support the competitive marching but do not grasp the concept of band being a learning environment in which certain goals and objectives should be worked for and accomplished. 11. Differentiating instruction also demands that we differentiate assessment, as not all students assess equally using any one method. 12. Directive from the Director of Curriculum. 13. discussion with collegues outside the school district 14. Each player is at a completely different level of progress at this point. My bottom line is to see improvement in individual performance. With such a tiny grop this is possible. 15. Establishing high expectations for our program. 16. how much time I have to complete the assesment - how many days can we take away from playing together? 17. I am the only music teacher. I feel overloaded. I have four young children, so time is a concern as well. 18. I change my assessments based on what will help my students learn from them the most ... if a written test will only hold them back and make them think mechanically rather than musically, then I might give an oral/performance based test instead. 19. I teach music K-12 on an alternate educator license. I am learning several aspects of my job as I go. Some days I teach 9 classes. The scope of my responsibilities has a significant effect on implementation of many things I know are needed. In order to keep my sanity, I have to take one day at a time and add processes and 20. procedures gradually. This is a long term commitment on my part, so I don't expect myself to get it all together any time soon. 21. I use what I feel best benefits the program. 22. I will tend to test more if it appears students are not practicing their parts on their own. In other words I test instead of requiring practice records. If 163 it appears the class is making steady progress on their music parts, I will tend to relax the playing test frequency. If it appears they are not working on their parts, I will schedule a test to encourage practice. The goal is to get them to the point where they will practice on their own, not because they have to submit a certain number of minutes, but because they have to master their music. It often helps to instill intrinsic motivation. I don't have specific practice requirements other than that they do practice regularly and make steady progress. Whether they do or not becomes apparent in rehearsals. They are asked to practice their music enough to master their parts, and they are encouraged to practice other things not for class (lessons, solos, practice with their friends, fun music they pick, etc). If it becomes apparent that they are neglecting their honor code commitment of learning their parts or they seem to be not making very much progress, then I will schedule a test to provide the extrinsic motivation. 23. If I honestly assessed and graded each student on his/her capabilities and gave too low a grade or had too high expectations, students would drop my class. I have to balance my goals and assessments with community expectations. Much of my band grade is based on classroom participation (since most of the kids won't practice outside of class) and on participation in performances. In our very small school, my grading system has worked for me. Classroom is 50% of the grade, written and/or playing tests are 15% and performances are 35%. If kids miss class, they owe me make-up practice time. Most of the kids in my band are also in choir, they are the FFA leaders, they are the sports stars, and do anything else that is expected in our school. I have to tread lightly to keep them in band. Also, I have parents who are looking at kids keeping up a grade point average. I've had parents tell me if their kid was going to get a B or C in band, they would pull them out because it would affect the grade point average. My largest (and only) band also has to provide pep music for ballgames--and people and the kids much prefer this to concert band music. So I have to choose concert band music carefully to keep the kids' interest. 24. I'm not going to answer any of these. my last answer should tell you. i know your trying to get your doctorate, but these questions make me ask you one. EVER TEACH IN A PUBLIC SCHOOL? (and yes, it is relevant) 25. I'm not sure if I've answered question #30 accurately (or if I even understand the question?) Although I do understand the value and reason to access,I spend very little time actually assessing my students in an effort to justify our existance or worth.I enjoy the teaching/musical experience I provide for my students. *I am the band director 5-12. I'm fortunate to teach in a District that supports my efforts and I endeavor to provide the best/comprehensive music education I can. 164 26. In my school the band grade does not count on the students GPA. Students are in band because they really want to be there. 27. individual student availability throughout the day 28. It holds everyone accountable 29. Judges tapes, lack of facilities and help in assessment - dealing with a wide range of abilities in one group - minimal class time - 30. My desire to bring to the students a comprehensive music education through performance. 31. my efforts to continue to improve instructional methods and assessment methods 32. Observation of other high quality music programs in Northern California.Particularly those taught by long tenure veteran directors. 33. participation in all activities-observation intuition 34. Past experience with the students and performance schedule influence what I do. 35. Personal expectations of student performance 36. Playing level of the students and needs of the music selected. 37. Some students tend to hide behind each other. This is not allowed in our bands, as standards are set very high for each student. The thing that others in my school can't figure out is how I get the students that don't perform at high levels for them to perform at exceedingly high levels for me. It is all in setting the expectation. The kids will generally rise to the occassion. No excuses. 38. State & local rubrics used in festivals and competitions. In CA the WBA circuit uses a rubric with a bell curve point system that is then weighted! No one has been able to explain the philosophy or methodology for using these three very different assessment methods simultaneously! I find this system used by WBA to be detrimental to music and music learning, and no longer participate in their competitions because I am philosophyically opposed to their assessment methods and application of their system. 39. State and federal requirements are forcing us to do a lot of busy work that is largely unnecessary. This tends to take time away from more important tasks such as actually teaching! All this assessment and accountability stuff is just a smoke screen for administrators at all levels to cover their butts. Just make sure good teachers are hired and let them teach - period. 165 40. State contest schedule 41. student behavoir and need for supervision 42. student involvement/conflict in other organizations 43. student response 44. student scheduling student motivation to be in band: too much testing they would probably drop out. The students take band because they like to play instruments, not to push paper on pencil, take quizzes, keep journals, etc. 45. Students can attend the College of W. for undergraduate classes. Top band members can only attend 2 or 3 times a week, full band rehearsals. 46. Tailored to individual student and musical needs 47. The available time to evaluate recorded performance tests. (I don't know if that qualifies under one of the time sections above.) 48. the community's culture which does not regard music in any form nearly as important as athletics. 49. The level of acceptance students show to the work ethic asked of them. 50. The literature being addressed in class, schedule changes due to holidays, testing (schoolwide) or extra curricular activities. Our school band is made up of students from 7th grade through 12th grade. The figure of 200 students in school is based specifically on the high school. About 35 students are in band. School enrollment for 7th-12th would be about 300. 51. The possiblility that music may be cut from the curriculum due to funding. 52. There are two dominant obstacles that control the size of the band and the assessment methods used. The first is when principals place the band period in the schedule where 3/4 of the students are in classes required for graduation. The second obstacle is that studnts in small high schools are involved in everything and the sports teams take students from band three to four days a week. It is difficult to maintain a consistant assessment method when students can not take the class or are out of the class so much of the time. 53. Time 54. TIME after class to listen to student recordings, have lessons. Student expectations--If it doesn't happen in school, it won't get done! As long as the pep-band plays, things must be fine! 166 55. time and space 56. Time it takes to grade the assessments to give back to the students in a timely manner 57. TIME TIME TIME TIME 58. Time, seems to be the biggest single factor, theirs and mine 59. Time, Time, Time, Time, Time 60. Time: amount allowed by administration for teacher | amount students have to participate in music 61. We are doing projects to meet the National Standards. Beyond that, it's pretty much participation-graded. With schools this small, you take what you can get, then work at whatever level most of those kids are at. 62. We do not have a music supervisor per se. We do have a subject area coordinator who is also a teacher. This does not provide much time in creating a good relationship that helps students continue music learning in the classroom. 63. We do not have a pullout lesson program, so we created a jury system which grades the students on their individual performance for us on audiotape. We also use this as a reseating method for all three bands. The lack of pullout lessons leaves us with this mehod as the most legitimate way to judge a stduent's performance and progress. It's supposed to be performance oriented... 64. We do so much in our band - I give them an A if they have 0 unexcused absences during a grading period. 65. what best meets the needs of a particular group 66. What I believe to be important for the betterment of my students. 67. work load. I teach elementary general music and jr hi school band and choir also. Between that and extra curricular things like drama and sports, assessment becomes minimal. 68. Would like to have more frequent and possibly more individualized assessment opportunities but it is difficult with the numbers in the class and given class/rehearsal time as well as demands on my time schedule, teaching 4 performance ensemble classes as well as marching band. 69. Your passion for music & your passion for your students. 167 APPENDIX J INTERCORRELATIONS BETWEEN SELECTED VARIABLES Years teaching Independent variable Age Any subject or level High school band At current school Age ? .90 .79 .60 Years teaching Any subject or level ? .89 .69 High school band ? .70 At current school ? Note. All correlations were significant at p = .00. 168 REFERENCES Abeles, H. (1992). A guide to interpreting research in music education. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 227-243). New York: Schirmer Books. Acock, A. C. (2005). Working with missing values. Journal of Marriage and Family, 67, 1012-1028. Airasian, P. W. (1991). Perspectives on measurement instruction. Educational Measurement: Issues and Practice, 10(1), 13-16, 26. Allison, P. D. (2000). Multiple imputation for missing data: A cautionary tale. Sociological Methods & Research, 28(3), 301-309. Alreck, P. L., & Settle, R. B. (2004). The survey research handbook (3 rd ed.). Boston: McGraw-Hill/Irwin. American Federation of Teachers, National Council on Measurement in Education, & National Education Association. (1990). Standards for teacher competence in educational assessment of students. Retrieved February 4, 2005, from http:// www.unl.edu/buros/bimm/html/article3.html American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education. (1999). Standards for educational and psychological testing (rev. ed.). Washington, DC: American Psychological Association. Andrews, B. D., & Quinn, R. J. (2004). First-year teaching assignments: A descriptive analysis. The Clearing House, 78(2), 78-83. 169 Apple, M. W. (1988). Teachers and texts: A political economy of class and gender relations in education. New York: Routledge & Kegan Paul. Barlow, D. (2005). The teachers? lounge: Does ?24 th out of 29? really matter? Education Digest, 71(2), 64-67. Bauer, W. I., & Berg, M. H. (2001). Influences on instrumental music teaching. Bulletin of the Council for Research in Music Education, 150, 53-66. Biner, P. M. (1988). Effects of cover letter appeal and monetary incentives on survey response: A reactance theory application. Basic and Applied Social Psychology, 9(2), 99-106. Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80(2), 139-148. Blocher, L., Greenwood, R., & Shellahamer, B. (1997). Teaching behaviors of middle and high school band directors in the rehearsal setting. Journal of Research in Music Education, 45(3), 457-469. Bloom, B. S., Hastings, J. T., & Madaus, G. F. (1971). Handbook of formative and summative evaluation of student learning. New York: McGraw-Hill. Boyle, J. D. (1992). Evaluation of music ability. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 247-265). New York: Schirmer Books. Boyle, J. D., & Radocy, R. E. (1987). Measurement and evaluation of musical experiences. New York: Shirmer Books. Brandt, R. (1987). On assessment in the arts: A conversation with Howard Gardner. Educational Leadership, 45(4), 30-34. 170 Bray, D. (2002). Assessment in music education. In G. Spruce (Ed.), Aspects of teaching secondary music (pp. 79-93). New York: RoutledgeFalmer. Brookhart, S. M. (1994). Teachers? grading: Practice and theory. Applied Measurement in Education, 7(4), 279-301. Brookhart, S. M. (1997). A theoretical framework for the role of classroom assessment in motivating student effort and achievement. Applied Measurement in Education, 10(2), 161-180. Brookhart, S. M. (1999). Teaching about communicating assessment results and grading. Educational Measurement: Issues and Practice, 18(1), 5-13. Brookhart, S. M. (2004). Classroom assessment: Tensions and intersections in theory and practice. Teachers College Record, 106(3), 429-458. Burrack, F. (2002). Enhanced assessment in instrumental programs. Music Educators Journal, 88(6), 27-32. Chiodo, P. (2001). Assessing a cast of thousands. Music Educators Journal, 87(6), 17-23. Cizek, G. J., Fitzgerald, S. M., & Rachor, R. E. (1995). Teachers? assessment practices: Preparation, isolation, and the kitchen sink. Educational Assessment, 3(2), 159-179. Colwell, R. (1970). The evaluation of music teaching and learning. Englewood Cliffs, NJ: Prentice-Hall. Colwell, R. (2000). Tools for music assessment. In K. Seidel (Ed.), Assessing student learning: A practical guide (chap. 17) [CD-ROM]. Cincinnati, OH: Alliance for Curriculum Reform. 171 Colwell, R. (2002). Assessment?s potential in music education. In R. Colwell & C. Richardson (Eds.), The new handbook of research on music teaching and learning (pp. 1128-1158). New York: Oxford. Colwell, R. (2003). The status of arts assessment: Examples from music. Arts Education Policy Review, 105(2), 11-18. Colwell, R. J. (1991). Evaluation. In R. J. Colwell (Ed.), Basic concepts in music education, II (pp. 247-278). Niwot, CO: University Press of Colorado. Consortium of National Arts Education Associations. (1994). National standards for arts education: What every young American should know and be able to do in the arts. Reston, VA: Music Educators National Conference. Conway, C. M. (2006). Navigating through induction: How a mentor can help. Music Educators Journal, 92(5), 56-60. Conway, C., & Jeffers, T. (2004). Parent, student, and teacher perceptions of assessment procedures in beginning instrumental music. Bulletin of the Council for Research in Music Education, 160, 16-25. Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative and qualitative research. Upper Saddle River, NJ: Merrill Prentice Hall. Cutietta, R. A. (1992). The measurement of attitudes and preferences in music education. In R. Colwell (Ed.), Handbook of research on music teaching and learning: A project of the Music Educators National Conference (pp. 295-309). New York: Schirmer Books. 172 De Leeuw, E. D. (2001). Reducing missing data in surveys: An overview of methods. Quality & Quantity, 35, 147-160. Eisner, E. W. (1998). The kind of schools we need: Personal essays. Portsmouth, NH: Heinemann. Eisner, E. W. (2002). The arts and the creation of mind. New Haven, CT: Yale University Press. Elliott, D. J. (1995). Music matters: A new philosophy of music education. New York: Oxford University Press. Farrell, S. R. (1997). Tools for powerful student evaluation: A practical source of authentic assessment strategies for music teachers. Ft. Lauderdale, FL: Meredith Music. Field, A. (2005). Discovering statistics using SPSS (2 nd ed.). Thousand Oaks, CA: Sage. Frary, R. B., Cross, L. H., & Weber, L. J. (1993). Testing and grading practices and opinions of secondary teachers of academic subjects: Implications for instruction in measurement. Educational Measurement: Issues and Practice, 12(3), 23-30. Fowler, F. J., Jr. (2002). Survey research methods (3 rd ed.). Thousand Oaks, CA: Sage. Goolsby, T. W. (1997). Verbal instruction in instrumental rehearsals: A comparison of three career levels and preservice teachers. Journal of Research in Music Education, 45(1), 21-40. Goolsby, T. W. (1999). Assessment in instrumental music. Music Educators Journal, 86(2), 31-35, 50. 173 Goranson, D. G., Jr. (2002). The arts: A guide to K-12 program development [Electronic version]. Hartford, CT: Connecticut State Department of Education. Retrieved April 23, 2006, from http://www.state.ct.us/sde/dtl/curriculum/currart_guide1.htm Graue, M. E. (1993). Integrating theory and practice through instructional assessment. Educational Assessment, 1(4), 283-309. Groves, R. M. (1989). Survey errors and survey costs. Wiley series in survey methodology. Hoboken, NJ: John Wiley & Sons. Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to participate in a survey. Public Opinion Quarterly, 56, 475-495. Groves, R. M., Fowler, Jr., F. J., Couper, M., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Wiley series in survey methodology. Hoboken, NJ: John Wiley & Sons. Gullickson, A. R. (1985). Student evaluation techniques and their relationship to grade and curriculum. Journal of Educational Research, 79(2), 96-100. Gutsch, K. U. (1965). Evaluation in instrumental music performance: An individual approach. Council for Research in Music Education Bulletin, 4, 21-29. Hanzlik, T. J. (2001). An examination of Iowa high school instrumental band directors? assessment practices and attitudes toward assessment. Dissertation Abstracts International, 62 (03), 955A. (UMI No. 3009721) Heerwegh, D., & Loosveldt, G. (2002). Web surveys: The effect of controlling survey access using PIN numbers. Social Science Computer Review, 20(1), 10-21. Hewitt, M. P. (2005). Self-evaluation accuracy among high school and middle school instrumentalists. Journal of Research in Music Education, 53(2), 148-161. 174 Hewitt, M. P., & Smith, B. P. (2004). The influence of teaching-career level and primary performance instrument on the assessment of music performance. Journal of Research in Music Education, 52(4), 314-327. Hill, K. W. (1999). A descriptive study of assessment procedures, assessment attitudes, and grading policies in selected public high school band performance classrooms in Mississippi. Dissertation Abstracts International, 60 (06), 1954A. (UMI No. 9935693) Jaeger, R. M. (1988). Survey research methods in education. In R. M. Jaeger (Ed.), Complementary methods for research in education (pp. 303-330). Washington, DC: American Educational Research Association. Joint Committee on Testing Practices. (2004). Code of fair testing practices in education. Washington, DC: Author. Retrieved March 15, 2006, from http://www.apa.org/ science/fairtestcode.html Kohn, A. (1994). Grading: The issue is not how but why. Educational Leadership, 52(2), 38-41. Kotora, E. J., Jr. (2001). Assessment practices in the choral music classroom: A survey of Ohio high school choral music teachers and college choral methods teachers. Dissertation Abstracts International, 62 (12), 4102. (UMI No. 3036343) Lehman, P. (1992). Curriculum and program evaluation. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 281-294). New York: Schirmer Books. Lehman, P. (1998). Grading practices in music. Music Educators Journal, 84(5), 37-40. 175 Lehman, P. (2000). How can the skills and knowledge called for in the national standards best be taught? In C. K. Madsen (Ed.), Vision 2020: The Housewright symposium on the future of music education (pp. 89-101). Reston, VA: MENC: The National Association for Music Education. Leonhard, C. (1958). Evaluation in music education. In N. B. Henry (Ed.), Basic concepts in music education: The fifty-seventh yearbook of the National Society for the Study of Education (pp. 310-338). Chicago: The University of Chicago Press. Linn, R. L. (1990). Essentials of student assessment: From accountability to instructional aid. Teachers College Record, 91(3), 422-436. Linn, R. L., & Miller, M. D. (2005). Measurement and assessment in teaching (9 th ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall. Locke, L. F., Spirduso, W. W., & Silverman, S. J. (2000). Proposals that work: A guide for planning dissertations and grant proposals (4 th ed.). Thousand Oaks, CA: Sage. Lohr, S. L. (1999). Sampling: Design and analysis. Boston: Duxbury Press. Madaus, G. F., & Kellaghan, T. (1992). Curriculum evaluation and assessment. In P. W. Jackson (Ed.), Handbook of research on curriculum (pp. 119-154). New York: Macmillan. Mark, M. L. (1992). A history of music education research. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 48-59). New York: Shirmer Books. 176 Mark, M. L., & Gary, C. L. (1999). A history of American music education (2 nd ed.). Reston, VA: Music Educators National Conference. McClung, A. C. (1996). A descriptive study of learning assessment and grading practices in the high school choral music performance classroom. Dissertation Abstracts International, 57 (08), 3437. (UMI No. 9700217) McCoy, C. W. (1991). Grading students in performing groups: A comparison of principals? recommendations with directors? practices. Journal of Research in Music Education, 39(3), 181-190. McCreary, T. J. (2001). Methods and perceptions of assessment in secondary instrumental music. Dissertation Abstracts International, 62 (10), 3328A. (UMI No. 3030187) McMillan, J. H. (2001). Secondary teachers? classroom assessment and grading practices. Educational Measurement: Issues and Practice, 20(1), 20-32. McMillan, J. H. (2003). Understanding and improving teachers? classroom assessment decision making: Implications for theory and practice. Educational Measurement: Issues and Practice, 22(4), 34-43. McMillan, J. H. (2004). Classroom assessment: Principles and practice for effective instruction (3 rd ed.). Boston: Allyn and Bacon. McMillan, J. H., Myran, S., & Workman, D. (2002). Elementary teachers? classroom assessment and grading practices. The Journal of Educational Research, 95(4), 203-213. 177 Meisenheimer, B. (1996). Critical attributes of teachers who have become practitioners of authentic assessment. Dissertation Abstracts International, 56 (12), 4652. (UMI No. 9612421) Meltzoff, J. (1998). Critical thinking about research: Psychology and related fields. Washington, DC: American Psychological Association. Mitchell, M., & Jolley, J. (1996). Research design explained (3 rd ed.). New York: Harcourt Brace. Morrison, S. J., Montemayor, M., & Wiltshire, E. S. (2004). The effect of a recorded model on band students? performance self-evaluations, achievement, and attitude. Journal of Research in Music Education, 52(2), 116-129. Music Educators National Conference. (1998). Grading practices in music. Music Educators Journal, 84(5), 37-40. National Board for Professional Teaching Standards. (2001). NBPTS music standards: For teachers of students ages 3-18+. Retrieved October 13, 2005, from http:// www.nbpts.org/pdf/ecya_music.pdf National Center for Education Statistics. (2005). Common core of data, 2003-2004 [Data file]. Available from the National Center for Education Statistics website, http:// nces.ed.gov/ccd/ National Commission on Excellence in Education. (1983). A nation at risk: The imperative for educational reform. Retrieved October 29, 2004, from http:// www.ed.gov/pubs/NatAtRisk/risk.html Oosterhof, A. (2003). Developing and using classroom assessments (3 rd ed.). Upper Saddle River, NJ: Prentice-Hall. 178 Pallant, J. (2005). SPSS survival manual: A step by step guide to data analysis using SPSS for Windows (version 12) (2 nd ed.). New York: Open University Press. Parsad, B., & Jones, J. (2005). Internet access in U.S. public schools and classrooms: 1994-2003 (NCES 2005-015). Washington, DC: National Center for Education Statistics. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press. Presser, S., Blair, J., & Triplett, T. (1992). Survey sponsorship, response rates, and response effects. Social Science Quarterly, 73(3), 699-702. Reed, J. B. (2005, November 15). Music groups seek standardized tests: Organizations to push exam at education board?s meeting [Electronic version]. The News-Press. Retrieved December 1, 2005, from http://www.news-press.com/apps/pbcs.dll/ article?AID=/20051115/NEWS0104/511150441/1002/NEWS01/ Robinson, M. (1995). Alternative assessment techniques for teachers. Music Educators Journal, 81(5), 28-34. Saunders, T. C., & Holahan, J. M. (1997). Criteria-specific rating scales in the evaluation of high school instrumental performance. Journal of Research in Music Education, 45(2), 259-272. Schafer, W. D. (1991). Essential assessment skills in professional education of teachers. Educational Measurement: Issues and Practice, 10(1), 3-6, 12. Schlichte, J., Yssel, N., & Merbler, J. (2005). Pathways to burnout: Case studies in teacher isolation and alienation. Preventing School Failure, 50(1), 35-40. 179 Schonlau, M., Asch, B. J., & Du, C. (2003). Web surveys as part of a mixed-mode strategy for populations that cannot be contacted by e-mail. Social Science Computer Review, 21(2), 218-222. Scott, D. (1998). Tiered evaluation in large ensemble settings. Music Educators Journal, 85(3), 17-21, 47. Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14. Simanton, E. G. (2000). Assessment and grading practices among high school band teachers in the United States: A descriptive study. Dissertation Abstracts International, 61 (09), 3500. (UMI No. 9986536) SPSS. (n.d.). Missing data: The hidden problem. Retrieved February 23, 2006, from http://www.spss.com/registration/premium/consol056.cfm?WP_ID=12 Stiggins, R. J. (1991a). Facing the challenges of a new era of educational assessment. Applied Measurement in Education, 4(4), 263-273. Stiggins, R. J. (1991b). Relevant classroom assessment training for teachers. Educational Measurement: Issues and Practice, 10(1), 7-12. Stiggins, R. J. (2001). Student involved classroom assessment (3 rd ed.). Upper Saddle River, NJ: Prentice-Hall. Stiggins, R. J. (2005). Student-involved assessment for learning (4 th ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall. Sudman, S. (1976). Applied sampling. New York: Academic Press. 180 Taebel, D. K. (1992). The evaluation of music teachers and teaching. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 310-329). New York: Schirmer Books. Thorndike, R. M. (2005). Measurement and evaluation in psychology and education (7 th ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall. Tracy, L. H. (2002). Assessing individual students in the high school choral ensemble: Issues and practices. Dissertation Abstracts International, 63 (09), 3143. (UMI No. 3065486) Trice, A. D. (2000). A handbook of classroom assessment. New York: Longman. Umbach, P. D. (2004). Web surveys: Best practices. New Directions for Institutional Research, 121, 23-38. United States Census Bureau. (2005). Geographic areas reference manual. Retrieved October 10, 2005, from http://www.census.gov/geo/www/garm.html United States Department of Education. (2005). Choosing a school for your child. Retrieved October 10, 2005, from http://www.ed.gov/parents/schools/find/choose/ pub_pg5.html#learn Washington State Department of Health. (2005). Guidelines for using rural-urban classification systems for public health assessment. Retrieved October 10, 2005, from http://www.doh.wa.gov/Data/guidelines/RuralUrban.htm Webster, P. R. (1992). Research on creative thinking in music: The assessment literature. In R. Colwell (Ed.), Handbook of research on music teaching and learning (pp. 266-280). New York: Schirmer Books. 181 Whoriskey, P. (2006, March 22). Fla. to link teacher pay to students? test scores. The Washington Post, pp. A1, A14. Wiggins, G., & McTighe, J. (2005). Understanding by design (2 nd ed.). Upper Saddle River, NJ: Pearson Merrill Prentice Hall. Wolf, D. P. (1987). Opening up assessment. Educational Leadership, 45(4), 24-29. Zerull, D. S. (1990). Evaluation in arts education: Building and using an effective assessment strategy. Design for Arts in Education, 92(1), 19-24. 182