Provost & Academic Vice President
2005 - 2006 Graduate Program Assessment Summaries
College of Education and Human Services
Health and Health Services Management
Curriculum and Instruction-Master of Science in Education with a Major in El Ed
Curriculum and Instruction- Master of Science in Education with a Major in Reading
Educational and Interdisciplinary Studies- Master of Arts in Teaching
Educational and Interdisciplinary Studies-College Student Personnel
Educational and Interdisciplinary Studies-Master of Science in Education
Educational Leadership-Masters Degree Program
Educational Leadership-Educational Specialist Program
Instructional Technology and Telecommunications
Kinesiology-Masters of Science in Kinesiology
Kinesiology-Masters of Science in Sports Management
Law Enforcement and Justice Administration
Recreation, Park, and Tourism Administration
The purpose of this report is to describe the assessment plans as well as the current results of major assessment for 2005-2006. Department results are reported in alphabetical order.
- the assessing of individual and community needs for health education;
- planning effective health education programs;
- implementing health education programs;
- evaluating effectiveness of health education programs;
- coordinating provision of health education services;
- acting as a resource person in health education;
- communicating health and health education needs, concerns, and resources;
- applying appropriate research principles and methods in health education;
- administering health education programs; and
- advancing the profession of health education. These national standards are used to structure curriculum and serve as departmental student learning outcomes.
Direct and Indirect Measurement: none listed.
Results: Graduate students and faculty presented three research papers and posters at the following professional meetings: American School Health Association, Illinois School Health Association, and the 14th Annual Illinois HIV/STD Conference. During this review period, 95 percent of the graduate students received good to excellent scores on their portfolio presentations (HE 602). Eighty-five percent of health education graduate students completed their major courses during AY 2005-2006 with a grade of “B” or higher. During AY 2005-2006, internship supervisors rated the students’ professional contributions in the area of health education as good to excellent. During this review period, nearly 100 percent of students rated the internship experience as good or excellent. Graduate health education interns received the highest ratings in the areas of resourcefulness and leadership. Ninety percent of M. S. in Health Education students were overwhelmingly satisfied with departmental courses and career preparation. Approximately 30 percent of students were offered full or part time employment at their internship site.
Feedback Loop: A graduate alumni survey is being developed (60% completion).
- Revisit graduate curriculum to address identified weaknesses that include the degree to which assignments are completed on time and written and oral communication.
- Complete and implement the graduate site supervisors’ survey.
- Complete and administer the alumni graduate program needs assessment.
- Encourage student involvement in the profession through departmentally sponsored faculty-student participation in professional scholarly opportunities at preeminent professional organizations in health education (with the goal of enhancing involvement through research skills, leadership abilities, and oral and written expression).
- the student will demonstrate an adequate knowledge base in the eight areas of counseling: Human Growth and Development, Social and Cultural Diversity, Helping Relationships, Group Work, Career Development, Assessment, Research and Program Evaluation, and Professional Identity;
- the student will apply counseling knowledge and skills by successfully completing a field based practicum and internship;
- the student will demonstrate sound ethical reasoning and behavior during both classroom and field based portions of the program.
Direct and Indirect Measurement: Knowledge base is assessed through two external test instruments: National Counselor Examination (80% pass rate desired) and Illinois Guidance Certification Exam (90% pass rate desired). Faculty will use tests, papers, audio and videotapes of counseling sessions to assess student progress. Faculty will meet once a year to review every student. The department will conduct follow-up surveys once every three years.
Results: Four students took the NCE test on October 23, 2004, and 3 passed; fifteen took it on April 16, 2005, and 12 passed. Since July 2004 seventeen students have taken the ISBE exam for a Type 73 Certificate, and all passed. April 2006 surveys will be compiled in May 2006 and then reviewed by the faculty either during summer 2006 or at the beginning of the fall 2006 semester.
Faculty met on January 27, 2006, to complete the annual student review of all students admitted to the department. Academic progress was assessed, students’ evaluation on the criteria for admission to the department was reviewed, viewpoints and information were shared, and recommendations were made at this meeting. Students were notified of satisfactory or unsatisfactory progress in writing. Students whose progress was unsatisfactory were asked to meet with their advisors to discuss resolution of problem areas.
Changes planned in reaction to 2005-2006 assessment of student learning data: The Department will be preparing for re-accreditation in June 2007. Writing the self-study will need to begin in December 2006, so most needed changes based on assessment feedback/strategies have been made to improve student performance and to meet the current demands of the counseling profession.
Changes implemented from 2004-2005 assessment of student learning data: Ongoing revisions of our student handbook, practicum and internship manual, and site supervisor manual; continued documentation of improved contact between the site supervisor and faculty; curriculum changes: assessment evaluation developed; a form to document that Iowa Standards are met in Internship for non-teacher certified school counseling students was developed for use in CN 597.
- The student collected and interpreted research as a means for extending and developing knowledge and understanding about teaching and learning;
- The student applied theoretical and research knowledge to practice in elementary education or reading setting;
- The student demonstrated capability in taking leadership advocacy roles in schools or Programs; and
- The student demonstrated deeper understandings of a particular area or discipline.
Direct and Indirect Measurement:
Direct Measures: Assessment of student learning is determined as
each instructor of a mid-point and capstone course records the number of
students who meet or do not meet each learning outcome. Students
receiving incompletes in capstone courses are required to complete a
written contract for the completion of coursework.
Indirect Measures: Graduate students complete the exit survey.
Outcome 1: Forty-five students (100%) in the mid-point courses demonstrated their ability to collect and interpret research as a means for extending and developing knowledge and understanding about teaching and learning. In the capstone courses, 100% of those who completed the course demonstrated their ability to collect and interpret research as a means for extending and developing knowledge and understanding about teaching and learning. Eight students in three courses received an incomplete.
Outcome 2: Forty-five students (100%) in the mid-point courses successfully applied theoretical and research knowledge to practice in elementary education or reading settings. In the capstone courses, 18 students (100%) who completed the courses applied theoretical and research knowledge to practice in elementary education or reading settings. Eight students in three courses received an incomplete. Indirect Measurement: On the exit survey, 94% of the responses indicated that graduate students believed they could apply theoretical and research knowledge to practice in elementary education or reading settings. Of the remaining responses, 6% were neutral.
Outcome 3: In the mid-point courses, 43 students (100%) demonstrated capability in taking leadership or advocacy roles in schools or programs. In the capstone courses, 18 graduate students (100%) demonstrated capability in taking leadership or advocacy roles in schools or programs. Eight students in three courses received an incomplete. Indirect Measurement: On the exit survey, 94% of the responses indicated that graduate students believed they demonstrated capability in taking leadership or advocacy roles in schools or programs. Of the remaining responses, 6% were neutral. Indirect Measurement: On the exit survey, 94% of the responses indicated that graduate students believed they could collect and interpret research as a means for extending and developing knowledge and understanding about teaching and learning. Of the remaining responses, 6% were neutral.
Outcome 4: Forty-five students (100%) in the mid-point courses demonstrated deeper understandings of a particular area or discipline. In the capstone courses, 18 graduate students, 100% of those who completed the courses, demonstrated a deeper understanding of a particular area or discipline. Eight students in three courses received an incomplete. Indirect Measurement: On the exit survey, 94% of the responses indicated that graduate students believed they demonstrated deeper understanding of a particular area or discipline. Of the remaining responses, 6% were neutral.
Additional Results from Exit Surveys:
- Students are satisfied with their educational experience at WIU (100% overall)
- Increased and varied course offerings are appreciated, including online courses.
- Students rated faculty in their area of specialization highly (100% above average)
- Students were satisfied with all course offerings with the exception of EIS 500 (6% dissatisfied) and the capstone course (7% dissatisfied).
- Advising received mixed approval. Responses indicated 13% of students were dissatisfied.
Changes planned in reaction to 2005-2006 assessment data: More effective means of direct assessment of learning outcomes will be pursued. Faculty will seek to reduce incompletes in capstone courses. The department will continue to promote the graduate program, providing relevant information and publicizing the National Board strand. To enhance the quality of advising, the current tend to centralized advising by the graduate coordinator will continue. Increased satisfaction may not be apparent until 2007.
Changes implemented from 2004-2005 assessment data: The goals of the program were revised to align with National Board Standard. Revision of the assessment plan is ongoing. Alumni surveys were discontinued due to poor response and high cost of mailing. Key courses in National Board alignment strand are currently under review. Courses are scheduled to meet students’ needs where possible. A new course, SCED 509 was added in Spring 2006. ELED 560 and ELED 566 are now offered online and in Quad Cities.
- Graduate candidates in reading will demonstrate their knowledge of content;
- Graduate candidates in reading will plan effective instruction;
- Graduate candidates in reading will successfully implement instruction; and
- Graduate candidates in reading will demonstrate effects on student learning in the K-12 classroom.
Direct and Indirect Measurement: Candidates complete the Illinois Reading Specialist Certification Test. Information about candidates’ content knowledge is also obtained through the course exams and projects. Candidates create lesson plans in several courses and during practicum experiences. Their ability to implement instruction is evaluated by department faculty during the practicum. A case study is completed during the practicum, providing information about the candidates’ ability to demonstrate effects on student learning.
Outcome 1: While the number of candidates completing the Reading Specialist Certification Test is quite small, the data thus far indicate that the average scores are above the required 240 points needed to pass the test. Candidates scored equally well in the sub areas of Language, Reading, and Literacy; Reading Instruction and Assessment; and Professional Responsibilities and Resource Management. The data indicate that candidates performed less well in the sub area of Reading Research and Curriculum Design.
Outcome 2: This data indicates that candidates can plan effective instruction for individual or small groups of children participating in a tutoring program for struggling readers. In this setting, it is particularly important to use in-depth assessment data to plan instruction that addresses the needs and characteristics of individual elementary students, as well as their interests, strengths, and weaknesses. Most candidates met these requirements at the target level of performance. While the data above reflect candidates’ abilities to plan instruction that meets individual student needs, data will also be collected in 2006-2007 on their ability to plan reading instruction that addresses the multiple characteristics and needs of students in a diverse classroom setting.
Outcome 3: Faculty observed candidates as they tutored an individual or a small group of struggling readers. At least 97% of the candidates demonstrated a “target” level of performance in using a wide range of curriculum materials in their instruction and modeling reading skills and strategies for students. Struggling readers often lack self-confidence; therefore, it is essential that candidates display dispositions that support these students and enable them to begin to think of themselves as successful “readers.” The nature of the interaction of the candidates with their tutees reflected a high level of support.
Outcome 4: At least 96% of the Reading Specialist candidates exhibited acceptable or target levels of performance in the assessment of their ability to demonstrate effects of their teaching on student learning. Candidates earned the fewest number of “target” ratings when faculty evaluated their ability to select and administer assessments to emergent readers. Candidates demonstrated greater competency in the selection and administration of appropriate diagnostic assessments when working with a middle/secondary reader than with an emergent reader. All candidates earned a “target” rating for their ability to communicate assessment information to appropriate audiences.
Changes planned in reaction to 2005-2006 assessment data: Faculty will examine the extent to which test objectives for the sub area “Reading Research and Curriculum Design” of the Certification Test for Reading Specialists are adequately addressed in the required coursework for reading specialists. Reading faculty will revise the rubric for assessing planning in RDG 574 to more specifically address the ability of candidates to utilize research-based teaching strategies and techniques in their instructional plans for struggling readers. Faculty will revise the Practicum Observation Checklist to focus to a greater extent on candidates’ application of research based practices as they tutor struggling readers. Faculty will revise the rubric for the RDG 574 Case Study so that it provides data on a greater number of competencies that are needed to demonstrate the effects of candidates’ teaching on student learning. Faculty will examine the expectations for “target” performance for all competencies to insure that criteria reflect high levels of performance.
Changes implemented from 2004-2005 assessment data: As required by our professional association, common project requirements were created to insure that candidates have similar experiences and learning opportunities regardless of the section of the course they complete. Assessments and rubrics were created that assess candidates’ ability to meet the national reading specialist standards (IRA standards). Assessments and rubrics were created to address IRA program report requirements for data collection, aggregation, and analysis.
Learning Outcomes: Objectives of the program center around the eleven Professional Teaching Standards and general expectations regarding language arts and technology.
Direct and Indirect Measurement: Portfolios provide an assessment tool for WIU faculty to utilize in evaluation of students as they exit the program and for recommendation for initial certification. Surveys of alumni of the program are also employed.
Results: Faculty review of portfolios found strong alignment between student work and program goals and objectives. Portfolios will be utilized again during this coming year. The results of student surveys indicate high overall satisfaction with the program’s ability to thoroughly prepare individuals for work in this field. They also displayed general agreement between the impressions of both the graduates and their supervisors. The surveys continue to reveal some dissatisfaction on the part of some graduates with regard to exposure to legal issues in the field and knowledge of basic research methods, as well as advising. This is not surprising, however, since the changes in course work that address these issues have only recently been implemented.
Feedback Loop: While assessment is continuous and rigorous for MAT candidates, there is a need to address the issue of advising.
- Models and philosophies of student affairs practice;
- Historical and philosophical roots of the student affairs profession;
- Goals and activities involved in the various student affairs functional areas;
- Characteristics, needs, and values of college student from diverse backgrounds;
- Outcomes of college for students; 6) Growth and development of students;
- Organizational development and administrative practice as they apply to higher education and student affairs;
- Basic counseling theories and techniques as they apply in student affairs settings;
- Principles of group dynamics as they apply to student affairs;
- Historical development of higher education as well as the current role, functions, and structure, and issues facing higher education;
- College environments and their impact on students;
- Legal an ethical issues facing the student affairs profession.
Direct and Indirect Measurement: The program continues to assess for cognitive, affective, and behavioral outcomes utilizing both qualitative and quantitative measures. In addition to the all other measures listed in the established assessment plan, two valuable activities were the continuing evaluation of student portfolios intended to demonstrate alignment with programmatic objectives and the use of annual surveys sent to graduates and current supervisors.
Results: Faculty review of portfolios found strong alignment between student work and program goals and objectives. Portfolios will be utilized again during the coming year. The results of the surveys indicate high overall satisfaction with the program’s ability to thoroughly prepare individuals for work in this field. They also displayed general agreement between the impressions of both the graduates and their supervisors.
Feedback Loop: The transfer of the summer internship occurred during the summer of 2005. The need for additional and consistent program coordination support remains an issue.
- The opportunity to explore and investigate educational and related relevant topics, skills, programs and issues outside of one’s current area of professional focus.
- The opportunity to grow personally and professionally through in-depth analyses of issues and problems in contemporary education and related areas of concern.
- The opportunity to construct a discipline grounded knowledge base regarding contemporary psychological, sociocultural, and philosophical theories and research as they pertain to educational problems and related areas of concern.
- The opportunity to develop the knowledge and skills necessary for the consumption, analysis and evaluation of scholarly literature pertaining to specific programs and problems within one’s profession.
- The opportunity to extend, activate and apply ones knowledge and skills through either an action-based or theoretically driven culminating investigation of a student-selected problem, issue or program.
Direct and Indirect Measurement: Faculty evaluation of student papers, projects, and tests; the on-site supervisor’s assessment of the practicum and internship experiences, the percentage of students who are successfully obtaining positions in student affairs, the capstone course, faculty review of student progress, graduate surveys. The Department continues to assess learners at several stages in their program. A 2.75 cumulative GPA is required for admission, and learners must maintain the GPA established by the Graduate Office. Overall progress is assessed at the submission of a degree plan or at 15 hours. A draft portfolio or a research proposal must also be submitted before students have completed 28 hours in the program. A fully accepted portfolio or a completed action research project or thesis serves as the final assessment.
Results: In the past year, 100% of students submitting portfolios, theses, or action research projects successfully met expectations. Quantitative ratings indicate high levels of satisfaction with the graduate program. Qualitative comments are in the process of being transcribed.
Feedback Loop: Assessment of progress by students as they move through the program is traditionally accomplished by monitoring grade point average and number of C grades. The development of a more detailed and formal process of assessment of progress is a priority concern for the coming year.
Learning Outcomes: none listed.
Direct and Indirect Measurement: Informal exit interviews with interns.
Of the 1862 certification applicants that the State of Illinois tested in 2005, 20 (1%) were WIU EDL Master’s degree graduates/students. One of the WIU/EDL student failed the exam, while 244 (13%) failed across the state. The WIU/EDL student who failed the exam was an undergraduate English major who took the wrong test and not an EDL student, effectively resulting in a 100% pass rate. When sub-scores were examined a significant weakness was noted in test area 3.
Survey indicates that students feel prepared to lead a building, were pleased with the faculty, program format (weekends, registration process) – missing was their sense of the degree being cost-effective. Students were concerned with curriculum, communication between students and faculty outside of class, the need to have all assignments returned in a timely manner, and consistency of expectations across courses.
Feedback Loop: When sub-scores were examined a significant weakness was noted in test area 3. The department needs to examine the weakness identified in test area 3.
Learning Outcomes: none listed.
Direct and Indirect Measurement: Students in selected classes were surveyed regarding program effectiveness and State Certification Testing.
Results: Of the 201 certification applicants that the State of Illinois tested in 2005, 24 (11%) were WIU EDL Educational Specialist graduates/students. 100% of the WIU/EDL students passed the exam. Three sub-tests that are aligned to the standards are also reported. The reported sub-test scores were very strong with only 3 scores out of 18 not meeting/exceeding the state. There was no definable pattern in the scores to warrant programmatic impact/change. Two scores exceeded the State’s score by more than 20 points but again no discernable pattern was found.
Survey indicates that students thought program strengths included hands-on classes with practical application of theory, professors who are knowledgeable about problems facing schools, and students noted that a new strength included relevant readings.
Feedback Loop: New faculty orientation will include training on the standards, assessment plan, data collection procedures.
Learning Outcomes: The student will
- demonstrate proficiency in his/her area of specialization;
- demonstrate proficiency in communication skills;
- use instructional design principles to design or evaluate instructional and training programs, or the instructional materials developed for them.
Direct and Indirect Measurement: The major assessment tool is the student’s culminating project. Students may choose to complete a thesis, an applied research project, or a portfolio presentation that demonstrates proficiency in his/her area of specialization. Communication skills will be assessed using the writing examples present in the student’s chosen project. Criteria for each of those experiences are attached. The criteria detail expected learning outcomes and includes methods of measuring those outcomes.
Results: Twenty-one portfolios, six applied projects, and one thesis were completed this year. No other results of student learning listed.
Feedback Loop: As a result of the reflection papers included in the portfolios, some recommendations are being considered, including curriculum adjustments and ways to increase support for students.
Learning Outcomes: Students will demonstrate exit level competency by completing a capstone experience.
Direct and Indirect Measurement: Capstone experiences: Thesis, comprehensive exam, synthesis paper of internship, or approved certification examination.
Results: Five students completed a thesis. Three students enrolled in comprehensive exam, but none successfully completed it. Six students successfully completed all requirements for synthesis paper and internship. Four students successfully passed certification exams – three for the NSCA and one for ACSM.
Feedback Loop: Graduate faculty have discussed increasing the number of required courses for the Kinesiology degree to discourage students from taking most of the coursework for the Sport Management degree and then changing to Kinesiology. Students that have done so in the past have had difficulty passing the comprehensive examination capstone experience and are not prepared to meet one of the other options for the capstone experience. We expect to revise the Kinesiology degree program during the 2006-07 year to address this issue. With the addition of three new faculty members during the 2005-06 academic year and five more in the 2006-07 year, more graduate students may choose to complete a thesis as a capstone experience.
Learning Outcomes: Students will demonstrate exit level competency by completing a capstone experience.
Direct and Indirect Measurement: Capstone experiences: Thesis or synthesis paper of internship.
Results: No students enrolled in the thesis option. Nineteen students successfully completed all requirements for the synthesis paper and internship.
Feedback Loop: Faculty will re-consider requiring an internship for both the thesis and non-thesis options. An alternative to the internship may be considered for students that are currently working in graduate assistantships that provide them an opportunity to gain “real world” experiences in areas of athletic/sport administration/management programs. With the addition of two new faculty members over the last two years with expertise in sport management, faculty that teach in this degree program will re-evaluate the curriculum and may propose possible changes.
- Students will demonstrate mastery of the knowledge gained in the LEJA major, including the areas of law, policing, corrections, courts and security;
- Students are expected to demonstrate the ability to apply their knowledge of law, policy, corrections, courts or security in practical/practitioner settings.
Direct and Indirect Measurement: Thesis or comprehensive examination, focus groups and/or professional advisory groups (PAG), alumni survey.
Results: The assessment results have demonstrated only one significant weakness for criminal justice professionals in the area of an undergraduate deficiency in quantitative techniques, understanding, and skills. The department has exceeded recommendations from past assessment activities. These recommendations, such as the ability to write at an advanced level, have been demonstrated in our graduate students’ thesis projects and papers. Another area of accomplishment is our Police Executive Administration Certificate program, which has experienced a 90% completion rate. .
Feedback Loop: Our graduate students have been given the opportunity to take LEJA’s undergraduate quantitative techniques class to satisfy this deficiency. We have revised curriculum based on assessments by students, alumni, faculty and administrators, and the future needs of the criminal justice field. Several new sub-objectives are being addressed. These include preparing students for the possibility of completing a doctorate, compiling and disseminating research in the field, and/or becoming a teacher/educator.
Learning Outcomes: Learning outcomes referenced (see below), but none identified.
Direct and Indirect Measurement: Advising, student performance evaluation, informal review, formal interview, thesis, internship, formal exit interview, and alumni feedback.
Results: Strengths identified through assessment activities included student perception of quality instruction; diversity of graduate degree candidates; positive mix of theory and practice in the graduate program; relationship with the Peace Corps fellows Program; and relatively large number of assistantship opportunities. Weaknesses identified through assessment activities included limited continuity of an "entering class;" scheduling difficulties; inconsistent levels of classroom rigor; and potentially unmet demand in the Quad Cities region. Classroom performance was indicative of positive learning environments. Feedback from alumni, employers and doctoral program representatives validated high levels of student preparedness. Graduate internship performance, as self-reported by interns, evaluated by practicing professionals, and assessed by university supervisors, validated high levels of competency in relation to stated learning outcomes.
Feedback Loop: none listed.