Standard 4 - Continuous Improvement
The education unit develops and implements a comprehensive continuous improvement process at both the unit level and within each of its programs that identifies program and unit effectiveness and makes appropriate modifications based on findings.
Note: The materials and descriptions offered here have been adapted to discuss initial and advanced credential programs only. Advanced degree program descriptions (i.e., data sets, collection procedures, etc.), while a part of the College of Education Assessment system, are not included for the purposes of this report.
4.1 Both the unit and its programs regularly and systematically collect, analyze, and use candidate and program completer data as well as data reflecting the effectiveness of unit operations to improve programs and their services.
COE Assessment System : A Comprehensive Continuous Improvement Process
Governance
The Dean of the College of Education (COE) works with the Associate Dean and the Director of Accreditation and Assessment to ensure the College is meeting its ongoing accreditation and assessment requirements. The Associate Dean, the Director of Accreditation and Assessment, the Assessment Committee and the Administrative Analyst/Specialist comprise the core assessment and accreditation team of the unit. The Associate Dean and Director of Accreditation and Assessment co-chair the Assessment Committee, work with all programs, keep the Dean and the Leadership Team current on all accreditation and assessment matters, and serve as the College liaisons for all University assessment and accreditation needs.
Overview of Accreditation and Assessment Requirements
The College of Education is responsible for the management and coordination of accreditation and assessment requirements for all university credential programs that prepare teachers and other school professionals to work in pre-kindergarten through twelfth grade settings, regardless of where these programs are administratively housed. All credential programs are included in what is known as the Professional Education Unit. The Professional Education Unit at CSUF includes credential programs from five departments within the COE, and two programs housed in other colleges, or university programs.
| College - Department or Program | Credential Level | Reviewing Agency* |
|---|---|---|
| COE - Educational Leadership | Advanced | CTC |
| COE - Elementary & Bilingual Education | Initial/Advanced | CTC |
| COE - Literacy & Reading | Advanced | CTC |
| COE - Secondary Education | Initial/Advanced | CTC |
| COE - Special Education | Initial/Advanced | CTC |
| CSUF Extension - Induction (General Educ) | Advanced | CTC |
| COMM - Communicative Disorders | Advanced | ASHA/CTC |
| HHD - School Nurse | Advanced | CCNE/CTC |
*CTC = Commission on Teaching Credentialing
*ASHA = America Speech and Hearing Association
*CCNE = Commission of Collegiate Nursing Education
Note: All colleges that house credential programs are also reviewed as part of the University accreditation process completed by the Western Association of Schools and College -Senior College and University Commission (WSCUC).
The COE uses a comprehensive system to evaluate candidate and program performance for all credential programs. Those programs which are not housed in the COE take primary responsibility for conducting candidate performance assessments and program evaluation processes according to their home college systems. However, these programs are supported and monitored by the unit assessment team for all CTC assessments and reporting requirements, which include:
- Annual COE Closing the Loop Program Reports - COE programs only
- Annual COE Strategic Goal Reports - COE programs only
- Annual Reports to the Office of Assessment and Institutional Effectiveness - All Programs
- Annual CTC Title II Reports - Initial programs only
- Annual CTC Data Submission Reports - All Programs
- CTC Program Review Reports - All Programs
- CTC Unit Common Standards Report - All Programs
College of Education - Assessment System Design
The assessment system is designed to manage the flow of data collection and analysis and works to close the assessment loop by reporting results that are used to determine strengths and weaknesses for program and unit improvement. As designed, the system supports on-going data-driven program reviews and unit studies that examine: (1) alignment of curriculum, instruction, and assessment with unit, state, and professional standards, (2) efficacy of courses, field experiences, and programs, (3) candidate’s development of content knowledge, skills and dispositions that lead to effective student learning or school leadership, and (4) effectiveness of program and unit operations.
Methodology
The system allows for the collection of multiple data sources, both direct and indirect, to monitor candidate performance and to manage and improve program effectiveness. These data sets are analyzed and used to determine if candidates have met the requirements necessary to matriculate through specified program levels and are collected at four (4) transition points:
Transition Points for Initial and Advanced Credential Programs
Initial Programs
- Admissions to Programs
- Admission to Initial Clinical
- Admission to Final Clinical Practice
- Exit from Program
- Post Program
Advanced Programs
- Admissions to Programs
- Program Continuation
- Qualifying Culminating Experience
- Exit from Program
- Post Program
See each program’s Transition Point Chart
for specific assessments and requirements.
Data Collection
The operational success of the assessment system designed for data collection, analysis, and use is dependent on the coordination of many individuals, offices, and agencies both internal and external to the professional education unit. This coordination is detailed in the following tables, outlining individual responsibilities for the collection of data, the timeline for collection, who receives data reports, and ways data results are used.
-
Collection and Flow of Integrated Data Sets for Initial Program Candidates
-
Collection and Flow of Integrated Data Sets for Advanced Program Candidates
Closing the Loop
Raw data are collected by the COE Data Analyst, analyzed, charted, and reported back to departments/programs through dropbox. Department chairs and/or designees examine the data. Findings are used to complete annual reports using the Closing the Loop Report template.
These reports note program strengths and challenges, and goals for program improvement.
Closing the loop reports
are shared with faculty and appropriate stakeholders per department protocol. Findings and identified improvement goals from these reports are also submitted and reviewed by the Office of Assessment and Institutional Effectiveness
to meet university assessment requirements.
Raw data are archived by the Data Analyst and used provide data for different purposes dependent on program, department, and college need.
Candidate assessment results are shared with program and unit leadership as part of the closing the loop process. These data along with other data sources are used to identify program and unit strengths and challenges and determine improvement goals. These sources are detailed in the following tables, outlining individual responsibilities for the collection of data, the timeline for collection, who receives data reports, and ways data results are used.
Collection and Flow of Integrated Data Sets for Program and Unit
Continuous Improvement of the System
The Assessment Committee is responsible for the continued evaluation and modification of the assessment system to ensure that it remains viable, comprehensive, informative, and effective. Committee members include assigned representatives from all unit programs who serve as liaisons for communication between the unit, program faculty, and other stakeholders.
Since our last accreditation and development of new strategic goals, there have been many changes that affect the assessment system. As a part of the continuous improvement process recent and ongoing modifications* include:
- Conceptual Framework Revision to better reflect the unit’s focus on diversity, equity,
and inclusion, including three guiding outcomes (CFOs)
- Development of specific student learning outcomes for each program that (PSLOs) align with the new Conceptual Framework outcomes.
- Development of program specific key assignments and rubrics to evaluate program student learning outcomes.
- Improving the process of monitoring the assessment of programs outside the COE to facilitate the completion of CTC and university requirements and
- Planning the redesign of the unit infrastructure to better serve candidates and faculty through all program transition points.
*Modification Timeline
on improvements to the components of the Assessment System
4.2 The continuous improvement process includes multiple sources of data including: 1) the extent to which candidates are prepared to enter professional practice; 2) the quality of the educational services provided to students during supervised practice; and 3) feedback from key stakeholders such as em ployers and community partners about the quality of the preparation.
As described in the initial and advanced program transition points and data charts, the assessment system includes opportunities for candidates to assess their quality of preparation at various times throughout the program. All results are used for program and unit review and improvement.
Initial Programs:
-
Placement Survey
: During the program candidates give feedback quality of preparation offered in the various stages of clinical practice placements, including the quality of mentorship received.
-
Supervisor SOQ ratings:
After each clinical practice placement candidate’s give feedback on the quality
- of their site-based supervisor in helping to prepare them as future teachers.
-
CSU Chancellor’s Office Exit Survey
: As candidate’s exit their programs, the exit survey gathers candidate’s opinions of the quality of their preparation as newly credentialed professionals in their fields.
-
CSU Year-Out Chancellor’s Office Graduate Survey:
The graduate survey provides the opportunity for past candidates to again rate how well they felt prepared, after being in their teaching positions for one-year.
- In addition to candidate perception of readiness, employers of our candidates are surveyed after the beginning educators first- year of employment. These results are used to inform program review and improvement.
-
CSU Year-Out Employer Survey:
Employers of first-year teachers from CSU Fullerton programs are offered the opportunity to provide feedback on the quality of preparation the program provided their new teacher(s) as they entered the field.
Surveys from 2018-2021 show that over 90% of employers would recommend our teacher preparation programs to future educators.
Advanced Programs:
Exit Survey – As candidate’s exit their programs, the exit survey gathers candidate’s opinions of the quality of their preparation as newly credentialed professionals in their fields. For COE candidates these surveys are designed and administered by the unit assessment team. The School Nurse Program designs and administers their own exit survey. Speech-Language Pathology conducts in-person exit interviews.
-
COE Exit Survey – Reading & Literacy, Administrative Leadership (PASC and CASC), Education Specialist Induction
-
School Nurse Exit Survey
-
Induction: Internal exit survey beginning 2023
Graduate/Alumni Survey – Most programs* offer alumni an opportunity to provide feedback on well they feel their program prepared them after working in the field. For COE candidates these surveys are designed and administered by the unit assessment team. Outside programs design and administer their own surveys. These are sent out every 1-3 years dependent on program. *School Nurse program does not survey graduates.
-
COE Graduate Year-Out Survey– Reading & Literacy, Administrative Leadership (PASC and CASC), Education Specialist Induction
-
Speech-Language Pathology Graduate Survey
-
Induction: Internal graduate/alumni survey beginning 2023
Employer Survey – Most programs* offer employers of our graduates an opportunity to provide feedback on how well they feel our programs prepared our graduates for their educator positions. For COE candidates these surveys are designed and administered by the unit assessment team. Outside programs design and administer their own surveys. These are sent out every 1-3 years dependent on program. *Speech-Pathology and School Nurse Programs do not survey employers.
Examples of how data has been used to inform and lead to program improvement can be found in the Data Based Changes Chart
provided. These charts represent some, but not all, program changes which have come from the collection and analysis of data.
