A Comprehensive System for the Evaluation of Schools	William J.Webster Dallas Independent School District

The District uses the CIPP evaluation model (Stufflebeam, et.al., 1971). Figure 1 provides a schematic depicting how the school improvement process functions within the parameters of site-based decision-making. Each school receives an annual needs assessment specifying school levels on important outcome variables. These data are available in July for use in developing the School Improvement Plans and District Improvement Plans. The important outcomes of instruction are determined through Districtwide assessments of all of the groups involved in the educational process. School program planning is implemented at the school level by the School Community Council. Planning focuses on determining the best method to proceed from current levels of important outcomes to desired levels of those outcomes and culminates in the production of a strategic plan, the School Improvement Plan. Standards of performance are established based on the District’s most effective schools from the previous year.

Specifically, once the needs assessment has identified needs, school staff must prioritize those needs and focus on reducing the discrepancy between desired and existing outcomes by establishing goals for those needs that receive highest priority. Schools are aided in this process by data on how well the District’s most effective schools have done in achieving specific goals of instruction. Once priorities are established, schools must determine methods of resource utilization for accomplishing program goals.

Site-based management does not assume that local building staffs necessarily know how to solve all of their problems. It does, however, place decision-making responsibility and accountability at the local level. Central staff become resources to the schools whose function it is to provide viable alternatives to solving school problems. The principal is ultimately responsible and accountable for meeting the important objectives of instruction. Central staff is responsible and accountable for providing viable alternatives for consideration by school staff.. This procedure is the input evaluation phase of the school improvement process and will only work if Central Office Divisions are competent and can supply the needed expertise. If the needed expertise does not reside in the appropriate Central Office Divisions, schools will not request needed services and the entire system will probably fail to produce the continuous improvement that it is designed to accomplish.

After the collection of relevant input information feeding a preliminary program planning stage, school level decision-making groups determines whether or not sufficient resources are available to make the desired changes. Quite often, sufficient resources are not available and some compromise is necessary. In many cases, the lack of resources is not limited to the realm of cost and political feasibility, but rather stems from an insufficient base of knowledge. Thus, educators are often in the position of having sufficient material resources but insufficient information resources. Once these decisions are made, the School Improvement Plan is complete.

The program implementation phase is then entered and each individual school staff is responsible for providing continuous formative feedback relative to program implementation. This feedback falls primarily into two categories -- process evaluation and interim product evaluation. Process evaluation has three major objectives: (1) the detection or prediction of defects in procedural design or its implementation during program implementation stages, (2) the provision of information for programmed decisions, and (3) the maintenance of a record of the implementation procedure as it occurs. Thus, process evaluation information keeps the School Community Council informed of the extent to which program implementation conforms to specifications and, from an evaluation standpoint, guards against the evaluation of fictitious events. It also provides a record of implementation that can be cross-indexed to program effect.

Much of the process evaluation which was at one time implemented by evaluation personnel now must be implemented at the local level. This is consistent with the accountability emphasis that is currently the philosophy of District management and the community. Since process evaluation is extremely expensive and is crucial to the improvement of instruction, most of it is currently implemented by school staff as part of their ongoing instructional delivery system. Process evaluation is thus focused on obtaining information for improvement.

Interim product evaluation provides periodic feedback to the schools relative to the attainment of specific sub-objectives during the implementation phase. Thus, process and interim product evaluation reports inform program management as to implementation and goal attainment levels while program adjustments are still feasible. Much of the interim product evaluation can be done through prortfolios of student work, performance testing, protocol analysis, and teacher-made tests; measures that are not available through systemwide data. In cases where serious needs are identified by interim product evaluation reports, tactical plans are developed as supplements to the School Improvement Plan to meet these needs.

An important part of the Dallas Independent School District’ interim product evaluation is the Diagnostic Skills Profile (DSP). These instruments are focused on the objectives of instruction in reading and mathematics and provide systematic feedback to teachers on how well their students perform relative to those objectives. The DSPs can be administered over a one month period, produce profiles of student learning, and are scored and returned to the teacher within two weeks. They are administered by teachers in the Fall and Spring in grades K-9, in the Fall in grade 10, and are available in Spanish in grades K-6. Results are used strictly for instructional improvement. District teacher groups were the major impetus for the development of this system.

Local school staffs are encouraged to design, implement, and interpret action research studies. With the movement of the District to site-based management and the related reduction of Central Office Staff, it is impossible to supply school staffs with information produced centrally pertaining to all of their many and varied needs. Action research is a process for problem-solving that is designed and implemented at the local building level. It is a process of taking and studying action and its corresponding consequences so that more effective action may be taken (Lewin, 1946; Town 1973). Expressed sequentially, action research requires a continuous recycling through four steps: (1) identification of needs, (2) development of plans of action to address these needs, (3) execution of these plans of action, and (4) formative evaluation of these plans. In open organizations such as schools, the strength of action research lies in its implementation by the organizations' members in their respective work sites. In effect, members of the organization actively learn while they study problems in contexts that they generally perceive as relevant and important. The results are used to supplement the more formal information available from the District's Evaluation Department.

Upon completion of a given cycle of program implementation, usually one year, a series of summative product evaluation reports are prepared. These reports take the form of the Special Report on Pupil Achievement (REIS91-102), a school-level report that provides up to four years of disaggregated data on all relevant outcome and input variables and is used to determine whether or not schools met their School Improvement Plan goals for the previous year and to establish goals for the coming year, School Effectiveness Indices, and program evaluation reports disaggregated by school. In addition, data are provided in a user friendly format that addresses the degree to which schools have met or failed to meet their School Improvement Plan goals. These reports, as well as relevant action research studies compiled by school staff, become the needs assessments for the next year's program adjustments.