Part 1: FAST Executive Summary
Study Development and Review Process
Texas fared better than many other states during the recent recession. Nevertheless, when the Texas Legislature convenes in January 2011, its members will face some difficult budget decisions.
School districts that operate efficiently – achieving strong academic performance while keeping costs low – offer valuable examples for other districts.
Public education spending consumes a large and expanding share of state and local revenues. Its growth has consistently outpacing enrollment growth and inflation. Some school districts, however, manage to achieve strong student performance while keeping spending growth to a minimum.
School districts that operate efficiently – achieving strong academic performance while keeping costs low – offer valuable examples for other districts. Widespread adoption of their strategies could help our state and local governments slow the rapid growth of educational spending while ensuring that Texas high school graduates are ready to succeed in college or the work force.
Project Overview
The 2009 Legislature’s House Bill 3 directed the Comptroller to “identify school districts and campuses that use resource allocation practices that contribute to high academic achievement and cost-effective operations.”1 In response, the Comptroller’s office created the Financial Allocation Study for Texas (FAST) to examine district and campus resource allocation – and the relationship between these allocations and student achievement.
This proved to be a complex task, as many forces influence student learning, including factors both in and outside school. Similarly, the cost of education is influenced by many factors, some beyond the districts’ control.
Expert Consultants
The research team began by assessing the data sources needed to perform the study required by the 2009 Legislature’s House Bill 3 (H.B. 3). This assessment involved collaboration between the Comptroller’s office and recognized experts in the field, including researchers at some of the state’s top institutions of higher education.
- The University of Texas at Dallas (UTD) Texas Schools Project provided detailed student data compliant with the federal Family Educational Rights and Privacy Act, allowing for analyses of student performance that cannot be made with publicly available data from the Texas Education Agency (TEA).
- UTD’s Dan O’Brien Ph.D., Jim Parsons and Kurt Beron, Ph.D., worked with Comptroller staff to develop new academic outcome measures based on scores from the Texas Assessment of Knowledge and Skills (TAKS) exams. These new indicators measure student academic growth from year to year, allowing for more accurate assessments of student progress.
- Lori Taylor, Ph.D., of Texas A&M University provided expertise on school district costs and produced groupings of fiscal peers for comparative purposes.
- Harrison Keller, Ph.D., of the University of Texas at Austin provided guidance on educational policy and assisted in the development of the study’s methods.
Texas Education Leaders
A Superintendent Advisory Committee representing school district leaders from across the state provided valuable input and practical suggestions for this study. Superintendents and their staffs formed working groups to discuss a series of topic areas important to this study, including:
- school cost drivers, including those outside district control;
- useful and reliable indicators of student performance;
- ways in which districts and campuses can be grouped for comparison; and
- the identification of best practices in school operations.
The Comptroller’s Superintendant Advisory Committee included:
Superintendant | District |
---|---|
David Anthony, Ed.D. | Cypress-Fairbanks ISD |
Frank Belcher (now retired) | Canadian ISD |
Keith Bryant | Bullard ISD |
Gene Buinger, Ed.D. | Hurst-Euless-Bedford ISD |
Jesus Chavez, Ed.D. | Round Rock ISD |
Mike Feinberg | KIPP Houston (Charter) |
Cynthia Garcia, Ed.D. | Driscoll ISD |
Lorenzo Garcia, Ed.D. | El Paso ISD |
Karen Garza, Ph.D. | Lubbock ISD |
Roland Hernandez, Ph.D. | Waco ISD (now with Corpus Christi ISD) |
Michael Hinojosa, Ed.D. | Dallas ISD |
Daniel King, Ph.D. | Pharr-San Juan-Alamo ISD |
Duncan Klussmann, Ed.D. | Spring Branch ISD |
Richard Middleton, Ph.D. | North East ISD |
Sylvester Perez, Ed.D. (now retired) | Midland ISD |
Carrol Thomas, Ed.D. | Beaumont ISD |
The Comptroller also met with a school board advisory group of Texas school trustees to discuss the study methodology and provide direction:
Trustee | District |
---|---|
Jim de Garavilla | Silsbee ISD |
Karen Ellis | Richardson ISD |
Carol Fletcher, Ph.D. | Pflugerville ISD |
Israel Hinojosa | Jim Hogg County ISD |
Mark Miller | Sealy ISD |
Lynn Ramsey | Shamrock ISD |
Sarah Winkler | Alief ISD |
Cindy Warner | Coppell ISD |
The FAST project used technical teams and peer-review panels to validate its methods and findings.
The research team met with teachers, principals and other education groups to discuss and address their concerns regarding this project. Those who contributed include:
- Association of Texas Professional Educators
- Bill and Melinda Gates Foundation
- Michael & Susan Dell Foundation
- Regional Education Service Center diectors
- Texas Association of School Administrators
- Texas Association of School Boards
- Texas Association of School Business Officials
- Texas Association of Secondary School Principals
- Texas Charter School Association
- Texas Classroom Teachers Association
- Texas Education Agency
- Texas Elementary Principals and Supervisors Association
- Texas Federation of Teachers
- Texas Institute for Education Reform
- Texas High Schools Project
- Texas State Teachers Association
“While our reform efforts at the state and national level have rightly focused on student achievement, we must now look at how well we serve students in the context of how well we use precious tax dollars. This project… will help spur needed improvements in the use of resources so that they can be best deployed to improve education for all Texas students.”
– Margaret Spellings, Former U.S. Secretary of Education and CEO of Margaret Spellings & Co.
Independent Review
The FAST project used two types of teams, technical teams and peer-review panels, to validate its methods and findings.
The technical teams provided guidance on the development of academic and financial performance indicators. These teams primarily comprised Texas academic and financial experts, including:
Academic Measures Team
- Chrys Dougherty, Ph.D., National Center for Educational Achievement
- Jon Lorence, Ph.D., University of Houston
- Jim Van Overschelde, Ph.D., Texas Education Agency (now with E3 Alliance)
- Lori Taylor, Ph.D., Texas A&M University
- Dash Weerasinghe, Ph.D., Plano ISD
- Victor Willson, Ph.D., Texas A&M University
- Gloria Zyskowski, Ph.D., Texas Education Agency
Financial Measures Team
- Tom Canby, Texas Association of School Business Officials
- Jim Dyer, Ph.D., McCombs School of Business, University of Texas at Austin
- Timothy Gronberg, Ph.D., Texas A&M University
- Kathy Hayes, Ph.D., Southern Methodist University
- Jim Parsons, Texas Schools Project, UT-Dallas
- R. Anthony Rolle, Ph.D., Texas A&M University (now with University of South Florida)
After the study methods were developed, they were submitted for analysis to a pair of independent peer review panels, one for academic progress and the other for financial and efficiency measures. These panels provided recommendations and comments on the draft methodologies. Their members included:
Peer Review Panel on Academic Measures
- Joan Herman, Ed.D., University of California, Los Angeles
- Michael Podgursky, Ph.D., University of Missouri
- Steven Rivkin, Ph.D., Amherst College
- William Sanders, Ph.D., SAS Institute
Peer Review Panel on Financial Measures
- William Duncombe, Ph.D., Syracuse University
- Stephen Frank, Ph.D., Education Resource Strategies
- Shawna Grosskopf, Ph.D., Oregon State University
- Jennifer Imazeki, Ph.D., San Diego State University
- Andrew Reschovsky, Ph.D., University of Wisconsin
- Amy Schwartz, Ph.D., New York University
Methodology
The FAST team produced methods to place Texas campuses and districts on a level playing field for comparisons of academic performance and spending. Any such measure must weigh certain factors, such as geography and demographics, which are beyond schools districts’ control. The measures of academic growth used in this study control for several of these factors.
The FAST report and an online web tool also provide multiple “lenses” through which to examine campus and district performance. Districts and campuses with similar characteristics can be grouped together for comparison across dozens of academic and financial performance indicators.
The FAST methodology also adjusts for a number of demographic, economic, geographic and other characteristics that affect academic performance and spending. The FAST methodology for measuring academic performance includes 32 control variables while the spending methodology adjusts for eight.
Academic Measures
The FAST team produced methods to place Texas campuses and districts on a level playing field for comparisons of academic performance and spending.
- The FAST analysis uses a value-added model that measures achievement by controlling for the varying characteristics of students, campuses and districts to estimate how much a district or campus contributes to student learning.
- Using the value-added model, the FAST report measures annual progress in reading/English Language Arts and math.
- The research team developed a composite academic progress rating by combining measures of math and reading progress.
- All academic progress measures are shown in percentiles ranging from one to 99, with 99 representing the most academic progress relative to other districts in the state.
- By controlling for factors outside of teaching that influence student performance, the research team is able to compare academic progress among districts and campuses on a level playing field.
- To further ensure fair comparisons, academic progress is averaged over three years.
- Thus, if a district has a math progress score of 60, it means that during the last three school years, the district’s students showed as much or more progress on math TAKS than 60 percent of districts statewide.
Spending Index
To alleviate the possibility of one-year anomalies in spending having undue influence, the research team used three-year averages for spending comparisons.
- Texas districts and campuses operate in a variety of “cost environments” – socioeconomic and geographic characteristics that influence the cost of education and are often beyond a school district’s control.
- The research team evaluated financial data for each district and campus by comparing them to “fiscal peers” – districts and campuses that operate in similar cost environments, are of similar size and serve similar students.
- To ensure the validity of financial comparisons, the research team employed a technique called propensity-score matching to identify up to 40 peers for each Texas school district and campus, based on common cost factors such as wages, school district size and geography and student demographics.
- After a group of fiscal peers is identified for a school district, the district is then assigned a “spending index” based on its spending relative to its fiscal peers.
- In creating the spending index, FAST compares district core operating expenditures per pupil, adjusted for geographic wage variations.
- A district’s spending index is determined by identifying the spending quintile in which it falls relative to its fiscal peers. The quintiles range from very low to very high, with very low indicating the lowest relative spending in the fiscal peer group and very high representing the highest.
- A similar process is used to create a spending index for each campus.
- There are, however, no uniform standards for districts to follow when allocating expenses to their campuses.
- Some districts allocate most of their central administration activities to specific campuses, while others do not.
- Operating expenditures for campus-related activities (instruction, instructional services, school leadership and student support services) are more consistently defined across campuses.
- Because of this, the FAST spending index for campuses is based only on campus-related expenditures.
- To alleviate the possibility of one-year anomalies in spending having undue influence, the research team used three-year averages for spending comparisons.
- Finally, the review team created a FAST rating that integrates the academic progress and spending measures to identify districts responsible for strong and cost-effective academic growth.
- Each district has received a FAST rating ranging from one to five stars, with half-star increments (Exhibit 1).
- A five-star district has a composite progress rating between 80 and 99 and a spending index of “Very Low.”
- A one-star district has a composite progress rating below 20 and a spending index of “Very High.”
- A district with “Very High” spending and a composite progress rating of 80 to 99, and a district with “Very Low” spending and a composite progress percentile below 20, both earn three-star FAST ratings.
- This rating does not make any judgment of the relative value of spending versus academic progress, recognizing that different school districts have different priorities and different constraints.
FAST Rating
Go Behind the Numbers
A detailed look at the methodology and calculations used to produce the FAST ratings is available in the FAST Study Appendix.(PDF, 4.7MB)
Exhibit 1
Academic Progress Percentiles + Spending Index = FAST Ratings
Spending Index | ||||||
---|---|---|---|---|---|---|
“Very High” | “High” | “Average” | “Low” | “Very Low” | ||
Composite Academic Progress Percentile |
80-99 | 3 STARS![]() |
3½ STARS![]() |
4 STARS![]() |
4½ STARS ![]() |
5 STARS![]() |
60-79 | 2½ STARS![]() |
3 STARS![]() |
3½ STARS![]() |
4 STARS![]() |
4½ STARS ![]() |
|
40-59 | 2 STARS![]() |
2½ STARS![]() |
3 STARS![]() |
3½ STARS![]() |
4 STARS![]() |
|
20-39 | 1½ STARS![]() |
2 STARS![]() |
2½ STARS![]() |
3 STARS![]() |
3½ STARS![]() |
|
LESS THAN 20 | 1 STAR![]() |
1½ STARS![]() |
2 STARS![]() |
2½ STARS![]() |
3 STARS![]() |
All links were valid at the time of publication. Changes to web sites not maintained by the office of the Texas Comptroller may not be reflected in the links below.
- 1Texas H.B. 3, 81st Leg., Reg. Sess. (2009).