Filipe Recch, Ph.D. candidate in International Comparative Education/SHIPS at Stanford University
Management practices and sub-state administration value-added – evidences from the state of São Paulo/Brazil
School systems are complex organizations that influence student learning. Decisions regarding educational policies along with their implementation through school district level management affect school quality and consequently the effectiveness of the system. Thus, understanding the degree to which different policies and their implementation affects student learning is valuable.
The impact of school leadership and school management on school effectiveness and student achievement has been widely discussed by scholars. While the main results show that there is a significant, yet sometimes indirect, effect of leadership and management practices on student achievement; others suggest that these studies fail to capture the independent effects of school management (Branch, Hanushek, and Rivkin, 2013; Fuller and Hollingworth, 2014; Grissom, J. A., Kalogrides, and Loeb, 2015; Hallinger and Heck, 2004, Witziers, Bob, Bosker, and Krüger, 2003). A lot of emphasis is also put on the role of national governments and its effects on educational development (OECD, 2016). However, middle level management’s – state, regional or district levels – influence in improving schools’ and students’ results is often overlooked (Carnoy, M., and Rothstein, R. 2015). Yet, as an essential part of policy implementation, mid-level management may directly influence policy outcome. This paper’s goal is to provide evidence that middle level management is indeed central to the development and delivery of effective educational policies and schools.
Brazilian states are relevant and useful sites to study this topic because of the organization of their educational systems (Marenco, 2017). In Brazil, public schools are governed either by a state administration or a municipal one. Therefore, within each state, and sharing overlapping territory, both municipal and state administration units have separate control of their parts of public schooling. While municipal school policies and management vary across municipalities, state-run schools are subject to the same policies statewide. Yet, state-policies are implemented and state schools are overseen by sub-state regional departments. These are not autonomous to develop their own policies, but the process and degree of implementing those mandated by the central administration varies. Thus, state schools in Brazil are governed by policies developed at the state level and implemented through sub-state administrations at the regional level.
Municipal schools, on the other hand, are managed locally and have their policies mandated by the municipal administration—they are not subject to state level policies, although in some states, state policies can be very influential in municipalities, and state-initiated reforms can be implemented with considerable state management leadership (Carnoy et al, 2017). As the municipal administration has authority over a much smaller territory than the state, they do not, usually, have sub-municipal administrations. Consequently, the implementation of the policies does not vary within each municipality. Across municipalities, though, both policies and implementation might vary.
In order to test the assumption that mid-level management practices are important to foster student learning, I look at the state of São Paulo/Brazil. São Paulo is a particularly interesting state for this analysis due to its size and economic importance, most populous and richest of the country. Furthermore, the size of the state population creates the necessity of great decentralization of the administrative structure of the state system. There are 91 regional offices of education centrally coordinated by the State Government, these are responsible for implementing state policies in the more than 600 municipalities of the state. On the other hand, the low level of cooperation between most municipal administrations and the state government provides a valuable scenario to test the validity of the estimates of sub-state administration value-added.
Using both state and municipal schools in the state of São Paulo/Brazil, I analyze the variation in student gain in test scores between the 5th and 9th grades and estimate the effect of sub-state regional administrative offices on these gains. Sistema de Avaliação de Rendimento Escolar do Estado de São Paulo (SARESP), a standardized test taken by more than 85% of state public schools’ students, is used to track individual students over time and follow their test score growth. Based on a production function, controlling for municipality, school, teachers, and students’ characteristics, I estimate the value-added by the 91 sub-state (regional) departments of education in the state to student gains in test scores. To estimate the effect of middle levels of management, I focus on the estimates for regional dummies (fixed effects). The model is applied for various cohorts; therefore, it is possible to estimate the relative ranking and absolute average value-added of each sub-state regional department for different groups of students.
Additionally, municipal schools within each sub-state regional office administrative territory can be used to validate the value-added estimates. As these schools are not subject to state policies, in the case of the state of São Paulo, the estimated fixed effect should not show significative value-added for students in municipal schools. Likewise, if the fixed effects are capturing broader territorial and regional influences that are not related to management practices at the state level, the value-added estimates would be similar to the ones of state students. Hence, the comparison between the value-added to state students and municipal students is a good validity test.
Preliminary results show (1) that regional fixed effects are stable across various cohorts of students, suggesting that regional administrations play an important role in schools’ value added. Raking the fixed effects estimated, the same sub-state departments of education concentrate at the higher end of average gain in tests scores, even after controlling for municipality, school, teachers, and students’ characteristics. The same happens at the lower end of the growth in test scores. The results found are consistent in both mathematics and Portuguese language test scores. Additionally, (2) the estimated value-added of sub-state regionals to municipal students are concentrated around zero and, thus, different from the ones estimated for state students.
These findings will be supported by qualitative evidence gathered through interviews with the leadership of four regional administrative offices of São Paulo state. The interviews consider a variation of the World Management Survey (WMS) framework to assess the differences in management practices between high performance and low performance regional offices. Two high-performing and two low-performing regional offices are used as references to describe the different management practices conducted within these regionals.
