FAQ: Opportunity Index

  • What is the Opportunity Index?

    The Boston Public Schools (BPS) Opportunity Index is a composite index that incorporates a range of data representing factors that are outside of the schools’ control, yet are predictive of students’ academic outcomes. The data include “place-based” measures related to students’ home neighborhoods – as defined by Boston’s 177 U.S. Census tracts – as well as measures specific to individual students and their families. By rolling multiple measures into a single, more accessible metric, BPS is better equipped to direct resources and supports to the schools and students who need them most.

    Why did BPS develop the Opportunity Index?

    Closing opportunity and achievement gaps is the Boston Public Schools’ driving priority. It is the catalyst for the hard work we are doing to replace the structures, practices, and mindsets that perpetuate inequities. Every student, in every classroom, in every school of the BPS system should have the same unconstrained opportunity to achieve the greatness within him or her. We know that students come to school with differing experiences, opportunities, and needs, but as a district, we historically have lacked a nuanced way to account for these differences and mitigate inequities that are out of schools’ immediate control. The Opportunity Index is an innovative tool created to help quantify these differences so we can make decisions and allocate resources more equitably, helping to close gaps in opportunity.

    How was the Opportunity Index originally developed?

    The Opportunity Index is designed to identify schools with higher concentrations of students in need of additional resources and supports to overcome opportunity gaps. To develop the Index, BPS partnered with researchers from Northeastern University and Harvard University through the Boston Area Research Initiative (BARI). This work was mostly done during summer and fall 2017 in preparation for the FY19 budget cycle.

    How often will the Opportunity Index be recalculated?

    The Opportunity Index will be calculated once for each school year. The initial methodology and calculations were used for decision-making in school year 2018-2019 (fiscal year 2019), and adjustments were made (outlined below) for school year 2019-2020 (fiscal year 2020). Every year, the district will recalculate the Index, using essentially the same model but incorporating any newly-available student and neighborhood data, and any changes to the indicators or their relative “weight” in the formula.

    Did the district take into account feedback raised by school leaders and community members for FY20?

    Yes. After the initial launch of the Opportunity Index in December 2017, feedback was collected from school leaders, School Committee members, partner organizations and the larger BPS community. Several themes emerged from this feedback, including gentrification, immigrant students and the exclusion of students with other needs, such as students with disabilities and English learners. This feedback was considered during the revision process through summer and fall 2018, resulting in an updated OI model for 2019-2020 (fiscal year 2020).

    Changes to the OI for FY20

    Indicator

    Level

    Description

    Status

    Academic Attainment

    Neighborhood

    Percentage of adults living in the census tract who have earned a Bachelor’s degree or higher.

    In both FY19 and FY20 model

    Neighborhood Safety

    A composite variable comprised of statistics related to gun use, domestic violence, and other crimes committed in the census tract.

    In both FY19 and FY20 model

    Socioeconomic status

    A composite variable of economic variables, including rates of family poverty, median household income, public assistance, and unemployment within a census tract

    Removed from FY20 model

    Median Household Income

    Median income for all households in the census tract.

    Added to FY20 model

    Custodianship

    Frequency of residents’ use of 311 to report issues in the area, an indication that residents have a sense of agency regarding their neighborhood and securing city services.

    Removed from FY20 model

    Physical Disorder

    Deterioration and denigration of neighborhood structures and spaces – both private and public – determined through 311 reports.

    In both FY19 and FY20 model

    Foreign Born

    Percentage of individuals in a census tract who were born outside of the U.S.

    Added to FY20 model

    Economic disadvantage

    Individual

    Participation in one or more of the following state-administered programs: Supplemental Nutrition Assistance Program (SNAP); Transitional Assistance for Families with Dependent Children (TAFDC); Department of Children and Families' (DCF) foster care program; and MassHealth (Medicaid). (Also known as direct certification).

    In both FY19 and FY20 model

    Residential Mobility

    Number of times a student moved, as measured by the number of times their geo code has changed in the last five years.

    Added to FY20 model

    Public Housing

    If the student currently lives in public housing (binary variable).

    Added to FY20 model

    Recent Immigrant

    A metric tracking if the student reported moving to the U.S. from another country in the last three years (binary variable).

    Added to FY20 model

    Chronic Absenteeism

    Past Performance

    Whether a student had an attendance rate of 90% or lower (binary variable). (Replaced by “Attendance Rate”).

    Removed from FY20 model

    Attendance Rate

    Annual student attendance rate (continuous variable). (Replaced the “Chronic Absenteeism” variable).

    Added to FY20 model

    Course failures

    Percentage of English Language Arts and Mathematics courses failed.

    In both FY19 and FY20 model

    MCAS failures

    Percentage of MCAS exam failures in English Language Arts and Mathematics.

    In both FY19 and FY20 model

    Ever Suspended

    A binary variable tracking whether a student had ever been suspended. (Replaced by “Number of Suspensions”).

    Removed from FY20 model

    Number of Suspensions

    A continuous variable tracking the total number of times a student was suspended from school. (Replaced the “Ever Suspended” variable).

    Added to FY20 model

    Each of the student-specific past performance indicators is calculated using the student’s previous educational level. That is, for high school students, the OI uses measures from the middle grades, and for middle school students, the OI uses measures from the elementary grades. The individual factors included use data from the current school year for each student. Neighborhood variables rely on the most currently available data, which often lags behind by two or three years. However, because neighborhood change usually happens gradually, neighborhood metrics are generally not substantially outdated.

    Calculating the OI

    How are these factors combined to generate the Opportunity Index?

    Using multi-level statistical analyses, we  calculated a “weight” for each of the indicators based on how predictive the indicator was of a students’ future academic outcomes. That is, if course failure was twice as predictive of chronic absenteeism was twice in predicting MCAS performance, it would be twice as important in determining a students’ OI score.

    We calculate a composite score for each student in the school using the available variables (for example, new students will not have historic data). We then standardize these scores across the district relative to their peers at the same educational level (elementary, middle, or high). That is, each student is given a percentile score from 0.01 to 0.99, representing their relative level of need within BPS.

    Each school’s Opportunity Index score is the average of all the individual student scores. Opportunity Index scores range from 0.01 to 0.99, with higher numbers indicating a higher average level of student need. Because of this, a school’s OI score represents their average student -- that is, a school with an OI score of .63 has a typical student with higher needs than 63% of BPS students. Of course, each school contains a range of students, and even some of our lowest overall need schools have some of the highest need students in the district.

    How are homeless students counted in the Index?

    Homeless students are associated with the census tract from their last known home address.

    Why are immigrant students only counted for their first three years in the U.S.?

    The immigrant definition used in the OI aligns with the federal definition of an immigrant, which BPS is required to report to the MA DESE annually. BPS does not track whether or not the student is a legal or illegal immigrant, only that the student has recently moved to the US.

    What information will schools receive about their Opportunity Index calculation?

    Each school has received a customized “scorecard” that includes its overall Opportunity Index score and a breakdown of the various indicators that contributed to the score. The report is designed to provide each school with an overview of the average level of student need, rather than specific information about individual students or their home neighborhoods. 

    Why did my school’s score change from FY19 to FY20?

    Similar to WSF, every year a school’s OI score should change slightly in response to enrollment shifts. In a typical year, we expect nearly all schools to see their OI change by less than 0.05 with some schools not seeing any change.  Schools with more significant enrollment changes may see greater swings. We realize that these enrollment changes will result in funding changes and are working with the Budget Office to work to smooth out these changes to allow for stable planning. However, while these swings can be disruptive, the OI is designed to automatically move funding from schools that become lower need to those that are becoming higher need.

    As this was the second year of the model, we made improvements based on stakeholder feedback (see the table above). Many of these changes are small tweaks (such as switching from whether or not a student was chronically absent to their actual attendance rate) while we also added some variables and removed others. As a result of these changes, the model has become slightly more accurate than it was last year, while remaining highly correlated to last year’s results. We expect to continue to make small refinements to the model over time, but we expect future changes to be smaller in magnitude than the changes we made this year.

    In general, if your OI score changed, about half of the change was due to enrollment changes at your school and half was due to changes in the model. There is rarely, if ever, one simple reason why a school’s OI score might have changed - it is almost always the combination of changes to the model and enrollment changes.