Sources & Methodology : Why We Measure What We Measure

Looking for the Sources and Methodology for a specific analysis?  

• 2021 Legislative Data One-Pager
• 2023 Legislative Data One-Pager

The mission of the Commit Data Dashboard is to democratize data to improve decision making in public schools. This raises a fair question: how do we determine which data to democratize?

There’s no shortage of publicly available data about the state of our schools, coming out of local school boards, state education agencies, and the U.S. Department of Education. Because of our emphasis on local and state policy, we’ve chosen to limit the scope of our data portal to the state of Texas, with options for viewing by school district, county, legislative district, or statewide.

We’ve also chosen to focus on twelve indicators across each of our data tools, as well as our annual Community Scorecard Report. Each represents a key benchmark in a student’s educational journey, from Pre-K to postsecondary. Taken together, these indicators provide a holistic picture of the health of the entire education system.

Each indicator meets all of the following criteria:

• Valid measure of the outcome
• Available consistently over time
• Produced by a trusted source
• Similarly measured across regions and school districts
• Feasible to gather and report
• Easily understandable to local stakeholders
• Malleable to a significant degree by local action
• Useful in the day-to-day work of collaboratives that are working to improve student outcomes

With this in mind, let’s look at each individually:

Pre-Kindergarten Enrollment

Definition

Percentage of eligible 3 and 4 year olds enrolled in Pre-K.  Eligibility is determined by income, language proficiency, special needs, military status, foster status, and/or experiencing homelessness.

Source

Texas Public Education Information Resource

Significance

90% of the brain is developed by the time a person is five years old. Research shows a strong correlation between quality early learning and regional prosperity. For every $1 invested in early childhood,a community saves $7 in long-term costs such as educational remediation, criminal justice, and welfare. And quality Pre-K programming has been shown to close achievement gaps for low-income students and students of color. 

Quality early learning for all children can change the economic outcome of our community in one generation. When we ensure that all students receive the best education, from the very beginning, we set them up for success for the rest of their lives.

Kindergarten Readiness

Definition

Percentage of students that qualify as “ready” based on an individual school district’s assessment and cut score

Source

Texas Public Education Information Resource

Currently, individual districts choose their own assessment to determine kindergarten readiness. A standardized and age-appropriate measure to be used by all districts across Texas is currently in development, as mandated by House Bill 3. Kindergarten Readiness report provided by the Texas Education Agency (TEA)

Significance

Kindergarten readiness is closely linked to future academic achievement and life success, though indicators for this outcome area often differ across school districts and states. The Texas Education Agency does not mandate a single kindergarten readiness standard across school districts, but it does mandate that districts assess literacy among incoming Kindergarteners.

Kindergarten literacy levels, in turn, predict reading ability throughout a child’s educational career.  Across states, school-entry academic and attention skills are highly correlated with later academic achievement in elementary school.

3rd Grade Reading

Definition

Percentage of students who achieve the “Meets Grade Level” standard on the STAAR 3rd Grade Reading exam.

Source 

State of Texas Assessment of Academic Readiness (STAAR) data provided by TEA

Significance

In the early grades, children begin to transition from learning to read to reading to learn. At these grade levels, the reading curriculum becomes more complex in both meaning and vocabulary.

Data shows that disparities in literacy during the early grades are linked to persistent achievement gaps. If children are behind by third grade, they generally stay behind throughout school. One longitudinal study found that students who do not read at grade level by third grade are four times more likely to drop out of high school than proficient readers. Conversely, students at or above grade level reading in earlier grades graduate from high school and attend college at higher rates than peers reading below grade level.

4th Grade Math

Definition 

Percentage of students who achieve the “Meets Grade Level” standard on the STAAR 4th Grade Math exam.

Source 

State of Texas Assessment of Academic Readiness (STAAR) provided by TEA

Significance

Elementary math lays the foundation for all future mathematics, as students need a stronger understanding of arithmetic and finite number sets to effectively tackle more complex concepts, such as algebra and infinite number sets.

Research indicates that an early understanding of math concepts is the most powerful predictor of later school success. Knowledge of fractions and division uniquely predicts subsequent knowledge of Algebra and overall math achievement more than four years later. And students in the lowest quartile of math achievement at ages 6, 8, and 10 are less likely to attend college than students who struggle in other subjects.

8th Grade Science

Definition

Percentage of students who achieve the “Meets Grade Level” standard on the STAAR 8th Grade Science exam.

Source 

State of Texas Assessment of Academic Readiness (STAAR) provided by TEA

Significance

A strong foundation in math and science at the middle school level can dramatically impact future workforce opportunity.

According to the U.S. Department of Labor, “there were nearly 10 million workers in STEM occupations” in 2021, a total “projected to grow by 11%” in the following ten years. Earning potential in this field is also significantly higher, with median wages in STEM occupations over twice the amount of non-STEM median wages.

Algebra I

Definition

Percentage of students who achieve the “Meets Grade Level” standard on the STAAR Algebra 1 exam.  

Note: This is the percent of students who take and pass Algebra 1. This does not take into consideration students who are eligible for Algebra 1 but take a lower level math course instead.   For more on this subject, see this Commit blog post.

Source 

State of Texas Assessment of Academic Readiness (STAAR) provided by TEA

Significance

Basic algebra has been referred to as the “gatekeeper subject” because of its correlation to both college and career success. Taking algebra in middle school opens the gateway to completing advanced mathematics courses in high school which, in turn, is highly valued for admission to many four-year colleges and universities.

According to analysis of PEIMS data from E3 Alliance, students who take two math classes beyond Algebra II in high school are three times more likely to graduate college. This makes having access to Algebra I by 8th grade especially important. Economists estimate that if the U.S. could raise math proficiency to Canadian levels, economic growth would improve by 1.5% annually and over the long run add $75 trillion to the U.S. GDP.

College Readiness

Definition

Percentage of annual graduates from all Texas public schools who scored "At or Above Criterion" on the College Board's SAT and/or ACT, Inc.'s ACT assessment. 

"At or Above Criterion" is defined as follows for each exam:

SAT: 480 on evidence-based reading and 530 on mathematics

ACT: 19 on English section, 19 mathematics, and 23 composite score

Source

Texas Academic Performance Reports

Significance

SAT/ACT exams administered in the 11th and 12th grade help us determine if a student has developed the academic foundation necessary for a successful transition from secondary to higher education. As nationally normed tests, these indicators allow our community to compare the progress of our school systems against those from around the country.

58% of students who do not require remediation upon entering college ultimately earn a bachelor’s degree, compared to only 17% of students enrolled in remedial reading and 27% of students enrolled in remedial math.  A strong correlation exists between SAT/ACT scores and first year college GPA and retention rates; SAT/ACT scores also help identify the need for remedial work. The three year graduation rate at Texas community colleges for full time students requiring remedial work is only 10%.

High School Graduation

Definition

Percentage of students who graduate high school within four years

Source

Texas Academic Performance Reports

Significance

High school graduation is not only a precursor to college enrollment; it is also a major indicator for earnings throughout adulthood. On average, high school graduates make  $300,000 more in lifetime earnings than individuals who did not complete high school. 

If the number of high school dropouts in the 50 largest U.S. cities were cut in half, the extra earnings of those high school graduates would add up to $4.1 billion per year. High school graduation has been shown to predict health, mortality, teen childbearing, marital outcomes, and crime. Increasing the educational attainment of one generation improves the next generation’s academic and social outcomes.

College Enrollment

Definition

Percentage of students who enroll in a public Texas institution higher education within 1 year of high school graduation

Source 

Texas Academic Performance Reports

Significance

Postsecondary enrollment marks one of the critical transitions in the cradle to career pipeline. By 2020, 62% of Texas jobs  will require some form of postsecondary education. Post-secondary credentials are not limited to four year universities; two-year programs and technical certifications are also valuable in this knowledge-based economy.

Research has shown that “low income students, even those with high academic performance levels, are less likely to enroll in college, more likely to attend two year colleges when they do enroll, and less likely to apply to more selective institutions compared to their more advantaged peers with similar academic preparation.” Texas students experiencing economic insecurity enrolled in College at an average rate of 39%, compared to 54% of students not experiencing economic insecurity.

College Persistence

Definition

Percentage of Texas public and private, two- and four-year higher education students who enrolled in college and continue onto their second year.

Note: Currently, there is not publicly available college persistence data that can be disaggregated by race, socioeconomic status, etc.

Source

National Student Clearinghouse Reports provided by individual school districts (Community Scorecard Report)

*College persistence is not reported on the Commit Data Dashboards.

Significance

Often students begin higher education with strong ideals but the transition proves very difficult. Supporting students during this critical period is essential for improving degree attainment.

As many as 1 in 3 students nationally do not return for their second year of college. Just over 1.2 million first-time, full-time, degree-seeking students began their undergraduate careers at public four-year colleges and universities in the fall of 2010. Six in ten (62.4%) completed their degrees within six years. Of the 1 million first-time students who enrolled at two-year public institutions in fall 2010, only one in four (39.3%) completed degrees or certificates from their starting institution within six years.

College Completion

Definition

Percentage of college freshmen who graduate within 6 years of enrollment

Note: Currently, there is not publicly available college persistence data that can be disaggregated by race, socioeconomic status, etc. This also does not take into account the students who do not enroll in college.  

Source

National Student Clearinghouse Reports provided by individual school districts (Community Scorecard Report)

Texas Higher Education Coordinating Board (Commit Data Dashboard)

Significance

The benefits of post-secondary education span across social and economic domains. An associate degree holder earns nearly $500,000 more in lifetime earnings over a high school graduate. A bachelor’s degree holder earns over $1 million more in life earnings over a high school graduate.

According to the Lumina Foundation, the number of jobs requiring an associate degree has grown by 1.6 million and the number of jobs requiring a bachelor’s degree has grown by 2 million since the end of the recession. Society benefits from a more educated population, as well, through lower instances of child abuse, lower rates of criminal behavior, and fewer teen pregnancies among children of college-educated parents.

Living Wage Attainment

Definition

Percentage of young adults (ages 25-34) earning above a living wage.

Living Wage Attainment is evaluated at the county and state level by considering individual earnings of young adult residents, age 25-34, as reported in the US Census American Community Survey (ACS) Public Use Microdata Sample (PUMS) datasets.  Dallas County and Texas overall rates may be evaluated using the ACS-1yr survey results, while disaggregated estimates and counties of lower population are evaluated using ACS-5yr survey results.

The Threshold for Living Wage Attainment was researched and informed by the MIT Living Wage Calculator to be initially established in 2018 as $50,000 individual earnings per year for young adult residents.  To account for inflation, that Threshold level is adjusted annually using Consumer Price Index (CPI) data provided by the US Bureau of Labor Statistics (BLS).  The Living Wage Attainment Threshold rate for 2022 US Census data currently stands at $58,784 per year.

Source

U.S. Census Bureau, American Community Survey, Public Use Microdata Sample

Significance

Analysis of the Dallas job market suggests that a degree or certificate may not be enough to prosper in the local economy, especially for Black and Hispanic/Latinx residents. A postsecondary credential is necessary, but not sufficient, for success in the Dallas County economy. Eliminating barriers across the workforce pipeline is essential in order to achieve an inclusive and prosperous region.

Ready to dive into the data? Check out how your community is doing on these key indicators on the Commit Dashboard. 

Want to see other organizations across the country who are also measuring these important metrics?  Check out the Strive Together Network.