Do you really know if your doctor -- or child's teacher -- is doing a good job?
In education and healthcare, job performance data are lacking – but that's changing.
Philadelphia — There are few things more important to society than educating children and providing healthcare to families. If either fails, the economic and social impact is devastating.
Yet do we know whether our doctors and teachers are doing a good job?
Persistent blind spots in these critical fields hurt all of us. Thankfully, efforts are under way across America to change this.
Healthcare reform bills passed by both the House and Senate (and now on hold) include nearly $100 million to develop healthcare performance measures. The Obama administration's Race to the Top education grants require that teacher evaluations be based on "multiple measures."
Performance measurement is not the stuff of campaign commercials. Yet in a down economy, there is growing interest in getting the most out of shrinking budgets, so a "measurement movement" may be in the making.
In teaching, there has been a remarkable absence of measurement. Evaluations are typically done by a principal or assistant principal who may not use defined measures. Instead, they often make judgments based on observations of teachers at work – a highly subjective process fraught with charges of unfairness.
Many experts agree that standardized tests fall short in evaluating teaching and learning; it's like looking through a keyhole to identify the totality of what students learn from their teachers. Recognizing these limitations, The Bill & Melinda Gates Foundation invested $45 million to develop "fair" and "reliable" measures of teacher effectiveness. Its two-year national enterprise is evaluating 3,700 teachers using an array of measures, including videos of teachers interacting with students, student surveys, examples of students' work, more tests given more often, and an assessment of a teacher's ability to know when a student just isn't "getting it."
Performance measures in healthcare are increasingly common. More than 500 are used daily by health professionals to assess everything from hand washing to mortality and readmission rates. But the number and deemed "fairness" of measures are ultimately much less important than whether they lead to improved patient care and student learning.
Where measures matter most: lives of patients
The way to get to excellence is to measure performance along the way, says Don Berwick, CEO of the Institute for Healthcare Improvement (IHI). If that sounds easy, it's not. Mr. Berwick says healthcare measurement too often generates fear that either paralyzes or games the system. That's why he showers accolades on organizations like Palmetto Health in Columbia, S.C., that face bad news and improve.
Dr. Shawn Stinson, vice president for clinical quality and patient safety at Palmetto Health, says corporate leadership and hospital staff were shocked in 2006 to learn that the mortality rate for their patients was high. "We had in our minds that we were ... all above average. It shook us up to find out that we weren't," says Dr. Stinson. Stung but determined, Palmetto Health leaders set a daring goal to cut mortality rates by 20 percent in three years. They accomplished the goal in 18 months and continue to improve by relentlessly focusing on measurable results and by refining the practices that led to those results.
"As a doctor, the feedback to us is really quite minimal. We don't really know how we are performing," Stinson says. Palmetto physicians now use a dashboard that lets them compare their records against the national average and top performers. "They don't want to find out that other [doctors' patients] have a shorter length of stay, lower costs, and mortality rates. These are things that will really drive performance," says Stinson.
Similar data-driven discoveries are helping to revitalize educators' efforts. The California Partnership for Achieving Student Success (CalPASS) has a database of more than 355 million student records from kindergarten through college. Using "business intelligence" software traditionally used for sales and finance, educators have found some startling revelations.
Seeing the light in preparing students
In one study, students who stopped taking English courses after 10th grade required the same level of remediation in community college as students who continued to take advanced English courses through 12th grade.
"[Teachers] looked at the data and they freaked out," says CalPASS executive director Brad Phillips. "Upon learning this, they asked: What are our standards, what are our assignments, what are the expectations in high school compared to community colleges? While we all call it English, the expectations are clearly different."
Mr. Phillips says educators learned that high school courses emphasized literature, while community college courses covered writing and grammar, and four-year colleges emphasized analysis and argumentation. As a result, officials changed high school teaching to create better alignment.
While students these days take more tests, it is difficult to connect a student's performance to the teacher who taught him or her. According to the Data Quality Campaign, only 24 states are able to make that connection. Resistance stems from a fear that student tests will be used as the dominant measure of good teaching, determining tenure, promotion, and payment.
Berwick calls himself a transparency nut. "I just have a lot of confidence that measurements will be well used if we put them out there, and that they will get better by use," he says. The Carnegie and Gates foundations invited him to share information about the IHI model and its potential application in education.
Both professions (not to mention students and patients) are well served by learning from consistent top performers and providing support to those in the middle and bottom. Phillips calls educators using his database "courageous leaders" for their willingness to reveal weaknesses and paths to improvement. While California repealed its law prohibiting the connection between test scores and teachers, CalPASS will keep individual-level teacher data anonymous. Continued participation requires trust and support. "Most of the time data is a hammer in education, when it should be a tool for doing better," Phillips says.
We all want better education and healthcare. But we cannot fix problems we are unwilling to examine. As the carpenter's adage goes – we will be wise to measure twice and cut once.
Becky Fleischauer is a freelance writer and founder of Navigator Communications, a firm serving education, healthcare, and environment clients. None of Navigator's clients is mentioned here.