Primary Colors, Smiling Children and KIPP Results

"The data is the key resource that tells us that we’re actually doing that [delivering our promises to kids]”. Richard Barth (KIPP CEO)
 

The KIPP Annual Report Card section of the website (under Results) presents a 98-page document accompanied by a 3 minute and 18 second video entitled “The Story Behind the Numbers”. I would like to start the critique of this section by examining this short video. Like the video “The Story of KIPP”, this video is meant to sell the KIPP model to parents by providing testimonials on its methods of evaluation from a variety of voices--students, teachers and administrators.

 

KIPP sells itself by emphasizing its focus on data based on the assumption that data consists of true hard facts and that KIPP data presents a holistic picture of a school model that is in good health—that others schools can and should look to for guidance. The guiding principles for KIPP data are called the Six Essential Questions. The capitalization of these three words-- the Six. Essential. Questions.-- exudes a certain visual impact that wards off further questioning as to how essential these questions really are---the capital letters tell us that these questions are powerful in determining truth.

THE SIX ESSENTIAL QUESTIONS
 1.  Are we serving the children who need us?
 2. Are our students staying with us?
 3. Are our students progressing and achieving academically?
 4. Are our alumni climbing the mountain to and through college?
 5. Are we building a sustainable people model?
 6. Are we building a sustainable financial model?

The video tells us that the Six Essential Questions are a “holistic way to evaluate” the school model which implies that other schools may only have One Essential Question which is “What do the state test scores tell us about each school?”.

After the video introduces the fact that they use a varied method of data collection, it goes on to present the importance of such data to different members of the school community.

Brandon (student): “The point of taking test is to find out what you’re weak at and strong at in a particular subject. It makes me learn from my mistakes.”

Teacher: “Data gives me power as a teacher by enabling me to really make informed decisions so not just I’m gonna teach this today because it’s the next thing in line, but I’m going to teach this today because my students are struggling with X.”

Richard Barth (KIPP CEO): “The data tells us in what ways are we fully living our values, fully living our beliefs and fully living our aspirations and in which ways we are falling short.”

These testimonials are very convincing as a whole. It is especially hard to argue with students and teachers who appear very committed to their learning and methods of evaluating learning and moving forward. All of these voices also emphasize words like “next steps” and “the future” which really brings home their message of the importance of using data to inform decisions. The video uses very powerful language to promote a holistic model of schooling that is reinforced by what they view as a rigorous model of data collection. Words like “aspirations, values and beliefs” are used to reinforce that the school is concerned with much more than improving test scores—it’s a model that improves all aspect of students’ experiences.

Now, let’s look at the 98-page report card---the hard data that is so boldly promoted in the video.

The first 27 pages of the report card are full of photographs of teachers and students accompanied by inspirational quotes presented in bright primary colors interspersed with information about the school model. The message is sell sell sell. It is not until page 28 where we see the first mention of the Six Essential Questions followed by a few pages of information on what the report card is, a few more quotations and photographs from students and leaders and some more self-promotional information/marketing. Finally, on page 35, the national results of the Six Essential Questions are presented. From page 43 to 98, the local results for the Six Essential Questions are presented where each locality receives two pages of reporting.

At this point, I’d like to take a step back and look at the KIPP Annual Report Card versus the Atlanta Annual Performance Report for 2012-2013.

Since KIPP’s key advertising strategy (in terms of results) is to convince the potential customer that KIPP diverges quite drastically from the norm in the sense that it looks at a multitude of other forms of assessment besides test scores, I thought it crucial to consider this in light of another school system’s annual report card. I chose Atlanta as a basis for comparison because it is one of the key locations for a KIPP school and is composed of a high percentage of students who qualify for a free/reduced price lunch (76%) and African-American population (79.5%)---both of which are key indicators/targets of KIPP schools (NCTQ 2013) .

The title of the Atlanta Performance Report may give away some of its mission within its title “The Atlanta Balanced Scorecard”. As its name suggests, the report explores three (balanced) different areas to evaluate according to the 4E's (Excellence, Equity, Ethics and Engagement (yes, capitalized!)) which are its guiding principles. The report aims to look at 11 district-level objectives according to 22 different measures. The three areas are as follows:
     1.     Students, Stakeholders and Learning Environment
     2.    Leadership and Talent Development
     3.    Financial and Operation Systems and Support

The Atlanta report card is a total of 27 pages of which only the cover page has any pictures of smiling children. In terms of being holistic, 26 pages of the report give information on the 22 different measures according to a range of areas. Importantly, the five of the Six Essential Questions are answered here---the one question that is missing is student attainment in college (Essential Question #4). The rest of the questions are answered in much more detail than in the KIPP model giving a much more nuanced perspective of the school situation in which data can be used to inform decisions just like in the KIPP model.  

KIPP’s report card on Atlanta’s indicators is a mere 2 pages of which the only information available is the following 9 indicators:

student enrollment, number of teachers, percentage of students eligible for free/reduced meals, percentage of student attrition, percentage of special education services, percentage of ethnicities, state testing results in reading and math, AP and ACT results,  and alumni attainment

These indicators do answer the Six Essential Questions, but do these questions have the impact and power that the testimonials in the video “The Story Behind the Numbers” suggests? Is KIPP more accountable than a public school? Why should I be wow-ed by this model? To me, it feels like the same old story with test results playing the role of the protagonist.  Perhaps the appeal of KIPP results is that it conforms to the standard, promulgated and powerful viewpoint that numbers can be trusted and KIPP presents the numbers that help them make their case a strong one. And well, it helps to have smiling children and primary colors to clinch the deal.