Positive Effects of Comprehensive Teacher Induction

Today, Mathematica Policy Research, Inc. released the final report of its IES/U.S Department of Education-funded randomized controlled trial (RCT) of comprehensive teacher induction. It shows a statistically significant and sizeable impact on student achievement in mathematics (0.20 standard deviations) and reading (0.11 standard deviations) of third-year teachers who received two years of robust induction support. That's the equivalent of moving students from the 50th to 54th percentile in reading achievement and from the 50th to 58th percentile in math achievement.

As a basis of comparison, I note that in 2004, Mathematica conducted a RCT of Teach for America (TFA). In that study, it compared the gains in reading and math achievement made by students randomly assigned to TFA teachers or other teachers in the same school. The results showed that, on average, students with TFA teachers raised their mathematics test scores by 0.15 standard deviations (versus 0.20 standard deviations in the induction study), but found no impact on reading test scores (versus 0.11 standard deviations in the induction study).

In another recent Mathematica report (boy, these folks are busy!), the authors note that "The achievement effects of class-size reduction are often used as a benchmark for other educational interventions. After three years of treatment (grades K-2) in classes one-third smaller than typical, average student gains amounted to 0.20 standard deviations in math and 0.23 standard deviations in reading (U.S. Department of Education, 1998)." In that report -- an evaluation of the Knowledge Is Power Program (KIPP), Mathematica researchers found a very powerful impact from KIPP: "For the vast majority of KIPP schools studied, impacts on students’ state assessment scores in mathematics and reading are positive, statistically significant, and educationally substantial.... By year three, half of the KIPP schools in our sample are producing math impacts of 0.48 standard deviations or more, equivalent to the effect of moving a student from the 30th percentile to the 48th percentile on a typical test distribution..... Half of the KIPP schools in our sample show three-year reading effects of 0.28 standard deviations or more."

Is it appropriate to compare effect sizes among RCTs or, for that matter, among research in general? I am told that it is, although certainly considerations such as cost effectiveness and scalability have to enter into the conversation. Implementation issues also must be attended to. With regard to teacher induction, the issue of cost effectiveness was addressed in a 2007 cost-benefit study published in the Education Research Service's Spectrum journal and summarized in this New Teacher Center (NTC) policy brief.

Disclosure: I am employed by the NTC which participated in the induction RCT, and I helped to coordinate NTC's statement on the study.

The NTC is "encouraged" by the study. However, NTC believes that "it does not reflect the even more significant outcomes that can be achieved when districts have the time, capacity and willingness to focus on an in-depth, universal implementation of comprehensive, high-quality induction. It speaks volumes about the quality of induction and mentoring provided and the necessity of new teacher support that student achievement gains were documented despite [design and implementation] limitations to the study."

UPDATE: Read the Education Week story by Stephen Sawchuk here. And the Mathematica press release here.

Hire This Teacher!

Welcome to a special new series of the Education Optimists: the Promising New Teachers Award! We aim to identify, praise, and help place a few incredibly talented young women and men who are seeking the opportunity to work for schools across the country.

Our inaugural choice is Stephanie Ake of New Hope, Minnesota.

Stephanie is a recent graduate of the University of Wisconsin-Madison's School of Education with a degree in elementary education. She has also completed three years of extraordinary training and service as the nanny to the children of the Education Optimists!

Stephanie has numerous qualities and dispositions that make her a stellar teacher. She is caring, patient, responsible and understanding -- skills necessary to manage a classroom of young children. She is reliable and trustworthy, always on time, always prepared, always ready. She is one of the most organized, and dependable people we have ever met, and at the same she is flexible and calm. It was a testament to her skills that when it came time to choose a preschool and a teacher for our son, Liam and I found ourselves comparing every candidate to Stephanie.

Stephanie seeks a teaching job in the Twin Cities area teaching preschool through 5th grade. We give her our highest recommendation. Please feel free to email her opportunities directly (ake@wisc.edu) or contact us with questions.

Keep An Open Mind

As discussions about the future of for-profit colleges intensify, my email inbox has begun to fill with inquiries. Why haven’t I weighed in? What do I think—is Congress on the right track? What does my recent conspicuous silence portend?

While I’m flattered (and a little confused) by a seeming desire to hear my opinion, the truth is I haven’t been ready to provide one. Over the past few months I’ve spent a lot more time thinking about the for-profits and the tough questions their growing presence in higher education raise. I’ve struggled with an intellectual exercise of sorts, attempting to set aside the financial interests associated with the sector and simply consider whether common objections to the industry would exist even if its colleges were not-for-profit. It’s not easy to sleep at night when wrestling with complex demons like that.

I’ve come to the conclusion that yes, objections would continue. We’d be worried about the quality of what’s being proffered, what students are actually learning, how hard the colleges are working to recruit students not really ready for college work, how much debt folks are graduating with relative to their new income, etc.

Here’s the rub: We should have the same concerns about our current public and private non-profit institutions of higher education. Many of us do have these concerns. We are just less vocal about them, perhaps because it is so much easier to object to treating people badly while making a buck, compared to treating people badly while not making a buck.

Our concerns are well placed, but they are also too narrow. We are looking for trouble only under a single lamplight, simply because that’s the spot illuminated. We need to look more broadly. There is a reason enrollment in the for-profit sector is growing, and it has at least partly to do with student demand. Our public colleges and universities aren’t sufficiently equipped to do the job—and blame for that is shared by states and localities, institutions, researchers, and taxpayers. It’s a little hard to know where the buck stops in that situation. It’s not so hard in the case of for-profits—so we disparage them more easily.

I’m not saying I’ve become a fan of the for-profits, or that my worries about how they are serving students have been allayed. Admittedly, the more I learn, the more I become somewhat more impressed--for example, by the innovative efforts of some to help transfer students and older students find a more fluid and efficient way to a credential. There are some examples of that kind of work at public institutions, but it feels a bit less "outside the box."

The current discussion in D.C. is worth having. It needs to be broadened and deepened. More voices need to enter the conversation. It’s in the interest of students all over the country for it to continue.

You Must Live Here

The Chicago Public Schools residency requirement has ensnared a district social worker and recent kidney donor, the Chicago Tribune reports. He may lose his job, unless CPS chief Ron Huberman intervenes.

We've written previously about such residency requirements being bad policy.

6/2/2010 UPDATE: So the man's job is safe (Chicago Tribune), but why necessitate special treatment? Why not eliminate the policy that restricts the district's ability to employ the best and the brightest regardless of whether they live within city limits or not?

Musical Elective of the Month: June 2010

Gaslight Anthem is an up-and-coming, New Jersey-based rock band. The band has two albums under its belt, including Sink or Swim (2007) and The '59 Sound (2008). It has been embraced by Bruce Springsteen, with whom it has performed, and lead singer Brian Fallon once lived four blocks from E Street in Asbury Park, NJ.

Gaslight Anthem's new album, American Slang, is released on June 14th, but both NPR and Rolling Stone are live streaming the album to give you an early listen. The music mag says that the Anthem's sound "gloriously combines the Clash, Motown and the Pretenders."

Visit the official web site for more information and concert listings.

Look what you started,
I seem to be coming out of my skin
Look what you've forgotten here
The bandages just don't keep me in

And when it was over, I woke up alone

And they cut me to ribbons and taught me to drive
I got your name tattooed inside of my arm
I called for my father but my father had died
While you told me fortunes, in American Slang

-"American Slang,' American Slang (2010)

Click here for past Musical Electives.

Race To The Top Applications Posted

Updated 6/22/2010 12:15 p.m. CDT

As predicted, 35 states plus DC submitted applications to the U.S. Department of Education today in Phase Two of the Race to the Top competition.

Please click on the hyperlink below to view that state's RttT Phase Two application:

Alabama [application]
Arizona [application] [appendices]
Arkansas [application]
California [application] [appendices]
Colorado [application]
Connecticut [application] [appendices]
District of Columbia [application] [appendix A] [appendices B-F]
Florida [application] [appendices]
Georgia [application] [appendices]
Hawaii [application]
Illinois [application] [appendix 1] [appendix 2]
Iowa [application]
Kentucky [application]
Louisiana [application]
Maine [application] [appendices]
Maryland [application]
Massachusetts [application]
Michigan [application] [appendices]
Mississippi [application]
Missouri [application] [appendices]
Montana [application] [appendices]
Nebraska [application] [appendices]
Nevada [application]
New Hampshire [signatures] [A] [B] [C] [D] [E] [F] [competitive] [budget]
New Jersey [cover letter] [A] [B] [C] [D] [E] [F] [STEM]
New Mexico [application] [appendices]
New York [application] [appendices]
North Carolina [application] [appendices]
Ohio [application] [appendices]
Oklahoma [application]
Pennsylvania [application]
Rhode Island [application]
South Carolina [A] [B] [C] [D] [E] [F] [budget]
Utah [application] [appendices]
Washington [application] [appendices]
Wisconsin [application]

Race to the Top: Picking Favorites

6/25/2010 UPDATE: Per the Capital Times story this morning about Wisconsin's chances, I stand behind my contention that the state is an unlikely Phase Two winner. As is the case with numerous states that fell in the middle or bottom of the pack in Phase One (WI was 26 out of 41 applicants), I don't believe that Wisconsin passed significant enough reforms since then to improve its competitive chances (as compared to Colorado, Connecticut, New York and Oklahoma, for example). However, state education leaders - including State Superintendent Tony Evers - deserve credit for authoring a much stronger proposal this time and for gaining widespread buy-in for the proposed reforms. Hopefully, many of those ideas can be carried forward regardless of the RTTT outcome.

Although this is a joint blog, this post is entirely my own and not Sara's. -- Liam Goldrick


Final Race to the Top Phase Two applications are not yet publicly available, so this may be a bit premature. But everyone likes fun parlor games, right? Plus, I hear that there may be a Race to the Top void to fill.

If one assumes that states with the highest Phase One scores will come in strong again in Phase Two -- and that's a significant "if" -- one can look at recent policy changes to determine which states may have strengthened their hand. Conventional wisdom suggests that Colorado, Louisiana and New York have moved on up as a result of recent legislative activity.

There are numerous wild cards, of course, including six new applicants (Maine, Maryland, Mississippi, Montana, Nevada, and Washington), reduced Phase 2 requested budgets that will change the elements of nearly every state's application, and stronger (or weaker) support for state applications from unions and school districts that may change the overall calculus.

Here is my early look at the competitiveness of states in Phase Two (with their Phase One ranking (if applicable) in parentheses):

FRONTRUNNERS
Colorado (14)
Florida (4)
Georgia (3)
Illinois (5)
Kentucky (9)
Louisiana (11)
Massachusetts (13)
North Carolina (12)
Ohio (10)
Pennsylvania (7)
Rhode Island (8)
South Carolina (6)

That's 12 states. If one believes the U.S. Department of Education's public statements that no more than 10-15 states will be funded in Phase Two, there's not much (any?) room left in the winners' circle, assuming none of these applicants are knocked out of the running.

CONTENDERS
Arkansas (17)
District of Columbia (16)
Maryland (n/a)
Michigan (21)
New Jersey (18)
New York (15)

DARK HORSES
California (27)
Connecticut (25)
Oklahoma (34)
Utah (19)


What's your take? Are there "frontrunners" from Phase One that shouldn't be considered as such? Are there other "dark horses" who should be on this list? Do any of the six new applicants have a real chance?

As state applications become available, we'll take a closer look and provide further analysis as time allows.

• Related Posts: Race To The Top's Dropouts