Jumping to Conclusions

In a recent post on Education Week's Inside School Research blog, Debra Viadero offers a caution about President Obama's support for community colleges. Pointing to her recent article on community college research that indicated how much more we need to know about how best to improve completion rates in that sector, she questions whether the president would be wiser to place his bets on career colleges. She says that a recent study by the Educational Policy Institute (EPI) and an ongoing program of research by James Rosenbaum and colleagues support her contention that community colleges ought to take cues from career colleges.

In my opinion, this talented reporter is jumping to conclusions.

Yes, the graduation rates at two-year for-profit colleges exceed those at two-year public colleges. No one disputes that. That does not necessarily mean, however, that career colleges are outperforming community colleges, or that community colleges should take steps to become more like career colleges. The plausible alternative explanations for the differences in results are numerous. For example, the students attending the two types of colleges may differ in important yet unmeasured ways, ways that are associated with chances of graduation (what researchers refer to as selection bias). Is one group more economically or educationally advantaged? More motivated? More apt to have a family, nighttime work, or receive tuition support from an employer? It's also possible that the differences in graduation rates stem from constraints that community colleges face but career colleges do not-for example, inadequate resources or a lack of control over mission or governance. It's one thing to point to differences in practices between the community colleges and for-profit colleges, and another thing to attribute those differences to variation in the "will" or intentions of practitioners, rather than attribute them to under-funding and all that comes with it.

Establishing that community colleges have poorer graduation rates than career colleges for reasons they can and should do something about requires evidence that the two differ on one or more key aspects that is causally linked to college completion. Say we knew that smaller classes caused better student retention-and community colleges have larger classes than career colleges. We'd then be able to say, there's something community colleges ought to fix. But we don't have evidence that that's the case.

Instead, research simply establishes that (a) career and community colleges have different graduation rates and (b) career and community colleges (sometimes) employ different institutional practices. Rosenbaum and his colleagues have done a nice job, as Viadero notes, of documenting the latter-using qualitative methods, mostly at colleges in the Chicago area. But they have not shown that those practices cause observable differences in graduation rates. Moreover, while they've produced one paper indicating that differences in the student populations at career and community colleges do not appear to account for disparities in outcomes, that analysis is based solely on a limited set of observable characteristics-and therefore don't rule out the possibility that different levels of student motivation, for example, are really the culprit. Just think about how students get to college-many at career colleges are actively recruited (sometimes off their living room couches) while many at community colleges effectively wander in the door. Why would we think, then, that career and community colleges are serving the same kinds of people and producing different results?

There's another consideration Viadero neglects, and that's college costs. Students at career colleges leave with far more debt than students at community colleges. Data from the 2007-2008 National Postsecondary Student Aid Study reveal that 61% of community college students graduate with less than $10,000 in debt, compared to only 22% of students graduating from 2 year for-profits. In contrast, 19% of graduates from 2 year for-profits have $30,000 or more in student loans, compared to only 5% of community college graduates. Nearly all students (98%) finishing at 2 year for-profit colleges have taken on a loan, compared to just 38% of community college graduates. Is that a problem? Is it offset by higher rates of graduation? The answers are far from clear. Absent better ones we shouldn't be relying on evidence like EPI's---a study of career colleges' high graduation rates that was supported by the Imagine America Foundation, formerly the Career College Foundation, established in 1982 as the research, scholarship and training provider for the nation's career colleges. Full-text of that study wasn't even placed online for researchers to fully vet!

Community colleges have a long, rich history of serving this nation. Sure, there's room for improvement, but without more solid evidence of which changes are needed let's not jump to conclusions and tout the for-profits as a model to which they ought to aspire. We might end up in a bigger mess than we're already in.

POSTSCRIPT:

I have now obtained a copy of the full EPI report. My suspicions were correct: the authors use nothing more than simple descriptive comparisons of students' characteristics and degree completion rates (calculated using NCES's DAS system, likely without propering weighting) to support their causal claims about the "benefits" of attending community college. For example, they write "The report suggests that career colleges work harder to provide appropriate student services and support" but present no data on institutional services or effort expended, particularly any tied to student outcomes. Their final conclusion -- "statistically, not only do students attending career colleges perform as well as or better than many other students attending comparative public institutions, but they persist in and complete their education while typically being more economically, educationally and socially challenged than other students"-- is based on nothing more than comparisons of sample means (no regression, no nothing). C'mon folks, this ain't the kind of research any consumer ought to be taking seriously. Glad to see Kevin Carey agrees.

State Teacher Policies Suck!

I'm sure glad that Kate Walsh and company weren't my professors in college. Damn! They are tough graders! With the exception of eight southern states (Alabama, Arkansas, Florida, Georgia, Louisiana, South Carolina, Tennessee, Texas) that received a 'C' and three northern states (Maine, Montana, Vermont) that received a 'F', every U.S. state received some version of a 'D' in the latest edition of the National Council on Teacher Quality's State Teacher Policy Yearbook. In grading the states, the authors look at five broad teacher quality areas (and numerous metrics within them): teacher preparation, expanding the pool of teachers, identifying effective teachers, retaining effective teachers, and exiting ineffective teachers.

While it is easy to poke holes at some of the National Council on Teacher Quality's seemingly ideologically-driven work (such as, I believe, its excessive focus on teacher pensions), much of its state policy analysis has a strong foothold in research and is one of the most comprehensive and regular analyses of state teacher policies. Like it or not, there is an increasing alignment between the NCTQ's scorecard and that employed by the U.S. Department of Education in the Race to the Top competition. The entire report should not be dismissed because of who they are (or are perceived to be). States should feel challenged by some of the analysis within the Yearbook and should consider looking to the "best practice" states identified under some of the metrics.

Here's a brief summary of the report's findings:

• State teacher policies are "broken, outdated and inflexible."
• Evaluation and tenure policies take too little or no account of classroom effectiveness. 47 states "allow tenure to be awarded virtually automatically."
  
• States are "complicit" on keeping ineffective teachers in classrooms. Only 1 state separates dismissal policy for poor performance from criminal and moral violations.
• Few states provide robust enough alternate routes into teaching.
• States' requirements for elementary teacher, middle-school teacher and special education teacher preparation are inadequate.
• There is too little accountability for teacher preparation in state policy. Only 5 states set minimum standards for teacher preparation program performance.
• States "cling to outmoded compensation structures," including the single salary schedule.
  

My primary quibble with the report is that it appears to completely and utterly discount the role of induction, mentoring and professional development in strengthening teacher effectiveness. Even if we prepare teachers better, recruit non-traditional candidates into the profession, retain them longer, compensate them differently, make evaluations more regular and meaningful, and find appropriate ways to terminate the small fraction of truly incompetent ones, it still will not be enough to maximize teacher effectiveness. There will continue to be a need for high-quality, individualized support upon entry into the profession and regular opportunities for data-driven, instructionally-focused professional development through a teacher's career. Professional development is not featured as a metric in the report at all and induction only enters as a criteria with regard to teacher retention, rather than teacher effectiveness -- which is where its most important power truly lies. That said, the evaluative criteria the report lays out about induction policy (on page 183-184 of the printed report) are worth noting and includes elements that states must attend to: mentoring of sufficient frequency and duration, mentoring provided at the start of the school year, and attentive mentor selection and high-quality training.

I won't beat this horse any further today, but check out these past posts for greater substance on what I'm getting at here with regard to the inadequate focus on the developmental needs of new and veteran teachers:

Race To The Top: Under The Hood
RttT: Redefining Teacher Effectiveness
Measurement Is Not Destiny

In other news, experts are doubting the likelihood of a 2010 reauthorization of the Elementary and Secondary Education Act, so these state teacher policies with an added dose of Race to the Top reforms is likely to be where it's at over the next year plus.

Making SAFRA Count

The end of last year was a busy time for me as I waited out the birth of my daughter who decided to spend an extra 10 days lounging in utero before emerging into the Wisconsin winter. I was so focused on strategies to promote her exit (sidenote: talk about an area in need of better research-give gobs of data on live births for hundreds of years, docs still refuse to hazard a prediction of labor occurring on any given night!), I virtually shut out the world of higher education policy. Imagine!

Thankfully, others were hard at work around and over the holidays, thinking about ways to make sure that the substantial, timely, and hard-won investment which will (fingers crossed) soon come to higher education via the Student Aid and Fiscal Responsibility Act (SAFRA) are most effective. Evidence of that work is contained in a December Lumina Foundation memorandum to the U.S. Department of Education, awkwardly (but accurately) titled "Structuring the Distribution of New Federal Higher Education Program Funding to Assure Maximum Effectiveness."

The memo gets it (mostly) right. There's great potential for this money to count, but also a real possibility it will do next to nothing if mismanaged. For example, if definitions of key terms like "college completion" are vague, and standards for "rigorous" research evidence ambiguous, then funds will likely go to continuing business as usual-for example, supporting programs that purport to increase college access while doing little to change rates of success-leading some to ask, access to what?

To avoid this the Department of Education needs a distribution system based first and foremost on one principle: keep it simple. It should make states define college completion and disseminate that definition-then stick to it. It's easiest to tell if plans are straightforward and consistent with intended principles if prospective grantees are forced to explain their ideas in a concise manner. Lumina gets this, and its team recommends a two-step process that requires a concept paper in advance of a full proposal.

So the good news is that this Lumina paper hits many of the key issues and makes some solid recommendations. That said, its authors missed an opportunity to address one important issue. The section titled, "How will the U.S. Department of Education know if these investments are actually helping to meet the President's goal?" is essential. It goes to the heart of one major goal of SAFRA-to increase the body of knowledge about what works in promoting college completion, and therefore the field's capacity to create lasting change.

As I recommended to ED's Bob Shireman early last year, we can do higher education a great service by holding a high bar for what constitutes research on college completion. Too often research in higher education hypothesizes that policies or practices advance desired outcomes, but utilizes insufficient methods to establish causal linkages between the two. As a result, we often don't know whether the results we see can be directly attributed to the new practice or investment.

In this case, ED should define "research" and "researchers" and "evidence," ideally in ways that are consistent with current practices at the Institute for Education Sciences; and require states to use those definitions. There should be a prescriptive process for selecting researchers (so as to make sure that truly independent evaluations are conducted) and proposals that allow for sustained research should be prioritized (e.g. those that leverage supportive foundation funding to continue the work to assess mid and long-range outcomes). I'd also like to see ED involved in increasing the capacity of researchers to do this kind of work, since it's far from clear how the demand for new work can be met by the current supply of higher education researchers. Maybe an IES pre- and/or post-doc training program targeted to postsecondary education?

Sure, this would require setting aside sufficient funds for the research side of the initiatives-but absent that investment, we'll likely never know whether the money spent on SAFRA-funded programs and policies had any real effect. That would, of course, be business as usual-precisely what we must avoid if we want to make this once-in-a-lifetime opportunity really count.

Goren Named New Director of Chicago Consortium

The Consortium for Chicago School Research just named a new Executive Director to replace John Easton who left to become director of the Institute for Education Sciences. CCSR made a wise choice in selecting Paul Goren, until now a senior VP at the Spencer Foundation.

I've done a lot of work with CCSR over the last few years, and been enormously impressed with their staff. I've also had the chance to interact with Paul, and believe he'll fit right in. He's firmly committed to improving urban education, and is willing to ask the hard, insightful questions required to make that happen.

Bravo, CCSR!

When Pigs Fly

It's not often I agree with the Wall Street Journal editorial page, but I guess pigs are flying today. The WSJ's take, in today's editorial ("Race To The Middle?"), on how the Race to the Top selection process should occur is on the mark:

To qualify, Mr. Duncan said states had to, among other things, lift caps on charter schools and remove barriers to using student records to identify good teachers and reward them. He's also said that "there will be a lot more losers than winners."

That's a good sign, but Mr. Duncan will be tempted to give more states less money in order to minimize political blow back and in the name of getting all states to make at least some, minimal progress. This is the Lake Wobegon school of education reform, where every state is above average....

But Race to the Top shouldn't be about rewarding a state for its grant-writing. It should use federal leverage to help remove barriers that stand in the way of state and local problem solvers.

The leading reform states are well known. Florida has superior data systems, thanks to reforms under former Governor Jeb Bush, and is upending collective bargaining provisions that prevent merit pay for teachers. Colorado has excelled at creating quality charter schools, while Massachusetts's academic standards are a national model. If states like these get Race to the Top cash, it will send a signal to the rest that they need to do more than mouth the right sentiments or pass a bill. They need to be break political china.

But as long as Race to the Top exists, Mr. Duncan ought to use it to reward only the very best reform states that want the money, perhaps only two or three in the first round.

Hat tip: This Week in Education

Speaking of Mr. Russo, he has provided an excellent resource for those of you who want to dive deep into the state RttT applications. His post provides a link to 22 (and counting!) of the 40 RttT applications filed on Tuesday.

D-Day

Today is the deadline for state applications in the first round of the Race to the Top grant competition. The easy prognostication to make is that the vast majority of the 39 states (and DC) that apply will have their initial applications rejected and all will reapply in round two, due in June. Most will fail then, too.

Despite the publicly released application scoring rubric, it is difficult to know exactly how the application scoring will play out, based upon who the reviewers are, whether Gates Foundation consultant funding helped certain states frame more compelling applications, stated or implicit pressures to fund only a certain number of applications (especially in round one), the importance lent to district and union buy-in from an implementation and sustainability perspective, and the strength of big-state applications versus small-state applications. To the latter point, there's ONLY $4 billion to be spread around, and the largest states could suck up as much as $700 million apiece. Florida, I believe, is very likely to be funded in round one. California and New York have much more of an uphill battle, and Texas, well, if Gov. Goodhair (thanks, Molly Ivins) has his way, may secede from the nation as well as the Race to the Top competition.

As I've said in a past post, my fervant hope is that states that have enacted 11th-hour bailouts of their Race to the Top prospects will not markedly benefit over states that have demonstrated historic commitment to education reform and the student outcomes that go along with it. Those states that have attempted to strengthen their chances by lifting charter caps, intervening in low-performing schools, raising academic standards, and enacting similar reforms should get some credit. But states that have taken these steps prior to Race to the Top influence should be recognized. By my estimation, states such as Delaware, Florida, Georgia, Louisiana, Massachusetts, North Carolina, Ohio, Tennessee and others would be appropriately rewarded for such sustained commitment and/or demonstrated results.

That all said, President Obama announced today the inclusion of $1.35 billion in his FY 2011 federal budget for a third year of the Race to the Top competition (WaPo story here). What this means exactly is still unclear, and may not be until the two initial rounds of competition are done. My hope would be that the Obama Adminsitration would use new resources to extend funding for leading states rather than broaden the competition and fund some of the reform leggards out there.

Photo courtesy of davidavery.wordpress.com

`````````````````````````````````````````

LATEST RACE TO THE TOP UPDATES

NATIONAL
Summary of state responses to the Race (Washington Post)

Last-minute resistance to the Race (New York Times)

District stances on Race to Top plans vary (Education Week)

Turning this Race into a relay (Eduflack)

Toothless Reform? (The Enterprise Blog)

CALIFORNIA
State files application (San Francisco Chronicle)

COLORADO
State files application, doesn't include new evaluation system, has union support (Denver Post)

New law will track teacher training programs grads (Denver Post)

DELAWARE
State board approves teacher evaluation changes (The News Journal)

ILLINOIS
Governor Quinn signs RttT bills (Catalyst Chicago)

Fear of winning Race? (New York Times)

IOWA
221 of 361 school districts sign on (Des Moines Register blog)

KENTUCKY
State board approves new performance measures (Louisville Courier-Journal)

MASSACHUSETTS
Governor signs RttT, ed reform bill (Boston Globe)

MICHIGAN
State application finalized (Lansing State Journal)

NEW JERSEY
Half of state's districts on board (The Star Ledger)

NEW YORK
Legislature takes no action on charter school bills (New York Times)

Bid goes forward, likely without lift of charter cap (Wall Street Journal)

Mayor Bloomberg signs off on RttT plan (New York Post)

OREGON
State application takes shape, union approved (The Oregonian)

RHODE ISLAND
Providence teachers' union sole union affiliate in state to support application (Providence Journal)

SOUTH DAKOTA
Charter school law proposed (Rapid City Argus Leader)

TENNESSEE
Student achievement will count for half of a teacher's evaluation (The Tennessean)

WISCONSIN
State submits bid (Milwaukee Journal Sentinel)

Governor: Wisconsin 'will likely miss out' because of 'a lack of reform in Milwaukee' (Governor Doyle press release)

Editorial: Milwaukee needs a mulligan (Milwaukee Journal Sentinel)

Excuses, Excuses...

Sara and I were talking in the car yesterday on the way to a very good children's museum in Rockford, Ilinois and bemoaning the fact that our blog doesn't have much thoughtful or updated content. Now, we have a ready excuse for that: Annie. All I've been able to muster lately are some cursory Race to the Top updates.

Nonetheless, our recent inclusion among the "Best Education Blogs for 2010" by the Washington Post's Jay Mathews and Valerie Strauss puts us in the mindset of wanting to live up to our billing. Give us some time - and we'll deliver.

-Liam

Updates on the Race: 01-15-2010

Round-one applications are due on January 19, 2010...

CALIFORNIA
Bay Area schools are Racing to the Top (Contra Costa Times)

ILLINOIS
70 percent of districts on board (Peoria Journal Star)

IOWA
RttT bills on the Legislature's agenda today (Des Moines Register Iowa Politics blog)

Seven largest school districts oppose Governor's plans (Des Moines Register)

KENTUCKY
Governor signs low-performing schools bill (Louisville Courier-Journal)

MASSACHUSETTS
Sweeping education bill passes legislature (Boston Globe)

NEW YORK
Unions opposing charter cap lift (New York Post)

RHODE ISLAND
No agreement between state, teachers' unions (Providence Journal)

TENNESSEE
Bill advances in House, headed for Senate vote (The Tennessean)

TEXAS
Editorial: Governor Perry is all 'rhetoric' (Houston Chronicle)

Op-ed: Perry's 'smokescreen' (The Dallas Morning News blog)

UTAH
Three fourths of school districts on board (The Salt Lake Tribune)

WASHINGTON
Governor's reform plan detailed (ABC News)

Editorial: Reforms needed to win round 2 (The Spokesman-Review)

Updates on the Race: 01-14-2010

COLORADO
RttT bill is fast-tracked (Denver Post)

CONNECTICUT
75 school districts on board (New Haven Register)

ILLINOIS
Legislation revamps teacher evaluations; governor's signature expected (The State Journal-Register)

INDIANA
93% of school districts join Race (Fort Wayne Journal Gazette)

IOWA
Senate approves RttT bill (Des Moines Register)

KENTUCKY
Governor signs low-performing schools bill (Louisville Courier-Journal)

MASSACHUSETTS
School bill ready for final vote (Boston Globe)

MICHIGAN
State teacher's union won't sign onto application (The Detroit News)

MINNESOTA
Over 250 districts, charter schools on board (Star News)

NEBRASKA
Governor: State's $122 million application includes 'Virtual High School' (Omaha World-Herald)

OREGON
112 school districts on board (Statesman Journal)

PENNSYLVANIA
Requiring local school board and union sign-off (Education Week Teacher Beat)

Editorial: Flexibility should accompany call for innovation (Harrisburg Patriot-News)

RHODE ISLAND
State, teachers union still at 'loggerheads' (Providence Journal)

TENNESSEE
Teacher evaluation bill clears latest hurdle (Chattanooga Times Free Press)

TEXAS
Governor Perry plays 'local control', 'Texas first' cards; rejects Race to the Top (Dallas News)

WEST VIRGINIA
Governor will call for special session if state fails in first round (Business Week)

WISCONSIN
Mayor/gubernatorial candidate offers compromise on mayoral control (Milwaukee Journal Sentinel)

Updates on the Race: 01-13-2010

NATIONAL
AFT chief vows to revise teacher-dismissal process (Education Week)

Strong applications versus stakeholder support? (Flypaper)

RttT fire drills ignore the fact that 52% 0f state application is based on PAST reform and achievement (Eduflack)

ALABAMA
Governor Riley links charters, Race chances (Dothan Eagle)

FLORIDA
53 of 67 school districts on board; only 5 with union backing (Orlando Sentinel School Zone blog)

GEORGIA
Governor Purdue pitches performance pay (Atlanta Journal Constitution blog)

ILLINOIS
Bill to strengthen educator evaluations passes state House (The State Journal-Register)

IOWA
Legislation needed to boost state's competitiveness (Des Moines Register)

Governor Culver presses for RttT legislation in State of the State (Des Moines Register blog)

Bill passes first legislative hurdle (Des Moines Register blog)

KENTUCKY
House passes low-performing schools bill (Louisville Courier-Journal)

LOUISIANA
State board endorses application (The Advocate)

MASSACHUSETTS
Editorial: Hold firm on education reform (Boston Globe)

MICHIGAN
State Board of Education uneasy about application (The Detroit News)

NEW YORK
Legislative action needed (Albany Times Union blog)

Editorial: The governor's desperate dash (Albany Times Union)

Lawmakers will vote on charter cap the day application is due (New York Daily News Daily Politics blog)

OHIO
Less than half of state districts sign on; union support expected (The Columbus Dispatch)

PENNSYLVANIA
Philly teachers support Race (Philadelphia Inquirer)

RHODE ISLAND
Lack of union support could weaken state's chances (Providence Journal)

TENNESSEE
Governor Bredesen unveils legislative plan (The Leaf Chronicle)

Governor Bredesen: "Seize the day" (Education Week via Chattanooga Times Free Press)

Governor, teachers reach compromise on teacher evaluations (The Commercial Appeal)

Union approves teacher evaluation based half on student achievement (Nashville Public Radio)

TEXAS
Out of Race (Austin American-Statesman blog)

WISCONSIN
Milwaukee Children's Zone a state focus (Milwaukee Journal Sentinel)

Updates on the Race: 01-11-2010

Playing catch up following the holidays and the birth of our second child ... here are the major Race to the Top updates a week before round 1 applications are due on January 19th...

NATIONAL
39 States and DC to apply in round one (U.S. Department of Education)

'Race To Top' Viewed as Template for a New ESEA (Education Week)

'Race To Top' Driving Policy Action Across States (Education Week)

Two State Unions Balking at 'Race To Top' Plans (Education Week)

CALIFORNIA
Assembly passes reform bill (Sacramento Bee)

Governor signs bill to improve state eligibility in Race; opposed by teachers' unions (Los Angeles Times)

Governor seeks to ease teacher firings (Los Angeles Times)

COLORADO
Summary of state plan (INDenver Times)

FLORIDA
Unions: State RttT plan is 'fatally flawed' (Orlando Sentinel)

Editorial: Unions must not walk (Miami Herald)

Editorial: Racing to the top (Orlando Sentinel)

ILLINOIS
Editorial: State legislature to be asked to pass teacher evaluation bill (Chicago Tribune)

KENTUCKY
State to be first to adopt new national academic standards (Lexington Herald-Leader)

LOUISIANA
Less than half of school district sign onto plan (New Orleans Times Picayune)

MASSACHUSETTS
House passes reform bill, would empower superintendents (Boston Globe)

MICHIGAN
Governor Signs Reform Legislation (AP)

MEA, AFT Leaders Told Their Support Isn't Needed (The Grand Rapids Press)

89 Percent of State Schools on Board (The Detroit News)

NEVADA
Governor proposes ban on collective bargaining, repeal of data firewall law (Las Vegas Review-Journal)

NEW JERSEY
Out-going, in-coming governors complicate state application (Eduflack)

Editorial: Race to the bottom (Bergen Record)

NEW YORK
Governor presses for education reforms, including lift of charter cap (New York Times)

City wants state charter cap lifted before signing onto RttT application (Gotham Schools)

OHIO
State seeks to avoid union controversy? (Flypaper via Teacher Beat)

RHODE ISLAND
Teachers unions balking at supporting application (Providence Journal)

TENNESSEE
Teachers' unions sign onto plan to tie student test scores to teacher evaluations (The Commercial Appeal)

Governor's education bills to be consider during special legislative session (The Tennessean In Session blog)

Special session on Race to the Top begins tomorrow (Memphis Daily News)

UTAH
Plan unveiled (The Salt Lake Tribune)

WISCONSIN
State lagging in Race to the Top (Milwaukee Journal Sentinel)

State seeks $254 million; application won't address mayoral control in Milwaukee (Milwaukee Journal Sentinel)

WYOMING
State finalizes application (Casper Star-Tribune)

The Newest Optimist

Happy New Year!

Liam and I are taking a bit of a break from blogging to spend time with the newest Education Optimist-- Annie Lucille Rae Goldrick! Annie joined our family on January 2, 2010, and is already a vigorous contender for the title of newest blogger in the family. She is a strong girl (weighing in at nearly 8 1/2 pounds) and has plenty to say on nearly every topic.

Expect us to return in a week or two...in the meantime, enjoy life! We sure are.

Sara