10 February 2006

Mandatory testing for colleges?

When I first saw this article, I immediately assumed it was a parody. I guess truth is sometimes stranger (and more radikyulus) than fiction:

A higher education commission named by the Bush administration is examining whether standardized testing should be expanded into universities and colleges to prove that students are learning and to allow easier comparisons on quality.

The Bush administration is going to make sure people in college are really learning? Someone reach through cyberspace and pinch me, please! I want to wake up!

Charles Miller, a business executive who is the commission's chairman, wrote in a memorandum recently to the 18 other members that he saw a developing consensus over the need for more accountability in higher education. "What is clearly lacking is a nationwide system for comparative performance purposes, using standard formats," Mr. Miller wrote, adding that student learning was a main component that should be measured.

Translation: "We should create a test that everyone can teach to that is filled with non-controversial 'facts.' Anyone who teaches anything else will effectively penalize their students and ultimately reduce funding for their programs." What I'd like to know is what the college accreditation boards have been doing all this time? Isn't this their job? Have all these people who were reportedly running from college to college evaluating programs really been vacationing in the Bahamas all this time? It's good to know that their detailed research into college programs can be replaced by a multiple-choice quiz.

Mr. Miller was head of the Regents of the University of Texas a few years ago when they directed the university's nine campuses to use standardized tests to prove students were learning. He points to the test being tried there and to two other testing initiatives as evidence that assessment of writing, analytical skills and critical thinking is possible.

Isn't this what the GRE and other assessment's test?

The Commission on the Future of Higher Education, appointed last fall by the secretary of education, Margaret Spellings, has until August to make a report on issues that include accountability, cost and quality. Educators are wary. "To subject colleges to uniform standards is to trivialize what goes on in higher education," said Leon Botstein, president of Bard College. "Excellence comes in many unusual ways. You cannot apply the rules of high-stakes testing in high schools to universities."

"Trivialize." A euphemism for "undermine."

In an interview, Mr. Miller said he was not envisioning a higher education version of the No Child Left Behind Act, which requires standardizing testing in public schools and penalizes schools whose students do not improve. "There is no way you can mandate a single set of tests, to have a federalist higher education system," he said.

We are entering the Twilight Zone. Is he saying you would have different tests yet be able to make valid comparisons? I'd like to hear about these new statistical tools that are able to make comparisons across completely different measures. And since colleges already have different tests, I'm a bit perplexed about the need to create a new set.

But he said public reporting of collegiate learning as measured through testing "would be greatly beneficial to the students, parents, taxpayers and employers" and that he would like to create a national database that includes measures of learning. "It would be a shame for the academy to say, 'We can't tell you what it is; you have to trust us,' " Mr. Miller said.

"What it is" is "education." And you don't have to trust anyone. If you don't like the college's curriculum, don't go there.

He said he would like the commission to agree on the skills college students ought to be learning — like writing, critical thinking and problem solving — and to express that view forcefully. "What happens with reform," he said, "is that it rarely happens overnight, and it rarely happens with a mandate."

I think the Bush administration needs to worry about itself. The country would be a hell of a lot better off if Shrub and his fellow neo-cons had engaged in critical thinking, problem solving, and an honest desire to get at the objective truth.

"It does happen with levers," Mr. Miller added, "and maybe the accreditation process will be one. Or state legislators. Or members of Congress." His push comes as college officials in an era of high tuition say they already feel pressure to justify costs. But university officials are wary of the notion that testing regimes should be used to measure all the different institutions that make up American higher education — small liberal arts colleges, large public universities, proprietary schools and religious academies — particularly if there is government involvement.

It's interesting. Even the arch-conservative Bloom has stated that the institutions of higher education need to be, in the positive sense of the word, "elitist" and stand apart from government and popular movements. This move would essentially put government in charge of deciding what education consists of.

David L. Warren, president of the National Association of Independent Colleges and universities, a group representing private, nonprofit colleges and universities, said: "What we oppose is a single, national, high-stakes, one-size-fits-all, uber-outcome exam. The notion of a single exam implies there are national standards, and that implies a national curriculum. Then we are on the way to a centralized Prussian education system."

"Centralized." Everything the Bush regime does has this singular motive: the endless push to centralize all power in the executive.

When Ms. Spellings, the education secretary, named the commission, she said that choosing a college was one of the most important and expensive decisions families make and that they were entitled to more information.

There's all sorts of information readily available, ranging from course curriculums, college rankings, and statistics on graduates' job success. If people want to see how students do on a multiple choice test, someone can compile stats on graduates' GRE scores when they apply for graduate school.

There is no unanimity on the commission, but some members also expressed interest in measuring student learning. Kati Haycock, a commissioner who is director of the Education Trust in Washington, which has supported standardized testing, said in an e-mail message: "Any honest look at the new adult literacy level data for recent college grads leaves you very queasy. And the racial gaps are unconscionable. So doing something on the assessment side is probably important. The question is what and when."

I think we should start testing government officials. It's only reasonable that tax payers know that candidates can really do the job. I'm betting that my grade-school daughter would beat the president in a basic geography test.

Jonathan Grayer, another commissioner, who is chief executive of the test-coaching company Kaplan Inc., said that with so many students in college and so many tax dollars being spent, "it is important for us to seek some type of knowledge about how much learning is going on." "What I am for is for institutions on their own or in groups to seek their own standards to show what they are achieving," Mr. Grayer said. "Whether that should be elective or mandatory, that is something the commission is thinking about."

Institutions do have "their own standards." Unless you're talking about UC Santa Cruz, these standards are put out in the form of something called "grades."

The question of how to assess higher education has been simmering for years. In the mid-1980's, the Department of Education directed the groups that accredit colleges and universities to include assessments of student academic achievement. College students have always been graded on exams, but there were relatively few standardized measures of the skills they had when they left college, except for licensing exams and graduate school admissions tests. And even those did not show how much the students had learned.

"The unanswered question in higher education is: How good is the product?" said Robert Zemsky, a commission member who is a professor of education at the University of Pennsylvania. "A growing number of people are beginning to want answers. What higher education is about to learn is that they can't play the 'trust me' game anymore."

It's the government that we need to stop simply trusting. We pay only part of the money that goes to colleges (and benefit from the positive externalities associated with education), whereas we pay all of the money that goes to government. I'd like to know what the government's doing without constantly being told "just trust us."

Part of what is driving the demand for accountability is money. Ms. Spellings has said that about one-third of the annual investment in higher education comes from the federal government and that officials know very little about what they are getting in return.
In addition, there has been growing attention to how many college students drop out and how poorly even graduates perform in the workplace and on literacy tests in an era of rising global competition. The National Assessment of Adult Literacy, given in 2003 by the Department of Education, found that less than a third of the college graduates it surveyed demonstrated that they were able to read complex English texts and draw complicated inferences.

I've heard that some Yale and Harvard grads have been particularly disappointing . . .

It is not clear whether the commission would recommend that funding be used as an incentive for testing or else withheld from colleges that refuse to use standardized testing. Although public universities seem most vulnerable to regulatory oversight because they are subsidized by state taxpayers, Mr. Miller points out that private colleges are subject to regulation, too. They are accredited by groups authorized by the federal government. And they must meet certain standards to qualify for federal grants and financial aid. "What we call public universities would be under the most pressure," Mr. Miller said. "But the question is, How public are some of the private universities? They depend a lot on public funding, too. And we have shifted more of the cost back to students. So I think consumers and other people will begin to ask questions like this more." "It would be O.K. with me," Mr. Miller added, if individual institutions like the elite universities in the Ivy League did not want to offer measures of student learning on standardized tests. "But would it be O.K. with everyone?" he asked, referring to their trustees, their donors, potential employers and others.

Of course, being sure that tests measure what students learn is difficult. Peter T. Ewell, a testing expert at the National Center for Higher Education Management Systems in Colorado, said it was hard for students to take tests seriously unless they were "embedded in the curriculum."

"You have to provide incentives for students to want to do it and to do their best," he said. Still, numerous colleges are experimenting. Mr. Miller, in his recent memorandum and in the interview, pointed to the recently developed Collegiate Learning Assessment test as a breakthrough. The exam, developed by the Council for Aid to Education, a former division of the RAND Corporation, asks students to write essays and solve complex problems. Mr. Miller also cited a recent demonstration project backed by the National Center for Public Policy and Higher Education and a computerized version of a test from the Educational Testing Service in New Jersey.

The University of Texas has worked for several years to address the Regents' mandate that its campuses use standardized testing to assess student learning. Pedro Reyes, the system's associate vice chancellor for academic affairs, said the campuses first tried to develop their own tests but concluded it was too time-consuming. Next, they turned to an Educational Testing Service exam, but found it "didn't quite get what we were looking for," Dr. Reyes said. The university then turned to the Collegiate Learning Assessment exam, which Dr. Reyes called "a much better tool not only to improve student learning but also to enable conversations about academic expectations and standards."

Not everyone on the Texas campuses is enthralled with standardized testing. John R. Durbin, a mathematics professor at the University of Texas at Austin and former faculty council officer, said in an e-mail message, "It would be a sad state of affairs if the people at the top had so little confidence in our faculty that they really believed outside bureaucrats and committees could help us raise standards."

Mr. Miller expressed confidence that the process would improve learning. "I think the process has been very effective," he said. "The surprising thing is that people who went through it, some of them reluctantly, all felt they had gained."


Ryan Mc said...

Wouldn't this test still be biased because there would be no way to police private universities.

BadTux said...

Education is not a product. Education is the inculcation of an ability to think rationally about a topic of concern. Being able to rote recite answers is not eductation. That is training. Training is necessary, but not sufficient, to have an educated populance.

It is clear that the actual target here is to remove the last vestiges of education from the American system. Apparently the powers that be have decided that they don't need even the small number of educated people who manage to stagger through over a decade of mind-numbing training and learn something in college besides how to recite rote facts. Those small numbers of educated people are beginning to be a nuisance and a hinderance to their goal of a Christopath United States, One Nation Under Pat Robertson, and must be destroyed. Any smart people they need? Well, just import'em from India or some other country stupid enough to have not destroyed its educational system.

Evil. Evil evil evil. These SOB's make Cthulhu look like a Boy Scout.

- Badtux the "Cthulhu is a piker" Penguin

drmagoo said...

Actually, colleges and universities are already evaluated by a process far more detailed than any standardized test. They have to get re-accredited every ten years. What does that entail? Well, the school has to (a) define what it does, (b) develop a set of assessments that measure whether or not students are learning in a broad range of categories, and (c) get visited by a team from a national accrediting organization which, during an extensive visit, reviews the report produced by the school, interviews a lot of people, and discusses the assessments. It's the procedure we should be using for other schools (with some modifications), if it wouldn't be so costly to do so.

In fact, colleges and universities developed the accreditation system in response to the government saying "either police yourself, or we will". So there's no need to get testing like this involved.hx

drmagoo said...

Actually, BadTux, education is a product - consumers pay for something, and then they receive it. It's a different sort of product than a widget one buys in the store, in that the value of the product comes largely in how much work the consumer puts into learning, but it's still a product, and operating a university is somewhat of a business (even at the smallish school I teach at, we've got a 40+ million dollar annual budget to deal with). The danger comes when people see the degree as the product, not what it took to get the degree, and are primarily interested in the cheapest degree they can get.

Karlo said...

I think what BadTux is getting at, and what I'd agree with, is that education encompasses much more than what can be tested on some standardized test. And you're absolutely right--colleges are already accredited, a process that's much more detailed than any test could be. If the loony idea of standardized college testing ever becomes a reality, I think we'll find that some skills are easy to test and some aren't, and as a result, the former will become much more important. Myself, I did a portfolio assessment for one of my degrees and it was one of the most valuable things I've ever done and I think it did an excellent job of showing what I learned and how I connected what I learned to my field. But the knowledge incorporated into a portfolio--which includes focuses particular to the individual--is hardly the sort of thing that could be put on a standardized test.

drmagoo said...

I agree with what you're saying, but it gets rather complicated sometimes. Education is something that is difficult, if not impossible, to assess entirely with standardized tests. We do it at the K-12 level because it's very expensive to do it any other way, and we don't have the national will to spend that kind of money. However, people spend an awful lot of money on higher ed, and accrediting boards, in response to that, have increased the pressure on schools to prove that they do what they say, which means that we need to assess student learning. Sometimes those tests are standardized, sometimes they're portfolio reviews. But in the end, if we can't convince our customers that we're giving them a product worth what they're paying for it, we go out of business.

Karlo said...

It seems to me that the "proof is in the pudding," as they say. If graduates enter job fields with inadequate skills, companies will gradually become less impressed with the particular degree, and the opposite is also true. Tests have a tendency to avoid conceptual constructs that are messy or controversial--just the sorts of things that a good university education should deal with. Most importantly, the really important skills involving analytical and critical skills are best seen in long-term projects, the longer and more extensive the better. Such projects demonstrate a student's ability to incorporate the little bits of information they have into larger schemes. This is why virtually all universities prefer theses as final projects instead of tests. Not that the latter don't have their place. But they don't cover enough, and as a result, do a poor job at characterizing students' overall learning.

BadTux said...

Education is not a product. Education is an outcome. Training is a product, indeed, the product most often purchased by customers of higher education. By Orwellianly redefining the word "education" to mean "training", you are playing into the American Taliban's hands, because they want to remove education (an outcome -- people able to think) from American soil. Because education is toxic to their dreams of one nation, under Pat Robertson, glory hallaleujah AMEN.

- Badtux the Snarky Penguin

Karlo said...

It's ironic. Just as Asian countries like Korea and Japan are abandoning the emphasis on rote learning in place or curricula emphasizing critical thought, the U.S. is moving to an emphasis on memorizing little bits of information and multiple choice tests.