Jon Boeckenstedt’s presentation took the form of a “Personal and Professional Evolution” in three phases. He began by noting that he was a “diamond in the rough kid” (high test scores and low grades; first-generation student) the sort that is often cited by the ACT and the SAT as benefitting from standardized tests. Boeckenstedt said that while he knows that “for some kids, these tests can be a path up” they aren’t worth it for the bulk of the country.
I’ve been a big fan of DePaul’s since I paid them a spontaneous visit five years ago (and I still wear the t-shirt they gave out that day), and for some of the same reasons that I find WPI to be so admirable: a clear focus on their mission and on trying to make their admissions policies match their stated goals. Boeckenstedt said that DePaul’s mission “explicitly discusses helping students who wouldn’t otherwise have access to college.” He said that as their admissions pool grew, they feared that “students we feel compelled to serve might be squeezed out”. Further, internal studies of first-year students showed that students with good grades who study and work hard are likely to be “achievers and succeeders”, and the ACT and SAT don’t evaluate those attributes.
Boeckenstedt observed that the natural way to form a class is to lop off the bottom, but doing so “quantitatively” often results in culling the “kids you want to help the most.” He said that there was a sense of self-loathing in discussing applicants as “3.7, 1220”, as opposed to a more individualistic way, and told us that “we wanted to engage more deeply with kids”.
“Phase two” of the story is what Boeckenstedt called “The Onslaught of Stupid”. He warned the audience that “if your school goes test-optional, be prepared for know nothings to air their opinions” (while showing a Chicago Sun-Times blog headline proclaiming the “dumbing down” of DePaul).
After careful study, DePaul found that “test scores co-vary strongly with social capital, which is not what we are trying to do.” Boeckenstedt stated that “so much of what matters (work ethic, ‘showing up’, not playing video games) can’t be shown in an application at all. And test scores don’t add anything to the equation. But they have a low rate of false positives, so selective schools like them.” He went on to note that “many colleges have a faculty full of people who got where they are partly due to test scores”. The admissions office spent a year presenting to all of their constituencies (including faculty) before settling on their test-optional plan. Boeckenstedt said that he was able to convince people by saying “I can guarantee you that we live by the idea of academic quality.”
Annually, 10-12% of applicants are non submitters. Most students who apply to DePaul come from states that require scores, so most send them. But “scores are not a big factor. They are looked at as an interesting, extra, non-required item, equivalent to a student who sends a book of their poetry, or additional recommendation letters.
“Phase three” consisted of “pushback from major testing organizations”. Boeckenstedt (whose harvesting of public data to explain trends in higher education has been a staple of my reading for years) says that Wayne Camara (top psychometrician at the ACT) accused him of “not being able to read data–one year after I was invited to Iowa City to teach ACT engineers how to use Tableau software to present data”! He also mentioned the refusal to let him ask questions at last year’s NACAC presentation on the book that was sponsored by testing companies and featured research by the College Board’s and ACT’s pet scholars and that seemed determined to provide evidence for why schools should not choose test-optional policies.
Boeckenstedt concluded by telling us that “it dawned on me that people are unaware that the College Board and ACT are less accountable to government and the people than the average hot dog vendor on the streets of New York City. Their goal is to create and drive curricula to build and promote testing down to kindergarten, but they are accountable to no one but themselves.”
“Why do we take this?”, he asked. “I don’t know. This doesn’t mean you shouldn’t require tests, if that right for you. But I’m actually just sorry you have to sit through this crap every year.”
As always, Jon Boeckenstedt proved himself to be an engaging presenter. What came through to me was the point (that reinforced what Andrew Palumbo told us) that institutions considering test-optional policies need to make sure that their choice reflects the institutional mission as well as the long- and short-term goals of the school and that all stakeholders have a chance to weigh-in, while being assured that the policy is being changed to make the school better, with more capable students.
What also came through (from all three presenters), is that even at these prominent test-optional schools, the majority of students still submit their scores. Figuring out why they do so goes beyond the scope of this article, but it seems clear to me that testing companies can definitely co-exist with test-optional admissions policies, and that we are far from a “tipping point” that will lead to the tests’ irrelevance. Whether that is good or bad is also a topic for another day, but please share your thoughts below.
A question and answer session was next, and there were microphones set up in the aisles for people to use. While questioners were asked to identify themselves when posing their queries, not all did.
The first question came from Chad Austin at Kean University in New Jersey. Austin said that they are considering test-optional, but they rely on the tests for merit aid, and asked the panelists for their thoughts. Boeckenstedt said that DePaul has an academic index computed in such a way that doesn’t need scores; Palumbo reminded Austin that “this could be a chance to revisit your approach”, as WPI did. Finally, Soule pointed out that “using test scores to award scholarships is a way to guarantee that most of your money goes to affluent kids”.
David Sheehy, of Boise High School, asked if there is any evidence showing it is worthwhile to have students send extra materials in lieu of test scores (which many test-optional policies require). The consensus was that none of the three panelists’ institutions asked for anything extra.
Tara Miller, from Austin High School urged Bob Schaeffer and FairTest to do a better job engaging counselors. She said that test agencies work hard to reach out to counselors every year, and that many of her peers don’t know how to explain test-optional to their communities. Schaeffer’s answer was that FairTest is a “three FTE (full-time equivalent) organization. We wish we could do more to reach new counselors and students, but in the meantime, look at our website, www.fairtest.org” I have to say, I had no idea that FairTest was so small–very interesting!
A fellow calling himself “Christopher from Honolulu” (who has listened to too much Larry King, I think) said that his school is “investigating the Mastery Transcript, and eliminating grades, but the word we are getting from colleges is that test scores will be needed to make sense of our students.” He asked what information the panelists would want to see on an “unusual transcript”. Whitney Soule told him that “colleges will want a method of assessing kids in context; maybe not test scores, but we will need a way to understand what their achievements are. Bowdoin had kids apply from 4,500 different schools: we know how to evaluate different environments, but we need the high schools to help us.” Andrew Palumbo agreed that “your students are better off with the more context you can provide.” Jon Boeckenstedt opined that “the Mastery Transcript is another opportunity to give privileged white kids more opportunity to stand out. Not that we shouldn’t reduce stress on kids, but the Mastery Transcript isn’t the way.”
As a graduate of a college without grades I certainly applaud any effort to reduce subjectivity under the false flag of numerical objectivity on the part of learning institutions. But as someone who worked with a wide range of students applying to a bewilderingly broad array of colleges, I know that the people deciding on our students’ applications need to be able to put kids into context one way or another. I wish Christopher had told us the name of his school, because I’d love to find out what they eventually choose to do.
The remaining questions were not very substantive. A woman who didn’t identify herself asked a confusing question about yield, which the panel didn’t seem to know how to answer. Someone called Dorothy who said that she works with the State Department helping students from overseas to apply to American colleges asked if there are any “implications” if international students apply test optional (she clearly wasn’t listening to Whitney Soule). Whitney courteously chose not to point out that she’d already discussed this, and noted that Bowdoin only asks for a test of English proficiency for applicants from outside the US, with which the other panelists concurred.
The final question came from a man who also did not identify himself. He was a bit pugnacious, and pushed the panelists to say how “transparent” their policies are and if “kids can call you to ask if they should or shouldn’t submit their scores”. Whitney Soule fielded this one, saying “we can’t really answer this in a vacuum. The point of our focus on ‘context’ and ‘holistic admissions’ is that a ‘good score’ might vary depending on the applicant.” This was a very polite answer. If I was on the panel I would have probably asked if he and his students even bother to read the admissions websites to learn their test policies; in my experience these are almost always spelled out clearly, which is pretty “transparent”.
I found this presentation to be very informative and that it made me think. Not having ever worked on the college side of admissions I don’t think that I can add anything to the discussion of whether or not a particular college/university should decide to go to a test-optional policy. But what leaped out to me as someone who has spent a long time advising students and parents is that the number of places that will consider applicants who do not send test scores is growing, and that there is likely to not be a stigma or implied weakness for those applicants. At the same time, the vast majority of students do send test scores, even to test-optional schools, so in many cases there is really no “wrong answer” to the question of “should I send my scores”.
As I mentioned above, Bob Schaeffer was gleeful about news that came out the day before that the University of California system was planning to evaluate whether ACT and SAT scores were actually predictive of college success (which is, of course, a major argument the companies make in favor of their products). Reading about the announcement, however, makes me somewhat skeptical that this will prove to be the death knell of standardized testing that Schaeffer seemed to be hoping for. In fact, it just seemed to indicate that the University of California system wants to study whether their policies align with their mission, which is what the panelists endorsed. [note: in his email, Bob Schaeffer clarified that “What I tried to convey is that, just as lobbying UC to adopt the SAT as an admissions requirement was a key part of the College Board’s campaign to make it a national test (see Nick Lemann’s history, The Big Test), so too could a UC decision to de-emphasize standardized exam scores (as a result of this study) encourage many other institutions to go test-optional.”]
Personally, I do not see any sign of the standardized test companies losing prestige or market share anytime soon. While test scores were designed to only be useful to colleges and universities, the companies’ customers are the students who take the tests, not the admissions office that interpret the scores (in the sense that it is students who pay for the tests). And with more and more states moving to making the ACT and/or SAT a mandatory assessment, the pool of students taking the tests isn’t going to shrink significantly in my lifetime. If two-thirds or more of applicants at these test-optional colleges and universities still choose to send their scores, the test companies will continue making healthy (non) profits for the foreseeable future.