What I Saw At NACAC 2018: A Summary of “Test Optional Admission: Current Trends, Best Practices, and Future Prospects”

I attended the 2018 National Association for College Admission Counseling conference in Utah in the last week of September. This was the fourth straight year I’ve been able to attend, thanks to my day job with Method Test Prep, a leading ACT/SAT preparation company. I was there to work, and consequently spent most of my time in the exhibit hall meeting current and future customers, but fortunately I was able to attend a session on the Thursday called “”Test Optional Admission: Current Trends, Best Practices, and Future Prospects”.

As a representative of a test prep company, and more significantly as a proud alumn of Hampshire College, which was test optional since its opening in 1970 (the college and I are the same age) and four years ago became the only “test blind” 4-year institution in America, I have had a deep interest in the “test optional movement” for decades. I was very interested to learn more about it, and I wasn’t disappointed. The presentation was a good one, with several thought-provoking moments.

 ****

The session met in a very large room with many hundreds of chairs. I had left my phone back at the exhibit hall which vexed me for two reasons: the pedometer didn’t count what must have been over a thousand steps to get to the room (the Salt Lake City convention center is surprisingly large!) and I couldn’t take a picture of the space. While the room was not “packed”, there were around 650 chairs and around half were occupied. My suspicion was that the session would attract more people from the college/university side of things, but a closer look (and a show of hands poll) made it seem that there were probably more high school counselors/independent counselors in the room. Interesting!

****

The presenters were:

Bob Schaeffer, a founding member and the public face of the National Center for Fair and Open Testing (FairTest).

Whitney Soule, Dean of Admissions and Financial Aid at Bowdoin College

Andrew Palumbo, Dean of Admissions and Financial Aid at Worcester Polytechnic Institute

Jon Boeckenstedt, Associate Vice President for Enrollment Management and Marketing at De Paul University

If you’ve been reading this blog over the years, you know that I am a big fan of Jon Boeckenstedt’s views on the social value of college admission, his facility with words and data, and his contributions to the field. Last year at NACAC I wrote about a session that featured a refusal to address his questions from the floor, and he follows me on Twitter, which makes me think I’m cool (in a geeky sort of way). I’ve met Bob Schaeffer at previous NACAC conferences and had chatted with him earlier that morning, during which conversation he laughed at the irony of a Hampshire grad working for a test prep company and seemed very excited about a news article from California that he would mention during the session. I have not previously interacted with Whitney Soule or Andrew Palumbo.

The session started on time and Bob was the moderator as well as a speaker. He told us that handouts would be available in the conference’s mobile app for us to download later, but they have yet to appear 10 days after the session. Unfortunately, my lack of a camera and my trust that the slides would be downloadable mean that some of the data I refer to below may not be wholly accurate, but represent my best efforts to type notes while the speakers were presenting. Any mis-statements are mine, and not the fault of the panelists. [Note: Bob Schaeffer sent me the speaker’s slides after I published this post, so I have re-edited it to make sure that numbers are correctly represented.]

******

Bob Schaeffer spoke first, introducing the speakers. He noted that Bowdoin is celebrating the 50th anniversary of becoming the first test-optional college, and that Whitney Soule and Andy Palumbo had spent their whole careers at test-optional schools, and that Jon Boeckenstedt would have “numbers and critical thinking” to offer us.

“49 years and some months since Bowdoin started the test-optional movement”, Schaeffer noted, “there are now 1,022 accredited, four-year test- optional colleges and universities, including more than half of the ‘Top 100’ national liberal arts colleges and many national universities.” Schaeffer mentioned the annual Inside Higher Ed survey showing that “a majority of counselors want to do away with the ACT and SAT”. I haven’t done a blog post on that article, but I did write a mini-Twitter thread on it, if you are interested. Schaeffer concluded his remarks by noting that “more than half of all the schools in the Northeast and a growing number in the mid-Atlantic” have test-optional policies, and he quoted the 2018 Fiske Guide as saying ” Students can avoid getting involved in the admissions test rat race and still have a range of colleges and universities from which to choose.”

Overall, Bob Schaeffer’s tone was similar to the triumphalist one that pervades almost every publication from FairTest, nearly all of his emails to the NACAC e-list, and I assume most of his conversations. While he surely knows better, Schaeffer seems to conflate “test optional policy” with “students don’t have to take these tests” and “the tests are dying out” (the latter two are not direct quotes, but the impression I get from him). I was a college counselor for over eight years and I always urged students to consider test-optional destinations if I thought that their scores might not represent their capabilities as students, but in many cases admissions policies are not in line with policies for awarding aid, or for sports eligibility, or other reasons (such as national origin of the applicant). All of which makes the test-optional scene a very pleasing range of grey shades, and not a black or white proposition. [note: after Bob Schaeffer read this post, he wrote that ” I always try to explain that ACT/SAT “optional” policies mean exactly that: applicants have the option — or choice — about whether to have their scores considered in the admissions process.  FairTest has never claimed that test-optional admissions is the same as ignoring standardized exam results in all circumstances…”]

******

Whitney Soule began her remarks by noting that her 27 years in admissions have all been test-optional. According to Soule, “multiple studies have shown that Bowdoin’s academic assessment is a good predictor of first year success with or without test scores.” She said that at Bowdoin, the transcript and school profiles put students into context, as do student essays and recommendation letters; if test scores are submitted, they are also used contextually.

Soule told us that Bowdoin became a test-optional college out of “principles of access”, and that over the years, typically 66%-75% of applicants do send test scores. As she said “we all know that anyone applying to Bowdoin has taken these tests multiple times”, but she was clear that Bowdoin performs a “holistic review” and that test optional is an important part of that.

“I know that counselors are skeptical that there isn’t an implied weakness for kids who don’t send scores,” Soule said. “We don’t hit a speed bump, we don’t stop and think and try to imagine what the applicants’ scores were, or their motives for not sending scores for us to review. We’re not asking because we make affirmative decisions derived from the materials students do send. Test scores are not heavily weighted, rather they are an extra piece of information, like an interview.”

Soule’s final point was that any school that uses test scores (whether they practice holistic admissions or not) “needs to understand what they are using them for, and that expectations match what the test was designed to do.” This was a thought-provoking statement, and also a great segue to the next speaker. I say “thought-provoking”, because in my line of work, I deal with many people who are not using the tests for their designed purpose. As you may know, dozens of states require all graduates to take either the ACT or SAT (and these highly profitable contracts are the biggest area of growth for the testing companies). Consequently, many schools are evaluated on the ACT or SAT scores of their students-even the ones who do not intend to attend a four-year college. A major problem at these schools is that the administration place great value on an assessment that many students find valueless, and that tests a lot of students on material that they have not had to master in their high school careers. I’m always glad to be able to help these schools to help their students, but we are definitely dealing with “off-label” uses of these college admissions tests.

******

Andrew Palumbo noted that, like Whitney Soule, he has only worked at test-optional schools but at his previous three workplaces (Union College, the Sage Colleges, and Plymouth State) he was part of the decision to switch to test-optional. “You can imagine my relief”, he said, “when I got to WPI and they were already test-optional.”

Palumbo observed that WPI became the first national STEM university to go test-optional in 2007, but that “test scores were never a major factor” at WPI. He told us that there are certain pillars of the WPI Plan, including:

•Collaborative learning

•Project Based learning

•Accelerated terms

•Non-punitive grading

and that they wanted their admissions process to reveal students who would “thrive” at WPI. He said that “standardized testing doesn’t match up” with such an individualized program, and that scores don’t show how well students work with others, or their independence. Palumbo said that the most common questions in admissions committee deliberations is “How would this applicant perform on a project team?” and that question cannot be answered by standardized test scores.

Palumbo told us that WPI’s goal is “a more effective and equitable admissions process”, and that in the decade since going test-optional, the student body grew by 41%, the number of women by 81%, under-represented students of color increased by 156% and under-represented women students of color by 232%. Besides these undeniably good results, Palumbo stated that Worcester Polytechnic has continued to review all of their policies; recently they stopped giving scholarships to National Merit students because it is based on standardized test scores that not all students will have or will want to submit.

I think that is super; I was already positively disposed to WPI thanks to the experiences of some of my former students there, but I really liked hearing how thoughtfully they work to make sure that their admissions policies are aligned with their institutional mission. One last thing that stood out was that Palumbo said that WPI had previously required students who applied without test scores to complete some other steps that students who sent scores did not have to do. The university realized that this might have inadvertently been a barrier to applicants, and in the first year after dropping these requirements, the number of “non-submitters increased three-fold”, as did diversity.

Bravo! While FairTest and mainstream media so often present “test-optional” as an elimination of standardized testing, closer reading of the press releases from schools reveals that this is not usually the case, and that it either isn’t true for all applicants, or that non-submitters have extra hurdles. I was very pleased to learn that WPI has eliminated those obstacles, and encourage other admissions offices to consider doing the same.

******

Jon Boeckenstedt’s presentation took the form of a “Personal and Professional Evolution” in three phases. He began by noting that he was a “diamond in the rough kid” (high test scores and low grades; first-generation student) the sort that is often cited by the ACT and the SAT as benefitting from standardized tests. Boeckenstedt said that while he knows that “for some kids, these tests can be a path up” they aren’t worth it for the bulk of the country.

I’ve been a big fan of DePaul’s since I paid them a spontaneous visit five years ago (and I still wear the t-shirt they gave out that day), and for some of the same reasons that I find WPI to be so admirable: a clear focus on their mission and on trying to make their admissions policies match their stated goals. Boeckenstedt said that DePaul’s mission “explicitly discusses helping students who wouldn’t otherwise have access to college.” He said that as their admissions pool grew, they feared that “students we feel compelled to serve might be squeezed out”. Further, internal studies of first-year students showed that students with good grades who study and work hard are likely to be “achievers and succeeders”, and the ACT and SAT don’t evaluate those attributes.

Boeckenstedt observed that the natural way to form a class is to lop off the bottom, but doing so “quantitatively” often results in culling the “kids you want to help the most.” He said that there was a sense of self-loathing in discussing applicants as “3.7, 1220”, as opposed to a more individualistic way, and told us that “we wanted to engage more deeply with kids”.

“Phase two” of the story is what Boeckenstedt called “The Onslaught of Stupid”. He warned the audience that “if your school goes test-optional, be prepared for know nothings to air their opinions” (while showing a Chicago Sun-Times blog headline proclaiming the “dumbing down” of DePaul).

After careful study, DePaul found that “test scores co-vary strongly with social capital, which is not what we are trying to do.” Boeckenstedt stated that “so much of what matters (work ethic, ‘showing up’, not playing video games) can’t be shown in an application at all. And test scores don’t add anything to the equation. But they have a low rate of false positives, so selective schools like them.” He went on to note that “many colleges have a faculty full of people who got where they are partly due to test scores”. The admissions office spent a year presenting to all of their constituencies (including faculty) before settling on their test-optional plan. Boeckenstedt said that he was able to convince people by saying “I can guarantee you that we live by the idea of academic quality.”

Annually, 10-12% of applicants are non submitters. Most students who apply to DePaul come from states that require scores, so most send them. But “scores are not a big factor. They are looked at as an interesting, extra, non-required item, equivalent to a student who sends a book of their poetry, or additional recommendation letters.

“Phase three” consisted of “pushback from major testing organizations”. Boeckenstedt (whose harvesting of public data to explain trends in higher education has been a staple of my reading for years) says that Wayne Camara (top psychometrician at the ACT) accused him of “not being able to read data–one year after I was invited to Iowa City to teach ACT engineers how to use Tableau software to present data”! He also mentioned the refusal to let him ask questions at last year’s NACAC presentation on the book that was sponsored by testing companies and featured research by the College Board’s and ACT’s pet scholars and that seemed determined to provide evidence for why schools should not choose test-optional policies.

Boeckenstedt concluded by telling us that “it dawned on me that people are unaware that the College Board and ACT are less accountable to government and the people than the average hot dog vendor on the streets of New York City. Their goal is to create and drive curricula to build and promote testing down to kindergarten, but they are accountable to no one but themselves.”

“Why do we take this?”, he asked. “I don’t know. This doesn’t mean you shouldn’t require tests, if that right for you. But I’m actually just sorry you have to sit through this crap every year.”

As always, Jon Boeckenstedt proved himself to be an engaging presenter. What came through to me was the point (that reinforced what Andrew Palumbo told us) that institutions considering test-optional policies need to make sure that their choice reflects the institutional mission as well as the long- and short-term goals of the school and that all stakeholders have a chance to weigh-in, while being assured that the policy is being changed to make the school better, with more capable students.

******

What also came through (from all three presenters), is that even at these prominent test-optional schools, the majority of students still submit their scores. Figuring out why they do so goes beyond the scope of this article, but it seems clear to me that testing companies can definitely co-exist with test-optional admissions policies, and that we are far from a “tipping point” that will lead to the tests’ irrelevance. Whether that is good or bad is also a topic for another day, but please share your thoughts below.

A question and answer session was next, and there were microphones set up in the aisles for people to use. While questioners were asked to identify themselves when posing their queries, not all did.

The first question came from Chad Austin at Kean University in New Jersey. Austin said that they are considering test-optional, but they rely on the tests for merit aid, and asked the panelists for their thoughts. Boeckenstedt said that DePaul has an academic index computed in such a way that doesn’t need scores; Palumbo reminded Austin that “this could be a chance to revisit your approach”, as WPI did. Finally, Soule pointed out that “using test scores to award scholarships is a way to guarantee that most of your money goes to affluent kids”.

David Sheehy, of Boise High School, asked if there is any evidence showing it is worthwhile to have students send extra materials in lieu of test scores (which many test-optional policies require). The consensus was that none of the three panelists’ institutions asked for anything extra.

Tara Miller, from Austin High School urged Bob Schaeffer and FairTest to do a better job engaging counselors. She said that test agencies work hard to reach out to counselors every year, and that many of her peers don’t know how to explain test-optional to their communities. Schaeffer’s answer was that FairTest is a “three FTE (full-time equivalent) organization. We wish we could do more to reach new counselors and students, but in the meantime, look at our website, www.fairtest.org” I have to say, I had no idea that FairTest was so small–very interesting!

A fellow calling himself “Christopher from Honolulu” (who has listened to too much Larry King, I think) said that his school is “investigating the Mastery Transcript, and eliminating grades, but the word we are getting from colleges is that test scores will be needed to make sense of our students.” He asked what information the panelists would want to see on an “unusual transcript”. Whitney Soule told him that “colleges will want a method of assessing kids in context; maybe not test scores, but we will need a way to understand what their achievements are. Bowdoin had kids apply from 4,500 different schools: we know how to evaluate different environments, but we need the high schools to help us.” Andrew Palumbo agreed that “your students are better off with the more context you can provide.” Jon Boeckenstedt opined that “the Mastery Transcript is another opportunity to give privileged white kids more opportunity to stand out. Not that we shouldn’t reduce stress on kids, but the Mastery Transcript isn’t the way.”

As a graduate of a college without grades I certainly applaud any effort to reduce subjectivity under the false flag of numerical objectivity on the part of learning institutions. But as someone who worked with a wide range of students applying to a bewilderingly broad array of colleges, I know that the people deciding on our students’ applications need to be able to put kids into context one way or another. I wish Christopher had told us the name of his school, because I’d love to find out what they eventually choose to do.

The remaining questions were not very substantive. A woman who didn’t identify herself asked a confusing question about yield, which the panel didn’t seem to know how to answer. Someone called Dorothy who said that she works with the State Department helping students from overseas to apply to American colleges asked if there are any “implications” if international students apply test optional (she clearly wasn’t listening to Whitney Soule). Whitney courteously chose not to point out that she’d already discussed this, and noted that Bowdoin only asks for a test of English proficiency for applicants from outside the US, with which the other panelists concurred.

The final question came from a man who also did not identify himself. He was a bit pugnacious, and pushed the panelists to say how “transparent” their policies are and if “kids can call you to ask if they should or shouldn’t submit their scores”. Whitney Soule fielded this one, saying “we can’t really answer this in a vacuum. The point of our focus on ‘context’ and ‘holistic admissions’ is that a ‘good score’ might vary depending on the applicant.” This was a very polite answer. If I was on the panel I would have probably asked if he and his students even bother to read the admissions websites to learn their test policies; in my experience these are almost always spelled out clearly, which is pretty “transparent”.

******

I found this presentation to be very informative and that it made me think. Not having ever worked on the college side of admissions I don’t think that I can add anything to the discussion of whether or not a particular college/university should decide to go to a test-optional policy. But what leaped out to me as someone who has spent a long time advising students and parents is that the number of places that will consider applicants who do not send test scores is growing, and that there is likely to not be a stigma or implied weakness for those applicants. At the same time, the vast majority of students do send test scores, even to test-optional schools, so in many cases there is really no “wrong answer” to the question of “should I send my scores”.

As I mentioned above, Bob Schaeffer was gleeful about news that came out the day before that the University of California system was planning to evaluate whether ACT and SAT scores were actually predictive of college success (which is, of course, a major argument the companies make in favor of their products). Reading about the announcement, however, makes me somewhat skeptical that this will prove to be the death knell of standardized testing that Schaeffer seemed to be hoping for. In fact, it just seemed to indicate that the University of California system wants to study whether their policies align with their mission, which is what the panelists endorsed. [note: in his email, Bob Schaeffer clarified that “What I tried to convey is that, just as lobbying UC to adopt the SAT asan admissions requirement was a key part of the College Board’s campaignto make it a national test (see Nick Lemann’s history, The Big Test), sotoo could a UC decision to de-emphasize standardized exam scores (as aresult of this study) encourage many other institutions to go test-optional.”]

Personally, I do not see any sign of the standardized test companies losing prestige or market share anytime soon. While test scores were designed to only be useful to colleges and universities, the companies’ customers are the students who take the tests, not the admissions office that interpret the scores (in the sense that it is students who pay for the tests). And with more and more states moving to making the ACT and/or SAT a mandatory assessment, the pool of students taking the tests isn’t going to shrink significantly in my lifetime. If two-thirds or more of applicants at these test-optional colleges and universities still choose to send their scores, the test companies will continue making healthy (non) profits for the foreseeable future.

I hope you found this to be an interesting post. If you are looking for more, here is the summary of the session from Inside Higher Education that includes some useful links.