Ekdahl described the effect of ORCA as being that "30,000+ of the most active and fired-up volunteers were wandering around confused and frustrated when they could have been doing anything else to help, like driving people to the polls, phone-banking, walking door-to-door, etc." Campaign workers were left "flying blind", as several put it, unable to identify non-voters or precincts which needed a last-minute robocalling campaign to drive up turnout. The targeted information promised by the campaign did not materialize and only the generic raw vote tallies were available in key areas. "Your numbers don't matter to us." but argued that the system had actually worked, despite the reports of problems: "We don't think Orca's problems had a material impact on the campaign, it was not election determinative. We had 30,000 plus volunteers across the country putting information into the system. We had 91 percent of all counties report into the system, 14.3 million voters were accounted for as having voted, and we received 5,397 reports on voting issues, such as instances where they ran out of ballots. The information came in, so you can't say it didn't work. You run into issues because it's so massive in scale." Conservative writer Joel B. Pollak suggested that ORCA had ended up suppressing Romney's own vote by tying up campaign volunteers at a critical time. He noted the narrow margin in the key swing states – only some 500,000 to 700,000 votes – and calculated that if each of the 37,000 ORCA volunteers had brought 20 voters to the polls in those states, the gap could have been closed. Ekdahl saw a "bitter irony" in the fact that "a supposedly small government candidate gutted the local structure of GOTV efforts in favor of a centralized, faceless organization in a far off place (in this case, their Boston headquarters)." The ORCA system had not received extensive
beta testing before election day, nor did the campaign know how it would interact with the data infrastructure in the TD Garden until the day itself. Sean Gallagher of
Ars Technica commented that the key failure was the dependency on automated testing rigs, which "can't show what the system's performance will look like to the end user. And whatever testing environment Romney's campaign team and IT consultants used, it wasn't one that mimicked the conditions of Election Day. As a result, Orca's launch on Election Day was essentially a beta test of the software – not something most IT organizations would do in such a high-stakes environment."
Slate writer Sasha Issenberg argued that the problems ran far deeper than ORCA's technical failings, as the Romney campaign had been left behind by the cutting edge of
data science. He noted that while a system like ORCA could not have changed the demographics, data science did make a great difference to the ability of the two campaigns to target and mobilize their voters. As he put it, "The Democrats have it and the Republicans don't." He suggested that ORCA's ability to affect the outcome had been over-hyped by the Romney campaign, as there was only so much that could be done on election day itself: "On short notice, you can send robocalls, reorder a call list and employ paid phone banks, but you are not radically changing the shape of the electorate. They acted like they had invented the wheel, but really all it would have been was a slightly better tread on the tire." ==See also==