Opening the curtain on Emily Glassberg Sands, Part I

Emily Glassberg Sands puts over her presentation. NY Times photo by Chester Higgins.

What felt like my personal "gotcha" moment with Emily Glassberg Sands (above, late of Princeton, now at Harvard) came halfway through my phone conversation with her a week or two ago. I'd just finished reading her study, "Opening the Curtain on Playwright Gender, etc." and I was in two frames of mind about it. I was greatly impressed by the thoroughness and forethought of its central portion (the much-discussed "script experiment"), but I was confused and uncertain when it came to the vague methodology of her final conclusions, in which she seemed to find that Broadway producers had closed female-written shows early, despite their above-average weekly revenues.

Now the ultra-articulate Sands had been in high gear from the very start of the conversation, but as I got closer to my concerns, she began to power-chatter at a nearly alarming rate. I kept trying to steer the conversation to what I thought should be the central question of her final chapter - were those closed shows actually more profitable than the male-written shows she was comparing them to? (Because profitability, or the lack thereof, is the reason producers close shows.) But every time I tried to phrase this question, Emily deflected it by claiming that I wouldn't understand her even if she explained her method, that I wasn't trained enough in statistics to comprehend what she was doing, etc., etc. Finally, I managed to blurt out the full question:

"Emily - do you or do you not have valid data on profitability?"

And suddenly the chatter ended abruptly; a long silence ensued. And ensued. For what seemed like minutes. "Emily? Emily, are you there?" I asked. "Yes, I'm still here," she answered quietly, but said nothing more. "Can you answer my question?" I then ventured. "I'm still here," she repeated. And again fell absolutely silent.

Then she began trying to end the call. "Look, I don't really have time for this," she said. "I've been working on this for more than a year, I've talked about it to a lot of important people!"

"Yes, but Emily - do you have valid data on profitability?"

She then let me know that the paper had been accepted for publication by the MIT Press; she'd already told me she would soon be on The Colbert Report. "I've been told that people like you would be calling me!" she said (which I took to mean people had predicted lonely, gay bloggers would soon be on her tail). "You're just going to write something scathing, aren't you," she went on bitterly. "I have to go!"

"But - "

"I have to cook, I have to clean! I have things to do!"

And so we went back and forth for awhile, during which time I just managed to sneak in my final question:

"But Emily, if producers are ending profitable shows, can you speculate as to why? If a producer were sexist, he wouldn't produce a show by a woman to begin with, would he? Why would he produce it, then shoot himself in the foot by killing his own profits?"

There was another long pause.

"Because there are always costs to taste-based discrimination," she finally offered.

"And that's what you think this is."

"Yes. I'm signing off now."

And that was that. I followed up with an e-mail (posted below) to try to tease out a more credible answer, but she hasn't replied, and somehow I don't think she ever will.

Because, of course, there is no reply. There's a gaping hole in the final chapter of her study, and smart girl that she is, Sands knows this as well as I do. And you don't have to be a statistics whiz to see it; indeed, I'd like to point out to all the dazzled liberal arts majors out there that Sands doesn't actually do anything too mathematically sophisticated in her study (although yes, its design and administration are admirably complex) and I had no real trouble following it until she suddenly began to fudge things to come up with her "profitability" proxy. Of course if you don't understand regressions and don't know what natural logs are, her equations look like some form of indecipherable magic. To someone like me, with a rusty knowledge of statistics and college-level math, they were at first heavy going, but after some glances back in the old textbooks, they pretty much came clear.

Because, of course, this is an undergraduate thesis, and functions in the same way undergraduate theses always have: it covers the techniques it's supposed to cover to demonstrate that the student has been paying attention and has a command of the material. I had to write one of these myself years ago (although it was nowhere near as complicated or impressive as Sands's!), and so I soon recalled the essential problem of all undergraduate papers on statistics: finding a data set appropriate to the techniques you know you have to apply.

What happened to Sands was that she wandered into the sights of a very high-powered crew with an obvious agenda. Her mentor was none other than Steven Levitt (looking geeky at left), co-author of the pop economics phenomenon Freakonomics - who was friends with Julia Jordan (looking sexily persecuted, below right), a fairly successful stage and television writer who had long told anyone who would listen that sexism must be holding back her career. Jordan wanted back-up for her personal theory of grievance; Levitt was no doubt looking for another public forum for the methods of Freakonomics, which had begun to seem a little irrelevant since its micro-analyses had served as a distraction from the macro-problems that had been brewing during its heyday (and about which it had little or nothing to say).

Of course the appeal of Freakonomics hadn't been hard to understand - its methods seemed to validate policy decisions while transcending traditional political biases. Not that it doesn't have an implicit libertarian/liberal bias - it does; indeed Levitt's first, eye-opening finding was that access to abortion correlated to later lower crime rates (since crime rates have risen recently, however, while abortion rates have remained stable, I'm not sure where that particular debate stands now). This of course, wasn't actually a political or moral argument - but its pure (if unspoken) utilitarianism seemed just as good as a moral politics to a lot of people. And to be fair, Levitt always reported his unexpected results - indeed, that honesty is central to the pleasure of reading Freakonomics, and its elevation as a kind of economics-lite substitute for political thought. No doubt that badge of honesty was also part of what lured Jordan to pursue a Freakonomics-style analysis to vindicate her claims of sexism in the theatre.

Enter Emily Glassberg Sands, in her own words "a young economist in the making" who is also rather clearly on the make. Only I'm sure Sands soon found the freaky catch in Freakonomics - it's quite hard to find data samples that match the requirements of its methods; indeed, the search for such data, I've read, has become a kind of cottage industry among economics grad students and post-docs. So Emily did what any smart undergraduate would do who'd already invested who-knows-how-many hours in a thesis topic:

She began fudging.

Only Sands eventually had to fudge a lot. Ironically enough, the "controversial" audit survey portion of her thesis seems pretty air-tight; the problem was that it had come up with contrarian results - it seemed to indicate that women, not men, were discriminating against other women. (OMG, Julia!) Still, if Sands had stopped there, she'd have already done enough to attract a considerable amount of attention.

But she went on to attempt to tease out evidence of sexism on Broadway - only to discover that she really couldn't do that with the data available. Sands says in her text that she's trying to apply Chicago price theory to the Broadway scene, but that requires marginal analysis, and she just doesn't have the numbers for that - indeed, she openly admits she has to infer marginal values from average values (a technical no-no). And frankly, even those average values are flawed, as Sands could never find actual data on production costs and profitability. So what we're looking at is a simulation of marginal analysis based on proxy variables (from a possibly flawed sample; see post below).

What's more, she's conceptually high and dry. One even begins to wonder whether "Chicago price theory" could ever be applied to as anomalous an economic scene as Broadway. Indeed, Sands herself wonders aloud at one point, "How would one define the 'marginal' male-written or female-written play on Broadway?" Good question; I've no idea either (Sands ponders 'an examination of plays just off-Broadway,' but that would bring up a raft of new problems). But I can tell that her entire final analysis is a kind of charade - a charade done while dutifully lifting the curtain over and over, and pointing out to her professors that she's using averages and dummy variables and proxies; but a charade nonetheless.

So is Ms. Sands guilty of fraud?

Well, when it comes to her auditors at Princeton - no. She clearly covers her ass in the text of her study - like so many undergraduate theses, her final chapter operates with an understood subtext of "I can't really do this with the data I could find, but if I could, I would do it this way, just like you showed in class!"

When it comes to the rest of us, however - yes, Ms. Sands has been a bit fraudulent. Or at the very least has allowed herself to be manipulated into giving a misleading impression. Because somehow all the provisos and explanations of her study have been dropped in her public presentations. In her initial public foray in New York, she even put up slides (one is reproduced below, note headline) that seemed to indicate that she actually had data on profitability, and every discussion I've read of her work in the press has implicitly or directly repeated that false claim (and she's had plenty of chances to add comments on these posts clarifying her position).



But if Sands did clarify her position, she'd be left saying the last thing that Julia Jordan wants to hear - that she found hard evidence indicating women believe sexism exists in the theatre, but no actual hard data proving that it does exist.

And what should we make of that possibility? More to come in Part II of this (at least) three-part series.