Still blocked on this chapter. I’ve resolved not to fight it too much, but to get some other things done, too, while I wait for things to work themselves out. In the meantime, what it is I’m doing when I study reception:
As I noted earlier, I want to know what people said about novels when they were first published, and what sort of strategies they had available for reading them. To be honest, that’s often enough for me—if my committee would let me stop at description, I’d read a whole bunch of reviews and describe away. But that generally falls a bit short of an actual argument, so the most important part of my job (and what’s been holding me up for a while on the new chapter, just like it did on the last one) is showing what it is about the reviews that’s significant to our understanding of the novel, the criticism, or some other broader issue.
In order to do that, the first thing I do is find every review I can of the book and read them all, labeling each one with the major themes that I see recurring. Scrivener is particularly good at handling the practical side of this, but that’s a different post. Then I read them again, and again, paying attention to the different strands running through them, and then I keep reading them until I have some sense of the story they tell. Then I try to tell it.
Not very systematic, right? It generally takes a whole lot of re-reading and writing a bunch of crappy notes before I figure out what it is about the reviews that’s interesting and significant. Once I see the seeds of the argument, it gets a bit easier. Then I’m putting together a case and providing evidence to support my claims that we should interpret the reviews in the way I’m proposing.
The evidence I provide is textual—direct quotation, paraphrase, and analysis of the reviews. And this is where it starts to get sticky, because how much evidence is enough evidence? None of the patterns I point out are going to be present in all the reviews. The impulse then is to count—tally up how many review deal with x theme and offer some percentages. But there are lies, there are damn lies, and there are statistics. What would the numbers actually show? That 36% of the reviews that I have available evidence this trend? My data set is “reviews I am able to find.” I’m able to find enough reviews that I can make some fairly confident arguments about the patterns and trends I see across them, but that doesn’t make them a statistically significant sample size of all the reviews of the book that were published. In this case, statistics would be damn lies, provided to give a sense of significance that mostly just takes advantage of our cognitive biases about certain types of data.
And that’s why the “so what” part of my argument is so important. I need to find compelling connections between the reviews, the novel itself, the historical context, and the criticism. If I can do that, then I don’t have to rely on some false sense of quantitative significance to justify my argument. Unfortunately, finding those connections takes a lot longer than crunching the numbers.
All of this puts me on the fringes of DH work and debates about quantitative analysis in literary study. I don’t do quantitative analysis, and despite the hypotheticals above, I’ve never attempted it, but I can’t say I haven’t thought about it. Most of the time, I have trouble seeing what data mining actually adds to the conversation, but in the face of curmudgeonly responses like Stanley Fish’s, I sometimes want to try it out just to be ornery. I’ve got some more thoughts on data mining that I want to work through at some point, but until then, Ted Underwood has a pretty thorough response to Fish’s grumpy rant.