Ever since reading Daniel Kahneman’s great book, Thinking, Fast and Slow, I have been intrigued by the issue of how the ideas about hiring and predicting success that he discusses might apply to the academic job market (I wrote some initial thoughts here). Recently I have been more directly involved in some academic job searches, and also having recently read Malcolm Gladwell’s David and Goliath (which deals with some of the same issues of predicting success) I thought that this was an issue worth exploring a bit further.
The process of hiring for an academic post, it seems to me, really needs to be broken down into three separate issues: (1) the qualities that the academic unit seeks in an applicant; (2) the evidence necessary to determine if the applicant has those criteria; and (3) the actual nuts and bolts process. This post will deal only with the first issue, with subsequent posts on the other two. This is a series targeted primarily at those who, like me, are academics involved in hiring, although it should also interest those who are seeking academic positions. Please note that these are just my personal thoughts; they do not necessarily represent (and in some cases emphatically do not represent) the views of my colleagues or my employer.
A word might also be in order about why this issue interests me. The central question for most job applicants – at least it was for me when I applied for jobs – tends to be, “What are they looking for?” After watching this process from the other side, at three separate academic institutions, I realized that “they” mostly aren’t entirely sure themselves. It is one thing, for example, to specify that we seek an excellent scholar, but it is another to evaluate quite specifically what that means. This often results in fuzzy discussions around factors that are easier to evaluate but perhaps a bit peripheral to the actual qualities being sought. I think, for example, of superlatives in reference letters (e.g., “she has a lively mind and is the best student I have taught in twenty years”) that really privileges the skills and reputation of the letter writer over the demonstrated qualities of the candidate. Or an engaging public lecture that becomes the basis (rather than the written record) of a candidate’s scholarship. Because these discussions are inherently vague and based on subjective evaluations, they can sometimes generate heated, and unpleasant, disagreement within an academic unit. I have often wondered if there might be a more analytical, transparent, and rigorous way to do this.
Here are a few thoughts about what a list of qualities for a place like Brown University might look like:
- Soundness of Scholarship
- Does the written scholarship ask questions that are important within the specialty?
- Are the answers important or significant within the specialty?
- Is the methodology sound?
- Do the questions, answers, or methodology demonstrate originality?
- Does it demonstrate command of the appropriate skills and languages?
- Accessibility of Scholarship
- Is the written scholarship accessible to scholars outside of the candidate’s specialized field?
- Is the written scholarship likely to be significant to scholars outside of the specialized field?
- Can the candidate communicate the results of her/his scholarship to a wider community of scholars?
- Productivity
- Has the candidate produced a quantity of scholarship appropriate to her/his stage of career?
- Trajectory
- Are the future projects significant, interesting, and promising?
- Breadth
- Does the candidate possess sufficient breadth of knowledge and interest so that s/he:
- Can contribute to the intellectual community of the unit;
- Can offer a variety of courses outside of the specialization
- Does the candidate possess sufficient breadth of knowledge and interest so that s/he:
- Ability to Contribute to the Curriculum
- Is the candidate qualified to teach the courses expected of the position?
- Is the candidate broad and flexible enough to develop courses that would be of interest to our students?
- Communication and Teaching Skills
- Can a candidate design a sound course?
- Does the candidate reflect on his or her teaching; take is seriously; and want to improve it?
- Can the candidate lecture effectively?
- Can the candidate lead an effective discussion?
- “No Asshole Rule” (see the book by that title)
- Is the candidate likely to be an asshole?
This is simply a base list. Some academic units might find some of these criteria irrelevant, while others might add several others. Note, though, that I have not attempted to weight these criteria; different units will put different values on each one. One academic unit, for example, might value breadth while another is looking for excellence in a small sub-specialty. It makes little sense to value a candidate’s ability to deliver effective lectures if the preferred mode of dissemination in that unit is in writing and seminar formats.
Once weighted, this approach could lend itself to quantification. I will return to this issue later this series. For now, though, I think that the take-away is the benefit of frank and specific discussion about specific criteria prior to a search. What are you looking for, and how do you value it? Is there a consensus within the search committee and academic unit, or are there differences that can be discussed before actually evaluating the applicants? Can these agreed-upon expectations be shared more widely, maybe even with the candidates? The cost of this approach is that individual faculty members will find themselves with less flexibility down the line should they personally prefer a candidate (perhaps due to similarity of field) who does not fulfill the agreed-upon criteria. To my thinking, though, this is perhaps better thought of as a benefit.
How do you evaluate the extent to which the candidate fulfills the criteria? That has to do with the data collected and its relevance. I will discuss that in the next post.