Hiring for an Academic Job (2): Data

In the last post I discussed the benefits of a search committee or (better) entire academic unit determining, in advance and as specifically as possible, the criteria that will be used in the process of an academic job search.  Here, using for illustrative purposes the criteria developed in the last post, I want to discuss the kinds of data collected during the search, and how they are and (more importantly) are not useful.  My discussion is here also personal and idiosyncratic; I want to sketch how I, personally, read a file.

1. C.V.  This is rather obvious, but serves as a quick roadmap through the application.  More specifically it provides information about topics of interest and expertise; productivity; teaching experience; awards and other recognition; and professionalization, as shown by conference presentations.  In general I do not find that conference presentations (which pad many early career C.Vs) are useful indicators for any of the criteria and I tend to ignore them.
2. Statement.  This gives me a sense of the significance of the candidate’s scholarship (at least as s/he understands it), both within and outside of the the specialization.  I also look here for future trajectory and engagement with teaching.
3. Writing sample.  This is an important indication of the candidate’s current scholarship and scholarly promise.  When I read these I try to assess, as far as I am able, the soundness, significance, and accessibility of the scholarship.  I read these humbly, as they are often outside my field of expertise, and I need to constantly remind myself that accessibility is not the same as significance, and just because I don’t understand something or its significance does not mean it is not brilliant.
4. Letters.  These are the trickiest to evaluate.  It is important to remember that these are most often written by those with some kind of vested interest in the candidate and who have their own personal (and well-meaning!) agendas.  Letter writers also can be better or worse writers, and it is important to separate that from the qualities of the actual candidate.  I try, to the best of my ability, to disregard vague praise and the like and to try to use the letters to better understand the candidate’s scholarship and its significance.   While comparisons to others are useful for tenure, they are less so in early career job letters, unless the writer provides a full list of all previous students and compares the candidate to them.
5. List of courses to be taught.  This helps to judge whether a candidate can contribute to the curriculum.
6. Syllabi, preferably with annotations.  I used to believe that a statement of teaching philosophy was useful until I read a pile of them only to find most of them banal.  Syllabi, especially if they describe the thinking behind them, provide a better window into course design and breadth.
7. Preliminary interview.  This relatively short conversation gives the committee an opportunity to clarify the file and probe a bit deeper.  If there is information otherwise missing (e.g., the next scholarly project; language competency) it is an opportunity to gather it.  If a candidate’s scholarship is not accessible in writing, I want to see if s/he can make it accessible orally.  If it becomes clear (as, alas, it sometimes does) that the candidate could not succeed in a campus interview, we want to know it now.
8. Campus interview.  This is even trickier to evaluate than the letters.  The campus visit gives the candidate an opportunity to demonstrate communication and, given the right conditions, teaching skills.  It provides an opportunity to provide a sample of the candidate’s scholarship to those outside of the unit who will not have read the file and to solicit their feedback.  Discussions with the candidate are helpful for judging issues of breadth; scholarly significance; and  whether they would be destructive members of the unit.  Assuming that the candidate is not an incompetent teacher or an asshole (disqualifying traits) I tend to see it as contributing about 25% to the total evaluation, with the rest coming from the written materials.

One might also solicit teaching evaluations, but I leave that off the list here.

Here is more or less the same discussion, repackaged according to the criteria to which the appropriate data apply:

• Soundness of Scholarship
  • Does the written scholarship ask questions that are important within the specialty? [2, 3,4,8 (in a more limited way)]
  • Are the answers important or significant within the specialty? [2,3,4]
  • Is the methodology sound? [3,4]
  • Do the questions, answers, or methodology demonstrate originality? [2,3,4,7,8]
  • Does it demonstrate command of the appropriate skills and languages? [1,2,3,4,7]
• Accessibility of Scholarship
  • Is the written scholarship accessible to scholars outside of the candidate’s specialized field? [3]
  • Is the written scholarship likely to be significant to scholars outside of the specialized field? [2,3,4,7,8]
  • Can the candidate communicate the results of her/his scholarship to a wider community of scholars? [3,7,8]
• Productivity
  • Has the candidate produced a quantity of scholarship appropriate to her/his stage of career? [1,4]
• Trajectory
  • Are the future projects significant, interesting, and promising? [2,4,7,8]
• Breadth
  • Does the candidate possess sufficient breadth of knowledge and interest so that s/he:
    • Can contribute to the intellectual community of the unit; [1,2,3,4,7,8]
    • Can offer a variety of courses outside of the specialization [5,6,7,8]
• Ability to Contribute to the Curriculum
  • Is the candidate qualified to teach the courses expected of the position? [1,4,6,7,8]
  • Is the candidate broad and flexible enough to develop courses that would be of interest to our students? [2,5,7,8]
• Communication and Teaching Skills
  • Can a candidate design a sound course? [5,7,8]
  • Does the candidate reflect on his or her teaching; take is seriously; and want to improve it? [2,7,8]
  • Can the candidate lecture effectively? [8]
  • Can the candidate lead an effective discussion? [7,8 (perhaps)]
• “No Asshole Rule”
  • Is the candidate likely to be an asshole? [2,7,8]

The “criteria and data” approach has important ramifications.  In my experience, the campus interview tends to be more heavily weighted than the written record.  If the soundness of scholarship, though, is valued more heavily than accessibility, this might well be a mistake.  We might avoid hires based more on the reputation and skills of letter writers than on the candidate himself or herself.  It would mean that the specific and appropriate evidence is brought to bear on the criteria.  There will still be disagreements, but they will be contained and targeted, constrained by agreement about what evidence is relevant.

Is such an approach too labor intensive?  I don’t think so.  In the next post, I outline a procedure of implementation.