At most colleges and universities, the granting of tenure and promotion is based on achievement in three key areas: scholarship, teaching, and service.  Measuring scholarship and service is not exactly straight-forward, but is still a relatively clear process that usually involves some combination of tallying things up and soliciting input from both external reviewers and those who can comment on a candidate’s contributions.  Measuring teaching, though, is particularly difficult.  Student evaluations, upon which many such recommendations are based, are after all one dimensional and notoriously difficult to parse.  Is there a better way?

Many institutions of higher education have grappled with this problem.  Some time ago I led an ad hoc committee commissioned by three academic units at Brown to consider how we might better, more fairly, and more transparently evaluate teaching.  I posted that report here.  It took a long time for those recommendations to wind themselves down into policy but recently one of the commissioning units of the original report did exactly that, revising its “Standards and Criteria” document (each academic unit has its own set of criteria for tenure and promotion) to accord with the ad hoc report.  In the interest of transparency (and should this be useful to anyone else) the revised section (very slightly redacted) is below:

B. Teaching
Assistant professors will be assigned an on going mentor with whom to discuss issues of pedagogy and personal development as a teacher. We recognize that excellence in teaching can take different forms. All candidates for promotion are expected to address in their statements their own goals, definitions of teaching excellence, and how they assess and reflect upon their effectiveness.
The following evidence will be used by the review committee to evaluate teaching effectiveness:
1) A teaching statement which includes the candidate’s teaching philosophy, classroom approach, and reflections on effectiveness;
2) Completed course evaluations from all the courses taught by the candidate since the date of hire or the last promotion;
3) Syllabi for all the courses for which there are course evaluations;
4) Teaching observation reports, to be completed at least once each year. These will follow a protocol: the faculty member whose teaching will be evaluated will attend a class meeting of the senior faculty member conducting the evaluation; an initial conversation between the two faculty members follows, including consultation about which course and course meeting to observe; then a review of the syllabus before the class meeting, the observation itself, the writing of the report, and the sharing of the report with the faculty member being evaluated;
5) A teaching portfolio that documents teaching effectiveness (including the candidate’s teaching statement, comments on student papers and revised papers, and annotated syllabi; Sheridan Center guidelines for the preparation of a portfolio might be consulted).
In their determination of the value of the candidate’s work submitted for evidence, the review committee will consider such matters as:
1) Quality of course design: Do the courses have appropriate learning goals and are they constructed in a manner that advances those goals? Do they demonstrate knowledge of the state of the field and do they include appropriate assessment techniques? The main evidence used for determination is class syllabi.
2) Contributions to curricular development: Does the candidate seek to create courses that contribute to our curriculum and/or that students might find attractive? Is the candidate willing to teach beyond her/his specialty if necessary? The main evidence used for determination is the candidate’s history of offerings and responsiveness to annual reviews.
3) Advising and mentoring: Has the candidate been willing to advise students on whichever levels are most appropriate? Has the candidate been an effective mentor and thesis advisor (where relevant)? Does the candidate mentor graduate teaching assistants? The main evidence used for determination is institutional data (i.e., the number of advisees); student evaluations; and the candidate’s statement.
4) Professionalism: Does the candidate demonstrate appropriate behavior in the classroom (e.g., comes to class prepared; does not belittle students)? Does the candidate treat graduate teaching assistants with respect? The main evidence used for assessment is student evaluations, classroom observation reports, and the candidate’s statement.
5) Professional development: Is the candidate self-reflective about his/her teaching and does she/he seek to improve it? The main evidence used for assessment is the candidate’s statement and (if provided) the teaching portfolio.
6) Effectiveness of teaching: We recognize that effective teaching and learning takes many forms and that it is very difficult to evaluate, especially in a relatively short time frame. Nevertheless, to the extent possible we would like to know whether students are learning in the candidate’s courses and the extent of that learning. Does the candidate teach in a manner that is inclusive? The main evidence used for assessment is the candidate’s statement; observation reports; student evaluations; and (if provided) the teaching portfolio.
7) Contributions to scholarship on pedagogy: This is not expected of all candidates but active participation in the scholarship of teaching and learning (SOTL) counts to the candidate’s credit.
8) Leadership: Does the candidate help to support and foster an environment that supports the learning and assessment of colleagues? While not expected of all candidates, activities that demonstrate leadership might include helping to observe others and/or organizing sessions about pedagogy.

It should be noted that in my opinion, at least, the goal of evaluation is not strictly evaluation: it is to help to  support a community of practice in which educators support each other in our common mission.  Teaching is hard and involves, in ways that scholarship and service don’t, making oneself vulnerable (a reflection on that here) and ideal work environment is one in which faculty feel free to discuss their failures and well as successes and to help each other to become better.  Evaluation, at its best, is simply a natural outgrowth of sustained reflection and practice.