Law school courses are graded. Grades are stupid. They purport to
reduce the evaluation of learning to a single number on a short,
obscure scale. In order to be useful to students, evaluation has to
measure more than one quantity, and must be expressed qualitatively,
directed at helping the student to learn more and derive more
satisfaction from learning. Grades therefore cannot possibly serve
the ends of the people to whom they are “given.” On further
consideration, it therefore becomes apparent that grades are “given”
only in the sense that they are imposed.
Students have grades imposed upon them, and are taught to care what
they are “given”: The obscenity of injustice is that the dominated are
taught to kiss the rod. Evaluation should take the form of
sympathetic and effective counsel: students should be
mentored, else
the Law School's insistence on replacing centuries of apprenticeship
is a sham that reduces the quality of legal training.
But we are told that employers require grades in order to evaluate
potential employees. Never mind that these supposed employers have
presently no jobs to offer, having been caught with their pants
entirely down amidst a permanent industry restructuring they
completely failed to foresee. Never mind that we should no longer be
teaching students to work for institutions such as these
grade-consuming employers: instead we should be teaching students to
work for
clients, who are people and institutions selecting
expertise rather than retaining brokers of commodity lawyer-hours, and
for whom grades are no more useful in the process of selecting a
lawyer than they are to the students who “earn” them. Never mind
anything, in fact, that would compel us to reexamine the basic
mechanisms of our practice.
Naturally, educational evaluation should involve sympathetic, critical
mentoring. But that takes time. We have created a culture in which
law professors declare themselves superior to the mere demands of
teaching, where the customer is always either right or wrong, but
never worth spending too much time on. Grades are an expression of
contempt for the people who pay for the educations we are supposed to
be helping them to get.
Of course the system manufactures its own consent: that's the purpose
of the obscenity of injustice. Even once it is apparent that grades
are stupid and their imposition is unethical, involving continuous
acts of disrespect towards those we have an obligation to assist, the
system refuses to accept even the possibility, let alone the
imperative, of change.
My goal in these wikis is to begin the remodeling of the relationships
at the heart of legal education. Not after a stack of committee
reports and a long debate about how much change is consistent with the
self-importance of the bloviators, but now. What my students and I
are doing together will always remain far from perfect. But it
refuses to surrender entirely to bullshit, which is something of a
start.
Objectives
The evaluation policies in the courses that are conducted here are
inherently imperfect: they continue to lead to the final absurdity of
a grade. But we coexist with the grading system as a community under
occupation: we cannot pretend that the system has our best interests
at heart, or that it will adhere to reason. We seek to achieve our
objectives for the interaction between teacher and students as a whole,
creating both workflows and principles of evaluation that further
those objectives:
- Continuous revision
- All work in these wikis, and in the courses they contain, may be revised by any student at any time. Comments and edits by other colleagues, or the fruit of reconsideration, may lead to changes, as may the experience of intervening writing. Neither the work nor the evaluation of the work ends at semester boundaries—law school is a continuous process rather than a series of discrete high-stakes resolutions.
- Collaborative learning
- Although students have a right to private submission of their personal essays—which the access control system of these wikis facilitates—preference is given to public submission, revision, and evaluation of collectively-edited material, thus allowing the entire class to participate in editing and learning from one another's work. Collaborative learning in wikis implies everyone's freedom to edit all of the common product. By comparing versions we can simultaneously evolve optimal contents and sensitively evaluate each individual's contributions to the team effort.
- Accountable evaluation
- Evaluation is a continuous editorial process, visible in the common product to all readers, and subject to the usual mechanisms of discussion and revision. Issues such as favoritism, bias, cultural astigmatism, etc. are dealt with organically, transparently, and respectfully, in a forum expressive of our common social values, rather than through inflexible and absurd procedural limitations that turn the whole richness of evaluative process into the sterile, archaic dichotomy of “papers” and “exams.”
Standards
Of course the narrative evaluation comprised of multiple editorial
interactions—ranging from formal commentary on successive drafts
of personally-authored essays, to revising edits on a
collectively-authored page introducing a scanned and translated
historical document, to slight comments in moderation of a
thoughtfully-revised debate among multiple interlocutors—will
have to be reduced eventually, under the watchful arrogance of the
occupying forces, into a single grade. This process, which by its
very nature is entirely without intellectual integrity and ethical
propriety, is usually conducted in a standardless fashion, which
renders it even more banal and even more infuriating. Counting
correct answers to questions is only one partial measure of the more
and less successful aspects of an individual's progress in learning.
The most sought-after examination grade, for example, will almost always fail to
distinguish between an unimaginative thinker with a superb memory for
the material covered and a bold creative reasoner with a penchant for
occasional inaccuracy. Sidney Carton, Sherlock Holmes, and Harry
Blackmun alike will (if the grader is lucky enough to get it right on
the basis of a single, usually anonymous, experience) meet
incongruously at the top of the class.
This is a charming result, but from the scientific point of view it's
detestable. If the occupation is to be dealt with at all, it should be
on a basis that preserves self-respect. So there should be some
standards for this fallacious measurement.
We measure here three qualities: effort, commitment, and improvement.
Little need be said as to the first: That diligence is part of the
craft we are teaching should be instilled as insistently as any other
aspect of the business of representing clients. Whatever we may do
with Carton, we do not tend to retain him. Commitment in any team,
even a large one, is palpable, The nature of the student's
intellectual engagement is best measured through the instructor's
editing of the personal essays, where the student's choice of question
and mode of pursuing both the first draft and the broad-scale revision
(all of which can occur multiple times in a semester if one limits the
article length to 750 or 1,000 words) defines the student's strengths
and weaknesses sensitively enough to enable the instructor to make
concrete suggestions for immediate improvement. Improvement is
documented in all the versions of all the materials the student has
collaborated in writing and editing, and is therefore subject to
effective measurement.
Grades
Reduction of multi-variate qualitative measurements to grades implies
an algorithm implementing a model for collapsing space by throwing
away information. Each turns complex surfaces captured in the
evaluations in the wiki into a single numerical score, by turning a
series of qualitative judgments into quantities and then operating on
a vector of quantities to produce the score. Algorithms used vary
depending on course size and other factors, but for each course some
transparent process generates a rank ordering. That rank ordering is
then normalized by the Law School's “upperclass elective
curve” for the relevant course size. (Even
LawContempSoc is subject to the upperclass elective curve, for
the information of those 1Ls who are already afraid of this curiously
foolish but inoffensive device for curbing the non-problem of
“grade inflation.”) Because no ranking algorithm fails to
produce some misjudgments, I make adjustments based on qualitative
judgments of effort, commitment and improvement, acting within the
limits set by the Law School's curve deviation allowances, so that all
grade distributions are as generous as, but no more generous than, the
occupying forces allow.
Aspiration
No system of evaluation that culminates in a grade can escape
complicity with dishonesty, or achieve pedagogical goals in a
fully responsible manner. The replacement of grades by collaborative
evaluations should be an early and uncontroversial aspect of our
adjustment to new technologies of teaching. But the occupation drags
on, as all the banal forms of injustice do. When students demand
change they will get it. Until they do, the decadence of business as
usual will remain undisturbed.