Category: Exams

Larry’s Note:  As I mentioned before, the following piece was originally scheduled to be published in Above The Law.  Ultimately they published only half of it, as they had a problem with some content (which was, I admit, a bit over the top). 

So what follows is a complete version of the text I had written (they only published part 1 of the 2 pieces I sent, ghosting me after that).   This is not original text as I’ve scrubbed it a bit since I sent it ATL.  I took out some (not all) of the profanity (who says I never admit that I’m wrong?), but I re-included links to the posts of a particular anonymous law prof  whose milquetoast advice inspired my piece. 

Original Title: Rainbow vomit, violent ejaculation, and . . . law professor advice

Hey, 1Ls and LLMs:

So, . . . confused much? Having fun yet?

Not to worry, law profs are here to the rescue!

Recently, at Above the Law, an anonymous professor, styled “lawprawfblawg” offered pearls of wisdom on law school success here, here, here, and here.

These pieces – summarizing quickly  – contain the normal bromides on how to succeed in law school from law profs:

Go to class! Brief cases! Make your own outlines (don’t use commercial outlines)! Make love not war! Sleep! Fart quietly and blame the least attractive person sitting closest to you! Have a life outside of class!

Sweet Christmas, as Luke Cage would say, that’s great advice . . .

Great advice . . . for me to poop on (to mix video metaphors).

To be clear:  This is not just another “law-professors-are-good-professors-for-me-to-poop-on” piece.

And lawprawfblawg’s pieces are not particularly obnoxious compared to other victim-blaming classics in the law prof advice genre.  (Some of lawprawfblawg’s advice is even good in my view: e.g., I totally agree 1Ls should start outlining at the start of the semester, and not towards the end.)

But I am going to poop a little bit.

In short, law professor advice on how to study in law school is useless, for two reasons:

1. The professor conflates his expertise in judging good exam answers with expertise on how to study so you can write a good exam. Of course the professor is an expert in what he likes on exams, as he grades them. But how do you know he knows how to study to write an exam he likes? Law profs are, in the sense, like food critics: Just because you’re good at judging how food tastes does not make you an expert in how to cook food that tastes good.

2. This situation is exacerbated by the central incoherence in law school teaching (usually unacknowledged by law profs in their advice):

• Almost all in-class instructional and intended outside study time in law school is spend on doctrinal analysis (reading appellate cases and arguing outcomes where facts and laws are given and known).
• Most exams assess a wider range of skills – attempting to identify which laws studied apply to completely new facts. More than application, you are graded on how well you analyze the application of law to new facts, and your judgment in coming to conclusions that are probabilistic rather than certain (“It depends on the meaning of what “is” is.”).

Let’s deplore the sin and love the sinner here: Lawprawfblawg, I love you, whoever you are.

But I adamantly and solidly poop on (most of) your study advice.

*   *   *

Brief interlude: Why listen to me?

My side hustle is tutoring law students. Mainly 1Ls. From the gamut of schools: top law schools (HLS, SLS, NYU, CLS, Chicago, Texas, Berkeley, etc., but no YLS, UVA or Michigan), next-20 schools, regional schools (almost all NY), TTTs, and even online schools (suck it, Concord; you too, Northern California).

Point is, I serve individual students, not whole classes. My students want to do well. If they don’t look good, I don’t look good.

In my work, I’ve seen thousands of practice exam answers, lots of outlines, and hundreds of students in person and online. I talked with many about precisely how they study and prepare for exams. (Average law prof sees 200-400 exam answers a year, no? Some, more.)

I’ve seen many of their results.

I’ve begged and cajoled students to do more practice exams.

I’ve begged and cajoled them not to brief cases and to avoid other stupidities.

So that’s my perspective. I am not a law prof, but I’ve worked with a lot of students and see things more from their perspective.

*   *   *

Part I: Law professor as food critic (not a trainer of chefs)

Painful analogy: You’re in a cook-off. Something like Iron Chef. Two people offer you advice:

• Gordon Ramsey, he of the three Michelin Stars and anger management issues, screams at you: “This kitchen is a fucking mess, you’re burning the skate, just burning it!!!; no, now it’s raw!!!”(throws skate at your head), etc.
• Anton Ego, fictional food critic voiced by the late Peter O’Toole, hisses advice at you on how to prepare skate, and the best skate he ever tasted, and things like that.

What do we know?

• Gordon spent his life in kitchens, cooking for tyrants, and teaching cooking. He is an expert in reproducing excellent taste consistently and teaching people the same (even if they need years of therapy after working with him).
• Anton judges meals. He is the greatest (fictional) arbiter of taste in the food universe.

And incidentally, Anton Ego is the sole judge of the cook-off.

So who do you listen to?

False dichotomy, right?

Of course, since Anton’s the judge, you cook to his tastes. You listen when Anton tells you what he likes and doesn’t.

But why would you listen to Anton over Gordon when Anton tells you what pots to buy, how to set up the kitchen, how to cut celery, how much butter to use?

Does his good taste inform those mechanical questions?

So that’s the analogy. Imperfect and pedantic!

My point: Your prof is the Anton Ego of exams. He is an expert in judging exams. Listen carefully for what he says, openly or impliedly, about his tastes.

But, weirdly, law schools don’t have Gordon Ramseys, really.  There is not an expert to explicitly teach you the mechanics of answering issue spotting law school exams. (Sometimes TAs, and sometimes profs in review sessions.)

Here’s my argument, more straightforwardly (but still rambling a bit):

1. The law professor is the only person who grades your exams. (Obvious point.) He alone decides who gets the all-important A, and who does not. He is — by definition — an expert in what he likes on an exam.

2. The law professor grades your exam blind. By design, he does not know (at the time of grading) who he is grading while he is grading, so he can focus on the quality of exam answers themselves.

3. I do not think law professors know, in any systematic or near comprehensive way, what their students do to study and prepare for exams. A practical way to prove this is the following challenge:

• To law profs: if you have this data, please send it to me or (if you don’t want to share) write me to tell me you have it (larrylawlaw at gmail dot com).
• To law students: This week, go to office hours to any prof and ask “How do your students study?” Politely interrupt when he tells you what they should do, and repeat your question: “OK, I get what they should do, but what do they actually do?  Especially your top students?” Write me if your prof gives you an answer.

I am not sure that most law professors have the time or means to find out how their students study, even if so inclined. Career advancement is based on “publish or perish.” Great teaching reviews don’t help.  They have even less incentive to find out how students actually study and whether those study methods correlated with good exam outcomes.  Doing that advances none of their career goals.

4. Combine 2 and 3, and it is clear that most law professors have no comprehensive basis for saying what study methods produce superior exam outcomes.

5. So, when law professors give advice on study methods, what is that advice grounded in?  Three sources, I think:

• Anecdotal evidence:  inferences they can make on the text of exam answers themselves (hint: bad ones stand out; good ones fit the prof like a glove)
• Anecdotal evidence:  self-selected students show up to office hours AFTER their exams to complain about their results (I had only one good student show up after his exam to ask his professor why he did so well on the exam.).
• Anecdotal evidence: Law profs’ own study methods that let them succeed in law school.

But I think this anecdotal evidence, based on the travails of struggling students, skews law professor advice to the general population.

Example: Standard law professor advice is not to use commercial outlines; lawprawfblawg argues this (who, to be fair, he or she has more intricate advice on this point, but this is the crux): “Unless you have a commercial outline written by someone whose name your professor recognizes, then, in my opinion, you should probably avoid it like the plague.”)

Some profs I have read or spoken with give this advice because they graded terrible exam answers that clearly cribbed from Emanuel’s or Gilbert’s.

Law professors who tell students “don’t use commercial outlines” believe that those who use them do so to the exclusion of going to class, listening to the prof, or doing the reading.

But the causal questions that goes unexplored:  Did commercial outlines cause student failure?

Or (leading question here), is the obvious (meaning, apparent to a law professor on reading an exam) use of commercial outlines, made apparent on an exam, a symptom of study failure?

Ooh, here comes another terrible analogy!

Suppose you are starving to death.

It’s your fault. You haven’t left the house for weeks to shop for food because of, I dunno, Worlds of Warcraft, or something even less seemly.

You are so weak that you can’t leave the house. You ransack the house and all you can find that is close to food is a bottle of Flintstones vitamins, and a big bottle of Vitamin E.

You gobble them down and pass out.

A medic later finds you vomiting rainbow-colored sand and violently and copiously ejaculating.

You jack-knife on the floor, retching from both ends, to borrow from Cervantes.

(Note: Wikipedia tells me bizarre or horrible things are more memorable. You’re welcome)

When you wake up, the medic berates you: “Dude, Flintstones and Vitamin E are not food!  Bad you!”

But Dr. House arrives and berates the medic: “Of course they’re not food! He ate Flintstones and Vitamin E because there was no food, dipshit!”

To be even more pedantic:

• Flintstones and Vitamin E are supplements, which help people when used properly. Similarly, Emanuels or Gilberts are supplements which help law students when used properly.
• Flintstones and Vitamin E are not food replacements. Similarly, Emanuels or Gilberts are not replacements for going to class and preparing properly (I won’t say “preparing as your prof asks).
• People who use Flintstones or Vitamin E correctly are hard to spot because they do not walk around in broad daylight, vomiting colors and nutting violently. Similarly, law students who use Emanuels or Gilberts correctly may be hard to spot — they may write good exams tailored to the prof and know well enough not to crib from commercial outlines or apply law never studied in class. (And others — such as my law prof friend Eric Johnson — have advocated ”>the use of commercial outlines as a starting point to understand the law before turning to cases.

6. On top of all of this, law professors themselves had law professors who believed that legal analysis is a gift from God/Crom/Ctuhulhu/Chuck Norris.

The corollary of this is: legal analysis is a God-given gift cannot be taught.

This is self-validating for professors, but a destructive, self-fulfilling prophesy that excuses professors from teaching the precise skills that result in exam success. Maybe not to the small number of people who would get it in any case, but what about everyone else?

Consider this quick, assholish fake FAQ I wrote on behalf of law professors:

Q: Who has two thumbs and kicked ass when he went to law school?

A: [Thumbs pointed inward.] This guy!

Q: Who has two thumbs and is teaching at a law school because he kicked ass in law school!

A: [Thumbs pointed inward.] This guy!

Q: Who has two thumbs, is teaching at a law school, and kicked ass in law school because he worked very hard to master teachable methods that can be reproduced by diligent students and that I would now like to share with you!

A: [Droopy thumbs + cough] Wait, no, I’m a natural, dude! Like my professors before me!

If you believe in innate specialness, well, you can’t teach what you think is simply given. (See: Babe Ruth, John Holmes.)

And in a way, I think this is the legacy of Christopher Langdell, who first created the muddled, bow-tie, cold-calling, some-students-are-special-snowflakes-and-all- others-get-Bs system we have now.

I turn to that now.

Part II: Bait and switch

A lot of what I say above sounds like it is vilifying law professors. It is a little.

But deeper down, what my assholish comment mask is my own belief that law profs are generally good people coping with a messed up system.

Perhaps not coping well.  But maybe there are no good ways of coping with something messed up.

Fundamentally messed up.

The underlying problem with professors giving advice is that they do not acknowledge (or, worse, may not be aware of) the central incoherence of how law school has been taught since Langdell:

Nearly 90-95% of your time in law school is spend on doctrinal analysis – reading cases and analyzing what outcomes should be given known facts and known law.

BUT

100% of your grade on final exams is based on an assessment of a wider range of skills, of which close case analysis is but one. And none of the skills other than doctrinal analysis that you need on an exam are explicitly identified or taught, really, in your core law school classes.

Professors advise you — in late November(when it’s too late, in my view) — to take practice exams. Most don’t say more than mentioning IRAC (Issue Rule Analysis Conclusion) on how to do issue spotting. Most don’t teach how to spot issues, which you need to apply IRAC in the first place.

But your grades — and your professional future — rides on you acquiring those skills somehow, despite this minimal or non-existent guidance.

Let’s dig deeper.

During and outside of class you:

• Read cases and try to understand them without guidance (you know now there is no textbook explaining the law; you figure it out on your own).
• Answer questions on the facts of the case to make sure you did the reading like good
• boys and girls: “Mr. Hart, please recite for me the facts of Hawkins v. McGeeeeeee!”
• Answer questions about procedural or other aspects of the case.
• Answer questions about whether case outcomes are correct in light of controlling or persuasive precedent.
• Sometimes argue different sides of a given case.
• Listen to windbags (sometimes professors, always gunners) bloviate.

These case analysis skills involve assessing case outcomes, with set facts, laws and arguments.

Maybe you debate the propriety of outcomes given the facts or previous cases. Maybe you’re asked to develop your own arguments based on facts and law. And sometimes you’re asked about outcomes on hypothetical facts not far removed from the case, but still fanciful (“OK, same case, but here a giant robot molests an emu attacking Mr. McGee? What result?”)

On the final exam you must:

• Identify colorable issues, i.e., laws that may colorably apply to specific facts that you have never seen before and that may be incomplete.
• Create arguments: once you identify issues, you make the best arguments you can on both sides, filling in gaps with common sense or inferences based on the facts. (This is harder than rearguing appellate issues that you might do in analyzing a case in class.)
• Apply objective case holdings to offer probabilistic conclusions on which arguments you created: You offer even tentative conclusions based on objective holdings of cases you’ve read.
• Exercise legal judgment: This is more than coming to correct legal conclusions. It is a “meta” skill that shows the professor that you can sort wheat from chaff that involves all sorts of other inquiries:
  • How certain were you about a conclusion?
  • Should you have been less certain given the complexity of the facts?
  • Did you spend too much time on an easy issue?
  • Not enough on a hard issue?
  • Make a totally insane argument?
  • Neglect a glaringly relevant fact? Suggesting filing useless motions in civ pro?
  • Fail to raise colorable claims, but focus only on clear winners (that everyone else noticed, too)?
• On exams, facts are completely new to you. They are not completely given (you have to fill gaps). Laws are not given to you. Or that is, the laws you should apply are not clearly marked – – you might have an open book exam and get to bring your outline, but the prof does not tell you in advance which laws apply to particular facts – you decide that.

This is a big gap, between what professors spend most of their time teaching, and what you are actually graded on.

And class room discussion can be confusing — what with the hypos involving giant robot and emus, how can you discern what your prof is going to do with something that crazy?

In fact the professor does care that you know actual case outcomes and there is a more or less objective way to apply them to say if a given claim (that you constructed from scratch) would likely to succeed or fail.

More contrasts:

Class and study time is for criticism and close reading of cases. Important skills.

Exams test your ability to create and objectively assess your creations: to spot issues, construct legal arguments, and resolve them in reasonable way (in the prof’s eyes) in an insanely short amount of time. Important skills.

SO where was the class time dedicated to teaching those important skills?

How fair is it to students — or the professors that implement this insane regime — to test skills not taught in class?

To have grades and, not to be dramatic, future careers depend on how well students can navigate the tension between what is taught and what is tested?

Two sets of thoughts to leave you with:

1. For self-interested students:

The gap between what is taught and what is tested creates an opportunity to get good grades by deciding not to follow standard law professor advice on studying. The herd will follow that advice and do nonsensical things like brief cases, avoid commercial outlines, etc. By choosing an alternate study methods aimed at exam taking rather than case analysis, you can get top grades. I’ve seen it.

Still, listen carefully to law professors when they tell you (wittingly or not) what they like on exams:

• When engaging in Socratic dialogues in class, what types of arguments do the like and dislike?
• What are their pet theories? (I once got an undeserved A in crim pro just by repeating the phrase “representation reinforcement” a million times on an exam.)
• What specific phrases do they prefer or hate? (Ex: My crim law prof wanted you to use the shorthand phrase “Leningrad Drunk” or “LD” — a reference to a crawling-on-all-fours, black-out night he had as a grad student — rather than “the intoxication defense”).

2. For law professors:

No one of alone you can fix the system, but do any of you (other than Eric Johnson) find this whole system insane? Is there no other way than The Paper Chase in the 21st century?

More specifically:

• Why bait and switch students? Why teach cases but test issue spotting (and not even acknowledge that these are not the same thing)?
• What is the pedagogical value of making law students — from day one — figure out the law on their own from reading the holding of appellate cases?
• Why suggest to students that briefing cases is important? What do they get out of this stupid text-dissection exercise?
• What is the long-term value of making students outline? What does that accomplish that just giving them an outline and lots of practice or real exams doesn’t do?
• Why only one final exam? Why not several through the semester? No educational theory says the best way to assess students is give one exam and disappear forever.

On the other hand, nevermind. I am arguing against self-interest here.

Without this insane set up, I would have no customers. Keep it up!

PS. No, dammit, in the end, I change my mind.  Those better angels of my nature win.

Law profs: please fix this fucking system. For the sake of your students.

No, don’t ask me how.  I’m not that bright.

You all (law professors) are, though.  You are the best and brightest.

So please do something about this insanity.