QotD: Open Book
Posted January 25th, 2004 @ 09:56am by Erik J. Barzeski
Question: How do you feel about open book tests?
My Answer: I despised them. 10% of what I learned in college came from books. They didn't encourage you to study - instead, the encouraged you to literally memorize what information can be found next to what pictures. The high grades went to the people that wrote the fastest and knew how best to use an index. How is that "getting an education?" It's true that open book tests usually had harder questions, but only because they were able to justify focusing on the minutae of any particular subject. The grand scheme of things? The major concepts? Who cares!
I always did better than most on open book tests, but I knew the tricks. Doesn't mean I can't enjoy them. I like tests that forced me down to a B. They resolved my will to study harder on the next one. Joke classes were just that - a joke. Open book tests were a joke too.
You are encouraged to answer the Question of the Day for yourself in the comments or on your blog.
Donate Life
Posted 25 Jan 2004 at 10:54am #
I am 100% for them. When I'm a teacher, most if not all of my exams will be open book. "Open Book" is how it works in the real world. You can look things up. The real test of knowledge, imho, is knowing how to look up and use the information. Now, true, with something like a maths class this can be as simple as plugging variables into a supplied equation. But what if the student is asked to prove something? Or explain how a process works? These are tests of real knowledge, not plug-and-chug.
I think teachers who refuse to give open book tests on grounds simliar to yours are those that are lazy. Too lazy to actually test the students on real information, they'd rather see the students memorize the equations and definitions.
I agree though, that most open book tests are written so that it is just a matter of writing fast and using an index. However it is possible to write an open book test that actually tests knowledge. Most of my college math classes were open book, students were welcome to have the formulas and definitions in front of them. However, we were then tested on proofs and explanations of applicability or extensions. Ideas that weren't in the book, but built upon things that were.
Posted 25 Jan 2004 at 11:26am #
"Open Book" is not how it works in the "real world" where you also have Google and - ahem - your peers for answers. Tests should test one's knowing and understanding of the basics. Engineers should not have to hit up Google or a book every time they want to figure out some basic tenet of their field.
Too lazy? I think open book teachers are lazy. After all, what are they actually forced to teach the students? Nothing - it's all in the book. Why meet three or four times a week? Why not just assign chapters and give tests? Lazy? I think you've got it backwards.
"Open Book" tests are very different than "Index Card" tests. An "Index Card" tests is one on which students can write formulas or "whatever fits on one side of an index card." Rote memorization of formulas isn't practical - knowing how to apply a specific formula is.
Posted 25 Jan 2004 at 11:33am #
Open book tests
Over at NSLog(); the question of the day is how do you feel about open book tests. Ambivalent at best
Posted 25 Jan 2004 at 11:59am #
Open book (which I use to mean 'the ability to look up specific facts you may not know offhand') is how it works in the real world (books/Google/peers). If I can't remember the formula for the integration of arcsin, I look it up. The idea that in a calculus class one would have to memorize every possible integration formula is ridiculous. Thats not how it works, when you don't know it, you go ask someone what it is. Knowing out to use it (what I suggest open book tests really test) is whats important.
An index card test appears like a compromise, but in my experience its nearly the same as an 'open book' test. Most of my index cards have been almost 50% blank after putting all the information I need on them. I can't imagine needing anymore, there simply isn't that much in the book/notes. So what difference would it make if I had used the book? I've seen people photocopy & shrink pages of the book to fit on an index card. Why not just let students use the book? Many students would feel more comfortable with the book.
Of course, the problem may be that the exam would be too easy with the book. So if the teacher actually puts effort into the test (rather that just a bunch of plug & chug problems or definitions. Furthermore, its crazy to think "it's all in the book" Its not. There are notes that need studying, classmates that one can/should study with, and ideas that are not in the book. So yes, while a student may be able to get away with just the book, attending class to learn what the fuck the book is talking about might be of use.....
Posted 25 Jan 2004 at 12:04pm #
Actually, the trackback here mirrors my opinions rather well:
Posted 25 Jan 2004 at 12:48pm #
Open book tests that allow you to drill the response in an index or copy some straight fact from the book are not well conceived. The ones that are a race against time where the people that bring the most stuff and have memorized the places where the facts are so they can copy them to paper aren't good either.
Open book tests allow you to test understanding rather than memorization. I've had countless exams where I basically knew the answer but couldn't get it 100% right because some keywords slipped from my mind. You want to focus on questions that make people associate concepts and challenge their understanding. See if they gave some thought to the concepts, not if they memorized them.
Closed book tests are in my experience usually memorization tests and I've seen many people that haven't got a clue about a subject in a course getting good grades because they could "vomit" all the definitions or rules.
I'd have to say that there's no definitive answer and it depends on the course and objectives. Specifically in engineering and science courses I believe that the open book approach is usually better.
Posted 25 Jan 2004 at 12:56pm #
Do either of you have a grasp of studies conducted on "what open book tests test?" My mother is a teacher, and a pretty good one at that. Do either of you actually know anything or should you be prefacing everything with "it is my personal opinion based on nothing more than my own experiences that..."?
Posted 25 Jan 2004 at 2:43pm #
Yes, it is my personal opinion based on my own experience and the experiences others have shared with me. I got the impression that you original post was along this lines too. I don't consider my opinion to be final or correct.
My point is that open book tests can somewhat factor out detailed memorization, which I don't believe is the primary objective in most engineering and science courses at the university level. They can concentrate on "the grande scheme of things" and the "major concepts" as you put it, of course they can also be along the lines you described them.
Please do point out any good studies or discussions about this you know about, I find this to be an interesting question.
Posted 25 Jan 2004 at 3:34pm #
I think that there is a time and a place for open book tests. I took a Business Law class that had an open book exam, and that made sense... in practicing Law there is an over-abundance of information and being able to find it quickly is an aspect of what being a lawer is all about.
I've also had Literature exams that were open book. We could all bring the texts and any notes written in them when we took the exam. Once we got there we had a few hours to write about five different essays that we'd never seen before. Most of the time we wouldn't use the readings, because we knew the stories pretty well, but it is helpful to be able to open the book again to double check things.
Obviously, foreign language exmas shouldn't be open book... we can all agree on that, right?
History classes are another example of a class that probably shouldn't be open book.
I think the issue really centers around what a teacher is trying to test. If you are trying to test knowledge, then keep the books closed. If you are trying to test thinking ability, reasoning, or the ability to quickly and coherently throw together a thought or a case then why not have the knowledge available -- especially if there is way too much to memorize?
I guess what I am trying to say is that open book tests are a good tool that, like any tool, becomes a bad thing when it is overused or used in the wrong situation.
Posted 25 Jan 2004 at 3:49pm #
Drivel:
David, that's crap. You can easily test "thinking ability, reasoning, and the ability to quickly and coherently throw together a thought or a case" without a book. Why? Because the knowledge should be in your head.
Think about what you said - that knowledge exists only in books? C'mon.
My definition of education does not involve merely learning how to process "knowledge" but includes the gaining of knowledge along with the ability to process it. Some of the best tests I've taken ask you to take some small kernel of knowledge - easily learned - and to apply it to a situation not contained in the book. Having a book at that point is a hindrance.
I despised rote memorization. I tackled the "idea" of the subject being taught and came out ahead of everyone for having done so. Open-book tests allowed less-capable students (and I'm quite comfortable saying this, given the fact that there is an established scale not of my own creation or imagination) to achieve abnormally high grades.
Several studies could be linked to if I wasn't sidetracked by the folklore.org site and some projects. Look for them yourselves if you want. I don't even know how many there might be online, but I've read enough to support my belief that very few teachers have the capacity or ability to properly create or administer an open-book test, and that poorly going open-book is not nearly as good as poorly going closed book.
David, you've really gotta start formatting your longer comments in a more readable fashion.
Posted 25 Jan 2004 at 5:15pm #
My understanding is he didn't say you couldn't test "thinking ability, reasoning, and the ability to quickly and coherently throw together a thought or a case" without a book. He was making the point that if you want to do that then there's no reason not to have a book, especially if there is many information to memorize. Knowledge should be in your head, but instead of focusing on having the "whole thing memorized" knowledge you can focus on understanding and questioning the concepts without fear of forgetting some small detail in the middle of the test.
Sure, if they're like the ones you described where you just have to look things up I can see this happening. If they test your understanding as opposed to your capacity to retain large amounts of information I don't see how less-capable or less dedicated students could fare very well.
This article/interview from the Centre for Development of Teaching and Learning, from Singapore, makes some good points.
Posted 25 Jan 2004 at 8:42pm #
Usually I prefer an index card. First of all, it's stupid to miss a problem because you forget one term in an equation that you normally would look up if there was any uncertainty. Fact of the matter is that in the real world, you only use one particular branch of your college learning on a very regular basis. It is good to be familiar with everything else, but it's also best to keep a good reference on hand to check if the formula's right. I don't think that no notes/no book exams really test working knowledge, rather they test the ability to cram and spit out minutia in one hour. Any test that I have ever crammed for, I promptly forgot the material by the next semester.
Posted 25 Jan 2004 at 9:37pm #
I think I'm going to have to agree with the index card field of thought here. I just graduated from MIT, and a lot of my exams were:
"1 [or 2] pages front and back of notes."
It is actually a pretty good compromise between having to memorize everything (very not real world) but not letting you slack off competely by not studying. You actually had to learn the material pretty well in order to even condense a month or sememster's worth of classwork onto a sheet of paper. And honestly, more often than not, if I had done my job preparing the note sheet, I hardly needed it for anything other than one or two long formulas which would be readily available in the real world anyway.
On a somewhat related note however, I got VERY good at photshop. All of our lecture notes were .pdf files, which scale very nicely to about 1.5 inches wide and arragned on a new .pdf file. When printed out at 600 dpi, you now have hundreds of sheets of paper nicely complying with the "2 sheets of paper front and back" criteria for our notes :).
Posted 25 Jan 2004 at 10:32pm #
For most classes I am against open-book tests. As you stated above, it becomes a test of how well you can look things up. For some classes, such as those that rely on formulas, it seems reasonable to either allow one page of notes or provide the student with the raw equations.
For many English tests though, I think it is appropriate to be able to have access to the text, such as a novel. This is because English tests are generally not hard facts, but interpretations of the work. I do not think it is appropriate to allow other documents, such as Cliff's Notes, though.
Posted 25 Jan 2004 at 11:49pm #
I had a Physics class with open book tests. It made us worry about learning the concepts (the things that non-physics majors like me should remember) not the details.
This not only made the class more interesting, I remembered a whole lot more. No cramming for formulas that get forgotten the next day.
I get hired to code PHP, but I sneak a peek at PHP.net many times during every project. Why? It's not because I don't know PHP. It's because I know the concept of what I'm doing. When I need the details of a functions (the switches for date() and such), I look it up. It's open book and in real life. That's how it works.
Posted 26 Jan 2004 at 6:05pm #
(Hopefully) more readable fashion:
Open book tests are a specific testing tool of the teacher, who is trying to discern a student's set of abilities. You are saying toss out that tool. I am saying to recognize that the tool is specialized, and therefor it has strengths and weakness. Keep the tool for the situations in which it works best.
Open book tests do swap the talent or skill of memorization for the ability to look things up (quickly.) Both are real-world skills.
Your lesser of two evils point is a good one. I totally agree that poorly using a more generalized tool for assessing student learning is much better than poorly using a more specialized one. Perhaps that was what happened in your educational experience, and if so, I can see why you'd be upset with botched exams.
I should have noted your word choice in the beginning of your post ("I despised them,") realized that you were quite biased, and not bothered you with an offering up of a more balanced perspective. I feel this way because you were being a bit brash towards me in what seemed to be an emotionless intellectual discussion.
Posted 27 Jan 2004 at 2:06pm #
These days I sit on the other side of the exam table and we actually give open book exams (as well as the 1-2pg of notes variety). It actually lets us ask much more difficult and in-depth questions than a closed book exam would allow. Now, if a student misuses a formula or displays a fundamental misunderstanding of the course material, the excuse "I forgot" doesn't really hold water. If your classes were jokes, blame the prof (or the TAs) for being lazy (Not that I can blame them too much, exam writing is a complete pain in the ass even if you're only helping to edit the exam).
Posted 27 Jan 2004 at 10:42pm #
Of course, lazy teachers can misuse any tool.
A lazy teacher can write a closed-book test where most of the questions are taken straight from the textbook chapter. (For example, "Multiple inheritance allows a class to be derived from multiple base classes" would turn into "What is the term for the derivation of one class from more than one base class?")
Likewise, a lazy teacher can write an open-book test that is a variation of the same but with more minutiae, turning into nothing but a scavenger hunt through the textbook. ("What was the first language to have multiple inheritance, and in what year was it developed?" (By the way, a little Googling shows the answer to probably be Flavors, developed in 1978, according to footnote 3 on the first page of this PDF.))
A good test should measure how well the student is able to use critical thinking skills to apply the knowledge they should have learned from the course. This is the kind of test that a good teacher creates.
Most open book tests that I've taken are ones in which the instructor was confident enough that the test was not simply one of regurgitation to let us have our books. The tests could have probably been taken closed book, too, but we had the books as reference material, if necessary. The focus on these tests were on application, so whether or not we had books was largely irrelevant.
A good computer science test will ask students to write a program that does something, possibly with certain restrictions as to the implementation. Let students have the language reference if they want to, or even a textbook about implementation of different algorithms. A student will do poorly unless they are familiar enough with that syntax and those algorithms to be able to apply them, along with critical thinking, to solving a new problem, like writing an indexer for a search engine, or an AI that can play Mancala.
If on a science test, you ask somebody to design an experiment to answer some question, they're not going to find the answer in a textbook (as long as the design for that experiment isn't already in the textbook). However, you're not going to necessarily expect them to remember the half-life of Uranium-234, or the speed of sound through Nitrogen. If you just told them these things as part of the test question, you'd be leading them on to a particular answer, rather than just giving them a reference source, and let them figure out how to design the experiment.
While this is my personal opinion and is founded on my experiences, the last biology teacher I had did her thesis on alternative methods of teaching and assessment. She was much better at creating open book tests than closed book ones. Many students complained about her open book tests, because they weren't just about regurgitating knowledge, but about thinking critically. Her final consisted of several journal articles about several different topics. She would ask us to draw conclusions based upon the findings of the articles and the knowledge we already had, to fabricate theories to explain some aspect of the data arrived at in these articles, and to create new experiments that would better test our theories. She was more interested in our thinking and defense of our conclusions than at us arriving at some predetermined correct answer. With that kind of test, it matters not one whit whether you can look in the textbook or not.
It's really hard to talk simply about open book or closed book tests without actually looking at how textbook access or lack thereof plays a part in a particular test.