Unintended consequences in education and teaching

Case 1
In a paper published recently [1], Job, Dweck, and Walton provide evidence that contradicts the theory that willpower is a limited resource.  They show that willpower is correlated with a person's underlying belief about whether willpower is a limited resource, so that one's belief about willpower is kind of like a self-fulfilling prophecy.  If you believe willpower is not limited, then your willpower will not be limited.

I don't know if that's true, since it goes against other research showing that willpower is a limited resource that can be depleted.  For all we know, the belief that willpower is unlimited may simply increase one's amount of limited willpower.  Whatever the case, the true nature of willpower has a big impact on how students should be taught in the classroom, and how they should be taught to study on their own.

If willpower is truly a limited resource, then teachers should take that into account in their lesson plans, and also teach students to take breaks to recover their willpower store.  The latter is dangerous advice, of course, since the unintended consequence is that it licenses students to take frequent breaks in the name of recovering willpower and attention-span, even when such breaks are unneeded.  Perhaps the trend of college students studying less and less is a one of the unintended consequences.

On the other hand, if it is true that believing willpower is an unlimited resource will enlarge one's willpower store either to some larger limit or to become truly unlimited, then of course this should be taken into account in teaching as well.  Students would have to be influenced into believing that willpower is an unlimited resource, and that the apparent diminishing of willpower is merely a symptom of a limiting belief.

We know willpower is important to academic achievement, as shown in studies described by Lehrer, so these contradictory studies really puts teachers in a bind.  What is a teacher to do in practice?

I think the prudent course of action is to assume that willpower is limited only by limiting beliefs that willpower is limited.  If the recent study by Job, Dweck, and Walton holds up to scrutiny, then students need to be taught to replace their limiting beliefs about willpower with more effective beliefs, in addition to being taught techniques for concentrating on the task at hand.

In my own professional experience, I have dealt with at least two specific cases where after I taught a student specific techniques on concentration, their ability to focus on a task for extended periods of time increased phenomenally.  So at least for some students, I know the willpower to focus on a task at hand can be taught.

Funny aside:  According to so-called "Brain-based learning", student's attention span on a single task is roughly as many minutes as their age.  So a 12 year old student can have singular focus on a task for roughly 12 minutes, on average, and this rule of thumb apparently holds for up to 20 minutes or so.  In real life, I have designed lessons where where entire class of students focused for an entire 50 minute period without carrot nor stick.  Another teacher who was observing my class noted, perplexed, how this is contrary to "brain-based learning" tenets.  Well, yes, and in science, we would call this evidence that refutes a tenet of a theory.  Having done some research in cognitive science now, I'm shocked that teachers are being persuaded to follow so-called "brain-based learning" principles at all.

In any case, if it turns out the received wisdom that willpower can be depleted is true, then by teaching students techniques for improving their willpower and by teaching them (what turns out to be false) beliefs about the unlimited nature of willpower, we would have at least avoided the unintended consequence.  That is, students won't have the excuse to continually procrastinate and delay real work by taking all-too-frequent breaks to "recuperate" their lost willpower.

It's best we don't underestimate how good students, and indeed everyone, are at rationalizing their procrastination — we should know, as we all do it [2]. 

Case 2
Take another example, the spread of computer technology in education.  As a computer science researcher, I obviously love technology.  I think there's so much information technologies can do to help students learn.  But just because a little bit of I.T. used strategically is a good thing, doesn't mean more of it is a good thing.  As a study reportedly showed, home computer access is correlated with lower test scores.

We want students to have computer and internet access because it gives them the ability to look up words in online dictionaries, read up on any topic in online encyclopedia, learn programming skills in various programming communities, etc.  But should we be surprised that this also lets them get even more distracted away from studying by making accessible Facebook, Twitter, MySpace, iTunes, etc?  This combined with possibly depleting willpower and it's no wonder we even need a research study to tell us home computer access will not, in itself, improve educational outcomes!

That's the unintended consequence of advocating for more student I.T. access.  Now imagine the challenges of teaching with every student being on laptops with open internet access in the classroom.  I don't have to imagine, of course, since I have experience with this.  Beyond the wrecked keyboards (why do students pick at the keys???), adding I.T. use into a lesson planning doesn't necessarily make it better.

Every time someone says students should have open internet access with their iPhone, iPad, laptop, etc, so that they can research for information right in the classroom, you should also think of the unintended consequences implied.  Students will also use their I.T. hardware to access Twitter, Facebook, and whatever else.  Yes, the students may suffer the natural consequence of not having as good a research project done, or not have their assignments completed, but unfortunately for many students, this is not a salient consequence at all.

I think I.T. use in the classroom and at home is a great thing, when it is properly managed and strategically employed.  This is not a trivial matter, and unintended consequences abound.  The proper use of such rich information sources, be it educational or merely entertaining information, as computer and internet technologies will be a topic for a future post.

Case 3
For a long time, researchers have known that test-taking helps learning.  Two recent papers reported by Munger and by Timmer show just this fact.  So if testing is great for learning, more testing is better, right?  And if we're going to test the students, the test marks should be a useful indicator of student performance too, no?

Unfortunately, when the test scores are used to determine consequences like teacher pay, school funding, etc, the unintended consequence is that the tests themselves become an obstacle to be bypassed rather than an obstacle to be confronted for the purpose of learning.  "Bypassing" here means teaching students "to the test", so to speak, and worse yet, outright tampering.

I recall reading about Soviet Russia (I've forgotten where I read this though), when glass production factories had quotas to meet.  In order to meet the quotas quickly, workers started making thinner glass panes that were faster to solidify.  This is reminiscent of using test scores to determine pay.  When pay and other high-stake benefits are determined by the number of correct answers on tests, then the number of correct answers will increase, regardless of whether students actually learn anything of use or not.

As Peter Drucker says, "What's measured improves".  But be very careful, because what's being measured might not be what you think you're measuring, and the unintended consequences can sometimes be worse than not measuring anything at all.

Another unintended consequences of frequent testing, even if the test scores are not used to determine high-stakes benefits, is the negative effect it has on the teacher/student relationship.  The teacher/student relationship isn't like the researcher/fake-student relationship that's established in the research studies.  The teacher has to essentially "live" with the students on a daily basis for 5 to 10 months, depending on the semester system in use.  As an adult, how would you like it if your manager reviews your work in detail on a weekly or bi-weekly basis? You'd cry that your manager is micro-managing, and breathing down your neck!

This doesn't mean tests shouldn't be used, or that it shouldn't be frequent.  But again, their use must be strategic and careful in avoiding the negative unintended consequences.  A rough separation to keep in mind is that of summative versus formative assessments, but this categorization isn't really fine-grained enough in practice.

Instead of thinking of assessments of various types, and then thinking of how they can be used, a more effective approach may be to think of the pedagogic purposes that needs to be satisfied, and then think of whether an assessment instrument of some type or other is the most effective means to that end.  There's so much more to assessment, but that's for another post.

Case 4
There is a false, but often subscribed to, belief that a master teacher can make all the difference in the classroom for students who has accumulated a learning and knowledge difference of years compared to their peers.  Like all occupation, there are the great ones and there are the really poor ones.  Of course, the worse of the worse teachers should be let go, but it's also impossible to have every single working teacher to be a master Jedi at teaching.  It's a simple statistical fact that the vast majority of teachers will be pretty much average in effectiveness.

But films like Stand and Deliver peddle the idea that students can neglect their studies for years, then with the right teacher, can go from arithmetic to calculus in a single year [3].  It's not just a conjecture that this idea is being spread around, but I have had parents basically demand the same happen for their child.  On a smaller scale, many students seem to think that they can neglect their studies for months, then cram for the final and get 90% (which they imagine getting in order to bring up their 55% average.  Needless to say, many are disappointed in the end).

It's probably true that there exist a few teachers out there who are so amazing that they can in fact deliver on this promise of near-instant student success after years of neglect.  But then this idea spreads and suddenly everyone is demanding every teacher to be able to do that.  As I said, it's a simple statistical fact that the vast majority of teachers will be pretty much average in effectiveness.

The unintended consequences are that students neglect their studies and expect a miracle from their teacher, that parents would take their children on month long vacations during the school year and expect their children would keep up with their studies [4], and that teachers-in-training are taught to believe that they, too, can be the master Jedi who will produce miracles in each of their students.  The sweat and hard work, over years of learning and teaching, that should be encouraged, are therefore de-emphasized, and unreasonable expectations spread like a virus.

Possibly a Case 5?
I love this article by Carey, reporting on some of the latest in cognitive research that has immediate application to learning and teaching.  One of the most interesting suggestion is to study in alternative rooms in order to mix up the contexts as a student studies some fixed material.  This is in contrast to the old advice, often repeated in study skills classes, of sticking to one specific place and to one scheduled time to study.

I'm not doubting the research findings, but calling the old advice "flat wrong", as Carey does, misses one of the reasons why the old advice might be effective.  I believe teaching students this new research-backed advice without the student knowing some context, and without them already possessing some reasonably good study habits, will invite negative unintended consequences.

The old advice about having a fixed location and time for studying is really an encouragement to study at least some amount regularly.  The fight isn't between less effective, but still useful, studying practice, versus more effective studying practice.  Rather, the fight for many educators is between persuading students to study some amount, versus students not studying at all.

Telling students to move from room to room, going from one location to another, invites students to waste the what little valuable studying time that many students will commit.  A one hour studying session, pretty standard suggestion to secondary school students (although completely inadequate), will turn into effectively only 40 minutes of studying if the student changes room twice.  Just think of all the "friction" in switching locations that might "reasonably" come up: packing up, setting up the new location, get a drink of water from the kitchen or cafeteria, plus a washroom break, etc.  That's a good 10 minutes to switch studying location!  We're all just very good at procrastination, and we're all very good at rationalizing what we do.

Every advice educators provide must first have its upside and downside weighed properly.  Sometimes, more efficient isn't more effective, as Dale Carnegie has noted.

[1] Job, Dweck, Walton.  (2010). Ego Depletion—Is It All in Your Head?: Implicit Theories About Willpower Affect Self-Regulation. Psychological Science.  See also, Gorlick.  (2010).  Need a study break to refresh? Maybe not, say Stanford researchers.  Stanford Report.

[2] Even as I write this blog post, I should be doing something else...

[3] Jesness.  (2002).   Stand and Deliver Revisited.  Reason.com.

[4] I've had parents get mad at me when I note the fact — in good faith and also while simultaneously noting that I understand these things do happen — that taking students out of their studies for a month-long vacation during the school year will negatively affect their studies.  That's as ridiculous as having a parent get mad at you when you note that encouraging the child to smoke may cause the child to have cancer.  I should add that I did continue to have a very positive relationship with those parents afterwards, so maybe they were having a bad day.  Who knows.