Friday, January 27, 2012

This has gone far enough

Author’s note: I am biased. For religious and philosophical reasons, I reject the idea that any person is incapable of anything. Some people have predispositions towards or against certain ways of thinking or performing and some people have disabilities. But I still believe, at least in general, that anyone who wants to learn something can learn it.

There’s a paper that has been quoted and linked quite a bit for the last little while. It describes an attempt to understand the bimodal distribution computer science teachers have found, no matter how they teach, for many years. All of the evidence seems to indicate that some students can program and some can’t.

The authors designed a test, showing a very small, very simple excerpt from a Java program, like the one that follows:

int a = 10;
int b = 20;
a = b;

The test essentially consisted of recording students’ responses to questions about the effects of the program. Of course, the authors were not so simple as to try to test students on their programming abilities to determine if they could become capable programmers. Instead, their conclusions stated that students who demonstrate consistency from question to question were more likely to succeed in an introductory programming class after the fact. By consistency, the authors mean forming a mental model for what each statement does and using that same model for each statement in each program, in order.

The paper shows tabular results from several different tests. The tables validate the authors' conclusions - and several other conclusions. Importantly, they show that students who have prior programming experience are likely to do very well on the test - much more likely, in fact, than those who have no such experience.

Since the test includes no instruction about programming, students are left to wonder how the statements interact with each other. Most students who have no experience in programming are likely to use algebraic rules to interpret the program; after all, algebra is the only system most of them would know that uses notation that looks like this. An algebraic evaluation, however, leads to a contradiction for our sample program (as 10 is not equal to 20). There is also no reason to assume that students who have no prior experience would assume (correctly, in this case) that the statements take place in sequence and not all at once. It is likely, but not certain, that students who do assume sequence will assume (again, correctly) that the sequence occurs in top-down order.

In short, the test seems to favor those who already know the answers. Those who don't are unlikely to succeed in programming classes.

It seems likely that a host of factors play a part in these findings. Students who do well in math are generally less intimidated by mathematical notation and so can think more clearly on the test and in the class. They also are more likely to have prior experience. Those who have prior experience are more likely to have habitual thought patterns that match other programmers; as long as programmers typically come from some demographic, their work will reflect their polity. This means that language design will favor those who use it because it has to make sense to them. This also means that professors in computer science, who are themselves successful programmers (we assume), are likely to think along the same lines. They are likely, then, to teach in a way that reflects their habitual thought patterns. This favors like-minded students and makes life more difficult for those who approach things differently.

To be sure, programming requires people to learn to think in new ways. It's hardly as if people are born with the right thought patterns in mind. But some students have a head start because of predisposition and prior experience. Those who don't, it seems, tend to do poorly in their classes.

There's quite a bit of irony here. Programmers are, by necessity, logicians (some better than others). We ought to be able to spot logical fallacies easily. But no one seems to have noticed that this study essentially begs the question: if students know programming, they'll do well in the class that ostensibly teaches them programming. If they don't know it, they won't do well.

It seems obvious to me that the problem is in the way we teach; after all, the students who don't know programming but sign up for an introductory class are really the ones the class should target. For example, do we make enough time for the students who don't know about programming to learn? Do we assume that all students will make the same assumptions we make (after decades of training) about sequence? Do we make other similar assumptions? Do these assumptions detract from our teaching? Does the large number of students in an introductory lecture make it impossible for the students who don't know how to program but who want to learn to ask questions and get answers?

In short, does our educational system fail to help those who need its education most?

I think it does.