Testing aptitude is a tricky thing, because testers do different kinds of things that all have their own kind of aptitude. There’s a big difference between being a world-class tester and a key member of a world-class testing organization.
For instance, I am considered by many who don’t know me (but have heard of me) to be a great tester. And it’s true that 1) I hope that I am, 2) I’ve worked hard to be great, 3) many people who HAVE seen my work would testify to my competence. But even so, there are some important aspects of testing that I’m not very good at. For instance, most of my admirers would admit that my diplomacy skills are not great, nor do I have much patience for managers who don’t know shit about testing, and yet presume to tell testers how they should do their jobs. (They make me… sooo… angry….) Um, anyway, I’m at my best when I’m working with people whom I can vent to and who will forgive me for my venting.
One charming, docile, patient, stable person + me = a much better test team than me alone. So, we should not be too excited about any narrow idea of allowing only great puzzle solvers into our teams.
Having said that, I would suggest that these are some interesting aptitudes of a tactical software tester (meaning someone whose main job is designing and performing tests):
- A talent for gently, cheerfully, and insistently getting information out of people through interviews, observation, mind-melds, truth serums, or whatever means necessary. The ability to ask questions in a way that makes people want to answer. I evaluate this by posing testing questions that are insufficiently specified, and observing the questions that are asked.
- A talent for thinking laterally. Technically, I mean a talent for (and patience for) producing and using multiple alternative mental models to interpret and cope with technical situations that occur. This aptitude often shows itself by another name: a sense of humor. I evaluate this by posing realistic puzzles about the behavior of a product or the state of a spec, and observing how the tester entertains different possibilities during the course of analysis.
- A talent for thinking logically. This means reasoning from within the constraints of a particular model. To evaluate this, I give testers a problem in test framing. I get them to describe a test, and then I walk them forward and backward through the test, asking them to explain exactly why they did what they did, and what the implications would be of doing it differently.
- A talent for loving puzzles. Notice I did not say a talent for solving puzzles. This is because many puzzles we must cope with are too difficult to solve. And yet we must try, anyway. Testers who don’t have the patience and courage to throw themselves at puzzles are doomed to miss a great many bugs.
- A talent for rapid learning. Great testing can be described as rapid learning. Some obscure technology is thrown in your direction, and you have to jump on it like a hungry piranha. You must be excited about learning new things. To evaluate this, I look for evidence that the tester has taught himself something complicated and interesting. However, I would prefer to observe this directly by giving the tester a few hours to test a website I specify.
- A talent for story-telling. The credibility of a tester comes, often, through their ability to explain things that happened. To evaluate this, of course I interview. But I’m looking for someone who is able to tell compelling technical anecdotes.
- A talent for identifying and using tools. This does not mean programming, but programming skill certainly speaks to this.
- A talent for writing. This is hard for a lot of testers, since a lot of testers are writing in English, and English is not their first language. Furthermore, testers who got into testing via the computer science route tend to hate writing. This is partly why I’m suspicious of companies that are gleefully technocratic, such as Microsoft and Google.
- A talent for service. Testing is a support activity. A tester who loves to help others do better work (as opposed to attacking people for making mistakes) will be treasured by the programmers. Personally, I’m pretty good at this. I give developers who work with me a 13 point document outlining my unilateral commitment to their success.
I don’t need to see all these aptitudes to be excited about a tester. But I need to see at least a few. Out of all of them, I think rapid learning is the most important… Though perhaps that’s my bias, considering I’m a high school dropout who wrote a book about education. Self-education seems, to me, like a key that unlocks many doors.