Sunday, November 18, 2012

Testing Towers

I typed into Google the following:

"We test software to"

And read the results. I took some ideas from these results, and some of my own, and started to write down reasons to test. Things that motivate testers. Then I put them in an ordered list... looking at the list now it looks like I put them in (arguably) order of lowest to highest risk, the most important (to some imaginary business developing the software) at the bottom and what is considered the "icing on the cake" at the top. I looked for priority, and I looked for context, and I came up with a lot of different building blocks. Here are some:

Good User Experience (product is easy and/or fun to use, even if it doesn't do what is required)

Product Solves Problem (product solves the problem it was set out to solve, in that the functionality it has actually fixes the problem that existed that the product was made to fix)

Product Has Fewest Number Of Bugs (the highest possible number of bugs has been found, leaving the fewest number remaining in the software)

Product Won't Do Bad Things In Production (the product doesn't do anything that would

Product Is Reliable (product doesn't crash, shut down, run slowly, fail)

Product Won't Kill People (doesn't fail, causing death or serious injury)

Product "Works" (is operational. The buttons work, the functions do something)

I'm Learning (I know nothing directly about the "quality" of the product as a result, but I know more about the product, so I understand better what a "quality" product would mean)

Manager Isn't Bothering Me (ignore the software, I have my manager off my back)

Fulfils Requirements Doc (does what the documentation says it should)

Product Is Testable (product can actually be tested)

Based on context (software development methodology, culture, strategy, resources, and a host of other things) these could be stacked in multiple ways. You might want "product won't kill people" in there specifically because you're testing a safety-critical product. But if you're testing military weaponry perhaps you want your product to be able to kill people. It could be that Product Is Testable is on the bottom as it's required to test that it Fulfils Requirements in documentation and has the Fewest Number of Bugs which thus Solves The Problem. Personally I think that would be appalling in all sorts of ways, but it might be what your testing tower looks like. Like this:

Solves Problem
Fewest Bugs Fulfils Requirements Doc
Product Testable

Unfortunately I'd say that the Fewest Bugs block doesn't really exist. I'd say that the idea of finding a quantum quantity of bugs in an infinite test space is nonsensical. *POOF* it's gone, and the Solves Problem brick is unstable and could fall at any second! I'd say also that Fulfilling Requirements Doc does not support the solution of the test problem, but only the solution of the requirements doc, which is not the same thing. *CRACK*, it's broken. And the whole (if overly short for this example) testing tower comes tumbling down. Oops.

We all test to investigate the software, but what are you investigating? What is your aim? Something worthwhile I hope.

So, what are your bricks? Are they strong ones that support the layers above? Are they in the correct order, so as to stop your tower being top-heavy? Do they even exist in your tower (apply, and logically follow, in the real world)? It's an interesting way to see your processes, and the dependencies in your workflow, and what you're trying to achieve by testing the product, and for whom you are testing it. I wouldn't use it for modelling your desired practices, personally, but it does give some insight into these things, and possibly even the culture and practice of your company, and where things could be improved. Build your tower, and see if it has a worthwhile top to climb to. Then throw logic at it and see if it's still standing.

If you're looking for more (or better) bricks you might turn to something like James Bach's CRUSSPIC STMPL quality characteristics heuristics, or his CIDTESTD project environment heuristics. You could take stock of your resources, or learn about a software development methodology that you don't use.

What does your testing tower look like?

Thinking about this recently I've used the blocks to identify the divergence from why we should test to why we DO test and the importance of company culture. Strategy, planning, learning... all vital to set out what we should do. Add a layer of culture, habit and company practices and you get what we ACTUALLY do.

Tuesday, November 13, 2012

Learn to Code or Learn to Stack Shelves

Okay, that was a little inflammatory. I'll calm it down.

I haven't read much on the subject of testers learning to code, but people seem to be in one of three camps. The "nearly all testers must be able to code" camp, the "nearly all testers don't need to be able to code" camp and the "I kinda sorta agree with both" camp, which is actually the second camp but without the strong sense of reaction to the first.

I think one problem is that there is a great deal of diversity in testing, including automation coders, developers who write unit tests, exploratory testers, script checkers... and let's face it not EVERYONE needs to learn to code. We're not dragging "Introduction to Java" books into every vaguely IT role and telling them they need to learn this stuff. I'm next to certain my managing director does not know how to code. And I would say that the variance in testers (and checkers) and the context in which they work, is so widely varied as to bear the weight of my hyperbolic example.

Let's try it this way. Ask yourself this question:

Would learning to code improve my odds on remaining employed and/or ability to test what I'm testing?

If "no", then don't learn to code. If "yes" then ask yourself this question:

Would learning to code be the best use of my time and energy it would take?

If "no", then don't learn to code. If "yes"... I'm sure you can work out the rest.

Coding vs Testing?

Okay, so it's a bit facetious, but you get the idea. There is no answer to "should testers learn to code", and anyone that says there is a universal answer is either being deliberately obtuse or has failed to understand the non-universal nature of testing. So I suppose the only thing now is to give some ideas to the benefits and drawbacks to learning to test.

Firstly I must point out that if all you do is execute scripts then you are a checker, and you might be in trouble. Sorry.

What learning to code will cost you

Firstly the downsides.

1. Time

Learning to code takes time. That's about it. You could use that time 

2. Effort

Learning to code takes energy and focus which could be better directed to learning your product, or learning more about testing or related subjects (goodness knows there's enough to keep you busy!)

3. Choice

Choosing to learn to code in one language does not mean that you are immediately familiar with others (although it does help). So choosing one language to code in will not be much use to a tester if you find yourself in a position that you can no longer use it.

4. You might be awful at it

The mind of a developer is a confusing place, as I'm sure you probably know, and this difference isn't accidental. Not everyone is good at coding. It can be tiring, frustrating, repetitive and boring. Some people love it, but I would venture that a great tester would not make for a great developer.

5. Coding can be dull

You're probably a tester, and you're interested in what's new and interesting. Different angles and approaches, alternative ways of thinking, exploration, creativity and drive. Personally I find coding at any sort of professional level my own personal idea of hell; the same syntax every single day, building non-physical creations with a virtual soupy bucket of strands that must be built, re-factored and honed to a solution. Not for me. Although I do know how to code in some languages, and I find it quite fun... but only knowing that I don't have to be exact or precise or impress anyone in particular. Just as I enjoy occasionally writing poetry but I'd be crying with frustration into my notepad if I were paid to do it every day for a critical audience... wow, I suddenly feel a wave of empathy with developers!

What learning to code will get you

1. Insight

I've found knowing how software is made (from a blank screen to a simple set of functions) helps me understand the mistakes and shortcuts that occur. I use this information to perform quick-attack tests. If you are in a position to apply creativity to your testing then knowing how the people that make the bugs make the bugs will give you an array of tools to help you find them.

2. Alternative employment

A dodgy claim, but knowing how to code gives you another trade to fall back on. To be quite honest if you have a lot of experience as a tester and no experience as a coder you will very likely have a lot of trouble getting this sort of work, but.. it might help.

3. Automation

Automated checks with any worth usually require some level of testing experience. Chances are there's something you could automate, or partially automate, to help you reach your testing objectives. 

4. Tool building

I use my coding ability to build tools to make my job easier. I am a true computer user, in that I am essentially lazy. I spend a little time to save a lot of time in my testing, and reporting. For example, I've coded systems that detect the version and build of the system I am testing and copy bug reporting information to the clipboard because I'm too lazy to copy, paste and fill in a bug report's environment details. And I do that many times a day. Now it's never victim to copy/paste errors, and it saves me a lot of frustration and a bit of time.

Okay, so we broke that down into parts... Where do we go from here?

Well, if you feel that you'll get something out of learning to code, and that you have some passion for it, and the time and energy to spend, then go for it! I think it's vitally important that you have an interest in coding to be even vaguely successful at it. If I came to the belief that I needed to eat broccoli every day to keep my job I'd start looking for another job... even if I was very good at it without any need for broccoli. I do coding as a hobby some times. Mind you, I pick locks, play Dwarf Fortress and have Magic tournaments with my girlfriend, so maybe I'm not a yardstick of entertainment. Either way, I think the main point in all this that the phrase

"Testers need to learn to code"

or similar are rife with hidden presumptions and worrying overtones. The meaning of "Tester", the meaning of "need" and the meaning of "code". The variance of testers I've already covered, and I hope I've gotten across some of the needs (or wants!). Of course "code" means so many different things... and these (and other points) have been far better covered by Rob of "The Social Tester", here.

So will you learn to code? Or perhaps there's another book on epistemology, general systems or game theory that you have left unread on your shelf that would provide you more insight and use. I won't advocate any need to learn to code, but if you want to I'd be the last to tell you not to. Whatever you take away from this, don't forget to learn something every day.

I've come to realise that the "you need to learn to code" accusation is levelled at people whose testing value isn't sufficient as to keep them employed. Perhaps it's not the "testers" (checkers) that need to learn to code, but the higher-ups that need to learn how to get value of their testing, and therefore learning how to advocate good testing (and admonish bad testing) would be more valuable in keeping one's job than learning to code.