Scripts Part I - What is a script?

Let's Define!

My current definition of a script is:

A set of instructions made explicit by one agent such that another (or the same) agent can later interpret them in order to replicate a process to achieve a desired outcome or behaviour.

So a script is, essentially, a communication tool. It can be verbal or written, it can be instructions to follow in order or a set of unordered instructions, but it is always done with the intention that the instructions will be followed by someone or something.

Here are some examples of scripts:
A computer program
A food recipe
A shopping list
A checklist

Communication

Implicit information in communication

When humans talk to each other we express two elements of communication. In pragmatics they are known as explicature and implicature. Explicature is what is actually said; for example "We went shopping, had a meal and ended up in the cinema". Implicature is what is implied by what is said. In the above example one might assume that going shopping, having a meal and ending up in a cinema happened in that order, but the sentence remains logically true even if the order is changed. There's also an implication that going to the cinema involved watching a film as part of a day out but the sentence remains logically true if the group stepped into a cinema then turned around and immediately walked out.

These two elements of communication are important when considering the audience for a script - the script has to be interpreted in order to be followed and that's what will affect the outcome of behaviour that actually happens (as opposed to the one desired by the script author)

Computers vs People

A computer program is a kind of script. It's a set of instructions made explicit by a programmer that the computer follows to execute the intentions of the programmer. Computers only pay attention to explicature in script instructions. This can be translated into the more common phrase "computers only do what you tell them to do".

Computer scripts may fail because computers are very picky about their explicature (they must be syntactically accurate). It may fail because the computer will execute the script as it exists while ignoring implicit desires. Just because you write a comment in the script saying "this function will return the square root of a number" does not mean it will. Just because you used a variable name called "square_root_of_input" does not mean that's what the variable contains - and your computer cannot care. We write unit tests in code as a way to try to ensure that the behaviour of the code we write matches our intended purpose - those are the lengths we go to to compensate for how mindless computers are and how literally they interpret our scripts.

Computers ignore implicature. They ignore context. And this is why they're so difficult to communicate with - human language is rife with implicature. We put up with this communications issue because computers are able to perform tasks (even ones that we are bad at or find impossible) in volume at an enormous rate.

A food recipe is a kind of script. It is a set of instructions made explicit by a person so that a (often) different person can follow them in order to end up with similar food. People have many powers that computers lack. They can understand implicit and tacit information. A recipe may fail because of a misunderstanding concerning implicit information. In the UK "crumble 3 plain biscuits" probably means McVities Digestives. In the US "biscuits" are basically a small bread a bit like a nearly tasteless scone. Try making a biscuit base with those.

Because it's followed by a person a recipe can have certain implicit or tacit information in it. For example if I'm told to use "self-raising flour" and I don't have any perhaps I might use plain flour and a leavening agent. The explicature is "add 200g self-raising flour". The implicature is "add something that has similar properties of 200g self-raising flour in the context of baking the pie this recipe describes". I know I can probably skip the "Add almonds" step but I can't very well skip "Bake at 180C for 40 minutes" because I can guess how differently they will affect the quality of the end product. This is the power humans have over computers.

Scripts in Testing

Automated Testing

Computers will interpret only the literal truth of explicature in scripts. This is the essence of checking to me - ignorance of implicature. This is a price we pay in exchange for speed - computers are much faster at making these checks.

Manual Scripts

Humans will interpret the implicature of a script. Humans are UNABLE to ignore implicature, we are simply not wired for it. It's part of our emotions. It's how we can process information in a heuristic way - not always accurately but with the efficiency to be evolutionarily successful. It's how we communicate complex ideas and their impact. It's what allows us to create art.

This means we cannot perform checks (under the strictest definition of check). A human check, I suppose, is the absence of knowledge to guide implicature in specific observations while making an evaluation in a product - we still make certain natural inferences but we can try to ignore parts of the context or try to behave as if we don't know the context in order to make a simple confirmation of the truth value of the explicature of an instruction (or what would be the explicature of an instruction were it uttered).

But why try to ignore inferences when we can use them?  Humans have enormous power over computers to investigate, learn, observe, interpret, infer, follow up, ask for information. So why write scripts to tell them what to do? If they're powerful enough to investigate a product surely it's cheaper and easier just to train them on how to look for problems rather than explicitly tell them what problems to look for. There is an answer to the above question which I hope to get to in a separate post.

What Is/Is Not A Script (In Testing)?

Things That Are Scripts (In Testing)

Automated Check Scripts

Automated "tests" are a set of instructions (the code) made explict by one agent (coded) such that another agent (the computer/compiler/interpretter) can later interpret (run/compile/interpret) them in order to replicate a process (making many checks quickly and forming some kind of output with the data) to achieve a desired outcome (the results of specific observations of the system in a way that a human can easily interpret, understand and evaluate)

Things That Aren't Scripts (In Testing)

Notes

Notes are not usually scripts. Notes are simply anything made explicit for the purposes of later reference. Scripts more specific - they refer to a set of instructions to be later interpreted to achieve a desired behaviour or outcome. Not all notes consist of a set of instructions and not all notes are designed to achieve a desired behaviour or outcome. All scripts are notes. Not all notes are scripts - in fact so few that we can make a abductive leap and say that when someone says "notes" they don't mean "scripts" because if they did they'd likely have said "scripts" to differentiate it from "notes".

Missions/Charters

A mission or charter is similar to a script in that it contains some explicit information and is used to communicate something in order to achieve a desired outcome or behaviour. The difference is that the communicated information is not a list of instructions to replicate a process. The instructions are not made explicit - the desired outcome or behaviour is. The steps taken (the set of instructions that replicate the process) is left up to the agent. It could be that a mission or charter contains a checklist (a type of script) so missions and charters can use the power of scripts. It may note a list of things that must be tested, information that's especially important, instructions on how to configure the system for the testing, and so on.

So while a recipe is a kind of script a note that says "We need a sponge cake. Plenty of lemon icing, please, and that fancy swirly decoration thing on the top."  is a mission, or charter. The recipe is left up to the chef - or maybe the recipient of the note will go out and buy a cake, or commission a custom cake.

Things That Might Be Scripts (In Testing)

Checklists

A checklist is simply a list of checks. The problem here is that it depends on how much information is left implicit in that list of checks that will determine if the checklist matches my definition of a script. Remember, a script is "a set of instructions made explicit by one agent such that another (or the same) agent can later interpret them in order to replicate a process to achieve a desired outcome or behaviour". This means that if the checklist has the intention and capacity to replicate a process then it's a script. If the process is left to the agent interpreting the checklist to achieve the desired outcome or behaviour then it's not. I'd go as far to say that most checklists are not scripts.

I'll update the list above as I find more things that are confused with scripts.

Thanks for reading, now please disagree with me!

- Kinofrost

Bad Testing Definitions

Life has become busy. I'm giving presentations and talks, I've taken up morning running, I'm getting involved with local groups and so on.  As a result I haven't had any time to blog, tweet or otherwise social media anything. So... here's my list of previously-tweeted "Bad Testing Definitions"!


#1: Focus/Defocus - what your eyes do after staring at the same page of the interface for an hour
#2: Exploratory Testing - Taking your work laptop with you when you go camping
#3: Risk-based testing - deciding your next test action using test techniques on post-it notes and a dartboard
#4: Test Coverage - what you wear while testing
#5: Test Cases - the capitalization of your text-based test data
#6: White-Box Testing - testing light-coloured modal windows in the UI
#7: Stress Testing - all testing up to and including 1 week before release
#8: Cross-Browser Testing - When you’re web testing and the browser throws a tantrum
#9: Touring - Moving companies once every 6 months
#10: Risk Impact - What you feel when you realise you didn’t test something important
#11: Claims Testing - Tests performed on functionality a customer has complained about
#12: Performance Testing - Making a song and dance about reports and metrics instead of doing any testing
#13: Test vs Check - How much testing you do for what you get paid each month
#14: Regression Testing - Testing while invoking the knowledge and experience of past lives
#15: Model-based testing - Fashion show focus groups
#16: Acceptance Testing - coming to terms with the state of the software
#17: Alpha Testing - Winning arguments online about testing term definitions
#18: User Acceptance Testing - confirming receipt of payment from customers
#19: Conformance Testing - Telling the project manager what they want to hear
#20: Volume Testing - Testing while sitting near to development on a Friday afternoon#21: Sanity Testing - Installing the fifth broken build in a row
#22: Fuzzy Penetration Testing - Some of these write themselves.#23: End to End Testing - Eating at a new Indian restauran
#24: Agile Testing - Dodging questions about the state of the product
#25: Best Practice
#26: Exhaustive Testing - Running out of coffee
#27: Black Box Testing - Using electronic devices while on a flight
#28: Static Testing - Touching the product whilst wearing a polycotton onesy
#29: All-Pairs Testing - The male component of a test team
#30: Backward Compatibility Testing - Idiot-proofing

Mysteries and Puzzles

National security expert Gregory Treverton (2009) wrote for Smithsonian Magazine in 2007 the following:

"Even when you can't find the right answer, you know it exists. Puzzles can be solved; they have answers.

But a mystery offers no such comfort. It poses a question that has no definitive answer because the answer is contingent; it depends on a future interaction of many factors, known and unknown."

The article is worth reading at some length if you haven't already done so (see references).

And I think within this definition lies a great deal to say about attitudes towards software testing. You understand the infinite test space and the necessary uncertainty of the quality of the systems you are testing, but I believe a lot of confusion, from certain people who are usually in non-testing roles, stems from their misunderstanding of the difference between determining good perceived quality through testing as puzzle solving and as mystery investigation.

The essential differentiating factor, as I see it, between a puzzle and a mystery is it's relationship to the quantity of information at hand. An unsolvable puzzle is one that simply requires more information to be added. Or as Malcolm Gladwell (2009, pp.160) puts it in his book What the Dog Saw: "A puzzle grows simpler with the addition of each new piece of information". A mystery cannot be solved purely through the addition of more information - in fact as we increase focus on collection of information then any further information we gain will contain a majority of noise in which more problems can hide; making the issue not necessarily worse, but necessarily bigger.

A bug found in production by a user is a puzzle; We can identify the problem, and immediately see what the desired behaviour is. We can trace it back, identify its root cause (or near root cause) and solve it. Preventing bugs going into production (basically: software testing) is a mystery; it's a cognitive process of questioning and re-framing with no (as yet) known problems and confusion over desired behaviour. We have nothing from which to trace back and we're executing the root causes (or near root causes) for problems that we haven't found yet.

This is one reason that we can only make predictive insights into user-experienced software quality; true quality is un-knowable. We have an understandable obsessive focus with solutions in business, and it's easy to see why, but treating software testing as a solution is risky if we identify the wrong problem to be solved. Our biggest problem is not a lack of information about a product, but too much. We have documentation, implementations, previous experience, relationships to the product, relationships to the software provider and its employees, the structure and culture of the company, the users, the customer, and I'm sure you can think of more. The problem we have is the attempt to identify what is useful, questioning both the product and the project. It's not about learning and applying a requirements document (like some developers do), it's about understanding and evaluating it; interpreting data into information and refining information into heuristic oracles. The investigation into the mystery of software bugs. To be better at software testing it isn't usually more initial information we need, but better analysis of the information we have (meaning we gather better information, not just more).

This is exactly why testing is a human-centric process and why it requires application of thought and engagement. As Gladwell (2009, pp.166) states: "Puzzles are 'transmitter-dependent'; they turn on what we are told. Mysteries are 'receiver-dependent'; they turn on the skills of the listener". And it's our skills in challenging what we are told, and more generally as information listeners, that make us good software testers. We are in the practice of a kind of Bayesian analysis of information, adjusting our belief in the subjective to rationally account for new evidence.

The logical solution to the puzzle of software testing is simply the collusion of information collection and persistence. The mystery of software testing requires experience, insight and investigation. Puzzles require us to think. Mysteries require us also to think about the thinking. This is why bug fix verification is easy, but boring. It's no longer a mystery to solve, but a set of puzzles... but if we continue to apply the mystery of software testing through Rapid and Exploratory techniques such as "branching and backtracking" (Bach, J., 20xx, pp.20) areas of opportunity then when we do bug fix verification we immediately add value to the testing in higher risk (due to the necessary recent code changes to fix the bug) areas.

Business has become good at solving puzzles, but seems to ignore or avoid mysteries; and as Philip Herr (2011) says: "And if we have been especially successful at solving puzzles, we may be tempted to define all business problems as such. (As the saying goes, “If all you have is a hammer, everything begins to look like a nail.”).". Our over-reliance on problem solving skills in business, combined with a desire to demarcate everything into metrics and simple, usable facts, has lead to a misunderstanding of the complexities and puzzle-resistant elements of software testing and therefore how testing can be best used to add value to any project. If you've ever worked in a company that views testing as a "necessary evil" or an exercise in compliance then you'll understand the point I'm making.

Dealing with useful information on and around a product being mixed together with a greater quantities of useless information, misleading information, incorrect information, and observation posing as objective fact requires us to look at finding important bugs quickly as a mystery, not a puzzle. As Gladwell (2009, pp.168-169) says, "If you can't find the truth in a mystery - even a mystery shrouded in propaganda - it's not just the fault of the propagandist. It's your fault as well."


References:

Bach, J., 2006. Breaking Down (and building up) Exploratory Testing Skill. [online] Available at: <http://www.quardev.com/content/whitepapers/ET_dynamics_Google_3_06.pdf> [Accessed 16 December 2012]

Gladwell, M., 2009. What the Dog Saw: and Other Adventures. [e-book]. Penguin UK. Available at: play.google.co.uk <http://play.google.co.uk> [Accessed 16 December 2012].

Herr, P., 2011. Solving Puzzles Delivers Answers; Solving Mysteries Delivers Insights. Brand News, [online] Available at: <http://www.millwardbrown.com/Libraries/MB_POV_Downloads/MillwardBrown_POV_Solving_Puzzles_Delivers_Answers.sflb.ashx> [Accessed 16 December 2012]

Tiverton, G., 2007. Risks and Riddles. Smithsonian Magazine, [online] Available at: <http://www.smithsonianmag.com/people-places/presence_puzzle.html> [Accessed 16 December 2012]

Testing Towers

I typed into Google the following:

"We test software to"

And read the results. I took some ideas from these results, and some of my own, and started to write down reasons to test. Things that motivate testers. Then I put them in an ordered list... looking at the list now it looks like I put them in (arguably) order of lowest to highest risk, the most important (to some imaginary business developing the software) at the bottom and what is considered the "icing on the cake" at the top. I looked for priority, and I looked for context, and I came up with a lot of different building blocks. Here are some:

Good User Experience (product is easy and/or fun to use, even if it doesn't do what is required)

Product Solves Problem (product solves the problem it was set out to solve, in that the functionality it has actually fixes the problem that existed that the product was made to fix)

Product Has Fewest Number Of Bugs (the highest possible number of bugs has been found, leaving the fewest number remaining in the software)

Product Won't Do Bad Things In Production (the product doesn't do anything that would

Product Is Reliable (product doesn't crash, shut down, run slowly, fail)

Product Won't Kill People (doesn't fail, causing death or serious injury)

Product "Works" (is operational. The buttons work, the functions do something)

I'm Learning (I know nothing directly about the "quality" of the product as a result, but I know more about the product, so I understand better what a "quality" product would mean)

Manager Isn't Bothering Me (ignore the software, I have my manager off my back)

Fulfils Requirements Doc (does what the documentation says it should)

Product Is Testable (product can actually be tested)


Based on context (software development methodology, culture, strategy, resources, and a host of other things) these could be stacked in multiple ways. You might want "product won't kill people" in there specifically because you're testing a safety-critical product. But if you're testing military weaponry perhaps you want your product to be able to kill people. It could be that Product Is Testable is on the bottom as it's required to test that it Fulfils Requirements in documentation and has the Fewest Number of Bugs which thus Solves The Problem. Personally I think that would be appalling in all sorts of ways, but it might be what your testing tower looks like. Like this:

Solves Problem
Fewest Bugs	Fulfils Requirements Doc
Product Testable

HTML Tables

Unfortunately I'd say that the Fewest Bugs block doesn't really exist. I'd say that the idea of finding a quantum quantity of bugs in an infinite test space is nonsensical. *POOF* it's gone, and the Solves Problem brick is unstable and could fall at any second! I'd say also that Fulfilling Requirements Doc does not support the solution of the test problem, but only the solution of the requirements doc, which is not the same thing. *CRACK*, it's broken. And the whole (if overly short for this example) testing tower comes tumbling down. Oops.

We all test to investigate the software, but what are you investigating? What is your aim? Something worthwhile I hope.

So, what are your bricks? Are they strong ones that support the layers above? Are they in the correct order, so as to stop your tower being top-heavy? Do they even exist in your tower (apply, and logically follow, in the real world)? It's an interesting way to see your processes, and the dependencies in your workflow, and what you're trying to achieve by testing the product, and for whom you are testing it. I wouldn't use it for modelling your desired practices, personally, but it does give some insight into these things, and possibly even the culture and practice of your company, and where things could be improved. Build your tower, and see if it has a worthwhile top to climb to. Then throw logic at it and see if it's still standing.

If you're looking for more (or better) bricks you might turn to something like James Bach's CRUSSPIC STMPL quality characteristics heuristics, or his CIDTESTD project environment heuristics. You could take stock of your resources, or learn about a software development methodology that you don't use.

What does your testing tower look like?

Edit:
Thinking about this recently I've used the blocks to identify the divergence from why we should test to why we DO test and the importance of company culture. Strategy, planning, learning... all vital to set out what we should do. Add a layer of culture, habit and company practices and you get what we ACTUALLY do.

Learn to Code or Learn to Stack Shelves

Okay, that was a little inflammatory. I'll calm it down.

I haven't read much on the subject of testers learning to code, but people seem to be in one of three camps. The "nearly all testers must be able to code" camp, the "nearly all testers don't need to be able to code" camp and the "I kinda sorta agree with both" camp, which is actually the second camp but without the strong sense of reaction to the first.

I think one problem is that there is a great deal of diversity in testing, including automation coders, developers who write unit tests, exploratory testers, script checkers... and let's face it not EVERYONE needs to learn to code. We're not dragging "Introduction to Java" books into every vaguely IT role and telling them they need to learn this stuff. I'm next to certain my managing director does not know how to code. And I would say that the variance in testers (and checkers) and the context in which they work, is so widely varied as to bear the weight of my hyperbolic example.


Let's try it this way. Ask yourself this question:

Would learning to code improve my odds on remaining employed and/or ability to test what I'm testing?

If "no", then don't learn to code. If "yes" then ask yourself this question:

Would learning to code be the best use of my time and energy it would take?

If "no", then don't learn to code. If "yes"... I'm sure you can work out the rest.

Coding vs Testing?

Okay, so it's a bit facetious, but you get the idea. There is no answer to "should testers learn to code", and anyone that says there is a universal answer is either being deliberately obtuse or has failed to understand the non-universal nature of testing. So I suppose the only thing now is to give some ideas to the benefits and drawbacks to learning to test.

Firstly I must point out that if all you do is execute scripts then you are a checker, and you might be in trouble. Sorry.

What learning to code will cost you

Firstly the downsides.

1. Time

Learning to code takes time. That's about it. You could use that time 

2. Effort

Learning to code takes energy and focus which could be better directed to learning your product, or learning more about testing or related subjects (goodness knows there's enough to keep you busy!)

3. Choice

Choosing to learn to code in one language does not mean that you are immediately familiar with others (although it does help). So choosing one language to code in will not be much use to a tester if you find yourself in a position that you can no longer use it.

4. You might be awful at it

The mind of a developer is a confusing place, as I'm sure you probably know, and this difference isn't accidental. Not everyone is good at coding. It can be tiring, frustrating, repetitive and boring. Some people love it, but I would venture that a great tester would not make for a great developer.

5. Coding can be dull

You're probably a tester, and you're interested in what's new and interesting. Different angles and approaches, alternative ways of thinking, exploration, creativity and drive. Personally I find coding at any sort of professional level my own personal idea of hell; the same syntax every single day, building non-physical creations with a virtual soupy bucket of strands that must be built, re-factored and honed to a solution. Not for me. Although I do know how to code in some languages, and I find it quite fun... but only knowing that I don't have to be exact or precise or impress anyone in particular. Just as I enjoy occasionally writing poetry but I'd be crying with frustration into my notepad if I were paid to do it every day for a critical audience... wow, I suddenly feel a wave of empathy with developers!

What learning to code will get you

1. Insight

I've found knowing how software is made (from a blank screen to a simple set of functions) helps me understand the mistakes and shortcuts that occur. I use this information to perform quick-attack tests. If you are in a position to apply creativity to your testing then knowing how the people that make the bugs make the bugs will give you an array of tools to help you find them.

2. Alternative employment

A dodgy claim, but knowing how to code gives you another trade to fall back on. To be quite honest if you have a lot of experience as a tester and no experience as a coder you will very likely have a lot of trouble getting this sort of work, but.. it might help.

3. Automation

Automated checks with any worth usually require some level of testing experience. Chances are there's something you could automate, or partially automate, to help you reach your testing objectives. 

4. Tool building

I use my coding ability to build tools to make my job easier. I am a true computer user, in that I am essentially lazy. I spend a little time to save a lot of time in my testing, and reporting. For example, I've coded systems that detect the version and build of the system I am testing and copy bug reporting information to the clipboard because I'm too lazy to copy, paste and fill in a bug report's environment details. And I do that many times a day. Now it's never victim to copy/paste errors, and it saves me a lot of frustration and a bit of time.

Okay, so we broke that down into parts... Where do we go from here?

Well, if you feel that you'll get something out of learning to code, and that you have some passion for it, and the time and energy to spend, then go for it! I think it's vitally important that you have an interest in coding to be even vaguely successful at it. If I came to the belief that I needed to eat broccoli every day to keep my job I'd start looking for another job... even if I was very good at it without any need for broccoli. I do coding as a hobby some times. Mind you, I pick locks, play Dwarf Fortress and have Magic tournaments with my girlfriend, so maybe I'm not a yardstick of entertainment. Either way, I think the main point in all this that the phrase

"Testers need to learn to code"

or similar are rife with hidden presumptions and worrying overtones. The meaning of "Tester", the meaning of "need" and the meaning of "code". The variance of testers I've already covered, and I hope I've gotten across some of the needs (or wants!). Of course "code" means so many different things... and these (and other points) have been far better covered by Rob of "The Social Tester", here.

So will you learn to code? Or perhaps there's another book on epistemology, general systems or game theory that you have left unread on your shelf that would provide you more insight and use. I won't advocate any need to learn to code, but if you want to I'd be the last to tell you not to. Whatever you take away from this, don't forget to learn something every day.


Edit
I've come to realise that the "you need to learn to code" accusation is levelled at people whose testing value isn't sufficient as to keep them employed. Perhaps it's not the "testers" (checkers) that need to learn to code, but the higher-ups that need to learn how to get value of their testing, and therefore learning how to advocate good testing (and admonish bad testing) would be more valuable in keeping one's job than learning to code.

Looking for Test Tools?

Are you looking for tools to help you in your testing?

You are probably in one of two situations.

Situation One

You have a problem for which you hope there is a tool to make your life easier. This is usual, and fine, and I wish you well in your journey. The way I usually do this is to identify the problem I have, and search Google for potential solutions, and you're probably not even reading this.

Situation Two

You are perusing a page of tools used by, and suggested by, other people. There is nothing intrinsically wrong with this, but here is where you need to be wary, and there are two main reasons that I believe you should:

Your context is not their context

What works for one person doesn't work for all others. This is fairly obvious, but you shouldn't confuse the process of testing with the tools of testing - if you try to save effort by changing the way you work to fit in with a tool then you're allowing the tool to potentially reduce how well you can work. You need to temper your lust for "smarter working" with a stoic judgement of the benefit the tool is going to provide.

Tools don't do any work

If I went out and bought a trowel and bricks and cement mixer and so on all I would end up with is well-prepared ingredients for a brick construction, because I do not know how to lay bricks. Anything I made would be slanted, unbalanced, weak and possibly dangerous. The tools we have to help us perform tasks are not replacements for a complete job but a short-cut to save time, effort or frustration in reaching our goal. If you seek, or use, a tool to try to do your testing for you ensure that you are comfortable with the fact that you are no longer testing. When you use a tool to help you you are taking yourself out of that part of the equation. This, on balance, may be an acceptable loss given the constraints on your time or sanity, or the lack of need for your personal touch for the task, but it is a loss nonetheless.


So, in conclusion, if you want to use a tool to help you do your testing then you need to balance the negative with the positive. The tasks you are replacing, your newly found lack of attention, your assumption the tool is doing the job, if the tool will work for your context, the maintenance costs and so on versus the time, effort and frustration you will save by using it. Don't confuse tools with processes.

Exploratory Testing, Management and Tools

Disclaimer: You wouldn't eat with a hammer and you wouldn't bang a nail in with a fork. Tools are options. They are a bag of tricks you keep by your side. A mechanic doesn't use the whole tool chest to check your breaks, and you shouldn't let software tools be your dogmatic methodology. Stay light, simple and efficient is my advice. Your own set of tools will be dependent on your own context. But if you want some more brushes in your paintpot here is a list of some exploratory and session-based management tools:

Second Disclaimer:

You will find a better list here: http://www.ministryoftesting.com/resources/software-testing-tools/

Exploratory Testing & Management Tools

Note Taking

Paper and Pen

Paper & pen is flexible (in both senses), versatile, expressive, cheap, easily accessed, and useful. It stores well in a suitable filing system, it's portable, and generally ubiquitous. Plus it doesn't need a computer, or even electrical mains power or batteries to use. Not to be ignored.

Notepad

A limited version of paper & pen for the computer! Your OS probably has an equivalent.

Rapid Reporter

Most ET practicioners have heard of this, in my experience. It is a "note taking application that aids in the reporting process of SBTM". It doesn't take up much real-estate, it sits always-on-top, and has a transparency slider, so it's very convenient to use. It's standalone (no installation), and outputs to the easy-to-parse CSV format. It's customisable, easy to control with the keyboard, and does its job very well, I highly recommend it.

http://testing.gershon.info/reporter/

Session Capture

Video Screen Recorders

Screenr, Camtasia, Hypercam

They all record everything you do, and if the disk space these videos use isn't an issue for you then they are a valuable asset. Once you have the video at hand you can answer those questions... Did it do that before? Did I click the button before or after? Why did it work the second time and not the first?...

qTrace

After a couple of suggestions I downloaded qTrace. I've played only marginally with this so far but it's remarkably good. You boot it up, select the application to record, hit record, then start testing. When you hit stop it presents you with every action you performed, and these trace files can be saved. You can even specify the number of screens you want kept so that you can raise the processes as a bug, and the software interacts directly with many tracking systems including JIRA (ours).. although you need to pay a small fee for this (and to remove a small ad). It also saves a very useful table of environment information (OS, browser version, CPU, RAM, Available RAM, current user...).
It has facilities to force a screen capture, add a notes to the steps, blur out sensitive information on the page, add arrows/boxes/text to your screens, and more.
Together with good notes I consider this a very viable candidate for our ET sessions. One thing it doesn't seem to do is track timestamps, so these would have to go in your notes.

Other Useful Tools

Mindmap Tools

FreeMind, XMind, MindMeister

JIRA Plugin: Bonfire [£]

A good-looking JIRA plugin that sits in your browser and handles private or public test sessions, bug report templates and more. Worth a look if you're planning on using session-based management.



Any further suggestions please tweet me @kinofrost, letting me know if you'd like direct credit for it (defaults to "no" for reasons of privacy and spam).

Many thanks to the many testers of Twitter, and a very useful RT from Software Testing Club for the suggestions.