I Failed The ISTQB Practice Exam - Part I

Series: Part I, Part II, Part III & IV, Part V, Part VI and Final Thoughts

Oh, the ISTQB! A source of confusion and marketing evil that the devil himself could learn from. Or maybe it's the one standard of the beginners of our industry that sets some common domain language bringing everyone together in a way that will eventually unify the world under a flag of peace.

Either way I wanted to take the ISTQB practice paper and see how I'd do if someone just thrust it in front of me. So here were the rules:

1. No looking anything up.
2. No reading of the ISTQB materials, syllabus, website, etc.
3. Try to answer honestly
4. Use this practice paper: http://www.istqb.org/downloads/finish/41/87.html (ISTQB CTFL Sample Exam Paper), marked using the marking sheet: http://www.istqb.org/downloads/finish/41/82.html (Answer Sheet for Practice Exam Version 2011)
5. If the ISTQB's marking sheet says I don't get marks then I don't get marks.

Note: The ISTQB is the source and copyright owner of this practice exam. I republish any materials therein under this statement from the exam: "Any individual or group of individuals may use this Practice Exam as the basis for articles, books, or other derivative writings if ISTQB is acknowledged as the source and copyright owner of the practice exam.

The first thing I notice is that the practice exam is 4 years old, but that's what I got from their website.

So how did I do? Well I failed. The pass rate is 26/40 and I got 21/40. Oops. Let's take a look at my answers!

Part 1- Fundamentals Of Testing

1. Which of the following statements BEST describes one of the seven key 
principles of software testing? 

a) Automated tests are better than manual tests for avoiding the Exhaustive Testing. 
b) Exhaustive testing is, with sufficient effort and tool support, feasible for all software. 
c) It is normally impossible to test all input / output combinations for a software system. 
d) The purpose of testing is to demonstrate the absence of defects.  

My answer: C

ISTQB says: C

This seems fairly simple to me. I don't know what "the Exhaustive Testing" is, though or why it might want to be avoided.

Edit: Should point out that I also don't know what the "seven key principles of software testing" are. This is not required information to answer this question.

Score: 1/1

2. Which of the following statements is the MOST valid goal for a test team?

a) Determine whether enough component testing was executed.  

b) Cause as many failures as possible so that faults can be identified and 

corrected. 

c) Prove that all faults are identified.  

d) Prove that any remaining faults will not cause any failures. 

My answer: B

ISTQB says: B

I feel guilty about this answer. I put it because I felt it was the most valid goal for most situations out of the ones provided, but honestly I wonder now if that is the answer. B is certainly not a particularly valid goal in any case as it doesn't specify what failures are being caused. As many failures as possible, or as many useful and informative failures? I mean I could create an almost identical failure every 5 minutes and report it every 5 minutes and achieve this goal while I play Angry Birds and sip a can of Pepsi.

Score: A guilty 2/2

3. Which of these tasks would you expect to perform during Test Analysis and 

Design? 

a) Setting or defining test objectives.  

b) Reviewing the test basis.  

c) Creating test suites from test procedures.  

d) Analyzing lessons learned for process improvement.  

My answer: I forgot to answer this question

ISTQB says: B

I skipped over this question and forgot to come back to it. Would I expect to set or define test objectives during test analysis and design? What is test analysis and design? Presumably they mean analysis of the situation to design tests. Well I'd probably want to define the objectives of a test I was designing (A). I don't know what test basis means - I think it's the thing that defines what the tests are going to be so we're talking about the context and the product and things I find while I'm testing so I will definitely do that (B). I might create test suites from test procedures, although that sounds very unlikely.

Score: 2/3

4. Below is a list of problems that can be observed during testing or operation. 

Which is MOST likely a failure? 

a) The product crashed when the user selected an option in a dialog box.  

b) One source code file included in the build was the wrong version.  

c) The computation algorithm used the wrong input variables.  

d) The developer misinterpreted the requirement for the algorithm.  

My answer: A

ISTQB says: A

I got this question wrong. ISTQB says I didn't, but again this is a guilty answer. They're all failures. A is a failure of the product. B is a failure of whoever put the file there. C is a failure of whoever input those variables or permitted them to be input. D is a failure of developer to interpret the requirement or of the person telling them about the requirement to make themselves understood (unless they also misinterpreted the requirement of course). I presume we're talking about an explicit requirement.

Score: A guiltier 3/4

5. Which of the following, if observed in reviews and tests, would lead to problems (or conflict) within teams? 

a) Testers and reviewers are not curious enough to find defects.  

b) Testers and reviewers are not qualified enough to find failures and faults. 

c) Testers and reviewers communicate defects as criticism against persons and not against the software product. 

d) Testers and reviewers expect that defects in the software product have already been found and fixed by the developers. 

My answer: A, C, D

ISTQB says: C

Okay, I misread the question. I actually answered the question "Which... could lead to problems...?" If testers aren't curious enough to find defects then test clients will be annoyed at the lack of service. If they communicate defects as criticism against, say, the developers this could cause a problem (but if they blamed the government perhaps not). If testers expect that defects have been found and fixed then what are they doing there? I imagine this could cause all sorts of misunderstandings.

The sentence "Testers and reviewers are not qualified enough to find failures and faults." makes absolutely no sense.

The question actually specifies that they would lead to problems, so the correct answer (in my eyes) is none of them would necessarily lead to problems.

Score: 3/5

6. Which of the following statements are TRUE? 

A. Software testing may be required to meet legal or contractual requirements. 

B. Software testing is mainly needed to improve the quality of the developer’s work. 

C. Rigorous testing and fixing of defects found can help reduce the risk of problems occurring in an operational environment. 

D. Rigorous testing is sometimes used to prove that all failures have been found. 

a) B and C are true; A and D are false.  

b) A and D are true; B and C are false.  

c) A and C are true, B and D are false.  

d) C and D are true, A and B are false.  

My answer: C (a and c are true, b and d are false)

ISTQB says: C

Software testing is obviously sometimes required for legal or contractual requirements. That's like saying "sometimes you need a buy a ticket to ride the bus". It's also obvious that finding and fixing defects can reduce risk of them occurring later on. Obviously the main point isn't to improve the developer's work.

There is a possible argument to be made for statement D being true. In a simple enough system with strictly defined parameters and with a simple enough definition of failure (say, subjective to a single user) one could define a scenario where testing proves all failures have been found. Interesting thought experiment... probably we'd have to define failures here as "product failures".

I'm fairly sure my mum could answer this question with minimum prior testing knowledge. I might try the next series as "My Mum Passes/Fails The ISTQB Practice Exam", to compare results.

Score: 4/6

7. Which of the following statements BEST describes the difference between testing and debugging?

a) Testing pinpoints (identifies the source of) the defects. Debugging analyzes the faults and proposes prevention activities. 

b) Dynamic testing shows failures caused by defects. Debugging finds, analyzes, and removes the causes of failures in the software.

c) Testing removes faults. Debugging identifies the causes of failures.

d) Dynamic testing prevents causes of failures. Debugging removes the failures.

My answer: B

ISTQB says: B

I don't think any of these answers describe the difference between testing and debugging. I just guessed what sounded most sensible. Obviously testing is so much more than showing failures caused by defects, but the first half of the statement is true. The second half isn't necessarily true. Arguments could be made for A (sometimes testing finds the source of a defect, sometimes debugging reveals prevention activities).

Score: 5/7

Well that's it so far. Come back next time for Part II - Testing Throughout The Software Lifecycle.