Lecture 8

• Life is never perfect
  • Internet fails to work
  • Phone gets disconnected
  • You team loses
  • Viruses kill people
• We need to test things before using them
  • Check internet signal
  • Check phone signal
  • Training sessions
  • Examine virus in lab

• If something fails we need to fix it
  • But to fix something, you need to know why it was broken in the first place
• In real life
  • Check internet cable, diagnose drivers, restart modem
  • Check airplane mode, move around the house, restart phone
  • Change tactics, change players, check equipment quality
  • Extract genome, test with animals, computer simulations

Problem Fixing

• Code sometimes always fails
  • Even code written by experts can fail
  • It is not possible to foresee every possible scenario
• You do not expect to write code in one run

Program testing can be used to show the presence of bugs, but never to show their absence!

Edsger Dijkstra

No amount of experimentation can ever prove me right; a single experiment can prove me wrong.

Albert Einstein

^^^ One sentence summary of the “scientific method”

• ensure code runs
  • remove syntax errors
  • remove static semantic errors
  • Python interpreter can usually find these for you
• have a set of expected results
  • an input set
  • for each input, the expected output

• intuition about natural boundaries to the problem
  • can you come up with some natural partitions?

def is_bigger(x, y):
  """ Assumes x and y are ints
  Returns True if y is less than x, else False """

• if no natural partitions, might do random testing
  • probability that code is correct increases with more tests
  • better options below
• black box testing
  • explore paths through specification
• glass box testing
  • explore paths through code

def sqrt(x, eps):
  """ Assumes x, epsilon floats, x >= 0, epsilon > 0
  Returns res such that x-epsilon <= res*res <= x+epsilon """

• designed without looking at the code
• can be done by someone other than the implementer to avoid some implementer biases
• testing can be reused if implementation changes
• paths through specification
  • build test cases in different natural space partitions
  • also consider boundary conditions

def sqrt(x, epsilon):
  """ Assumes x, epsilon floats, x >= 0, epsilon > 0
  Returns res such that x-epsilon <= res*res <= x+epsilon """

• use code directly to guide design of test cases
• called path-complete if every potential path through code is tested at least once
• what are some drawbacks of this type of testing?
  • can go through loops arbitrarily many times
  • missing paths
• guidelines
  • branches
    • exercise all parts of a conditional
  • for loops
    • loop not entered
    • body of loop executed exactly once
    • body of loop executed more than once
  • while loops
    • same as for loops
    • cases that catch all ways to exit loop

def isPrime(x):
  """ Assumes x is a nonnegative int
  Returns True if x is prime; False otherwise """
  if x <= 2:
    return False
  for i in range(2, x):
    if x % i == 0:
      return False
  return True

• What should we test for?

def isPrime(x):
  """ Assumes x is a nonnegative int
  Returns True if x is prime; False otherwise """
  if x <= 2:
    return False
  for i in range(2, x):
    if x % i == 0:
      return False
  return True

isPrime(0) # expected value: False

## False

isPrime(2) # expected value: True

## False

• Oops! For x = 2, the answer should have been True!
• We know we should test this because we could see the code.

def abs(x):
  """ Assumes x is an int
  Returns x if x>=0 and –x otherwise """
  if x < -1:
    return -x
  else:
    return x

• a path-complete test suite could miss a bug
• path-complete test suite: 2 and -2
• but abs(-1) incorrectly returns -1
• should still test boundary cases

• Exercise both branches of all if statements.
• Make sure that each except clause (we will learn later) is executed.
• For each for loop, have test cases in which
  • The loop is not entered (e.g., if the loop is iterating over the elements of a list, make sure that it is tested on the empty list),
  • The body of the loop is executed exactly once, and
  • The body of the loop is executed more than once.
• For each while loop,
  • Look at the same kinds of cases as when dealing with for loops.
  • Include test cases corresponding to all possible ways of exiting the loop.
• For recursive functions, include test cases that cause the function to return with
  • no recursive calls,
  • exactly one recursive call, and
  • more than one recursive call.

• When bugs enter electric equipment, they may break it
• When virtual bugs enter software, they may break it
  • Except, virtual bugs cannot crawl into software
  • You put them there!
• Debugging is the act of finding and removing bugs from software