These are currently popular methods to test software.

Fuzz Testing

• Input massive invalid, unexpected, or random data(Fuzz) to find errors or vulnerabilities in the software.
• Effective on blackbox testing
• Pros
  • Simple
  • Provides results with little effort
• Cons
  • May miss bugs that don’t trigger a full program crash
  • May be less likely to trigger bugs that are only triggered in highly specific circumstances

 

Stochastic Testing

• Input random sequence of datas 

 

Model-based Testing

• Generates meaningful test case models which can detect errors on the desired behavior.

 

Symbolic Testing

• Replace inputs of program with symbolic variables.
• Track symbols rather than concrete input
• When execution path diverges, fork and add constraints on symbolic values
• Find the feasibility of all paths in the program.
• Quite expensive for finding feasibility of path.

x = read();
if (x > 3) {
  y = 1;
  if (x < 0)
    y = 2;
} else y = 3;

After changing with symbolic variables, it checks the feasibility of paths. On above example, the expression (y = 2; on line #5) will not be reached.

• Challenges
  • path explosion
    • Exponential in branching structure
    • Loop on symbolic variables
  • Complex code and environment dependencies
    • System calls
    • Library calls
    • Recursive calls
• Possible Solutions
  • For “path Explosion”
    • Random search (e.g., randomly restart in a while loop, pick next path randomly), but check coverage implementation is required
    • Combined search
  • For “complex code and environment dependencies”
    • Simulate system call
    • Build simple versions of library calls
    • Run library an system calls with a concrete value
    • Summarize the loops

• Applications:
  • Detecting infeasible paths
  • Generating test inputs
  • Finding bugs and vulnerabilities
  • Proving two code segments are equivalent (Code Hunt)

 

Concolic Testing (Concrete + Symbolic)

• Input actual concrete value to the program, then generate new input by changing the path condition.
• Less expensive than Symbolic testing.

 

A/B (or split, bucket) Testing

• Comparing two versions of software which performs better
• Statistical analysis which performed better

e.g.) randomly showing two different versions to users, and see which one had more uses

 

Boundary Testing

• Extreme ends like Start- End, Lower- Upper, Maximum-Minimum, Just Inside-Just Outside values are called boundary values and the testing is called “boundary testing”.
• Input variables are selected at their:
  • Minimum
  • Just above the minimum
  • A nominal value
  • Just below the maximum
  • Maximum
• Boundary Testing comes after the Equivalence Class Partitioning.

 

Equivalence partitioning or equivalence class partitioning (ECP)

• Software testing technique that divides the input data of a software unit into partitions of equivalent data from which test cases can be derived
• In principle, test cases are designed to cover each partition at least once.
• This technique tries to define test cases that uncover classes of errors, thereby reducing the total number of test cases that must be developed.
  • Pros: reduction in the time required for testing software due to lesser number of test cases

 

 

References

• https://www.guru99.com/equivalence-partitioning-boundary-value-analysis.html
• https://en.wikipedia.org/wiki/Equivalence_partitioning