Black-Boxing the Simple App

Nearly every QA interview includes a roleplay scenario where you have to explain how you would go about black-boxing some kind of trivial Windows application. The hypothetical app could be purely the interviewer's brainchild, or the clone of some common utility (e.g. Calculator, Notepad, Paint). Whatever the particulars, let's refer to it as Simple App.

Despite the stunning simplicity of Simple App, resist the temptation to begin immediately rattling off the first 100 tests that pop into your head. This is not an exercise for you to come up with as many tests as possible. Yes, of course they want to probe your black-box testing expertise. But above all, they want to know whether your overall approach is methodical.

Here's how to tackle it. Tell them you have a 3-step approach:

1. You will conduct a smoke test. This has to come first, because you need an understanding of what Simple App basically does in order to write the test plan. Plus, should you encounter any severe test blockers, you can focus your planning effort on the features which actually work.
   
   Start with a clean VMware image. Enable system memory dumps and application crash dumps, then take a snapshot of the pristine OS. Launch Process Monitor and install Simple App. Note all the system modifications made by the installer, including files and Registry entries. Take a second VMware snapshot.
   
   Launch Simple App. Process Monitor should point out any config files and/or log files. Take an inventory of the app's major features. (At this point the interrogation begins in earnest, with the interviewer answering all the questions you have about Simple App. Take notes.)
   
   After you have an overview of the feature set, you want to peek under the hood. Launch Depends and examine the list of DLLs required by Simple App. You're looking for DLLs to reveal nonobvious functionality. Run Strings to find error messages, hard-coded paths, or other useful hints.
   
   
2. You will write a test plan. This is where to put all your feature-specific tests.
   
   No matter how brief, having a concrete plan is necessary for time management and tracking your testing progress.
   
   Your test plan should identify any likely risks to Simple App's quality. At the very least, this boils down to answering the question: What could go wrong with this app?
   
   Explain that you anticipate having to adjust your plan to changing test conditions, like drilling down as you encounter particularly buggy features. Also at this point, the interviewer often will play around with the precise features of Simple App. Roll with the punches.
   
   Without fail, your test plan must always include running under AppVerifier. Do not omit this.
   
   
3. You will follow the test plan, recording test results and any other notations as you progress. Mark down which tests have passed, failed, or been skipped. This would allow someone else to review or continue your work.
   

If you stick to an approach like this, you will demonstrate that you possess a methodical approach toward testing.