jessenoller.com

Some ran­dom thoughts that have been brew­ing in my head since fly­ing in on Wednes­day around how to build out a log­i­cal series of tests, and libraries within the Qual­ity Assur­ance space. (Sorry, this is a bit of a stream of con­scious­ness post)

First, let’s cover the assump­tions you need to make:

• All tests, and test cases, will be even­tu­ally deprecated
• Every Test has some value (even dep­re­cated tests)
• Hav­ing a clear strat­egy for dep­re­cat­ing tests and then migrat­ing those tests into a bin for “func­tional regres­sion tests” is key

Start­ing here, and think­ing about it, you have to make some log­i­cal par­al­lels, the core of which is “if each test case’s value even­tu­ally reaches slightly more than 0, but future test cases must build on those test cases, mod­u­lar­ity and dis­pos­abil­ity is key”.

That’s not any­thing ground break­ing, in and of itself — how­ever, how you actu­ally imple­ment this is key. What I see Perl devel­op­ers com­monly do (and most other peo­ple) is to iso­late core behav­iors within a core series of libraries. How­ever, I reg­u­larly see lit­tle mod­u­lar­ity within those libraries.

You must approach this from the stand­point of each test case is a series of Atomic “stand alone” actions keyed into a spe­cific sequence result­ing in a test case. If this is the case — shared libraries for core actions are key. How­ever, iso­la­tion and mod­u­lar­ity of these libraries is key.

If you approach this cor­rectly, your tests become ulti­mately dis­pos­able. Say you have a test case like this:

• write 5k files
• read 4.5k files
• delete 3.5 files
• Hash remain­der files

If you iso­late the write, read, delete and hash actions into a series of mod­u­lar libraries, the actual imple­mented test case becomes a sim­ple wrap­per around these actions. Break­ing up the libraries into log­i­cally sorted libraries means that you can eas­ily swap out/around those libraries out from under­neath the tests (com­monly, a result of refactoring).

Men­tion­ing refac­tor­ing brings up an inter­est­ing side point — IMHO, QA test code, and libraries should be refac­tored con­stantly, and mer­ci­lessly. They com­monly need to cover the width and breadth of the PuT (prod­uct under test), and prod­ucts grow and change. From what I have seen, you com­monly need 2x the core prod­ucts code base to effec­tively auto­mate the test­ing of the PuT, and this means you have a huge code base (and big code bases move slow) but with 2x the code, you have to move 2x the speed of the prod­uct just to keep up with new fea­tures, dep­re­cated fea­tures, or new actions.

I should refac­tor this post later — but in essence, I am sim­ply act­ing as a cheer­leader for QA to adapt highly Agile and RAD meth­ods of devel­op­ing tests and tools. If you have >85% automa­tion (as my cur­rent com­pany does) you have to be fast to recode/tool those tests, libraries and tools, as well as devel­op­ing new ones.

You must mod­u­lar­ize, you must refac­tor, and you must dep­re­cate. Just like a devel­oper. QA is a busi­ness that has mul­ti­ple mas­ters. You must act as a devel­oper, and think like one when writ­ing test code. But you also have to leave your QA hat one.

Iron­i­cally, large code bases within QA means you should prob­a­bly think about writ­ing tests for your tests. Oh, the fun.

Some ran­dom thoughts that have been brew­ing in my head since fly­ing in on Wednes­day around how to build out a log­i­cal series of tests, and libraries within the Qual­ity...