It’s time to stop testing

Testing, as it has traditionally been done, may no longer be a good idea.

I am talking here about end to end, usually UI based, testing. Even the automated kind.  The kind that I myself have specialized in for several years!

Once simple fact cannot be ignored:

  • Quality for customers is determined by application code quality
  • Tests, in of themselves, do not improve the quality of application code

These simple facts have bothered me greatly in my role as an automation specialist over the past few years.

One clarification – I’m not talking about Unit tests (including not talking about Unit Tests in the UI layer, i.e. Javascript).  Those are the tests that are written using TDD and thus must be written before the application code, initially failing, to drive the application code design and result in testable code that always has tests.  There is never a ‘no time for tests’ situation when you always write the test first.  The practice is harder to do than it is to write here but it can be adopted.  Those Unit tests are still essential and should never be skipped.

When working on application code itself, I have recently seen the considerable difference in quality due to different approaches in writing ES6+ functional style JavaScript code and it is quite remarkable the number of bugs that can be avoided by using the modern constructs, along with the huge increase in readability and maintainability that contributes to higher quality code and less bugs for customers.

For much of our industry, End To End testing has moved from manual to automated processes and yet at company after company the approaches I encounter still reflect waterfall and command and control approaches – the UI tests are written by someone other than the developer and the feedback to the developers comes days to weeks later and is done by someone else who is then in a defensive, checking, testing role, not a quality improvement role. ‘Our QA is now called QE and is embedded in the team’ is a common refrain.  Unfortunately the next comments are often about how hard it is for them to keep up with developers when they write tests. Not to mention the fact that their best QE just got “promoted” to (application) developer and now earns $45,000 more per year.  Actions always speak louder than words and 2nd class citizen syndrome becomes rampant and accepted by all. “The QA person” has quite a remarkable set of assumptions and triggers implicit biases (many based on real evidence) in our industry.  The other issue is that the testing code base itself will take more maintenance over time, quickly becoming an additional issue for code quality and needing tests to test the tests and even tests to test them.

There are several key approaches that need to be adopted to address this change.  These approaches are well known by many organizations, however they still struggle to realize the changes that are needed in existing processes including architectural approaches.

The key new approaches are:

  • CI – Continuous integration to run all tests in the cloud for all branches during development
  • TDD measurements as KPIs, reporting and compliance measures
  • Immediate feedback from production customers by automated means
  • Canary Releases
  • Blue Green Releases
  • Feature Flags
  • Speed – Avoiding testing suite time lengths that continually grow
  • Continuous Deployment reducing MTTR (Mean Time To Recover)
  • Teams that promote contributions from all members and pay equitably

It’s exceedingly hard to do the above because most organizations default to continuing the previous developed testing characteristics of

  • Manually run automation and manual testing
  • TDD compliance not monitored as a KPI
  • Measuring bugs and focusing on speed of response to bugs
  • Production customer real-time automated feedback KPIs not shown in-house on primary displays to development teams
  • Test Suites that grow in length every week
  • QA’s being failed or junior developers that are paid less

and doing those activities quickly leads to no time to do the previously mentioned ‘new approach’ activities that would actually be of more benefit in improving quality for customers.  Change is hard, especially when it appears to mean less testing and more risk.  When done correctly it can actually mean more testing (more unit, less UI) and less risk if many supporting parts are done correctly but moving to this model is very hard and the more established the company and their software development shop, the harder it is.  This is one of the key reasons that small companies and startups continue to disrupt, as it is generally easier to adopt new practices than to change existing practices that were successful in the past.

To summarize, improve quality for customers with

Less End to End testing…


and more quality activities such as…

Code Linting, Code Grading, Code Reviews, Code Education and Training, Immutable Objects, Internal Iterators, TDD Measurement,Well named objects and methods, avoiding premature optimization, Short methods, Short Classes, Single Responsibility, English readable, well stubbed and mocked, SOLID principle based code that is deployed in a modern CI/CD environment that provides immediate automated feedback based on customer activity and provides the facility to revert changes with minimal efforts within seconds or minutes.

Works on ALL my machines !

It’s a familiar situation at work where code ‘works on my machine’… but when another developer… or a staging deploy.. or a production deploy happens, it doesn’t work on that machine.  There are many practices to address this – virtual machines, docker, automated tests, etc, etc.

There is a similar situation in learning new technology skills: The same “I did it once, on my machine and it worked, but when I tried later to do it, it didn’t work.  I don’t remember exactly how I did it before and this time I encounter unexpected problems I didn’t experience before.

This leads to a number of issues:

  • I typed something once.  I’m unlike to remember that in a week
  • At some point I’ll try and use a different machine
  • Dependency hell – stuff seems to be ok but then on machine X it isn’t
  • I didn’t encounter any problems so didn’t learn how to get around them

To address this, I use a practice of switching machines 2-3 times a day.

This approach developed naturally over time to match my daily schedule, i.e. by working from home on a desktop, working from a cafe on a laptop and then working from home on the desktop again.  As with other coding activities I am addressing the pain point by doing the activity more often, not less.  This invokes the lazy programmer in me that will then address ephemeral issues such as local setup and get the experience I need to be able to walk into other situations and make progress having encountered and conquered many different setup issues while learning.  It also ups the amount of source code management that I do through git which is always good practice.  I recently stopping coding within a Dropbox directory ‘cos I need to exclude node_modules/and dropbox doesn’t allow that (you’d have to selective sync for every node projects node/modules directory which is waaaay too much config management for me.


git setups

It’s a small thing, but… when I get

There is no tracking information for the current branch. 
Please specify which branch you want to merge with. 
See git-pull(1) for details. 
git pull <remote> <branch> 
If you wish to set tracking information for this branch you can with
git branch --set-upstream-to=origin/<branch> master

it’s easy to fix cos in my ~/.gitconfig file I’ve got

setups = !git branch --set-upstream-to=origin/`git symbolic-ref --short HEAD`

which  lets me simply type

git setups

to get

Branch 'master' set up to track remote branch 'master' from 'origin' by rebasing.

Vim for JS

I’m having the time to really fix and set up my tooling and that is such a good thing.

Today a couple of seemingly basic tasks I’d not had time for recently.

  1.  Get vim paste working within javascript files
  2. Get javascript tabs as spaces working as expected

These are a pretty big deal now I’m immersed in the world of good looking es6 javascript.
The last thing I want is my carefully styled code looking like blahhhh in other formats, editors, etc. due to mixed tabs and spaces.

As if often the case, the fixes turned out to be much simpler than feared.  Lets face it, when you are changing your main tools configuration there is good reason (and for me experience) at what might happen if you mess it up.

For 1, the changes were to add a line to my ~/.vimrc file for javascript the same way I had previously done for Ruby.  For ruby I have:

autocmd FileType ruby setlocal ts=2 sts=2 sw=2 expandtab

so for Javascript I just added

autocmd FileType javascript setlocal ts=2 sts=2 sw=2 expandtab

for issue 2, the fact that pastes
kept getting
more and more indented

I found that the fix for that was to do [esc]:set paste before doing the paste.  Remember to then [esc]:set nopaste after pasting because I remember something else breaking later if you don’t reset it.

Now I know I’ll be much more prepared to share js code with my IDE fan friends !