So, how does this work in a world of artificial intelligence (AI)? To understand the future implication, it’s worth a glance at the past for context.

As a computer geek that wrote my first programme aged 6 on a Victor Sirius 1 in the early 1980’s (if you think workflow today is hard, try it on a Sirius running MS-DOS 1!), I’ve worked merrily in the arena of requirements in, a package of “if, then, else, loop – GUI and data” (hosted on ever more elaborate infrastructures) resulting in product out.

‘AI revolution, not evolution’

The tools have expanded in capability immensely in the last 35 years, but with AI technologies and the available computational power (queue Moore’s law), we’re in the midst of an AI revolution, not evolution. It’s very exciting! Let me explain with some examples:

When Deep Blue beat Gary Kasparov in 1997, that was done with (and impressive at the time) sheer computational grunt. I got to meet one of the Deep Blue machines on a recent trip to the computing museum in Palo Alto – it’s there, frozen in time, preserved, obsolete. It was a significant step.

The most recent equivalent (the better part of 2 decades later) was in South Korea, in March 2016 when DeepMind’s Alpha Go beat Lee Sedol (the winner of 18 world titles). That was a spectacular evolution.

Machine learning & deep learning

Now queue revolution. Just 1 year later, in October 2017, Google’s “AlphaGo Zero” AI taught itself to become world champion in only three days whilst being told only the basic rules of the game. That’s incredible given the nearly infinite combinations in Go. Anyone else sensing another Moore’s law here?

… right back on track… back to the world of “if, then, else, loop – GUI and data”, as well as DevOps with a sprinkling of AI…

At its simplest, adding a predictive mechanism based on an LSTM or another self-correcting machine learning regression technique (e.g. Random Forrest) doesn’t comfortably fit the normal test cycle for software (irrespective of a delivery model).

For instance, with an oversimplified test case pass of 1+2=3, how does this apply to a black box that constantly retrains itself? I.e. it decides how important is 1 or 2, should it even consider them as statistically significant based on the latest training?

Control systems

Now imagine you have 100000 dimensions and you are retraining your models every half hour! An interesting problem when you have a control system where you must demonstrate at every stage how you reached your result. (Queue the requirement for an incorruptible high-speed transaction audit – Blockchain anyone?)

Now consider QA a human which can’t solve a 100000-dimensional problem in 30 mins. Therefore, how do you know if you’ve improved? Maybe parallel run two of these models side by side? Let a neural net learn and decide how valid the output is in each circumstance? Again, great, but we’ve just moved the problem up a level! Who watches the watchers after all.

Sure, we can test and verify results over past partial training sets, look at errors, and tweak as part of the release cycles, but I do wonder just as the now-defunct Victor Sirius 1 is obsolete (although 896K or RAM, 10.6MB HDD and 800×400 display were not to be sniffed at in 1982 before the requirement for IBM compatibility crushed it!) how far into the future our current DevOps practices can extend?

Written by James Kelloway, Agile Delivery Manager at National Grid