{"id":7917,"date":"2016-05-03T10:09:19","date_gmt":"2016-05-03T10:09:19","guid":{"rendered":"http:\/\/www.devopsonline.co.uk\/?p=7917"},"modified":"2016-05-03T10:09:19","modified_gmt":"2016-05-03T10:09:19","slug":"test-in-the-devops-world","status":"publish","type":"post","link":"https:\/\/devopsnews.online\/test-in-the-devops-world\/","title":{"rendered":"Test in the DevOps world"},"content":{"rendered":"

Test is like the Rodney Dangerfield of DevOps \u2013 it gets no respect says Joan Wrabets, CEO of Quali. But that is all about to change. In a recent Gartner survey, over 50% of respondents listed test automation as a top enabler of DevOps success. This is really illustrative of the recognition that DevOps cannot succeed if only one part of the DevOps continuum is automated.<\/p>\n

If DevOps is a continuum, it starts with planning and development and ends with deployment into operations. Much of the focus in DevOps has been on application build automation, or development, and deployment of apps into operations. But for DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test.<\/p>\n

DevOps projects tend to start in only one part of the continuum. When the driver for DevOps is in the test organisation, it is called continuous integration. However, it is also important to work to continuously expand automation across the continuum. When test teams take the lead, they need to find ways to push automation back into development and forward into staging and production. In fact, the problem of test automation is particularly difficult and this means that the team that implements test automation is in a great position to lead DevOps initiatives beyond test.<\/p>\n

Continuous integration challenges<\/h2>\n

Why is automating the various testing steps (regression, performance, stress, GUI, QA, Interoperability and security testing) so difficult? There are five key components that must be automated successfully:<\/p>\n

    \n
  1. The infrastructure: It starts with one very important characteristic that is critical to testing which is not automated \u2013 the ability to automatically mimic the infrastructure in which the software (or hardware) being tested is expected to run in when it gets to production. Having the ability to give each tester (or set of automated tests) their own personal replica of the target production infrastructure \u2013 aka a ‘sandbox’ \u2013 is critical to ensuring that the software coming out of test can be automatically and continuously deployed into production. Automating tests is not nearly as difficult as automating the underlying infrastructure in which those tests run.<\/li>\n
  2. The production environment workload: Often not considered in automating testing is the requirement to mimic the workloads that would be common in production. This includes the network traffic, other application workloads, and network security profiles. These activities need to be automated within the sandbox to create a more realistic environment for testing.<\/li>\n
  3. The tests themselves: Of course, the tests themselves also need to be automated and there are a variety of tools to assist in this process. As software gets more complex, this continues to be a challenge, with mobile testing replacing GUI testing as the most challenging type of testing to automate.<\/li>\n
  4. Reporting and result analysis: In order to provide continuous automation from development through testing to delivery, it is important to see and analyse test results automatically.<\/li>\n
  5. Tool integration: Finally, automation into and out of testing needs to be enabled. This means that sandboxes and test automation suites need to be API-driven so that they can be started by a previous tool in the DevOps toolchain and can also initiate a tool that follows testing in the toolchain.<\/li>\n<\/ol>\n

    Sandbox is a key enabler for DevOps<\/h2>\n

    Sandboxes are self contained infrastructure environments that can be configured to look exactly like the final target deployment environment, but can be created and run anywhere.\u00a0For example, developers can create a sandbox that looks like the production environment \u2013 from network and hardware to OS versions and software to cloud APIs. They do their development in that sandbox for a short period of time and when they are done they tear down the sandbox.<\/p>\n

    Testers can do the same thing, and in addition, they can run a bunch of tests with the sandbox configured to look like their internal IT environment and then automatically re-configure the sandbox on the fly to look like the external cloud\u00a0environment and run more tests.\u00a0This allows them to test all of the possible environments that the application could run in without disrupting the actual production infrastructure.<\/p>\n

    Technically, what is a sandbox?\u00a0Intuitively we know that a sandbox is a protected space where you have complete control and others are allowed in only if you invite them.\u00a0You can bring your own toys into the sandbox and make anything you want in the sand.\u00a0Just stomp it out if you don\u2019t like it and start over.\u00a0Technically, sandboxes follow these same rules.\u00a0A number of vendors are now providing Sandbox solutions (some are called ‘Environment as a Service’) that have a simple interface for creating any target infrastructure environment and configuring it with as much control as you want.\u00a0They allow you to bring\u00a0applications, tools, tests and automated processes into that sandbox. They provide protections so that others cannot mess with any infrastructure that you are currently using in your sandbox. They also provide reservation and scheduling for many\u00a0people so that whole teams of developers and\/or testers can share physical and virtual infrastructure on-the-fly for hours, days or weeks at a time.\u00a0Finally, a good Sandbox solution can be triggered from the outside (for\u00a0example, from a DevOps tool).<\/p>\n

    In the world of hybrid clouds, applications need to be deployable on both on-premise infrastructure and on public cloud infrastructure. Sandboxes allow testers and developers to mimic both environments and define applications that can run successfully in both types of infrastructure.<\/p>\n

    Edited for web by Jordan Platt.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Test is like the Rodney Dangerfield of DevOps \u2013 it gets no respect says Joan Wrabets, CEO of Quali. But that is all about to change. In a recent Gartner survey, over 50% of respondents listed test automation as a top enabler of DevOps success. This is really illustrative of the recognition that DevOps cannot…<\/p>\n","protected":false},"author":2,"featured_media":7920,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"content-type":"","pmpro_default_level":"","footnotes":""},"categories":[2],"tags":[145,67,298,611,610,28],"yoast_head":"\nTest in the DevOps world - DevOps Online North America<\/title>\n<meta name=\"description\" content=\"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Test in the DevOps world - DevOps Online North America\" \/>\n<meta property=\"og:description\" content=\"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test\" \/>\n<meta property=\"og:url\" content=\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\" \/>\n<meta property=\"og:site_name\" content=\"DevOps Online North America\" \/>\n<meta property=\"article:published_time\" content=\"2016-05-03T10:09:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png\" \/>\n\t<meta property=\"og:image:width\" content=\"884\" \/>\n\t<meta property=\"og:image:height\" content=\"515\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"DevOps Online\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@DevOpsAmerica\" \/>\n<meta name=\"twitter:site\" content=\"@DevOpsAmerica\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"DevOps Online\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\"},\"author\":{\"name\":\"DevOps Online\",\"@id\":\"https:\/\/devopsnews.online\/#\/schema\/person\/de52473fff111f14d90763193184cb1e\"},\"headline\":\"Test in the DevOps world\",\"datePublished\":\"2016-05-03T10:09:19+00:00\",\"dateModified\":\"2016-05-03T10:09:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\"},\"wordCount\":923,\"publisher\":{\"@id\":\"https:\/\/devopsnews.online\/#organization\"},\"image\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png\",\"keywords\":[\"automation\",\"DevOps\",\"Gartner\",\"Quali\",\"Sandbox\",\"testing\"],\"articleSection\":[\"Featured\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\",\"url\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\",\"name\":\"Test in the DevOps world - DevOps Online North America\",\"isPartOf\":{\"@id\":\"https:\/\/devopsnews.online\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png\",\"datePublished\":\"2016-05-03T10:09:19+00:00\",\"dateModified\":\"2016-05-03T10:09:19+00:00\",\"description\":\"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test\",\"breadcrumb\":{\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/devopsnews.online\/test-in-the-devops-world\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage\",\"url\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png\",\"contentUrl\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png\",\"width\":884,\"height\":515,\"caption\":\"Testing in DevOps\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/devopsnews.online\/test-in-the-devops-world\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/devopsnews.online\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Test in the DevOps world\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/devopsnews.online\/#website\",\"url\":\"https:\/\/devopsnews.online\/\",\"name\":\"DevOps Online North America\",\"description\":\"by 31 Media Ltd.\",\"publisher\":{\"@id\":\"https:\/\/devopsnews.online\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/devopsnews.online\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/devopsnews.online\/#organization\",\"name\":\"DevOps Online North America\",\"url\":\"https:\/\/devopsnews.online\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/devopsnews.online\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2020\/03\/DevOpsOnline_email.png\",\"contentUrl\":\"https:\/\/devopsnews.online\/wp-content\/uploads\/2020\/03\/DevOpsOnline_email.png\",\"width\":198,\"height\":64,\"caption\":\"DevOps Online North America\"},\"image\":{\"@id\":\"https:\/\/devopsnews.online\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/x.com\/DevOpsAmerica\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/devopsnews.online\/#\/schema\/person\/de52473fff111f14d90763193184cb1e\",\"name\":\"DevOps Online\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/devopsnews.online\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/cf0ba37fb1f8baf226b40986afbe7f9f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/cf0ba37fb1f8baf226b40986afbe7f9f?s=96&d=mm&r=g\",\"caption\":\"DevOps Online\"},\"url\":\"https:\/\/devopsnews.online\/author\/test-magazine\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Test in the DevOps world - DevOps Online North America","description":"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/devopsnews.online\/test-in-the-devops-world\/","og_locale":"en_US","og_type":"article","og_title":"Test in the DevOps world - DevOps Online North America","og_description":"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test","og_url":"https:\/\/devopsnews.online\/test-in-the-devops-world\/","og_site_name":"DevOps Online North America","article_published_time":"2016-05-03T10:09:19+00:00","og_image":[{"width":884,"height":515,"url":"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png","type":"image\/png"}],"author":"DevOps Online","twitter_card":"summary_large_image","twitter_creator":"@DevOpsAmerica","twitter_site":"@DevOpsAmerica","twitter_misc":{"Written by":"DevOps Online","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#article","isPartOf":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/"},"author":{"name":"DevOps Online","@id":"https:\/\/devopsnews.online\/#\/schema\/person\/de52473fff111f14d90763193184cb1e"},"headline":"Test in the DevOps world","datePublished":"2016-05-03T10:09:19+00:00","dateModified":"2016-05-03T10:09:19+00:00","mainEntityOfPage":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/"},"wordCount":923,"publisher":{"@id":"https:\/\/devopsnews.online\/#organization"},"image":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage"},"thumbnailUrl":"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png","keywords":["automation","DevOps","Gartner","Quali","Sandbox","testing"],"articleSection":["Featured"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/","url":"https:\/\/devopsnews.online\/test-in-the-devops-world\/","name":"Test in the DevOps world - DevOps Online North America","isPartOf":{"@id":"https:\/\/devopsnews.online\/#website"},"primaryImageOfPage":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage"},"image":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage"},"thumbnailUrl":"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png","datePublished":"2016-05-03T10:09:19+00:00","dateModified":"2016-05-03T10:09:19+00:00","description":"For DevOps to pay off, there needs to be continuous automation from development all the way through to operations \u2013 and that means automating test","breadcrumb":{"@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/devopsnews.online\/test-in-the-devops-world\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#primaryimage","url":"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png","contentUrl":"https:\/\/devopsnews.online\/wp-content\/uploads\/2016\/05\/Screen-Shot-2016-05-03-at-10.46.25.png","width":884,"height":515,"caption":"Testing in DevOps"},{"@type":"BreadcrumbList","@id":"https:\/\/devopsnews.online\/test-in-the-devops-world\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/devopsnews.online\/"},{"@type":"ListItem","position":2,"name":"Test in the DevOps world"}]},{"@type":"WebSite","@id":"https:\/\/devopsnews.online\/#website","url":"https:\/\/devopsnews.online\/","name":"DevOps Online North America","description":"by 31 Media Ltd.","publisher":{"@id":"https:\/\/devopsnews.online\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/devopsnews.online\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/devopsnews.online\/#organization","name":"DevOps Online North America","url":"https:\/\/devopsnews.online\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/devopsnews.online\/#\/schema\/logo\/image\/","url":"https:\/\/devopsnews.online\/wp-content\/uploads\/2020\/03\/DevOpsOnline_email.png","contentUrl":"https:\/\/devopsnews.online\/wp-content\/uploads\/2020\/03\/DevOpsOnline_email.png","width":198,"height":64,"caption":"DevOps Online North America"},"image":{"@id":"https:\/\/devopsnews.online\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/DevOpsAmerica"]},{"@type":"Person","@id":"https:\/\/devopsnews.online\/#\/schema\/person\/de52473fff111f14d90763193184cb1e","name":"DevOps Online","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/devopsnews.online\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/cf0ba37fb1f8baf226b40986afbe7f9f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/cf0ba37fb1f8baf226b40986afbe7f9f?s=96&d=mm&r=g","caption":"DevOps Online"},"url":"https:\/\/devopsnews.online\/author\/test-magazine\/"}]}},"_links":{"self":[{"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/posts\/7917"}],"collection":[{"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/comments?post=7917"}],"version-history":[{"count":0,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/posts\/7917\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/media\/7920"}],"wp:attachment":[{"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/media?parent=7917"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/categories?post=7917"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devopsnews.online\/wp-json\/wp\/v2\/tags?post=7917"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}