Paul Gerrard introduces the Tools Knowledge Base.

The digital revolution is driving change in the software development and testing community. New processes, disciplines and the tumbling walls between silos mean tools are essential and the challenges of continuous delivery, DevOps regimes and shift-left are forcing testers to rethink how they test and where tools fit. The automation challenge has moved from selection and implementation of two or three tools for each discipline to selection and implementation of 20 or 30 tools for a project team using DevOps.

High volume of DevOps tools

Digital and DevOps are everywhere in the software blogs and media. Although DevOps is primarily a cultural change designed to reduce the delays in delivery and deployment, there is a focus on tools as an enabler. In principle, all of the processes to support continuous delivery are automated – now or at some point in the future. The tools required to provide this support include virtualisation and containerisation, source control, provisioning and of course, testing.

One key goal of digital is to continuously deploy to production to conduct experiments. The DevOps tools chain, as it is called, extends into production and includes logging, monitoring and analytics, so the range of tools required is much broader than we are used to. Nowadays, companies require twenty or so tools to be fully automated.

Tools to support digital programmes and DevOps.

But increasingly, requirements, story, release and task management and other collaborative tools are seen as part of the DevOps tools chain. The full tool chain might now have 30 tools. The schematic above attempts to place the tool types involved into a more meaningful structure.

The challenge of acquiring integrated tools chains is more complicated (and expensive) than before.

Curated listings of tools

Now, for as long as the web has existed, there have been websites that provide lists of references to tools that support for example, test automation. These web pages and sites have usually been set up by individuals, wishing to share their knowledge of software tools for their own communities. But it’s a burdensome task to create and maintain these lists. Vendors move webpages around, they rename tools, they merge and split tool functionality, they add new tools and new vendors and tools are popping up all the time. It’s really hard to maintain the accuracy of lists like these.

If you look around the various websites that provide such lists, this is what you tend to find:

• Listings do not provide much detail beyond simple categorisation, e.g. ‘Web’ or ‘Mobile’ test tools.
• Invariably, the lists are incomplete. Common tools are listed; less well-known tools are often missing.
• Most listings are dominated by proprietary tools. Open source tools are less well-represented, although some ‘free tools’ listings do exist, they are still incomplete.
• Many tools have functionality that spans multiple categories. Some are available in proprietary, some are open source and deployed on workstations, servers or SaaS platforms. Tools might be listed in multiple categories, but usually not.
• Tools listings often provide a link to a vendor web page and little else. Forums, training, supporting service companies or contractors are not usually listed and cannot be searched.
• Tools cannot be compared with respect to functionality, licensing, platforms or integrations.
• There are no tools usage statistics available; we have to rely on vendor marketing to gauge popularity.
• Not enough information, too much advertising.

The Tools Knowledge Base

The Tools Knowledge Base is a free-to-use service providing information on tools, vendors and the consultants and service companies that support them. It has been created by Gerrard Consulting working with TestOpera Limited and (among other features) provides:

• A searchable directory of over 2412 tools: Our focus is (broadly) DevOps, SDET and test activities. Each tool record stores limited data but links to the vendor or developer web page. This basic information and the content of the tools web page are downloaded and indexed nightly by our search engine.
• A sophisticated search engine: The search facility can be used to find tools using textual queries.
• A hierarchical tool type/features list: Every tool can be properly profiled and compared.
• Over 19,813 searchable blog posts: We download the content of blogs from over 300 bloggers each night. We do not store the blog posts, we provide a searchable index and link to the sources.
• Embeddable content: We offer a range of APIs allowing conferences, service companies and consultants to access and share our data on their own websites.

How do I use the Tools Knowledge Base? How is it kept up to date?

If you are tools user: we’d like you to register, and identify the tools you use. Your tools chain will appear on your profile. If a tool you use is not in TKB, then we invite you to create it. (If you want to embed the tools you use as a list in your website, there is an API for that).

If you are a tools expert or tools service provider: please see above, plus…we are looking for people who are knowledgeable enough to review or possibly edit the features listings for the tools you know well. The features hierarchy will grow and evolve over time – help us to perfect it. We will list you as a service provider on the tools you know best. It’s the least we can do.

If you are a tool vendor: we ask you to search for the tools you offer and check they are in the system and properly described. If your tools exist in the system and you want to manage the information we hold, that’s fine – we can make you the administrator (after a quick check on ownership). Alternatively, nominate a tools expert and we’ll invite them to keep your details correct. We can also maintain the data on your behalf.

If you own/contribute to an open source project: We make exactly the same offer as we make to the vendors. You are free to edit the information for your tools in the same way. You might already use GitHub, for example. We offer an extra publicity channel to reach a broader audience. People looking for tools often start their search with proprietary products and rarely see free tools listed side by side. Now is your chance.

If you are a blogger: Search for one of your recent blog posts and if you find it – your blog is already indexed in the system. If not, you can suggest the blog and register it yourself as long as it aligns with our scope (DevOps, testing and collaboration). Company blogs and general technology blogs are also acceptable. If your blog is indexed, let us know, and we will give you credit for it on your profile.

If you maintain your own online tools listing: please get in touch. We believe we already have more comprehensive listings than anyone else. We can provide ad-free, embeddable tool-type listings for your existing site. Join us as a partner and tools expert and help us to improve the data in our system to improve the value of your site. There are currently 183 tool type listings – all are available for free.

If you want a tools listing on your own site: Get in touch. We make the same offer as above. 

The Tools Knowledge Base is an attempt to help with the problem of tools research, evaluation and selection. We need your help as users, tool suppliers, developers and experts to make it the tools directory of choice for Digital projects and DevOps regimes.

Edited for web by Cecilia Rehn.