Data breaches are the new normal – according to the Identity Theft Resource Center, there were nearly 1,600 of them in 2017 in the US alone, exposing 179 million records.

Demonstrating the scale of the issue on the other side of the Atlantic, UK retailer Dixons Carphone admitted in June 2018 that it had suffered a hacking attack involving 5.9 million payment cards and 1.2 million personal records.

No wonder database administrators (DBAs) want to keep data confidential and protected, particularly in a world where companies can be fined 4% of their turnover for non-compliance under GDPR rules which protect the personal data of European citizens.

At the same time, however, the increasing pace of business means developers need to create and release code in much shorter DevOps timescales. And that includes the database too because Redgate’s 2018 State of Database DevOps Survey revealed that 76% of developers are now responsible for developing databases as well as applications.

Protecting data

This is where the snag arises because, in order to develop and test database code, developers need access to ‘real’ data. Redgate’s survey also showed that 67% of developers use production data in development and test environments, usually in the form of a copy of the production database.

So on one side, you have DBAs looking to protect data and account for every record, aiming for anonymity and confidentiality, and on the other, you have developers needing up-to-date, production-scale data to properly test changes.

An additional hurdle is the time and disk space eaten up which provisions copies of databases or backups for use in development. This can act as a blocker, either slowing down releases or be forcing companies to use out-of-date copies which are no longer representative of the production database.

The need for compliant provisioning

How do you solve this conflict? The database can’t be the bottleneck in DevOps, yet data must be protected. Organisations, therefore, need to adopt a four-stage process to keep everyone happy and development and test on track.

• Provision copies

Duplicating a full database to give a copy to developers is time-consuming and takes up lots of storage space, particularly given the growing size of production databases and the desire of many developers to have their own copy as a sandbox to test changes against.

Solutions have emerged, however, which ‘clone’ databases and enable copies to be provided to developers which are a fraction of the size of the original database. Importantly, they work just like normal databases and can be connected to and edited using any program, so developers don’t even notice the difference when using them.

Solutions like this smooth out the provisioning process and speed up the DevOps workflow for both DBAs and developers.

• Protect the data

The faster and more efficient provisioning of copies provides part of the answer, but it doesn’t remove the need to preserve the anonymity and confidentiality of customer data. That’s where data masking comes in, replacing sensitive data like personally-identifiable information with fictional data that retains the look and feel of the original.

This means the database remains perfectly usable for development and still operates in the same way as the real data for testing and analysis, but the content is secure, helping to meet regulations such as GDPR, HIPAA and SOX.

As well as development and test, this offers valuable advantages across a wide range of business areas including business analysis and cloud adoption – essentially any situation where there’s a privacy or security risk associated with using real, identifiable data. Companies should look for a solution that enables them to customise the rules which control which data is masked at scale, ensuring company-wide standards and compliance.

• Automate the process

Different developers and testers will want access to data at different times, probably when DBAs are at their busiest. Therefore, look at self-service tools that automate the process and make it easy for developers to receive copies of the database with information safely masked, when and where they need them. This frees up DBA time to focus on more business-critical activities.

• Manage access

One of the key requirements of regulations like the GDPR is that companies need to be able to document their processes and provide full audit trails of who had access to which data. Ensure you are able to create a central record of all your database copies so that you can track this, and look at role-based provisioning to prevent unauthorised access to sensitive information.

To conclude, the database needs to evolve faster for DevOps to achieve its full potential – it cannot be the bottleneck. At the same companies know the importance of protecting confidential information, and the consequences of failing to do so. Only by adopting a combination of efficient provisioning and data masking will they be able to balance these two demands, remove conflict and speed up development, test and release across their business.

Written by Steve Jones, Redgate Software