Zafar Ali, Director, IDB Solutions LTD, discusses how you can boost your success through the creation of a good BI testing strategy. 

Enterprises use various BI tools like Cognos, Oracle Business Intelligence Enterprise Edition (OBIEE), Tableau, SQL Server Reporting Services (SSRS) etc., to analyse swathes of data and convert raw data into a more presentable format like dashboard or key performance indicators (KPIs), in order to make well-informed decisions, but you need to make sure your testing strategy is good.

The success of any BI project depends on the trust of the data shown in the BI dashboard/reports, performance of the BI system, data integrity, data security, ease of use of the application and the graphical representation of data.

Testing of a large scale BI system faces challenges in the form of data quality assurance (DQA), metrics/aggregation rule test and data grain etc. The success of any BI system depends on the traceability of the BI functionality to the business requirement (BR), issue/anomalies fix log and how well the test cases are developed.

What is BI testing?

BI testing plays a pivotal role in the success of large-scale BI rollouts in enterprises.

BI testing ranges from: data validation, formatting, security, and performance to graphical user interface (GUI). Emphasis on a thorough BI is pivotal for improving the quality of data, user interface and meeting business requirements.

Developing a good testing strategy

The purpose of BI/DWH/ETL testing is to get credible data for your end-users. Making sure a comprehensive testing strategy is adopted can increase the credibility of BI.

A comprehensive test strategy is a stepping-stone for the effective test cycle of BI application. Your test strategy should cover any stage of data flow in order to make sure that the data input to data output is effectively tested.

To ensure testing readiness, the following key areas of your testing strategy should be focused on:

• Testing scope – testing categories and types to be used
• Test environment
• Test data availability
• DQ and performance benchmark
• Issue/enhancement log and traceability

The different testing categories

DWH/ETL testing:

ETL testing is quite different from conventional testing. There are many challenges we faced while performing data warehouse testing. Here is the list of few ETL testing challenges:

• Incompatible and duplicate data.
• Loss of data during ETL process.
• Unavailability of inclusive test bed.
• Testers have no privileges to execute ETL jobs by their own.
• Volume and complexity of data is very huge.
• Fault in business process and procedures.
• Trouble acquiring and building test data.
• Missing business’s data transformation rules.

Data is important for businesses to make critical business decisions. ETL testing plays a crucial part in validating and ensuring that the business information is exact, consistent and reliable.

Functional testing:

When a new report or dashboard is developed for consumption by other users, it is important to perform a few checks to validate the data and design of the included reports.

Verify that the new report or dashboard conforms to the report requirement / design specifications. Some of the items to check are:

• Verify that the report or dashboard page title corresponds to the content of the reports.
• For reports with charts, the axis should be labelled appropriately.
• The aggregation level of the data in the reports should be as per the report requirements.
• Verify that the report or dashboard page design conforms to the design standards and best practices.
• Validate the presence and functionality of the report download and print options.
• Where applicable, verify that the report help text exists and is appropriate for the report content.
• Verify the existence of any required static display text in the report, such as FOIA text.

Regression testing:

BI stress testing is similar to the testing of any other web/desktop application. The objective is to simulate concurrent users accessing reports with different prompts and to understand the bottlenecks in the system.

A typical BI user will login to the system and navigate to reports (or dashboards) and apply prompts and drills down to other reports. After the report is rendered, the BI user reviews the data for a certain amount of time, called ‘think time’. Conducting a stress test requires the simulation of the above BI users behaviour, concurrently, for different user loads. So it is important to have a list of different user logins for the stress test. When executing the reports, each user can pick a different set of prompt values for the same report.

Security testing:

BI applications also have authentication and authorization security requirements, often integrated with SSO (single sign on) or lightweight directory access protocol (LDAP) or other mechanism like oracle weblogic. The objective of security testing is to validate that the BI user’s access to the BI reports, subject areas and dashboards, is limited according to their access levels. Access to the reports is generally controlled by an application role or a LDAP-based security feature in the BI tool.

Single sign on is often used as the authentication mechanism for BI applications in large enterprises – objective of this testing is to ensure that users are able to access BI applications using their single sign on access (or windows authentication).

Report/BI testing:

BI tools, such as OBIEE and business objects, empower the business users by providing the capability for them to create their own reports without the help of a developer. These tools generate the database queries automatically for the reports, based on the measures and dimensions. For OBIEE the model is defined in the RPD, while business objects store the model in the form of a universe. Business users can select any combination of dimension and then measure the attributes available in the subject area in order to come up with their own ad-hoc report.

Edited for web by Jordan Platt.