Automated Testing Of Tableau Dashboard Using Kinesis CI

Vikas Agarwal

Business Intelligence, Tableau

Using Tableau to perform data analysis lets you make strategic business decisions to generate better results. Tableau has proven to be a market standard for Business Intelligence Software by visualization, representation, and analysis of large quantities of data. In large organizations, continuously updating and refreshing Tableau Dashboards is a day-to-day task to ensure correct data reporting by performing necessary modifications. Tableau generates reports that might need some changes due to

Unrelated functionality.
Lack of insightful information.
Reporting errors.

Hence, it requires testing to align reports per the organizations’ requirements.

Why test?

Testing measures the “Correctness of reporting” of business intelligence software such as Tableau. It is important to access the functionality of Tableau when the software analyses a large volume of data, as errors are quite predictable in the results.

There are two ways of testing the reports- Manual and Automated.

Manual testing is done by the business intelligence or data analyst team. Suppose you compare manual input with automated systems. In that case, automatic testing is a clear win-win situation, as manual tasks require manpower, time, and effort and increase the cost by stretching the data analysis cycle.

Hence, choosing an appropriate software to deploy the Dashboard of Tableau is the right protocol.
The automated testing compares the obtained values with a set of predetermined figures and references. Such testing is performed by a Continuous Integration Server rather than manual testing, which might consume your time.

Which software should you choose?

The top-notch software for deploying the dashboards of Tableau is software known as Kinesis.

This software works on the procedures of developing automated testing solutions and tools to check reports of Tableau.

Choosing Kinesis Continuous Integration to automate testing Tableau Dashboards save time, speeds up reporting processes, and reduces manual costs. It speeds up the operation of Business Analysts and developers by reducing their workload.

Why a continuous integration environment?

A continuous integration environment is the quickest way to obtain desired modifications in the reports. All the Developers place their code in the ashared source code repository several times a day. The system continuously checks for errors based on automated build and identifies necessary modifications and errors in the code before releasing the app. If errors are located, the developer rechecks, fixes, and updates the code to resolve the error. Once all the errors are fixed, the code is deployed in the application successfully. To run four-level tests of Kinesis CI, you must access continuous integration servers such as Jenkins and TeamCity. You can choose the CI server and perform implementation with Kinesis and Tableau to run automated tests on the dashboard. Thus, Kinesis CI saves you time, reduces downtime of your system with quick solutions, and cuts down manual testing costs.

Kinesis Software Architecture

Launched in 2016, Kinesis CI is new software to perform automated Testing on Tableau. The architecture consists of two components that communicate with Tableau and external tools.

Kinesis Designer
The Designer component of Kinesis CI works at the User Interface level of the software. For running development tests, there are some protocols and standards to follow. These criteria are commonly called Test properties. Before any code is uploaded into the source code repository, it must follow the test properties. Otherwise, errors would arise.

To help the development team run appropriate tests, Kinesis Designer generates a JSON file and embeds test properties and necessary information. The file is loaded into Command Line for further processing.

Command Line Interface (CLI)
Test Runner Command Line Interface receives the test properties embedded in a JSON File and opens it to run test cases for Tableau.

You must access the Tableau Dashboard to create snapshots for running test cases. A web driver extracts the metadata from Tableau Dashboard to run test cases.

The command line interface has two main components – Main Testing Components and Backend Testing Components.

The main testing component runs the following tests,

- Functional Test
- Regression Test
- Cross Environment Test
- Performance Tesr

Backend components

JSON Test Plan Parser
Web Canvas
Web driver
Text executor and report generator

Functional Testing
Testing Tableau’s functionality is crucial to determining the experience of the Business Intelligence team. The user experience includes user clicks, navigations, interactions, filters, switching tabs, and many parameters. If the business analyst teams spend most of their time analyzing data, you can think of running a functionality test.

The functional tests are designed to conduct each test one after the other.

For pre-existing data, you can test it against an expected data set or extract the real-time data displayed in Tableau for comparison.

Functionality tests contain a series of tasks to record user navigation on Tableau’s dashboard.

The user logs in to Tableau and opens a data visualization to publish.
Refreshes the data
Loads the data from the CSV file and extracts it.
Loads data to database table from CSV file.
Sets a parameter and compares all the data attributes to a predefined set of values.
Runs a command.

Regression Testing
Regression Testing lets you take the baseline at a point T in time. After a certain interval of T+t, compare the Tableau Dashboard to the baseline for measuring regression over time.

The baseline contains

Metadata
Worksheets
Data
Layouts
Filters
Parameters

These values are directly compared with the current values of the Tableau Server.

Tableau passes the test if the baseline values are equal to current values.
If the baseline values are not equal to current values, Tableau fails the test.

The failure of tests requires you to re-upload the baseline for running successful regression tests. Re-uploading of baseline follows an algorithm that repeatedly resolves errors while running weekly and daily tests.

Cross-Environment Testing
Cross Environment Testing lets you perform tests over two Tableau Dashboards, just as the name suggests. One dashboard could be the developer-side dashboard, and the other is the already deployed one.

This type of testing is carried on during the procedures of server upgradation. The old server runs parallel with the new server for direct cross-comparison between the dashboards. Vizzes are set to compare the snapshots of both environments. Developers can select the attributes for comparison, like worksheets, layouts, filters, etc.

If the source and deployed dashboards match, Tableau passes the cross-environment test.

If the source and deployed dashboards do not match, Tableau fails the cross-environment test. After failing, the algorithm runs to resolve the difference and proceeds with the test again.

Performance Testing
Performance Level testing is the highest logic form of software testing to identify Tableau’s downtime and errors introduced by modifications upon performing data analysis.

Kinesis CI runs Performance level tests based on Service Level Agreements (SLAs) and Apex Scores. It directly relies on audience engagement and assesses Tableau’s response times for multiple concurrent teams (could be hundreds of users) working on dashboards.

An Apdex Score is calculated to measure the performance of Tableau. This score incorporates Tableau’s availability and several performance metrics. The Apdex Score ranges between low-level (0) and high-level (1) logic.

Errors and latency in response reduce the Apdex Score, while fast loading times and fewer errors in accordance with service level agreements increase the score of Tableau Server.

Connection to Tableau Server
Kinesis CI must communicate with Tableau Server to write the code and import the dashboards. It is done by

REST API
JavaScript API
tabcmd
Web Instructions

Conclusion

With the ever-growing market size of software testing, it is very important to test Tableau with an automated system. Choosing a manual testing procedure consumes time and increases operating costs. But software like Kinesis CI directly imports Tableau Dashboards to support the existing functionalities. Kinesis CI offers compatibility with Enterprise solutions with custom login. It performs efficient automated Testing on Tableau Dashboards using various methodologies like Functionality, Regression, Cross-environment, and performance testing to extract errors and lags. Once errors are located, Kinesis uses Continuous Integration to resolve the issues on a daily basis, several times. These processes by Kinesis CI reduce Tableau’s inaccuracy, the overall testing cycle length, and your organization’s reporting costs.

Vikas Agarwal

Vikas Agarwal is the Founder of GrowExx, a Digital Product Development Company specializing in Product Engineering, Data Engineering, Business Intelligence, Web and Mobile Applications. His expertise lies in Technology Innovation, Product Management, Building & nurturing strong and self-managed high-performing Agile teams.