Automated Testing Of Tableau Dashboard Using Kinesis CI

Launched in 2016, Kinesis CI is new software to perform automated Testing on Tableau. The architecture consists of two components that communicate with Tableau and external tools.

Kinesis Designer
The Designer component of Kinesis CI works at the User Interface level of the software. For running development tests, there are some protocols and standards to follow. These criteria are commonly called Test properties. Before any code is uploaded into the source code repository, it must follow the test properties. Otherwise, errors would arise.

To help the development team run appropriate tests, Kinesis Designer generates a JSON file and embeds test properties and necessary information. The file is loaded into Command Line for further processing.

Command Line Interface (CLI)
Test Runner Command Line Interface receives the test properties embedded in a JSON File and opens it to run test cases for Tableau.

You must access the Tableau Dashboard to create snapshots for running test cases. A web driver extracts the metadata from Tableau Dashboard to run test cases.

The command line interface has two main components – Main Testing Components and Backend Testing Components.

The main testing component runs the following tests,

• • Functional Test
  • Regression Test
  • Cross Environment Test
  • Performance Tesr

Backend components

• JSON Test Plan Parser
• Web Canvas
• Web driver
• Text executor and report generator

Functional Testing
Testing Tableau’s functionality is crucial to determining the experience of the Business Intelligence team. The user experience includes user clicks, navigations, interactions, filters, switching tabs, and many parameters. If the business analyst teams spend most of their time analyzing data, you can think of running a functionality test.

The functional tests are designed to conduct each test one after the other.

For pre-existing data, you can test it against an expected data set or extract the real-time data displayed in Tableau for comparison.

Functionality tests contain a series of tasks to record user navigation on Tableau’s dashboard.

• The user logs in to Tableau and opens a data visualization to publish.
• Refreshes the data
• Loads the data from the CSV file and extracts it.
• Loads data to database table from CSV file.
• Sets a parameter and compares all the data attributes to a predefined set of values.
• Runs a command.

Regression Testing
Regression Testing lets you take the baseline at a point T in time. After a certain interval of T+t, compare the Tableau Dashboard to the baseline for measuring regression over time.

The baseline contains

• Metadata
• Worksheets
• Data
• Layouts
• Filters
• Parameters

These values are directly compared with the current values of the Tableau Server.

• Tableau passes the test if the baseline values are equal to current values.
• If the baseline values are not equal to current values, Tableau fails the test.

The failure of tests requires you to re-upload the baseline for running successful regression tests. Re-uploading of baseline follows an algorithm that repeatedly resolves errors while running weekly and daily tests.

Cross-Environment Testing
Cross Environment Testing lets you perform tests over two Tableau Dashboards, just as the name suggests. One dashboard could be the developer-side dashboard, and the other is the already deployed one.

This type of testing is carried on during the procedures of server upgradation. The old server runs parallel with the new server for direct cross-comparison between the dashboards. Vizzes are set to compare the snapshots of both environments. Developers can select the attributes for comparison, like worksheets, layouts, filters, etc.

If the source and deployed dashboards match, Tableau passes the cross-environment test.

If the source and deployed dashboards do not match, Tableau fails the cross-environment test. After failing, the algorithm runs to resolve the difference and proceeds with the test again.

Performance Testing
Performance Level testing is the highest logic form of software testing to identify Tableau’s downtime and errors introduced by modifications upon performing data analysis.

Kinesis CI runs Performance level tests based on Service Level Agreements (SLAs) and Apex Scores. It directly relies on audience engagement and assesses Tableau’s response times for multiple concurrent teams (could be hundreds of users) working on dashboards.

An Apdex Score is calculated to measure the performance of Tableau. This score incorporates Tableau’s availability and several performance metrics. The Apdex Score ranges between low-level (0) and high-level (1) logic.

Errors and latency in response reduce the Apdex Score, while fast loading times and fewer errors in accordance with service level agreements increase the score of Tableau Server.

Connection to Tableau Server
Kinesis CI must communicate with Tableau Server to write the code and import the dashboards. It is done by

• REST API
• JavaScript API
• tabcmd
• Web Instructions