Creating a custom chart

Body

I often get asked how to make some change to a chart in DASH when there isn't anything in our UI to get it done. Fortunately DASH's charting widget was designed to be extensible, and can be teased into doing almost anything you want with a little work. In this article I'll show you how to customize a chart by creating a heatmap chart with custom coloring. We'll learn a couple valuable techniques: how to create a new type of chart not currently available out of the box, and how to modify a chart's default rendering behavior. In the process, we'll touch on using DASH's MyBox feature for introducing new content into your system.

How charts are put together

Before we talk about creating a custom chart, we have to understand how the Data Visualization widget in DASH works. DASH charts are the result of combining a number of resources.

• The template spec: Under the covers, the Data Visualization widget leverages IBM's Rapidly Adaptive Visualization Engine (RAVE). RAVE visualizations are described in the VisJSON language, and each DASH chart starts with a partial implementation - a chart specification that describes the basic chart, but still needs some missing bits filled in to be complete.
• A file that describes changes to the chart depending on the current DASH theme.
• A file defining the supported color palettes.
• The chart's preferences, as set by the user when the chart was defined (the chart's title, the dataset, how the data columns are mapped to fields in the chart, legend placement, etc.).
• The data returned by the query to the dataset.
• A javascript module we call the merger that combines all this into a single complete VisJSON specification

A customized chart can be created by two providing a custom template spec, a custom merger, or both.

Let's see how this is done through an example. We will create a heatmap, a chart DASH does not ship out of the box, and customize the way its colors are allocated.

Create the custom heatmap chart

We create a customized chart like any custom widget, starting with DASH's Widget Wizard.

1. Open the Console Settings » Widgets page
2. Click on the  'New' icon. This opens the Widget Wizard and displays its Welcome panel.
3. Click  Next. This opens the Create/Base widget panel of the wizard.
4. Select the Data Visualization widget as the base widget.
5. Click Next. This opens the Create/General panel of the wizard.
6. Give the widget a name, like Colorful Heatmap (this is how it is labeled in the widget palette when building a new dashboard).
7. Give the widget the description such as A heatmap with custom coloring (this is the text in the popup description displayed when hovering over the widget in the palette).
8. Move the Dashboard Widgets catalog from Available Catalogs to Selected Catalogs to put our chart into that catalog. (If you want to, you could open the Console Settings » Catalogs page and create a new catalog for your custom widgets. For now, this will do.)
9. Click Next. This opens the Create/Security panel of the wizard.
10. Expand the Editor section of Selected Roles, then move "all authenticated portal users" into that role. This will let anyone that can edit dashboards also edit this new charting widget.
11. Click Next. This opens the Customize Page. Now we can define the new chart.
12. The Visualization Specification URL is the location of our custom VisJSON specification template. If you study up on the VisJSON specification language, you can use it to create and add completely new chart types to DASH. (I have to warn you though, it's a bit of a black art, and there are additional constraints necessary to make it work within DASH. That will be the topic of another article.) For now, it's best if you stick to customizing existing charts, and it just so happens that DASH ships with an experimental heatmap chart that we can use here. 
    Set the Visualization Specification URL to /DASHRaveWidget/specs/heatmap/spec.json.
13. Uncheck the This specification is complete checkbox. Leave this checked only when the specification URL points to an endpoint that responds with a fully formed VisJSON chart, complete with the data. In this case, we want to configure instances of the heatmap to render to user-specified data sources, so that's not the case.
14. The Custom merger URL is the location of our custom merger code. First we need to put it where DASH can serve it up. To that end, DASH has a feature called MyBox - a directory where you can put files you'd like to add to DASH.
    1. On your server, cd to <your profile>/installedApps/<your cell>/isc.ear/myBox.war
    2. Create a new directory, custom
    3. Put the attached to heatmapmerger.js(View Details) into this new directory.
15. Click Next. This opens the Summary pane of the wizard.
16. Click finish

Create a dashboard and put this new heatmap on it

1. Click the button to create a new page
2. Give the page a name, like Heatmap Test, choose your preferred page layout, and click OK.
3. In the dashboard builder's widget palette, click on the Dashboard Widgets catalog to open it.
4. Find our Colorful Heatmap widget, and drop it onto the page (Pro tip: you can use the arrow, page up/page down keys to move around in the palette, and Enter to put a copy on the page.)
5. Edit the chart.
   1. Select a dataset suitable for a heatmap. This will be one that can be used to create an XY grid, where each cell is given a color based on some value.  If you have the sample file provider enabled¹, enter CPU in textbox and click Search. Select CPU Utilization as your dataset.  This is the dataset we'll use for the rest of this article.
   2. In the Required Settings section, put Time on the X axis, SystemName on the Y axis, and use CPU as the value field.
   3. Click OK to save these settings and view our chart

If all went according to plan, you should have a chart that looks like this

(Don't worry that Time isn't in order. In this dataset the values are strings, not real date-time values. Real date-time data is ordered correctly.)

How did this work?

The template spec file defined the basic heatmap. For color mapping, the default merger will take the selected color palette and evenly distribute the colors from 0% to 100% of the data values. This means that if you have CPU percentage, like in our sample dataset, but the data ranges only from 50-55, then that is what gets mapped from 0%-100% for color. The default heatmap, using the Blue-Red Diverging color palette, winds up looking like this:

Not very appealing. Let's look at the heatmapmerger.js code that creates a better color mapping.

1. define(["dojo/_base/declare", "dojo/Deferred", "dash/rave/merger"], function(declare, Deferred, Merger)
2. {
3.     return declare(Merger, {
4.         constructor: function() {},
5.         merge: function(params, template_spec, data_section, grammar_section)
6.         {
7.             if(!params.data_options)    // working around a bug
8.                 params.data_options = {};
9.                 
10.             var p0 = this.inherited(arguments);
11.             return p0.then(function(spec) {
12.                 var p = new Deferred();
13.                 try {
14.                     element_color0 = spec.grammar[grammar_section].elements[0].color[0];
15.                     if("mapping" in element_color0) {                 
16.                         element_color0.mapping = [
17.                             {at: 0, color: "green"},
18.                             {at: 30, color: "green"},    // 0-30 is green
19.                             {at: 30.00001, color: "yellow"},
20.                             {at: 50, color: "yellow"},   // 30-50 is yellow
21.                             {at: 75, color: "red"},      // 50-75 is a linear gradient from yellow to red
22.                             {at: 100, color: "red"}      // anything over 75 is red
23.                         ];                      
24.                     }
25.                 }
26.                 catch(ex) {
27.                     console.error("Error modifying color mapping");
28.                     console.error(ex);
29.                 }
30.                 p.resolve(spec);            
31.                 return p.promise;
32.             });
33.         }
34.     });
35. });

The work is done in the merge function.

On line 10 we call the default merger's merge do handle the bulk of the work. Though the merge itself (at this time) is synchronous, it is part of an asynchronous process. This is why it returns a Promise. When the original Promise is fulfilled, we execute our code starting at line 12.

The color mapping at line 16 is defined just like you would define a linear gradient.  While the VisJSON spec says that you can place 2 color stops at the same value to effect a step change in color, there's a bug in RAVE that requires us to separate them by a tiny amount. Hence the values at 30 and 30.00001. The entries for 0 and 100 are unnecessary, as RAVE will color anything below the smallest value its color, and anything above the largest value its color, but I like adding them for symmetry.

At line 30 we resolve our promise, which will kick off the rest of the processing that gets the chart rendered on the page.

At line 31 we return the promise, so other processing may one day chain off this work.

Debugging

It may take some experimentation to get your modification of the VisJSON spec the way you want it. There are a couple ways to do that.

In your browser's debugger, find the div with an id that looks something like like ns__644481511__visWidget_vw. It's the one with _vw you need.  Then in the console, try:
window.sss=dijit.byId('ns__644481511__visWidget_vw').getSpecification()

Now you can inspect and fiddle with sss to your heart's content.
dijit.byId('ns__644481511__visWidget_vw').setSpecification(sss)

will push your changes back into the widget.

Another technique is to get the spec like I did here, convert it to JSON

JSON.stringify(sss)

then copy and paste it into the spec panel you get when clicking on any of the RAVE sample visualizations.

JSONLint is a useful tool for finding errors in your JSON.

Summary

Typically this is how a custom chart is created. Start with the spec of an existing chart and use a custom merger to manipulate the completed spec to do something special for you use-case. It's seems a bit long-winded, but the process is relatively straight forward.

You can find more information on using DASH's MyBox feature here, and more on RAVE here.

1. In <Jazz Home>/datasets is the file sample.properties. If it has the single line VISIBLE=true, then any csv file in that directory can be served as a dataset to DASH.