Create a JavaScript Scrolling and Sweeping Heatmap

Hello again!

In this article, we will create two spectrogram charts with two different animations:

• Scrolling heatmap
• Sweeping heatmap

We can use the XY Charts from the "LC JS" library to create a Heat Map chart. The grid series can easily handle datasets in a million data points range even on low-end devices with large amounts of RAM; even billions of data points can be visualized.

A heatmap allows you to analyze the magnitude of an event using two or more colors to represent the event's intensity in a range of values.

Before building the spectrogram charts, let's set up the working environment.

Setting Up Our Template

1. Download the project's template to follow the tutorial (.ZIP)

2. You'll see a file tree like this one:

3. Now, open up a new terminal

4. As usual in a Node JS project, run the npm install command.

This would be everything for our initial setup. Let’s code.

CHART.ts

This file will work as our main file. Here, we will create an instance of some common properties that we can send to the charts as parameters.

1. Import the charts TS files:

import {createScrollingChart} from './ScrollingHeatMap';
import {createSweepingChart} from './SweepingHeatMap';

2. Declare the constant lcjs that will refer to our @arction/lcjs library.

3. Extract required classes from lcjs.

const {
    lightningChart,
    PalettedFill,
    LUT,
    ColorHSV,
    emptyLine,
    AxisScrollStrategies,
    AxisTickStrategies,
    LegendBoxBuilders,
    Themes,
    LinearGradientFill,
    ColorRGBA
  } = lcjs;

const { createSpectrumDataGenerator } = require("@arction/xydata");

4. Creating the LUT properties:

const lut = new lcjs.LUT({
    steps: [
      { value: 0, label: "0", color: lcjs.ColorHSV(0, 1, 0) },
      { value: 15, label: "15", color: lcjs.ColorHSV(270, 0.84, 0.2) },
      { value: 30, label: "30", color: lcjs.ColorHSV(289, 0.86, 0.35) },
      { value: 45, label: "45", color: lcjs.ColorHSV(324, 0.97, 0.56) },
      { value: 60, label: "60", color: lcjs.ColorHSV(1, 1, 1) },
      { value: 75, label: "75", color: lcjs.ColorHSV(44, 0.64, 1) },
    ],
    units: "dB",
    interpolate: true,
  });

  const paletteFill = new lcjs.PalettedFill({ lut, lookUpProperty: "value" });

Let's review the properties.

LUT (Look Up Table) refers to a style class for describing a table of colors with associated lookup values (numbers). Instances of LUT, like all LCJS style classes, are immutable, meaning that its setters do not modify the actual object but instead return a completely new modified object.

• LUT properties: 
  • Steps: List of color steps (color + number value pair).
  • Interpolate: true enables automatic linear interpolation between color steps.

To learn more about classes, please refer to the documentation.

5. Creating the chart objects.

const chartScrolling = lcjs.lightningChart()
    .ChartXY({
      theme: lcjs.Themes.darkGold,
    })
    .setTitle("Scrolling Heatmap Spectrogram");

const chartSweeping = lcjs.lightningChart()
    .ChartXY({
      theme: lcjs.Themes.darkGold,
    })
    .setTitle("Sweeping Heatmap Spectrogram");

• setTitle: Sets the title for the chart. This title will appear at the top of the chart.
• Theme: a collection of predetermined themes are available. The color theme of the charts must be specified upon creating them. It cannot be changed afterward without affecting and having to recreate the component. See more in the color themes documentation.

6. Creating the charts.

createSweepingChart(lcjs,createSpectrumDataGenerator,paletteFill,chartSweeping);
createScrollingChart(lcjs,createSpectrumDataGenerator,paletteFill,chartScrolling);

The two create functions; we will execute all the code inside the charts' TS files. As you can see, we are sending the properties that we created before as parameters that will be used to give format to the charts.

SweepingHeatMap.ts

// Length of single data sample.
const dataSampleSize = 1000;
// Length of visible sweeping history as columns count.
const sweepingHistory = 250;

chart
  .getDefaultAxisY()
  .setTitle("Frequency (Hz)")
  .setInterval(0, dataSampleSize, false, true);

// Create heatmap series.
const heatmapGridSeries = chart
  .addHeatmapGridSeries({
    columns: sweepingHistory,
    rows: dataSampleSize,
  })
  .setFillStyle(paletteFill)
  .setWireframeStyle(lcjs.emptyLine)
  .setMouseInteractions(false);

• dataSampleSize: length of the single data sample.
• sweepingHistory: Length of visible sweeping history as columns count.
• Chart:
  • getDefaultAxisY:  Gets the Y axis.
  • setInterval: Set axis scale interval.
•  Parameters:
  • start: number. Starts the scale value.
  • end: number. Ends the scale value.
  • animate: number | boolean | undefined. Boolean for animation enabled, or the number for animation duration in milliseconds
  • disableScrolling: boolean | undefined. If true, disables automatic scrolling after setting the interval.

1. HeatmapGridSeries.

const heatmapGridSeries = chart
  .addHeatmapGridSeries({
    columns: sweepingHistory,
    rows: dataSampleSize,
  })
  .setFillStyle(paletteFill)
  .setWireframeStyle(lcjs.emptyLine)
  .setMouseInteractions(false);

• addHeatMapGridSeries:

Add a Series for visualizing a Heatmap Grid with a static column and grid count. Has API for fast modification of cell values. The HeatmapGridSeries is performance-optimized for handling massive amounts of data.

Reference performance specifications on an average PC:

•  Heatmap Chart with 1 million data points (1000x1000) is cold started in ~0.3 seconds.
• Heatmap Chart with 1 million data points (1000x1000) is re-populated (change data set) in ~0.050 seconds.
• Heatmap Chart with 16 million data points (4000x4000) is cold started in ~2.0 seconds.
• Heatmap Chart with 16 million data points (4000x4000) is re-populated (change data set) in ~0.5 seconds.

setFillStyle: adding the LUT properties to fill the chart.

setWireFrameStyle: A Wireframe is a line grid that highlights the edges of each cell of the heatmap. See more about line grid styles.

2. Create Band for visualizing sweeping updates.

const band = chart.getDefaultAxisX().addBand(true)
  .setStrokeStyle(lcjs.emptyLine)
  .setFillStyle(new lcjs.LinearGradientFill({
    angle: 90,
    stops: [
      {offset: 0, color: lcjs.ColorRGBA(0, 0, 0, 255)},
      {offset: 1, color: lcjs.ColorRGBA(0, 0, 0, 0)}
    ]
  }))
  .setMouseInteractions(false)

3. Now, add a legend box to the chart.

// Add LegendBox to chart.
const legend = chart
  .addLegendBox(lcjs.LegendBoxBuilders.HorizontalLegendBox)
  // Dispose example UI elements automatically if they take too much space. This is to avoid bad UI on mobile / etc. devices.
  .setAutoDispose({
      type: 'max-width',
      maxWidth: 0.80,
  })
  .add(heatmapGridSeries);

A Legendbox is a type of UI element that floats inside the chart. Users can freely move the legend box with basic mouse interactions. Its default position can also be established within the application code.  

The purpose of a legend box is to describe the series and other visual components of the chart, by displaying their names and colors. Hovering over a series' legend box entry will highlight that series, and clicking on the entry will toggle that series' visibility. 

Read more in the LegendBox documentation.

The SetAutoDispose property will automatically dispose of example UI elements if they take up too much space within the canvas application. This is done in order to avoid bad UI on mobile devices or smaller screens.

4. createSpectrumDataGenerator.

// Stream in sweeping data.
let iSample = 0;
let dataAmount = 0;
createSpectrumDataGenerator()
  .setSampleSize(dataSampleSize)
  // NOTE: Number of unique samples in example data.
  .setNumberOfSamples(12340)
  .setVariation(15)
  .setFrequencyStability(0.7)
  .generate()
  .setStreamRepeat(true)
  .setStreamInterval(25)
  .setStreamBatchSize(1)
  .toStream()
  // Scale Intensity values from [0.0, 1.0] to [0.0, 80]
  .map((sample) => sample.map((intensity) => intensity * 80))
  .forEach((sample) => {
    // Produce sweeping update effect by pushing new samples in by invalidating previous intensity values with sweeping motion.
    heatmapGridSeries.invalidateIntensityValues({
      iColumn: iSample % sweepingHistory,
      iRow: 0,
      values: [sample],
    });

    band
      .setValueStart(iSample % sweepingHistory)
      .setValueEnd(band.getValueStart() + 10)

    dataAmount += sample.length;
    iSample += 1;
  });

InvalidateIntesityValues (invalidate intensity values) produces a sweeping update effect by pushing new samples in by invalidating previous intensity values with a sweeping motion.

Invalidate the range of heatmap intensity values starting from the first column and row, updating coloring if a Color look-up table (LUT) has been attached to the series.

 "This function creates a basic progressive random generator and uses the Stream API to send the data to the console.

Calling generate() on any data generator returns a new instance of 'DataHost'. The generate() function can be called multiple times to get a new dataset with the same settings as before but with different values each time." 

Source

ScrollingHeatMap.ts

The Scrolling Map has the same starting logic, but the main change will be located in the sample mapping:

// Stream in continous data.
let dataAmount = 0;
createSpectrumDataGenerator()
  .setSampleSize(dataSampleSize)
  .setNumberOfSamples(1000)
  .generate()
  .setStreamRepeat(true)
  .setStreamInterval(25)
  .setStreamBatchSize(1)
  .toStream()
  // Scale Intensity values from [0.0, 1.0] to [0.0, 80]
  .map((sample) => sample.map((intensity) => intensity * 80))
  // Push Intensity values to Surface Grid as Columns.
  .forEach((sample) => {
    pushSample(sample)
    dataAmount += sample.length;
  });

As we can see, the .forEach function is simpler than the sweeping mode. The new values are pushed as new columns in the grid to get continuous streaming.

The following logic ensures a static sampling rate, even if input data might vary. This is done by skipping too frequent samples and duplicating too far apart samples. The precision can be configured by simply changing the value of [sampleRateHz].

let lastSample
let tFirstSample = 0
const pushSample = (sample) => {
  const tNow = performance.now()
  if (lastSample === undefined) {
      heatmapSeries.addIntensityValues([sample])
      lastSample = { sample, time: tNow, i: 0 }
      tFirstSample = tNow
      return
  }

  let nextSampleIndex = lastSample.i + 1
  let nextSampleTimeExact = tFirstSample + nextSampleIndex * sampleIntervalMs
  let nextSampleTimeRangeMin = nextSampleTimeExact - sampleIntervalMs / 2
  let nextSampleTimeRangeMax = nextSampleTimeExact + sampleIntervalMs / 2

First Condition: Too frequent samples must be scrapped. If this results in visual problems, then the sample rate must be increased.

 Second Condition: At least 1 sample was skipped. In this case, the missing sample slots are filled with the values of the last sample. 

if (tNow < nextSampleTimeRangeMin) {
      // Too frequent samples must be scrapped. If this results in visual problems then sample rate must be increased.
      // console.warn(`Skipped too frequent sample`)
      return
  }
  if (tNow > nextSampleTimeRangeMax) {
      // At least 1 sample was skipped. In this case, the missing sample slots are filled with the values of the last sample.
      let repeatedSamplesCount = 0
      do {
          heatmapSeries.addIntensityValues([lastSample.sample])
          repeatedSamplesCount += 1
          nextSampleIndex += 1
          nextSampleTimeExact = tFirstSample + nextSampleIndex * sampleIntervalMs
          nextSampleTimeRangeMin = nextSampleTimeExact - sampleIntervalMs / 2
          nextSampleTimeRangeMax = nextSampleTimeExact + sampleIntervalMs / 2
      } while (tNow > nextSampleTimeRangeMax)

      heatmapSeries.addIntensityValues([sample])
      lastSample = { sample, time: tNow, i: nextSampleIndex }

addIntensityValues: refers to the sample arrived within the acceptable, expected time range.

To display incoming points amount within the Chart title:

const title = chart.getTitle();
let tStart = Date.now();
let lastReset = Date.now();
const updateChartTitle = () => {
  // Calculate amount of incoming points / second.
  if (dataAmount > 0 && Date.now() - tStart > 0) {
    const pps = (1000 * dataAmount) / (Date.now() - tStart);
    chart.setTitle(`${title} (${Math.round(pps)} data points / s)`);
  }
  // Reset pps counter every once in a while in case page is frozen, etc.
  if (Date.now() - lastReset >= 5000) {
    tStart = lastReset = Date.now();
    dataAmount = 0;
  }
};
setInterval(updateChartTitle, 1000);

In the first condition, we calculate the number of incoming points/second. In the second condition, we Reset the counter every once in a while in case the page is frozen, etc.

NPM Start

Finally, to visualize the project, run the npm start command in the terminal. You'll receive the URL path to the local host (http://localhost:8080/).

Here's the final project:

Final Words

Scrolling and sweeping heatmaps are frequently used in vibration analysis to help detect errors in industrial equipment and machinery, e.g., air compressors. It is also possible to create a 2D and 3D heatmap and spectrogram application using NodeJS, TypeScript, and LightningChart JS.

These types of charts assist in monitoring machines' performances and are extremely useful for corrective and preventive maintenance.

Thanks for reading, and see you in the next article!

Need help with your code?

Contact me on LinkedIn, and I'll be happy to help.