Visualising Solar Generation Data in a Custom Histogram using D3.js

Using the “brush” feature of the D3 Javascript library again proves handy for creating an interactive, animated histogram.  This type of visualisation helps to analyse and explore the distribution of time-series data.

For this demo, home solar PV generation data has been obtained from United Energy’s Energy Easy portal in CSV format.  For the sake of convenience in dealing with the raw data which usually comes in half-hourly intervals, this data has been loaded in to a Pentaho data warehouse instance (more details in a later post, perhaps!) and converted to day-by-day figures.

Preparing Solar PV generation data with Saiku

Analysis can help explore whether solar panels are getting less efficient over time, or even determine what a “good” day of production is like in summer vs winter (by looking at the relevant frequency of each in the histogram).

Features:

• Drag and scroll date region which affects histogram above:
  
• Changeable histogram buckets:
  
  
• Snap-shotting of one selected date range for visual comparison with another (e.g. summer vs winter comparison):
  

Key techniques – how it’s achieved:

1. Data is embedded into the HTML page:
   

   <pre id=”csvdata”>
   Day,Quantity
   2013-01-01,9.288
   2013-01-02,10.494
   2013-01-03,7.376
   …
   2014-12-29,3.832
   2014-12-30,7.752
   2014-12-31,7.212
   </pre>

   It’s hidden visually using the CSS display:none directive…

   #csvdata {
   display: none;
   }

   …and then read into a Javascript variable for display and computation via D3.

   var raw = d3.select(“#csvdata”).text();
   var dataset_copy = d3.csv.parse(raw);

2. Maximum value is found in the selected region of the dataset:
   

   max = d3.max(dataset, function (d) {    return +d.Quantity;});

   …histogram is created based on maximum value:

   // Generate a histogram using uniformly-spaced bins.
   datasetHist = d3.layout.histogram()
   .frequency(false)

   .range([0, max])
   .bins(buckets)
   (dataset.map(function (d) {

   return d.Quantity;
   }));

3. An HTML form input slider is used for selection of number of buckets:
   

       240px; text-align: right; font-size: 10px;font-family: sans-serif;”>Number of buckets: 10

   </label>
   nBuckets”>

   Value of slider is passed into Javascript (to variable “buckets”) via this code:

   d3.select(“#nBuckets”).on(“input”, function () {
   buckets = +this.value;
   d3.select(“#nBuckets-value”).text(+this.value);
   clearSnapshot();
   render();
   });

Demo / code:

Try out the demo and check out the code here:
http://jsfiddle.net/hzmj24d1/

Advertisement

Visualising energy consumption profile (by hour of day) using D3.js

With the benefit of smart electricity meters it’s possible to obtain hourly data showing household consumption in KWh. I downloaded this dataset for my own house in CSV format from United Energy’s EnergyEasy portal.

With some massaging, the data can be formatted to a structure which which makes aggregation easier.  The excellent tool OpenRefine made this task easier, effectively unpivoting half-hourly measures which were in many columns into a single column, so that the data looks like this:

Day,Interval,Quantity
2012-01-01,0000,0.05
2012-01-01,0030,0.05
2012-01-01,0100,0.044
2012-01-01,0130,0.05
2012-01-01,0200,0.044
[...]
2013-12-31,2130,0.025
2013-12-31,2200,0.019
2013-12-31,2230,0.025
2013-12-31,2300,0.025
2013-12-31,2330,0.025

Using the D3 Javascript visualisation library, it’s possible to create an interactive visualisation which can interrogate this data in arbitrary ways to find patterns and answer basic questions about household energy consumption.  For example:

1. During which hours of the day is the highest average energy consumption? Is this different in summer vs winter?  Has this changed from 2012 to 2013?
2. Has the minimum energy consumption overnight changed?  Is the new (and slightly annoying) energy saving power board purchased in mid 2013 doing its job to reduce standby power use?
3. During which hours of the day is power usage the most variable?

Features

• Selectable date range – e.g. to compare a rolling 12 month period. This uses a “context” graphics section in D3.js with brush functionality to trigger realtime recalculation of data in the “focus” section when a user selects a range using their mouse.  The live update of the hourly consumption profile means it’s easy to see trends over time in the “focus” area of the screen (shown in the following point):

  

  D3 selectable time range using “brush” technique

• Plotting of max / min / mean / standard deviation of KWh consumption per hour of the day:

  

  D3 Mean Max Min and Standard Deviation calculations for each half-hourly time interval of the day

• “Snapshotting” of date range – e.g. to compare two consecutive years in an interactive way:

  

  D3 snapshot time comparison

Demo

Check out a live example here:
http://jsfiddle.net/ugxo00bu/7/