When is the R² Not the R²?

A recently received support email contains a riddle.

“When I build a regression model in Excel and remove the intercept, I get a different R² than when I do the same thing in MetrixND. Why does this happen?”

To be clear, the estimated regression model for both Excel and MetrixND excludes the intercept term as shown below.

So, why does Excel and MetrixND produce different R² values?

MetrixND Regressions and R² Calculation

In MetrixND, R² is calculated as shown below.

This is the standard definition for R² and is constructed as one minus the ratio of the sum of squared errors (SSE) divided by the total sum of squares (TSS).

The mathematical equivalent can be written as the model sum of squares (MSS) divided by the total sum of squares (TSS) as shown below. This is becomes important at the end (i.e. spoiler alert!).

Since I’m paranoid, I double check the equations by building a regression model and manually validating the MetrixND equations in Excel. Check. The manual equations confirm that MetrixND is calculating the R² according to the standard definition.

Excel Regressions and R² Calculation

If MetrixND is correct, then what’s wrong with Excel?

First, I need to run the Excel regression module. If you’ve never done it, you can do it with the following steps.

Step 1: Select Data, Data Analysis and pick the Regression option. This opens the Regression dialogue box.

Step 2: Fill out the dialogue box by identifying the X and Y data ranges and select your options. For my test, I select the check box for “Constant is Zero” to remove the intercept from the regression equation, highlighted in red:

Step 3: Select “OK” and I can see the answer.

Following this process, I compare the Excel and MetrixND R2 values, and indeed, I get different numbers. So what’s wrong?

I wade through the morass of internet posts and find a Microsoft confession: “… if you click the Set Intercept = 0 box, the R-squared value is always incorrect in any version of Excel…”

There it is. Microsoft says they are incorrect.

What is Excel Doing?

After a bit more digging, I am happy to confirm that Excel calculates R² using the following formula only when the intercept is removed (please note that Excel uses the standard definition which matches the MetrixND results when the intercept is included in the regression model).

When you consider this calculation, you can see that this version of R² is much like the mathematical equivalent version (MSS/TSS) I showed above. Here it is again for your reference:

If y ̅ = 0, then the equations are the same. Thus, when the intercept is removed, Excel changes the formula and re-centers the R² calculation around 0 not y ̅. In Microsoft’s words, this is “incorrect”.

When is the R² Not the R²?

When the R² formula is incorrect. If you find the word “incorrect” overly harsh, perhaps the word “different” is better. After all, it’s just a formula.

Want to learn more? Attend an Itron Forecasting Workshop!

The Smart Home Study

The vision of a smart grid is based on distributed energy resources dynamically interacting to create smart homes and smart loads that automatically respond to price signals to effectively balance energy supply and demand. Read more on Itron’s work with the Smart Home Consortium and the California Energy Commission.

Welcome to the New Itron.com

Today, we unveiled our new global website, which serves as
the front door to our company and allows visitors like you to quickly
understand who we are, what we do and what value we provide. With the new itron.com,
users will experience a new, lighter and brighter website with a more engaging
and modern interface.

The new site features updated company and high-level product
messaging and incorporates the product and solutions pages from the Silver
Spring Networks website.

Other new pages and key features of the site include:

  • Why Itron: Because we can’t assume you
    know who we are and our full breadth of product offerings, the Why Itron page highlights
    what’s unique about us.
  • Customers: Our customers are at the
    center of what we do. The customers page features key results and successes our
    customers have achieved using our solutions.
  • Solutions & Products: We updated our solution
    pages to make it easier for you to find what you’re looking for. We also
    enhanced and updated our product catalog, which features hundreds of products
    available in multiple regions and languages.
  • Use-case Portfolio: The use case
    portfolio provides a new lens into solutions by specific need, such as leak
    detection or demand response.
  • Resources: Itron has a rich bank of
    content, including customer case studies, white papers and videos.  On the resources page, you can more easily
    find key collateral, webinars and information in a central way.

If you have any suggestions or questions about the new site,
feel free to contact webinquiry@itron.com.

Itron Goes Arctic

It’s summer, and many of Itron’s employees are taking summer vacations to sunny destinations, but one employee will spend the next 12 days on a journey of a lifetime in the freezing temperatures of the Arctic. Matt Smith, director of product management at Itron, was selected by ClimateForce 2041 and Itron to head off on an epic adventure as part of the ClimateForce ‘Leadership on the Edge’ Arctic 2019 expedition.

Itron’s corporate
social responsibility
team sponsored this opportunity in collaboration with
the 2041 Foundation, whose mission
is to inspire, stimulate and connect organizations and communicates who seek to
protect our world and create a more sustainable, clean energy future. Founded
by polar explorer, Robert Swan, 2041 hosts ClimateForce expeditions to convene
leaders who drive corporate sustainability innovations.

Matt was drawn to the ClimateForce Arctic 2019
Expedition through his personal and professional passion to create sustainable
businesses that drive meaningful change on a global scale. He is honored to be
chosen and to see first-hand the dramatic changes happening in the Arctic. He
will use these learnings to inspire and build solutions that improve the
management of resources as well as helping utilities mitigate the impact from
the changes that have already set in motion.

As a ClimateForce ambassador, Matt looks forward to working with Itron employees, industry partners and customers to create a more resourceful world. To kick off the trek, Matt and other expedition attendees will meet in Oslo, Norway for three days of lectures, presentations, gear checks and an introduction to the Arctic. The crew will then set off to Svalbard and the beginning of their expedition.

At Itron, we aspire to empower our employees to
create a more resourceful world through the work we do every day. Together,
with ClimateForce 2041, we are thrilled to have one of our leaders on this
journey and look forward to sharing his experiences and learnings along the
way. You can join Matt by reviewing the full itinerary here and keep up with his adventure by following
Itron on Twitter and Instagram.

Rooftop Solar PV and Smart Streetlights Survey

Itron recently surveyed the load forecasting community concerning the possible impact of rooftop solar PV installations on utility operations. There were 88 respondents that spanned a broad range of small to large utilities and system operators.

Approximately 86% of the respondents indicated that behind-the-meter (BTM) solar generation impacted their organization. We also inquired about what other areas were impacted and posed additional questions regarding load forecasting and street lighting.

For more detail, read our short summary here.

IoT for Impact: How Smart Devices are Preparing Communities to Respond to Disasters and Save Lives

Smart metering networks can be leveraged to support a host of complimentary Industrial IoT applications to improve the performance, reliability and resilience of other critical infrastructure systems. On an average day, meters act as IoT sensors to provide utilities with high-resolution data from the edge of their network to precisely monitor energy usage, but when disaster strikes, they could truly save lives.

Increasingly, utilities are deploying multiple types of sensors on their networks such as methane detectors, pole tilt sensors, power line sensors and air quality devices. The data these sensors generate enables utilities to predict imminent danger, respond and recover quicker and more precisely, or, in some cases, even predict and prevent dangerous conditions all together. For example, by using methane detectors, utilities can sense the buildup of the dangerous gas at levels below the explosive limit, identify the location of potential risk of a gas explosion, turn off the gas and alert first responders to clear the area—all before an explosion occurs. When wildfires threaten regional air quality, utilities and communities can use distributed sensors to predict which areas will be most impacted and alert nearby populations.

What about less predictable disasters? Experts predict that a major earthquake is due to hit the state of California, but without knowledge of when or what its impact will be. IoT sensors such as pole tilt, electric distribution line, leak detection, air quality, and methane sensors among others enable a more resilient electric grid and can be leveraged to mitigate risk, minimize damage, and respond more effectively to save lives and speed recovery in the aftermath.

In a recent demonstration at IoT World created with our partner, RWI Synthetics, Itron showed the impact of an earthquake in Santa Clara, California. We were able to demonstrate how sensors working in tandem with predictive modeling were able to detect and impact outcomes during and after the quake, such as:

  • Detecting leaks and remotely shutting off gas and water lines;
  • Detecting if a pole is down to prevent safety risks and promote quicker recovery efforts;
  • Push out alerts to communicate quickly with citizens, providing the most up-to-date information to ensure their safety; and
  • Utilize intelligent evacuation planning to best direct traffic and get people out of harm’s way quickly.

Beyond how infrastructure might react, we were also able to demonstrate how people might react to disasters—and the still standing or crumbling infrastructure around them—through the creation of synthetic environments like those created by RWI Synthetics. RWI combines data analytics with extensive behavioral research to create details down to the stress levels and biometric responses of each synthetic individual.

By designing and trialing critical infrastructure systems, cities and utilities can use this information to help prepare for the unpredictable, improve system resilience and mitigate risk—while taking special consideration for citizen well-being. Doing so can effectively save countless lives and billions of dollars in avoided infrastructure repairs following a disaster.

Though these advances make a major impact on public safety, network resiliency and critical infrastructure for cities, big ideas and innovation are still needed to continue rapid improvement—and we at Itron are always looking to engage new great ideas and applications. To learn more, click here.

Learn more about our partner network here.

Accepting Nominations: 2019 Excellence in Resourcefulness Awards

In partnership with Frost & Sullivan, Itron is now accepting nominations for the 2019 Excellence in Resourcefulness Awards. Presented on Oct. 11 at Itron Utility Week in Marco Island, Florida, the awards recognize utilities who are committed to the sustainable use of energy and water.

The award categories include:

  • Excellence in Resourcefulness – Energy: This award recognizes either an Investor Owned Utility (IoU) or municipal’s ability to conduct successful technology implementation and behavioral change, producing results in reducing significant waste of electricity.
  • Excellence in Resourcefulness – Water: This award applauds a utility’s method for addressing resourcefulness through the implementation of behavioral change and technology implementation that results in significant reduction of water usage and water waste.

Frost & Sullivan will select a winner in each category after conducting in-depth research, interviews and evaluating utilities against industry best practices. Decision criteria for each category includes societal and business impact.

Considerations for societal impact include improving customer awareness and participation, creating changes to reduce waste through technology and consumer engagement and waste reduction outcomes that benefit the community. Business impact factors include a clear vision to reduce waste through technology implementation, achieving operational effectiveness as the result of a successful sustainability strategy and positioning a utility’s brand’s image as a leader in sustainability.

Nominations can be submitted here until July 29, 2019 at 9 p.m. PDT.

The Mile-High 2019 ISO Forecasting Summit

They have a saying in Denver – “If you don’t like the weather, wait five minutes.” While that is clearly an exaggeration, it’s not too far off from what the city’s residents and visitors might experience on any given day in the Mile-High City. And since it was the destination of this year’s 13th annual ISO/RTO/TSO Forecasting Summit, May 7-9, attendees from most of the North American and Australian ISO/RTO/TSO organizations got a taste of how volatile the weather can be sometimes. For example, at the start of the week, the weather was fairly mild as I saw people roaming the streets of downtown in t-shirts and shorts. However, by the end of the week, it was snowing with a bit of rain and cold wind.

While the mercurial weather taught us to expect the unexpected, the engaging presentations and discussions during the summit showed us how forecasters across this rapidly evolving industry are addressing the many pervasive challenges surrounding growing distributed generation (DG), granular load forecasting, long-term load forecasting, and weather and load forecast accuracy.

With increased DG from both renewable and non-renewable technologies, organizations are continually exploring methods of gaining visibility into DG in their service territories. Consequently, one of the big challenges has become determining a superior approach for integrating it into a load forecast. Many attendees shared their experiences using common practices such as load reconstitution and direct modeling and discussed the pros and cons of load forecasting at more granular levels.

In a world where the pressure to produce more accurate load forecasts is the highest it’s ever been, forecasters are perpetually looking to improve the performance of their statistical models. Attendees offered insight into the approaches and tools they’re using to improve forecast performance, and also demonstrated how load forecasters might be able to mitigate the impact of weather forecast error.

Finally, the load implications associated with the growing saturation of electric vehicle charging and cryptocurrency mining—among other items—was also a popular topic of discussion, as organizations look ahead to better understand the impact of these activities on system loads.

With so many of the brightest minds in the industry in one room, one could say the level of expertise was a “mile-high”. That’s certainly a trend that I’d like to see continue to grow.

Itron Collaborations Recognized at IoT World Awards

Last week at IoT World, Itron was shortlisted for its
nomination of Con Edison’s methane detectors for the Best Industrial IoT
Solution Award and for its
nomination of Utilidata’s grid-edge optimization application for the
Best Edge Computing Solution Award.

In the Best Industrial IoT Solution category, New York City’s
utility, Con
, was recognized for its deployment of 9,000 smart natural gas
detectors on Itron’s Industrial IoT network to proactively identify natural gas
leaks and improve safety. With these devices on its smart meter network, Con
Edison can detect natural gas leaks and quickly alert its emergency crews to
improve the safety of the communities they serve.

The battery-powered natural gas detectors were developed as
a collaborative effort between Itron, New Cosmos and Con Edison through Itron’s
developer program. The detectors are the first battery-powered solutions
deployed on Itron’s secure, standards-based industrial IoT platform. The
devices are easily integrated on Con Edison’s existing network infrastructure
from Itron, which they already utilize to manage 3.6 million electric meters
and 1.2 million gas modules.

Itron’s collaboration with Utilidata,
a software company committed to driving resiliency, sustainability and
efficiency for utilities, was also recognized in the Best Edge Computing
category. Through the collaboration, Utilidata will offer its industry-leading
AdaptiVolt™ voltage optimization capabilities as an application on Itron
Openway Riva meters. This grid-edge optimization application enables utilities
to leverage the real-time information and distributed intelligence capabilities
of Itron meters to optimize the delivery of electric power and reduce energy

By transforming meters into intelligent edge devices,
utilities and software developers can create the next generation breakthrough smart
grid applications. This third-party application will be the first available and
demonstrates Itron’s commitment to developer enablement and to providing an
open platform for third-party innovation.

Itron is committed to bringing innovative IIoT solutions to
cities and utilities to transform existing processes and deliver greater
safety, innovation and resourcefulness. Congratulations to Con Edison and
Utilidata for their recognition at IoT World.

Itron Showcases the Potential of Intelligent Streetlights and Smart City Use Cases at LIGHTFAIR International

Cities around the world are beginning to implement smart city solutions to ensure greater safety, efficiency and convenience for their citizens. For many cities, smart street lighting is the first step to creating a smart city, with its proven business case and ability to create a network canopy under which additional applications can be deployed, such as smart parking, water leak detection, natural gas leak detection, gunshot detection and more. With a connected network, cities can modernize their infrastructure with connected lighting, while providing a platform to manage additional smart city devices as their needs evolve.

Recognized as a global leader in smart streetlight solutions, Itron is collaborating with many cities, including London, Paris, Melbourne and Chicago, who are benefitting from our technology today. In fact, the City of Chicago recently installed its 100,000th light, an important milestone in its overall plan to upgrade more than 270,000 lights.

Itron’s partner Ameresco, a leading independent provider of comprehensive energy efficiency and renewable energy solutions, is continuing to drive momentum in this ambitious initiative, which is currently the largest municipally led smart streetlight project in the United States. With these streetlights on a connected network, the city will improve safety on the roadways while significantly reducing energy usage. The project is currently expected to save the City of Chicago over $10 million annually.

Intelligent streetlights deployed at scale unleash the potential of smart city solutions for its citizens. By starting with smart lighting, cities can lock-in immediate ROI while laying a foundation that encourages continuous innovation and accommodates an array of applications and use cases, from intelligent transportation, resilient and sustainable infrastructure, public safety and security, and municipal digital services.

Be sure to visit Itron at LIGHTFAIR International in booth #3731 this week to learn more about the potential of smart streetlights.

Improving Financial Analysis with AMI Data

Smart meter data is more granular and more timely than monthly billing data. Access to this data supports a paradigm shift in forecasting processes, allowing analysts to develop more powerful methods and to implement new approaches.

In our next brown bag, we focus on the shift from monthly to daily modeling and show how this shift can improve clarity and visibility in forecasting and variance analysis processes. Join Stuart McMenamin in this second brown bag of the year on Tuesday, May 21 at noon PDT for “Improving Financial Analysis with AMI Data”.

You can register for this brown bag and other forecasting events at www.itron.com/forecastingworkshops.

Unlocking Action at the Edge: Announcing Itron’s New Developer Tools for Distributed Intelligence Applications

The frontier of smart grid innovation is continuing to accelerate with more devices and applications proliferating across the electricity distribution grid. At Itron, we believe that open platforms will provide the connective tissue that enables all of these systems to work together in concert to deliver reliable, efficient and cost-effective services. Our developer program provides technology venders with the tools they need to connect their device—like power line monitors and smart inverters—to our proven, globally deployed networks. These solutions are already empowering grid operators with unprecedented visibility and control. True transformation will occur when these systems can work together in real-time, leveraging live data streams to generate insight and take action.

Now, we are excited to announce the expansion of the Itron Developer Program to enable developers to build applications that run on Itron meters.

Our Distributed Intelligence Software Developer Kit allows developers to tap into the compute power and real-time data available on Itron OpenWay Riva™ meters. Developers can create applications that leverage meter data in real-time and combine this with critical information like grid location-awareness. In addition to making the meter available as an app platform, Itron will also be providing an app store that enables utilities to securely deploy and manage a portfolio of distributed apps created by both Itron and its developer partners.

This week at IoT World in Santa Clara, our first distributed intelligence (DI) partner, Utilidata, will be in our booth demoing their Volt-VAR optimization application, the first-ever third-party application for Itron meters. The interactive booth demo will highlight how signal processing, embedded in the meter, enables agile grid management and operations through real-time responses to precise conditions and variables. As booth visitors adjust meter voltage, they will see how the technology reacts instantaneously and optimizes energy delivery. Additionally, visitors to the booth will learn how seamlessly this technology integrates into existing infrastructure, maximizing grid investments and powering a new approach to smart grid deployments. The demo will simulate a reduction and increase in voltage supplied in order to optimize grid performance and reliability, meet energy efficiency goals, integrate Distributed Energy Resources (DERs) and save energy.

The next wave of innovation has arrived and the possibilities are endless!

So, what will you do with the power of distributed intelligence? We can’t wait to see. Our DI SDK will officially launch in June.

Come visit us at IoT World this week, booth #602, to learn more.

I agree to have my personal information transfered to AWeber ( more information )
Opt in to receive notifications when a blog post is published. Don't miss the thought leadership, insight and news from Itron.
We hate spam. Your email address will not be sold or shared with anyone else.