Skip to Content

Recovery Act Smart Grid Programs

Consumer Behavior Studies

The Department of Energy’s (DOE’s) Smart Grid Investment Grant (SGIG) program is working with several SGIG award recipients to examine the response of consumers to variable electricity prices, referred to as time-based rate programs in conjunction with the deployment of advanced metering infrastructure (AMI) and associate technology. The effort presents an opportunity to advance the electricity industry’s understanding of consumer behavior by addressing unanswered issues and questions with highly rigorous experimental methods. This section provides the rationale and approach for conducting these consumer behavior studies.

Quick links to consumer behavior studies topics:

Historical Context for Electricity Time-based Rate Programs

As far back as 1894, the electric industry has been debating the issue of how to efficiently and optimally charge customers for consuming electricity. At that time, there were emerging but very contentious discussions among economists about the merits of pricing the new commodity differentially based on time. The early uses of electricity were for lighting between dusk and dawn, which meant that the generation facilities used to power incandescent lamps (the dominant use of electricity at that time) were left largely idle during the daylight hours. As a means to more efficiently utilize these power plants, two different types of pricing schemes were heavily debated: one based on a customer’s consumption coincident with the system’s instantaneous maximum demand (i.e., system peak) and another based on a customer’s consumption coincident with different pre-determined time periods (e.g., nighttime between 6 p.m. and 6 a.m. vs. all other hours). The challenge with both rate designs revolved around metering – cost-effective technology did not exist at that time to allow electricity consumption to be captured at the required level of granularity. Thus, virtually all customers were charged for their electricity consumption at a rate that was time-invariant (i.e., flat).

By the 1970s, the debate had moved beyond issues of economic efficiency and instead turned towards more practical concerns about consumer behavior – could mass-market (i.e., residential and small commercial) customers manage their electricity under time-based rate programs. The Federal Energy Administration, the predecessor to the Department of Energy, sponsored several studies in the late 1970s and early 1980s to determine if and how residential customers would respond to time-of-use pricing (e.g., one price during weekday hours of 12 noon to 6 p.m. and another price for all other hours). The results of the studies indicated customers were, in fact, capable of managing their electricity consumption by moving it away from the expensive “peak” period to the less-expensive “off-peak” period. In spite of this evidence, the lack of low cost interval or period-based metering would continue to limit the industries’ ability to expand the application of time-based rate programs at the residential level through the end of the 20th century.

Discussion on Time-based Rate Programs and AMI

Over the past ten years, the cost of interval meters, the communications networks to connect the meter with the utility, and the back-office systems necessary to maintain and support them (i.e., advanced metering infrastructure or AMI) have all dramatically decreased. The implementation of AMI and interval meters by utilities, which allows electricity consumption information to be captured, stored, and utilized at a highly granular level (e.g., 15 to 60-minute intervals in most cases), provides an opportunity for utilities and public policymakers to more fully engage electricity customers in better managing their own usage.

Utilities can now collect customer electricity usage data at a level that allows them to offer time-based rate programs, which provides customers with opportunities to respond to diurnal and/or seasonal differences in the cost of producing power (i.e., time-of-use pricing) and/or dynamically to deteriorating power system conditions, high wholesale power costs, or both (i.e., critical peak pricing, real-time pricing). Under these new "dynamic pricing" schemes, rates can change from hour to hour and from day to day. Customers also have the ability with AMI to better understand their own overall daily and even hourly usage patterns, whereas before only monthly consumption information was conveyed. By introducing more dynamic rate structures and providing customers with more detailed information about their usage patterns, AMI provides customers with a strong incentive to invest in control technology in order to facilitate altering their consumption patterns with real and predictable impacts on their overall electricity bills. For example, programmable communicating thermostats (PCTs) allow customers to pre-program an adjustment in the thermostat temperature setting in direct response to receipt of dramatic electricity price increases via the utility’s AMI. Otherwise, customers would have to receive the event notification signal through some form of mass communication method (e.g., email, phone, pager) and then be home to adjust the thermostat.

In regulatory arenas across the country, utilities are laying out the rationale to policymakers and stakeholders for undertaking investments in AMI. Three core issues have been consistently raised in these AMI business case proceedings: cost recovery of the investment, benefits from utility operational savings, and benefits from the introduction of dynamic pricing. In the first case, stakeholders (e.g., public utility commissioners, consumer advocates) want to understand what the actual cost to fully implement the utility’s AMI plan will be, what are the risks to ratepayers and shareholder if there are cost overruns, how such costs will be recovered from ratepayers (e.g, $/kWh, $/customer, $/kW) and how the costs will subsequently be allocated to different customer classes (e.g., residential, commercial, industrial). Second, stakeholders want to better understand how this investment will reduce utility expenditures on operations and maintenance efforts over time (e.g., elimination of people to read meters, reduced truck rolls, etc.). Often these operational savings in isolation are insufficient to fully justify undertaking AMI investments, but instead must be coupled with the implementation of more dynamic pricing in utility ratemaking – the last source of benefits. The timing and magnitude of changes in electricity consumption patterns by customers in response to dynamic pricing has real and measurable financial effects on a utility’s cost of service. Reductions in coincident system peak consumption may result in future deferral of new investment in electric generating facilities and/or distribution system infrastructure upgrades. Overall shifts in electricity consumption away from expensive periods may reduce the average price of electricity for consumers.

Since the overall cost-effectiveness of many AMI business cases rest on the financial benefits derived from the introduction of dynamic pricing, it is crucial to properly and accurately estimate these benefits which hinge on assumptions concerning the number of customers who take service under such rate designs (enrollment) and the degree of change each customer exhibits in its consumption of electricity (performance). Over the past 20 years, several studies have assessed how customers would change their consumption patterns in response to dynamic electricity prices coupled with greater access to usage information via Web portals, in-home displays, etc. and/or technology to better automate the control of electricity-consuming devices (see Faruqui and Sergici 2010 for more information). Collectively, most past studies of consumer behavior relating to these factors have focused on issues of performance over acceptance. Many have been technology trials, “proof-of-concept” trials, test-runs to work out the kinks for full-implementation, and/or they have utilized a wide array of different experimental design approaches of various levels of sophistication and rigor. Unfortunately, the diversity of results has led many jurisdictions to question the applicability of the results, requiring utilities to run their own pilots, thereby delaying implementation of AMI, to see if comparable results are produced.

The Need for Consumer Behavior Studies

The Obama Administration has pursued a national policy to update and modernize the country’s electric infrastructure. The American Recovery and Reinvestment Act of 2009 set aside $3.4B to fund the SGIG program to accelerate the transformation of our electric transmission and distribution systems by promoting investments in smarter grid technologies, tools, and techniques for immediate commercial use. The Funding Opportunity Announcement (DE-FOA-0000058) explicitly identified an interest in AMI projects that involved dynamic pricing and use of randomized and controlled experimental designs. DOE has subsequently broadened the scope for such experiments in the hopes that these studies would investigate the power of AMI in seamlessly integrating pricing, automation technology, and information technology in order to facilitate a change in consumer behavior. This will further our understanding of the magnitude of this change caused by each component as well as by key drivers that motivate this change resulting in actionable utility and industry efforts to achieve them on a wide scale. Ideally, these studies will provide more definitive answers to policymakers and stakeholders in the area of acceptance of and response to both these separate and combined components of the Smart Grid (i.e., pricing, automation technology, information technology) in order to expedite the modernization of the grid in the near future.

In the area of pricing, for example, the utility industry knows a fair amount about customer opinions of time-based rate programs after customers have gained some experience with them but little about what motivates customers to accept these rate offerings in the first place. The investment in AMI can provide an opportunity for researchers to study how this acceptance may differ across market segments, with more comprehensive access to usage information to better assess the value proposition derived from taking service under time-based rate programs, and with access to automation technology that may make changes in electricity consumption behavior in response to such rate offerings easier to pursue. Research in this area should also attempt to better understand the diversity of response to time-based rate programs across different market segments (e.g., low income, elderly, high usage).

New studies also should also assess customer acceptance of and response to control or automation technology at the end-use level, such as indirect utility control of a customer’s refrigerator, dryer, or other major appliances. A number of recent studies have shown that this type of technology holds substantial promise in helping the bulk power system mitigate renewable resource integration issues, but more needs to be understood about what motivates customers to adopt them and how this adoption may differ across market segments.

These new studies also need to assess customer responses to various time-based rate programs and how the increased access to information about electricity consumption, both at the household and end-use levels, as well as across different delivery mechanisms (e.g., bill comparisons, Web portals, in-home displays) can affect customer behavior and electricity use patterns. Many prior studies have shown that access to information alone does indeed induce a chance in electricity consumption, but little is understood about the persistence of these effects or how that change in behavior differs if time-based rate programs are jointly implemented. New studies should attempt to address these key gaps in our knowledge base.

Consumer Behavior Studies
Advanced metering infrastructure (AMI) provides the opportunity to expand a utility's offering of time-based rate programs for electricity, potentially changing customer behavior through the interaction of pricing mechanisms, technology and information systems in the home. The investment in AMI can provide an opportunity for researchers to study how acceptance of and response to these elements may differ across market segments.

DOE’s Approach to Studying Consumer Behavior

To achieve highly precise and credible estimates of the change in consumer behavior associated with the research questions cited above, DOE is requiring those SGIG recipients undertaking such studies to apply highly rigorous randomized controlled experimental designs to their studies. In theory, evaluations employing random selection and random sampling will possess more credible and precise estimates of effects (i.e., internal validity) that can be extrapolated to similar groups outside of the study (i.e., external validity) as compared to studies that do not use random selection. Many prior studies in this area, however, have opted to utilize experimental designs that differed from those being pursued here; the increased costs and technical effort associated with these more rigorous methods was of greater concern than any potential sacrifice in precision and applicability of results from pursuing less rigorous experimental approaches.

To overcome this concern, DOE formed technical advisory groups (TAGs) to provide technical guidance and support for the SGIG recipients undertaking such studies. TAGs will help to promote the use of more rigorous experimental design methods, which may provide results that offer more value to utilities, their stakeholders, and their policymakers. Composed of electricity industry experts from academia and the consulting world with years of experience in designing, implementing, and evaluating such consumer behavior studies, the TAGs work collaboratively with each SGIG recipient who is undertaking a consumer behavior study. In addition to working with each recipient to align DOE’s methodological approach with the practical realities of regulatory environment in which each utility operates, the TAGs attempt to make sure the study addresses issues of direct interest to the recipient and in a broader sense DOE, in such a manner that results from the studies are actionable at both the local and industry-wide level. To accomplish this, the recipient is required to develop a framework, or a Consumer Behavior Study Plan (CBSP), that encompasses the appropriate application of sampling and sample design, identifies study objectives for rat. and non-rate treatments, discusses how these objectives will be evaluate. and finally lays out how the evaluation results will be documented in a written report to DOE. The TAG is there to provide guidance to the SGIG recipients in all of these areas in a very pragmatic and collaborative manner.

Status of Consumer Behavior Studies

TAGs are currently working with roughly 10 SGIG recipients with the goal of having recipients’ develop a CBSP that meets the high and exacting standards of DOE. The table below identifies those recipients who have met these standards and have a CBSP approved by DOE. Due to various considerations, though, some recipients have submitted drafts of their CBSP that do not yet meet these standards. TAGs continue to work with these recipients to assist them in revising their study plans to conform to DOE standards, while accommodating each utility’s regulatory or other constraints. Ideally, the remaining SGIG recipients will have a CBSP filed and approved by DOE by the end of second quarter of 2011.

SGIG Recipient with an Approved CBSP

Given the amount of time it takes a utility to receive regulatory approval for the study and/or rate designs, create marketing and outreach strategies and materials for study recruitment efforts, update the back-office systems required to process and bill customers, etc., it is unlikely most recipients will get into the field for their studies’ before late 2011. Many recipients are proposing studies that last for two full years, which means final evaluation results for all SGIG recipients with an approved CBSP will not be forthcoming until late 2014 or early 2015.

Reporting Requirements

As indicated above, recipients who receive approval from DOE of their consumer behavior study plans are expected to undertake certain reporting requirements so that recipients can convey what was learned from their studies, but also that external parties (e.g., DOE, academics, etc.) can evaluate for themselves the recipient’s own project data to address additional research questions.

DOE is requiring each recipient to submit an evaluation report at the end of the study, as well as at an interim point if the study is designed to last for two years or more. These evaluation report. will contain at a minimum:

  • An identification of study objectives;
  • A description of how the study is designed to meet these objectives;
  • The analytical methods used to evaluate the study objectives;
  • A summary of the data collected for use in the evaluation effort; and
  • The results from the evaluation effort and a determination if the study objectives were successfully accomplished.

In addition, two detailed data sets will be reported to DOE by each of the SGIG recipients with an approved CBSP. The first is a set of impact metrics, which quantify the level of response to the different dynamic pricing treatments included in the consumer behavior study. The second is a very granular project data set containing masked customer-level hourly consumption and characteristic information that cannot be tied back to an individual customer name or address. This data will, for all intents and purposes, be comprised of the same data used by the recipients to evaluate their own studies. DOE has created a preliminary listing of the customer characteristics to be included in the project data set, but a final listing of data elements as well as reporting templates will be forthcoming once the database design is complete.

All of the required data to be reported by the recipients (i.e., impact metrics and project data) will be used by DOE to perform meta-analysis across all approved SGIG consumer behavior study projects. This effort will attempt to better understand what drives common results and what may explain differences in response, acceptance, etc. to time-based rate programs, information technology, and/or automation technology. Since most studies anticipate getting into the field by 2012, the meta-analysis effort will result in a report as well as a dedicated subsection in the “performance results” area of the Smartgrid.gov website summarizing lessons learned from these studies, (forthcoming in 2014 or 2015 depending greatly upon the timing of study commencement and data submission).

Preliminary Data Elements Tables

View and down load preliminary data elements tables: