Data Analytics in Power Markets

✍ Scribed by Qixin Chen, Hongye Guo, Kedi Zheng, Yi Wang

• Publisher: Springer
• Year: 2021
• Tongue: English
• Leaves: 292
• Category: Library

✦ Synopsis

This book aims to solve some key problems in the decision and optimization procedure for power market organizers and participants in data-driven approaches. It begins with an overview of the power market data and analyzes on their characteristics and importance for market clearing. Then, the first part of the book discusses the essential problem of bus load forecasting from the perspective of market organizers. The related works include load uncertainty modeling, bus load bad data correction, and monthly load forecasting. The following part of the book answers how much information can be obtained from public data in locational marginal price (LMP)-based markets. It introduces topics such as congestion identification, componential price forecasting, quantifying the impact of forecasting error, and financial transmission right investment. The final part of the book answers how to model the complex market bidding behaviors. Specific works include pattern extraction, aggregated supply curve forecasting, market simulation, and reward function identification in bidding. These methods are especially useful for market organizers to understand the bidding behaviors of market participants and make essential policies. It will benefit and inspire researchers, graduate students, and engineers in the related fields.

✦ Table of Contents

Preface
Acknowledgements
Contents
1 Introduction to Power Market Data
1.1 Overview of Electricity Markets
1.2 Organization and Data Disclosure of Electricity Market
1.2.1 Transaction Data
1.2.2 Price Data
1.2.3 Supply and Demand Data
1.2.4 System Operation Data
1.2.5 Forecast Data
1.2.6 Confidential Data
1.3 Conclusions
References
Part I Load Modeling and Forecasting
2 Load Forecasting with Smart Meter Data
2.1 Introduction
2.2 Framework
2.3 Ensemble Learning for Probabilistic Forecasting
2.3.1 Quantile Regression Averaging
2.3.2 Factor Quantile Regression Averaging
2.3.3 LASSO Quantile Regression Averaging
2.3.4 Quantile Gradient Boosting Regression Tree
2.3.5 Rolling Window-Based Forecasting
2.4 Case Study
2.4.1 Experimental Setups
2.4.2 Evaluation Criteria
2.4.3 Experimental Results
2.5 Conclusions
References
3 Load Data Cleaning and Forecasting
3.1 Introduction
3.2 Characteristics of Load Profiles
3.2.1 Low-Rank Property of Load Profiles
3.2.2 Bad Data in Load Profiles
3.3 Methodology
3.3.1 Framework
3.3.2 Singular Value Thresholding (SVT)
3.3.3 Quantile RF Regression
3.3.4 Load Forecasting
3.4 Evaluation Criteria
3.4.1 Data Cleaning-Based Criteria
3.4.2 Load Forecasting-Based Criteria
3.5 Case Study
3.5.1 Result of Data Cleaning
3.5.2 Day Ahead Point Forecast
3.5.3 Day Ahead Probabilistic Forecast
3.6 Conclusions
References
4 Monthly Electricity Consumption Forecasting
4.1 Introduction
4.2 Framework
4.2.1 Data Collection and Treatment
4.2.2 SVECM Forecasting
4.2.3 Self-adaptive Screening
4.2.4 Novelty and Characteristics of SAS-SVECM
4.3 Data Collection and Treatment
4.3.1 Data Collection and Tests
4.3.2 Seasonal Adjustments Based on X-12-ARIMA
4.4 SVECM Forecasting
4.4.1 VECM Forecasting
4.4.2 Time Series Extrapolation Forecasting
4.5 Self-adaptive Screening
4.5.1 Influential EEF Identification
4.5.2 Influential EEF Grouping
4.5.3 Forecasting Performance Evaluation Considering Different EEF Groups
4.6 Case Study
4.6.1 Basic Data and Tests
4.6.2 Electricity Consumption Forecasting Performance Without SAS
4.6.3 EC Forecasting Performance with SAS
4.6.4 SAS-SVECM Forecasting Comparisons with Other Forecasting Methods
4.7 Conclusions
References
5 Probabilistic Load Forecasting
5.1 Introduction
5.2 Data and Model
5.2.1 Load Dataset Exploration
5.2.2 Linear Regression Model Considering Recency-Effects
5.3 Pre-Lasso Based Feature Selection
5.4 Sparse Penalized Quantile Regression (Quantile-Lasso)
5.4.1 Problem Formulation
5.4.2 ADMM Algorithm
5.5 Implementation
5.6 Case Study
5.6.1 Experiment Setups
5.6.2 Results
5.7 Concluding Remarks
References
Part II Electricity Price Modeling and Forecasting
6 Subspace Characteristics of LMP Data
6.1 Introduction
6.2 Model and Distribution of LMP
6.3 Methodology
6.3.1 Problem Formulation
6.3.2 Basic Framework
6.3.3 Principal Component Analysis
6.3.4 Recursive Basis Search (Bottom-Up)
6.3.5 Hyperplane Detection (Top-down)
6.3.6 Short Summary
6.4 Case Study
6.4.1 Case 1: IEEE 30-Bus System
6.4.2 Case 2: IEEE 118-Bus System
6.4.3 Case 3: Illinois 200-Bus System
6.4.4 Case 4: Southwest Power Pool (SPP)
6.4.5 Time Consumption
6.5 Discussion and Conclusion
6.5.1 Discussion on Potential Applications
6.5.2 Conclusion
References
7 Day-Ahead Electricity Price Forecasting
7.1 Introduction
7.2 Problem Formulation
7.2.1 Decomposition of LMP
7.2.2 Short-Term Forecast for Each Component
7.2.3 Summation and Stacking of Individual Forecasts
7.3 Methodology
7.3.1 Framework
7.3.2 Feature Engineering
7.3.3 Regression Model Selection and Parameter Tuning
7.3.4 Model Stacking with Robust Regression
7.3.5 Metrics
7.4 Case Study
7.4.1 Model Selection Results
7.4.2 Componential Results
7.4.3 Stacking Results (Overall Improvements)
7.4.4 Error Distribution Analysis
7.5 Conclusion
References
8 Economic Impact of Price Forecasting Error
8.1 Introduction
8.2 General Bidding Models
8.2.1 Deterministic Bidding Model
8.2.2 Stochastic Bidding Model
8.3 Methodology and Framework
8.3.1 Forecasting Error Modeling
8.3.2 Multiparametric Linear Programming (MPLP) Theory
8.3.3 Error Impact Formulation
8.3.4 Overall Framework
8.4 Case Study
8.4.1 Measurement of STPF Error Level
8.4.2 Case 1: LSE with Demand Response Programs
8.4.3 Case 2: LSE with ESS
8.4.4 Case 3: Stochastic LSE Bidding Model
8.4.5 Time Consumption
8.5 Conclusions and Future Work
References
9 LMP Forecasting and FTR Speculation
9.1 Introduction
9.2 Stochastic Optimization Model
9.2.1 Model of FTR Portfolio Construction Problem
9.2.2 Scenario-Based Stochastic Optimization Model
9.3 Data-Driven Framework
9.4 Methodology
9.4.1 Clustering
9.4.2 Mid-Term Probabilistic Forecasting
9.4.3 Copulas for Dependence Modeling
9.4.4 Training and Evaluation Timeline
9.4.5 Scenario Generation
9.5 Case Study
9.5.1 Data Description
9.5.2 Comparison Methods
9.5.3 Statistical Validation of Quantile Regression
9.5.4 Scenario Quality Evaluation
9.5.5 Impact of Node Reduction with Clustering
9.5.6 Revenue and Risk Estimation
9.5.7 Sensitivity Analysis on the Number of Clusters
9.6 Conclusion
References
Part III Market Bidding Behavior Analysis
10 Pattern Extraction for Bidding Behaviors
10.1 Introduction
10.2 Assumptions and Proposed Framework
10.2.1 Model Assumptions
10.2.2 Bidding Data Format
10.2.3 Data-Driven Analysis Framework
10.3 Data Standardization Processing
10.3.1 Filtering Available Capacities
10.3.2 Sampling Bidding Curves
10.3.3 Unifying Data Length
10.3.4 Clipping Extreme Prices
10.4 Adaptive Clustering of Bidding Behaviors
10.4.1 Distance Measurement
10.4.2 K-Medoids Clustering
10.4.3 Adaptive Clustering Procedure
10.4.4 Clustering Algorithm
10.5 AEM Data Description
10.5.1 Description of Market Participants
10.5.2 Description of Bidding Data
10.6 Bidding Pattern Analysis
10.6.1 Parameter Setting
10.6.2 Bidding Patterns of DUs by Fuel Type
10.6.3 Comparison of Similar DUs
10.6.4 Discussion
10.7 Feature Analysis on Bids
10.7.1 Discrete Aggregation Feature
10.7.2 Probability Distribution Feature
10.7.3 Time Distribution Feature
10.8 Conclusions
References
11 Aggregated Supply Curves Forecasting
11.1 Introduction
11.2 Market and Framework
11.2.1 Market Descriptions
11.2.2 Forecasting Framework
11.3 Data Integration and Feature Extraction
11.3.1 Data Integration
11.3.2 Feature Extraction
11.4 ASC Forecasting
11.4.1 LSTM Model
11.4.2 Influencing Factors
11.4.3 Training and Forecasting
11.4.4 Evaluation Criteria
11.5 Case Study
11.5.1 Dataset Description
11.5.2 Feature Extraction
11.5.3 ASC Forecasting
11.5.4 Calculation Information
11.5.5 Methods Comparison
11.6 Conclusion
References
12 Learning Individual Offering Strategy
12.1 Introduction
12.2 Data-Driven Market Simulation Framework
12.2.1 Market Assumptions
12.2.2 Offering Data Clustering and Indexing
12.3 Individual Offering Strategy Learning
12.3.1 MFNN Model Structure
12.3.2 MFNN Model Inputs
12.3.3 MFNN Model Training
12.3.4 DNN-Based Model Structure
12.4 Market Clearing Simulation
12.5 Case Study
12.5.1 Basic Data
12.5.2 Individual Offering Behavior Forecasting
12.5.3 Market Simulation
12.5.4 Comparison with Current Price Forecasting Methods
12.5.5 Calculation Efficiency
12.6 Conclusions
References
13 Reward Function Identification of GENCOs
13.1 Introduction
13.2 Assumptions and Framework
13.2.1 Market Assumptions
13.2.2 Data-Driven Framework
13.3 Bidding Decision Process Formulation
13.3.1 Markov Decision Process in Wholesale Markets
13.3.2 Reinforcement Learning Process
13.3.3 Bidding Data Integration
13.4 Reward Function Identification
13.4.1 Deep Inverse Reinforcement Learning Algorithm
13.4.2 Discretization Methods for States and Actions
13.5 Bidding Behavior Simulation
13.5.1 DQN-Based Bidding Simulation Model
13.5.2 Value Function and Q-Network
13.6 Case Study
13.6.1 Dataset Description
13.6.2 Parameter Setting
13.6.3 Reward Function Identification
13.6.4 Bidding Behavior Simulation
13.7 Conclusions
References
Correction to: Introduction to Power Market Data
Correction to: Chapter 1 in: Q. Chen et al., Data Analytics in Power Markets, https://doi.org/10.1007/978-981-16-4975-21