Devops Mutating Webhook Project

Devops Mutating Webhook Project

Devops Automation Of Software Deployment Pipelin · ML Project

Python · Data Preprocessing · Model Development · Evaluation

Project focus: anomaly detection / classification using IoT sensor streams and timestamped device measurements.

Large Scale Probabilistic Simulation of Renewables Production

Mike Ludkovski∗ Glen Swindle† Eric Grannan†

May 11, 20

Abstract

We develop a probabilistic framework for joint simulation of short-term electricity genera-

tion from renewable assets. In this paper we describe a method for producing hourly day-ahead scenarios of generated power at grid-scale across hundreds of assets. These scenarios are con- ditional on specified forecasts and yield a full uncertainty quantification both at the marginal asset-level and across asset collections. Our simulation pipeline first applies asset calibration to normalize hourly, daily and seasonal generation profiles, and to Gaussianize the forecast–actuals

distribution. We then develop a novel clustering approach to stably estimate the covariance ma- trix across assets; clustering is done hierarchically to achieve scalability. An extended case study using an ERCOT-like system with over 5 solar and wind farms is used for illustration.

1 Introduction

The stochastic nature of renewable energy generation (primarily understood to be grid-scale solar

and wind farms) necessitates probabilistic analysis of future generation which is in turn used as an input for unit commitment and economic dispatch decisions. For example, reliability assessment and the need for reserves depend critically on potential deviations of actual generation from forecasted levels. To do so, one must be able to sample scenarios of realized generation obtained from a generative model, rather than, say, from bootstrapping a fixed set of weather scenarios arising from

a numerical weather forecast. In this article we tackle the problem of developing a stochastic model for generating system- wide scenarios of future renewables generation. Here we are concerned with the daily and hourly timescales, the prototypical context being day-ahead unit commitment that is performed by regional transmission operators. As a motivating example, given an asset-level forecast for the next 2 hours, our goal is to sample actual energy production across many renewable assets, again at the hourly

scale. The typical setting for deployment of this platform involves several hundred generation assets of two or more types. The key feature of our approach is the ability to construct a joint model across all the as- sets. Thus, each outputted simulation scenario is a large multivariate matrix, with rows indexing assets and columns indexing the 2 hours. These simulations are intended to be consistent both with single-asset behavior (marginals) as observed historically, as well as observed correlations be-

tween assets; the latter includes capturing correlation across asset types (solar vs. wind) and zonal laws. Accurately capturing the respective dependence across assets and time is the crux of our contribution. ∗ Department of Statistics and Applied Probability, University of California Santa Barbara, USA, 93106-3110; email: [email protected] ; † Scoville Risk Partners, 1 Carnegie Center Dr Princeton, NJ, USA, 085

The other fundamental challenge that we tackle is the relative paucity of historical data. One typically has access to a year or two of relevant forecasts/actuals, and as a result there are sub- stantial limitations to what statistical methods are applicable to calibrate the high-dimensional correlation structure. For example, in the context of unit commitment where the delivery time scale is hourly, the number of random variables being simulated for a single delivery day in the

ERCOT case study used for illustration below is ≈5 assets × 2 hours ≈ 1 . The data available for calibration is typically  10 days of historical forecast and actual volumetric data. Given the seasonal variation in the behavior of the asset types under consideration, the data relevant for calibration for a particular delivery day is a fraction (say 25%) of the total available data. This renders nontrivial both characterizing the dynamics of each individual asset and estimating cor-

relations between the assets. Indeed, with the length of the time-series p roughly matching the number of assets J, direct inference of the J × J covariance matrix is ill-posed. To this end, we propose a hierarchical approach that recursively breaks the problem into estimating smaller J` × J` matrices and then reassembling these into a block structure.

Beyond calibrating the cross-asset correlation, the context of renewable energy simulations

imposes several other demands on the stochastic engine. First, simulated power levels must not only respect the non-negativity constraint, but also satisfy the hour-, day- and asset-specific maxima. These are especially pronounced for solar assets, where the daily and annual cycles of solar radiation create multiple layers of seasonality. Second, the simulated scenarios must capture this mixed distribution, which becomes especially important when considering aggregated production across

multiple assets. Indeed, the distribution of realized generation exhibits significant point masses: there are nontrivial probabilities of both zero and maximum generation, along with a continuous distribution within those bounds. Third, because scenarios are generated conditional on a forecast, it is critical to capture the forecast-actuals dependence, such as the conditional heteroskedasticity. Finally, our scenario assessment is driven by the impact of renewable generation on the grid,

i.e. aggregate production across several assets as reflected in the transmission network and respective security constrained unit commitment (SCUC) and economic dispatch (SCED). Therefore, the quality of the joint scenario distribution is primarily based on how well it captures aggregate production, rather than abstract statistical metrics. Literature review: At the bird’s-eye level, our simulation platform can be placed in the land- scape of synthetic dataset frameworks, see for example [GGP16, LFPSP+ 19, RVCR21, ZKF+ 18,

BRM+ 21] for other ways to build synthetic solar and wind scenarios. We also refer to the com- petitions [HPF+ 16, HXB19]. To our knowledge, ours is the first platform to combine simulations across multiple asset types. The latter raises the challenge of the varying number of active hours (i.e. correlating solar assets that are only active in the daytime with wind assets that are active throughout).

While there are hundreds of articles on forecasting renewable generation, by and large these

focus on point predictions. For example, the emerging machine learning techniques are excellent in minimizing the predictive error but are not designed to quantify uncertainty around those predic- tions. For the probabilistic forecasts that we aim for, one may mention [LDP19, WDS+ 18, BRM+ 21] for solar generation and [SWWW17, RSWW20, LSF+ 20, PG12, GLW+ 06, MSGR13] for wind gen- eration. Note that some of the above works focus on predicting horizontal global irradiance (the dominant driver of solar energy production) or wind speed (which drives wind production), while

we concentrate on direct modeling of quantities measured in megawatt-hours. While applying the (nonlinear) production curve to map from weather inputs to MWh outputs is feasible for analysis of a single, fully known asset, it is extremely challenging to carry out in bulk, especially since not all asset characteristics may be publicly known. Consequently, we believe that working solely in MWh-universe is more appropriate; this choice does restrict some of the available forecasting

tools. A recent overview about integration of probabilistic forecasts in grid operations is provided in [LZ20]. Closer to the aim in our work, there is a literature strand addressing spatio-temporal prob- abilistic methods that simultaneously model renewable generation at multiple geographic loca- tions. Existing approaches include copulas [MWH17, PZH+ 20, TWX+ 18]; kriging or Gaussian processes [AYZW15, vdMSS+ 18, YLLQ18, WK13] and downscaling of weather forecasting ensem- bles [BRM+ 21]. Among copula approaches, one may distinguish the application of vine copulas

[WWL+ 17], tail copulas [MR20] and Bayesian copulas [PZH+ 20]. Relative to the bulk of the forecasting literature our setup has two critical distinctions. First, our primary interest is in probabilistic forecasts that provide the full joint distribution across all assets of interest. To that end, we concentrate on assessment using statistical scoring methods, rather than on minimization of predictive error. We utilize the framework of strictly proper scoring rules introduced by Gneiting and Raftery [GR07, GBR07, GK14] and summarized for our context

in Section 5.1. Gneiting already applied these to wind speed analysis in [GLW+ 06]; for an updated

overview in the context of renewable generation see [LDP19]. Related works that assess probabilistic

forecasts of renewables are in [WDS+ 18, RSWW20, ZS18].

Second, our motivating application is the sampling of day-ahead hourly scenarios conditioned

by a respective forecast. Such Numerical Weather Predictions (NWP) are received daily by the market participants and system operators and drive the downstream tasks of unit commitment, economic dispatch, and risk measurement. This day-ahead setup implies that we view all quantities as vectors, indexed by (active) hours and collected daily, rather than as time-series. Moreover, the non-trivial dependence between actuals and forecasts implies that looking at forecast errors on their

own is insufficient and that the dependence between conditioned realizations materially differs from the unconditional one. For example, the tail dependence between conditioned forecasts is much weaker than between unconditional ones. The rest of the article is organized as follows. Section 2 summarizes our platform and the data that we work with. Section 3 describes our clustering algorithm which is the main methodological contribution. Section 4 describes the de-trending, rescaling and calibration applied prior to the

clustering and then their application in reverse to generate scenarios given the fitted covariance structure. Section 5 presents a case study for a large system with 200+ each of solar and wind farms; Section 6 concludes.

2 Platform Overview and Data

2.1 Process Flow

At a high level, the proposed method is summarized in Figure 1 and conceptually involves (A) calibration, namely standardizing and normalizing raw data to render it Gaussian-like, whereby we extract the respective z-scores; (B) hierarchical clustering based on correlation-driven annealing; (C) simulation based on conditional Gaussian draws, which is then fed in reverse order to the calibration module. The platform accepts as input historical forecasted and actual power generation for each asset.

The following modules are then applied: (i) Asset Data Preprocessing: Organization of historical actual and forecasted volumes as well as metadata.

(ii) Asset Meta-Calibration: Parameterization of key operational attributes, notably maximum production and diurnal envelopes. This is separated from generic daily calibration due to the

Platform Structure

Asset Meta-

Asset Data Historical Asset Correlation Simulation of Asset

Preprocessing Calibration Calibration Estimation Deviates Simulations

Figure 1: Workflow of the simulation platform.

global nature of the attributes being parameterized and the associated time required.

(iii) Historical Calibration: Model calibration by asset for a set of dates spanning the historical date range—usually this is all dates. This yields a set of normal deviates for each asset associated with each hour of delivery and spanning the calibration date range.

(iv) Asset Clustering: Construction of hierarchical clusters used in the correlation representation using the results from step (iii).

(v) Asset Calibration: Model calibration by asset for a specific simulation date—this could be in the historical date range for backtesting or for a future date.

(vi) Correlation Estimation: Construction of intra-cluster correlations and propagation of specific asset normal deviates up the clustering hierarchy.

(vii) Simulation of Deviates: Propagation of simulations down the cluster hierarchy with condi- tional normal calculations completing normal deviate simulations within each cluster.

(viii) Asset Simulations: Conversion of normal deviates to production volumes using asset calibra- tion results, including production and diurnal envelopes.

Note that after steps (i)-(v), asset dynamics are represented by a set of historical normal deviates spanning the set of “active hours” for the asset. For wind the active hours are 1:24; for solar assets active hours vary by asset and time of year creating a nontrivial impediment for most correlation estimation procedures which is the key motivation for the proposed approach in steps (vi)-(vii).

J Number of assets i, j asset indices

d = 1, . . . , D; h Days, Hours t generic time index gd = (gd,h ) Actual production MWh αd,h actuals fraction ∈ [0, 1] fd = (fd,h ) Forecast production MWh βd,h forecast fraction ∈ [0, 1] gdmax Estimated daily max max gd,h Estimated hourly max µh (β) Conditional mean of α σh (β) cond variance of α

k = 1, . . . , K index for factors ψk , γk PCA factors

A corr matrix ρij entries of A

C cluster c(C) delegate of C

C clustering hierarchy l = 1, . . . , L hierarchy levels

η annealing param ` annealing rounds

E sim annealing energy Tc annealing schedule

|C| cluster size κ cluster size penalty φ year frac Θ window width Id = [sd , td ] observed diurnal boundaries [ŝd , t̂d ] estimated boundary

Id daily window Hd (·) daily rescaling function

Gnom nominal capacity

• There are J assets, indexed by i, j = 1, . . . , J;

• Days are indexed by d = 1, 2, . . . , 365;

• Hours are indexed by h = 1, 2, . . . , 24;

• Generic time index is t = 1, . . . , T ;

• To define distances between days, we use the corresponding year fraction φ(d) ∈ [0, 1];

• For a given day d, we use a time window of width Θ: Id = {d : |φ(d) − φ(d )| ≤ Θ};

• Actual generation is gd = (gd,h ) on day d—a vector of dimension 2 (the number of 5 minute intervals in a day). When averaged to hourly intervals, actual quantities are denoted by gd = gd,1 , . . . , gd,2 . Similarly fd = (fd,h ) denotes forecasted (hourly) generation. Both gd and fd are ∈ R .

• Daily diurnal production boundaries pertain to solar assets. Specifically, Id ≡ [sd , td ]; sd denotes the first interval (counted in 5-min units) of the day with positive production; td the last. These are historical realizations for each day. The corresponding estimated diurnal boundaries are Iˆd = [ŝd , t̂d ].

• Normalized generation ratios are αd ∈ [0, 1] and βd which correspond to gd , fd ;

• PCA factors are ψk , k = 1, . . . , with respective amplitudes γkj (d) indexed by days;

• The correlation of γ1i (·) and γ1j (·) is denoted by ρij . Below for a target date d, we compute correlation over a window Id so technically the correlation is ρij (d). The resulting covariance matrix is A.

Anacacho_Wind_Farm Actual Forecast 0.8

Generation Fraction

10-Feb 11-Feb 12-Feb 13-Feb 14-Feb 15-Feb

Figure 2: Left: Proposed renewable assets in ERCOT region. Symbol size is proportional to

nominal generation capacity Gnom . Right: time series of normalized generation αd,h at Anacacho wind farm over 5 days (1 observations) in February 2018.

2.2 Case Studies

Our test set spans the ERCOT market in Texas and includes both solar PV and wind generators.

The dataset was primarily created by NREL through re-analysis of numerical weather simulations

and covers the 20 and 20 calendar years. NREL re-analyzed ensembles of ECMWF (European

Centre for Medium-Range Weather Forecasts) models in order to extract predictions of solar irra-

diance and wind speeds at 10km spatial resolution and 5 min frequency. Interpolation was then applied to obtain respective physical quantities at the sites of the solar and wind farms. Next, the weather data was combined with asset characteristics and the respective power transfer curves to derive energy output via the WIND [DCHM15] and SIND [SXL+ 18, FYHZ19] toolkits. This type of re-analysis can be done both for existing assets, as well as proposed (i.e. “synthetic”) projects,

permitting study of high renewable-penetration scenarios [RHD+ 18]. We note that these weather models do in fact provide ensemble forecasts; however the ensemble dispersion is based on a differ- ent notion of uncertainty and requires separate pre-processing to yield calibrated range of actuals; in this work we therefore do not utilize ensembles and use only the mean point forecasts. The resulting ERCOT-wide dataset that we consider consists of two testbeds: (i) Existing, which includes 2 solar farms and 1 wind farms; (ii) Proposed, which adds more than 3

additional generators, for a total of 2 solar and 2 wind assets. The projects are throughout ERCOT, although Existing assets occur only in some of the ERCOT zones (for example Coast for wind contains no assets). ERCOT consists of 8 zones (Coast, West, Far West, North, North Central, East, Southern and South Central), which will be used later for aggregate analysis in Section 5. The left panel of Figure 2 shows the locations of the 4 assets in the Proposed testbed. Note that nominal capacities span several orders of magnitude from 1.0 MW to 12 MW, with a

median of approximately 1 MW and total nameplate capacity of 41GW for solar and 60GW for wind. In total, the dataset spans T = 7 days or 175 hours. Given the limited size of the dataset, we utilize all of it for training and concentrate on in-sample testing.

2.3 Stylized Features of the Data

Due to the widely varying nameplate capacity Gjnom and the dominant diurnal pattern in solar generation, analysis of raw produced MWh is not recommended due to obvious non-stationarity. Instead we normalize by de-trending and rescaling (see Sec. 4.2) to obtain production ratios that are always in the unit interval [0, 1]. This normalization is further combined with a shift-and-stretch transformation to yield data that are statistically i.i.d. across assets and days for the purposes of

correlation estimation. A production ratio of zero means that no power is produced and 1 means that maximum possible production (given hour, day, and asset capacity) is achieved. The right panel of Figure 2 displays a sample hourly time series for a wind asset forecast and actuals, αd,h , βd,h . The key feature is the nontrivial number of zeros and ones in the actuals αd,h —on many hours either there is zero wind and the turbine is not spinning, or the turbine spins at its maximum rate and realized generation

fraction is 1. For wind assets, there are about 4-7% of hours with zero generation, and about the same for max-generation. Additional daily patterns are manifest. There is much less wind during the day, so for example at noon, on 8-15% of days there is no wind energy produced at all (see mid-day of Feb 12). Similarly, strong wind primarily happens at night and at midnight 15-30% of observed generation is at maximum capacity (see the nights of Feb 1 and 1 in Figure 2). As could

be expected, forecasts tend to smooth out such point masses and hence are generally strictly inside the (0, 1) interval. In line with these features, joint simulation of realized volumes conditioned on forecasts must consider and account for the following: • The distribution of realized forecast errors d,h (the difference between actual and forecasted ratios) being dependent upon the forecast value. Such conditional dependence is particularly striking for solar assets. The left panel of Figure 3 shows the realized fraction of maximum

production, αd,h , for a solar asset scattered against the forecasted ratio βd,h . The variance of forecast error d,h is noticeably smaller at high forecasted levels —a manifestation of the uncertainty associated with potentially cloudy days, in contrast to relative certainty of a sunny forecast. Similarly, the right panel of Figure 3 scatters actuals against forecasted ratios for a wind asset —we observe low uncertainty on windless days (βd,h ≃ 0) and increasing uncertainty as wind forecasts pick up.

• Solar and wind asset dynamics are fundamentally non-stationary. For example, for solar assets 8AM in January is very different than 8AM in March. Rescaling needs to account for the maximal achievable capacity varying by hour and the fact that solar start/end periods are driven by the annual sunrise-sunset cycle. Similarly, there are seasonal and diurnal patterns in average wind generation and the respective variance. Consequently, fusing observations from different days in order to estimate trends and correlations requires multiple layers of

As we are interested in directly modeling electricity production, there are hard physical con-

straints on the simulated quantities. Assets have nominal capacity Gnom , but maximum achievable max varies by hour, especially for solar, where g max depends on the angle of the sun at capacity gd,h d,h max is not available directly and is rather statistically inferred from that instant. In our setup, gd,h the data, see Section 4.2.

3 Cluster Analysis for Correlation Structure

In this section we describe our procedure for inferring the conditional correlations of production

ratios αi and αj given the respective forecasts. The high-dimensional nature of the problem coupled

BlueBell Solar Production Ratio Scatter (Hour 2PM) Amazon Wind Production Ratio Scatter (Hour 1PM) 1 1

0.9 0.9

0.8 0.8

Low Forecast

Actual Production Fraction

Actual Production Fraction

Error 0.7 0.7

0.6 0.6

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Forecast Production Fraction Forecast Production Fraction

Figure 3: Left: actual α·,h vs forecast β·,h production ratios for a solar asset (Blue Bell) at h =2pm.

Right: same for a wind asset (Amazon Wind Farm) at h =1pm and a window of 2 days.

with the limited amount of historical data available for calibration means that some assumptions on the structure of the problem, particularly of the correlations between asset-delivery hours must be made. These are detailed below, but roughly speaking the approach is to construct a hierarchical relationship between asset clusters, thereby preserving high correlations between assets that have historically exhibited such behavior.

Our approach is based on constructing clusters that capture closely-related assets, and then

estimating the historical correlation matrix for each cluster. This is done recursively through selecting cluster “delegates” that are used for correlating at the parent level. Ultimately, cluster membership is used to build a structured correlation matrix that is fed into a multivariate-Gaussian framework. To this end, we address the questions of (i) defining the concept of asset similarity that underlies clustering; (ii) cluster construction; (iii) estimation of intra- and inter-cluster correlations.

Factor Representation: The approach used in all correlation and clustering that follows is based

upon a principal component analysis (PCA) representation of each asset’s standardized/normalized deviates. The latter are obtained from a joint model for (α·,h , β·,h ) visualized above and explained in Section 4. Figure 4 shows the first few eigenvalues and spectrum for a particular solar asset on January 1, namely the shape of the first 3 PCA factors h 7→ ψk (h) (note that h ∈ {8, 18} as the other hours feature zero solar production with probability 1, meaning there are only 1 factors in

total) and the corresponding eigenvalues λk , k = 1, . . . , 11. The typical empirical factor is manifest; the decay in the spectrum is useful. The interpretation of PCA factors matches the application of factor analysis in other situations. The first factor ψ (·) (blue) is qualitatively similar across all assets and corresponds to production values higher than expected conditional on the forecast. This statement does not imply a linear response. A higher loading γ of the first factor propagates back through the copula transformations

that rendered the residuals normal and on which the PCA was performed. However, monotonicity is preserved and increasing the loading on the first factor increases realized production. In a similar way, increasing the second factor loading γ increases production in the first active hours and decreases production in the later hours (red curve in Figure 4). All intra-asset correlation analysis is done in the above “factor space” rendering the joint analysis of assets with different sets of active hours easily implementable. This accomplishes two things.

First, it translates the functional correlation of the curves (rendered as vectors) αi , αj into the more interpretable correlation of the individual amplitudes γki (·), γkj (·). The latter are time-series indexed by d. Second, it allows us to consider correlation of vectors of different dimension, needed

Solar Factors

Factor Level

-0.4 8 9 1 1 1 1 1 1 1 1 1 Hours

Solar Spectrum

1 2 3 4 5 6 7 8 9 1 1 Factor

Figure 4: Sample factor analysis for a solar asset (Blue Bell). Factors ψk (h) (top) for k = 1, 2, 3

(in blue, red, orange, respectively), and eigenvalues λk (bottom). The calibration is to 1/1/2018.

both to jointly model solar production (a vector of 10-1 hours) and wind production (2 hours), but also solar production in different time-zones or longtitudes/latitudes (e.g. a vector of 1 hours and a vector of 1 hours). The common features of factors ψk across assets cease to be observed beyond the first few factors, and motivates our clustering on γ as ψ has an unambiguous sign across assets.

3.1 Clustering via Simulated Annealing

The key premise of the developed correlation structure is the assumption of a hierarchical form; one based upon a correlation-clustering of lower level asset clusters. Clustering is typically implemented by asset type, i.e. separate between solar and wind. The top level intra-asset clusters are handled specially as described below.

- All clustering currently uses the first factor amplitude, γ

- At any level of the hierarchy each cluster C is represented by the first factor amplitude of a specific “delegate” or “centroid” asset.

- At the lowest single-asset level this is just the first factor amplitude γ . - At each of the subsequent levels the “centroid” is the asset with the highest average correlation with other member assets.

• Clustering is via simulated annealing (SA) with an energy function of the form:  

X κ X

E(C) := 1 + (1)

|C| − 1

C∈C i,j∈C

where C denotes the set of clusters at step of the annealing process; |C| is the number of elements in cluster C. ρi,j = Corr(γ1i (·), γ1j (·)) is the correlation between first factor amplitudes

of members i and j. Finally, κ is tuned dynamically to target a user-specified cluster count reduction between levels. Note that there is an implicit competition between cluster size and average intra-cluster correlation reflected in the κ-term. One way to view this penalty is the product of the size of the cluster |C| and the average over all pairs of 1 − ρ2i,j . A given choice of κ enforces a soft constraint on the achieved cluster sizes {|C|}. We generally seek clusters of 2-5 assets. This implies that larger testbeds with more assets will end up with more

layers of the cluster hierarchy, compared to smaller ones. For example, we obtain 3-4 layers in the smaller Existing testbed and 5 layers in the larger Proposed testbed. Smaller κ results in flatter hierarchies. SA Perturbations: The idea of SA-based clustering is to gradually search for the best cluster assignments, namely those that minimize the energy function (1), via performing local perturbations on the cluster partition that are probabilistically explored. This is done via a loop over ` that moves

from a current partition to a new one as follows: - Generate a candidate partition C`0 by perturbing the current one C` ; - Compute the corresponding change of energy ∆E` = E(C`0 ) − E(C` ); - If ∆E` < 0, accept the perturbation; otherwise, accept the perturbation with probability exp(−T` ∆E` ) where T` is the current annealing temperature parameter. If the perturbation is rejected, then the new partition is the same as the current partition; - Lower the temperature by a factor η: T`+1 → ηT` .

We propose two types of local perturbations:

1. Merge: pick two assets i, j and move i to the cluster C` (j) of j. If i was in a singleton cluster

|C` (i)| = 1, that would reduce total number of clusters by 1;

2. Split: pick an asset i and split the cluster it belongs to, C` (i) (assuming it has more than 1

member) into 2 clusters, which would increase the total number of clusters by 1.

Note that depending on current partition, some perturbations are ruled out (for example, splitting

a singleton, or trying to add to a cluster that is already of maximal size). The SA algorithm is initialized with every asset being in a singleton cluster and proceeds for a fixed number of L steps. Among the tuning parameters is the annealing schedule η, the number of annealing steps L, and the cluster penalty κ. Furthermore, we have the form of the energy function where we can perturb the original correlations ρi,j in order to penalize deviations of high correlations more than

Figure 5 visualizes the cluster hierarchy for two typical days, for both solar and wind assets.

We aim to have a reduction factor of 0.3 (see κ above) for each level of the hierarchy. Centroids of higher level clusters are indicated with progressively larger symbols and lines indicate cluster membership. We observe a strong geographic contiguity of clusters across all levels of the hierarchy. In particular, the North/South and East/West divisions are clear, and for instance the wind assets in South and coastal Texas are usually in separate top-level clusters compared to the rest of the

system. In this case study there are many physical agglomerations of several assets very close to each other. These are almost always clustered together at the lowest level of the hierarchy, resulting in very tight (geographically) clusters that look like a single “blob” in Figure 5. The main role of the clusters is to construct a low-dimensional correlation matrix which can stably estimate the true correlations. To this end, we do not seek a sophisticated clustering ap- proach; moreover we do not interpret the cluster assignments as a “hard” partitioning of the assets,

SOLAR WIND

3 3

3 3

2 3

2 2 -1 -1 -1 -1 -9 -9 -9 -1 -1 -1 -1 -1 -1 -9 -9 -9 -9 -9

Long Long

Figure 5: Left: Clusters for 2 solar assets on Dec 1, 2017. Right: clusters for 2 wind assets

but as a scaffolding that regularizes the inference of A. b Thus, the plots in Figure 5 are primarily for illustrative purposes and do not directly reflect the quality of the clusters. In particular, there is limited practical meaning to a particular asset being designated a centroid.

Figure 6 shows the distribution of cluster sizes |C|, i.e. the number of assets that constitute

a cluster. By construction, the median cluster size is 3 (≃ 1/κ), however clusters can vary sub- stantially. About a quarter of clusters are singletons, i.e. the annealing algorithm did not find any suitable other assets to group them with. In that particular example for Feb 1, 2018, we start with

2 distinct assets; there are then 6 clusters at level 2, 2 at level 3, 8 at level 4 and finally 3 at

the top level 5, yielding a clustering tree structure with 1 clusters (nodes) at 5 levels, as shown in right panel of Figure 6. Based on the selected parameters, with a few hundred assets we end up with 4-5 layers of cluster hierarchy. Note that by construction simulated annealing involves randomization, i.e. running SA twice with exactly same inputs will yield slightly different results. Similarly, the user has a choice of building a clustering structure separate for each target day, or freezing the cluster assignment

across a range, or possibly across the entire year. In the latter case, we recommend computing PCA factors across the date range, in order to capture the typical average behavior.

3.2 Upward Correlation Propagation

Any attempt to render high-dimensional correlation estimation useful and computationally tractable

involves some decision about what correlations to keep. Our approach posits that:

– Factors within each asset class are similar in form—especially among the higher factors. More precisely, the magnitude of the inner products of factor k for assets i and j are high.

– Cross-factor correlations are low—the (by construction) zero correlation between different factors of the same asset extends to intra-asset: that is, the correlation between (asset,factor) (k , i ) and (k , i ) are low when i 6= i .

We choose to: a) keep same-factor correlations and b) zero cross-factor correlations across assets; both assumptions are observed empirically. The iteration proceeds as follows:

WIND

-1 -1 -1 -1 -1 -1 -9 -9 -9 -9 -9 Long

Figure 6: Left: clusters for 2 wind assets on Feb 1, 2018. Right: distribution of |C| on that day

- Every cluster has a set of member assets for which the correlation matrix of factor amplitudes (k) γkj is computed. This yields a set of correlation matrices AL,C (i, j) for cluster C at hierarchy level L and factor k with each component (i, j) corresponding to the correlation between the k th factor amplitude γk of members i and j.

- A subset of a user-specified number p of members of cluster C is selected to be used at the next clustering level (recall clusters were already defined before). The set of p members for upward propagation is selected to minimize the trace of the first-factor covariance matrix conditioned on the first factor amplitudes of such subsets. Essentially, the assets that persist to level L + 1 are those that span the first-factors of the entire set most effectively.

- Each asset type ultimately has a top level of clusters—recall that the clustering hierarchy is terminated once a prescribed cardinality is achieved (usually this is in the range of 1-

4 clusters). Cross-asset correlation structure is accommodated a final covariance matrix

constructed from the factor amplitudes from the first factors ψk of each member of the top level clusters. So, for example, under the setting where each top level asset can contribute its top two factor loadings γ , γ to the top-covariance, a cluster with 3 members would contribute

6 factor amplitudes. The result is a covariance matrix of dimension roughly half of the sample

size in the results that we present below. We note, however, that the effective dimension of this matrix is in the single digits, rendering covariance estimation tractable [KL17]. We have also explored use of GLASSO methods to yield sparse precision matrices.

4 Calibration and Simulation

Meta-Calibration: The purpose of the meta-calibration step is to estimate generation profiles

(maximum generation for solar; mean for wind) and diurnal boundaries for solar. The purpose of these estimates is to facilitate transformation of historical production quantities at nearby dates to values consistent with the statistical attributes of a particular target date being simulated. In other words, we take a transductive approach, where the model is fundamentally based on a particular prediction set and is then locally estimated as target dates change. This choice is driven by (i) the

strong seasonality at multiple levels; (ii) the non-constant dimension for the simulations, namely the varying number of active periods for solar generation both across days and potentially across

different assets due to latitude effects. We refer to [MR20] for a related discussion of calibration in the context of modeling GHI.

4.1 Seasonality

Solar and wind generation each exhibit nontrivial seasonality in production. This is visible for

solar assets at the daily time-scale for maximum achievable production in the left panel of Figure 7. Similarly, shorter time-scale variations are visible in the diurnal profiles in the right panel of Figure

7. The general approach to rendering a nearby date d˜ statistically consistent with a target date d

is to systematically rescale the observed forecast and actual production in day d. ˜ We first estimate daily maximum production levels gdmax by minimizing the asymmetric error:

Xh i

gdmax − Ǧd 1{gmax −Ǧd >0} − κM,1 gdmax − Ǧd 1{gmax −Ǧd <0} + κM,2 max gdmax − Ǧd , (2)


d d d d PK where gdmax = k=1 [αk,1 cos (2πkφ(d)) + αk,2 sin (2πkφ(d))] and we take K Fourier modes. A sample envelope of maximum achievable solar daily production, gdmax by date is shown in the left panel of Figure 7. We utilize K = 6 Fourier modes in order to fit relatively flat intervals in the

summer period. Once obtained, these envelopes are enforced on actual and forecasted quantities— namely it is assumed that there is zero probability of any generation in any hour on day d in excess of gdmax . Next we handle the diurnal production boundaries, i.e. the periods outside of which solar pro- duction is zero with probability 1 due to darkness. While production boundaries intuitively corre- spond to sunrise/sunset at the asset location, the engineering characteristics of the asset, its local

topography (e.g. mountains) and orientation of the panels prevent their direct association with any external geophysical datasets. As such, we infer the diurnal envelopes, denoted by ŝd , t̂d , via another minimization of an asymmetric error over the training set. For the “stop-gen” boundary t̂d this takes the form:

Xh

i

min t̃d − td 1{t̃d −td >0} − κD t̃d − td 1{t̃d −td <0} (3) t̃d d

where t̃d is a Fourier series (with K = 3 in our case) and κD the penalty for violating the boundary. Similar estimation pertains to the “start-gen” boundary ŝd . Generation is set to zero with proba- bility 1 outside of the estimated diurnal boundaries [ŝd , t̂d ]. The right panel of Figure 7 shows ŝd and t̂d across the year. For wind assets, there are no diurnal boundaries and maximum capacity is fixed throughout the year. Nevertheless, there is still a seasonal variation of production that we infer in order to rescale

and standardize different days. Indeed, if May tends to be more windy that April and we wish to consider April and May data jointly, we should be scaling May production down to maintain the same relative ranks. To this end, we estimate an hourly mean generation surface AvWd,h using a

Fourier series representation. This surface is employed to rescale production in a fashion similar

to that used for solar. Recall that our goal is to generate i.i.d. data across all calibration days. Thus, periods of the year where production tends to be lower should be inflated to match expected production on the target day, and vice versa. The same adjustment is done across hours of the day to make them identically distributed too.

4.2 Rescaling

Any reasonable attempt to use temporally local data to estimate asset behavior for a given target date must first transform the nearby data to be plausibly representative of the target date. The

Daily Interval and Outer Envelope Daily Start

3 0.4

Actual Actual

Forecast 0.3 Start Hour

Fraction Max Gen

2 0 5 1 1 2 2 3 3 Day

1 Daily Stop

0.8 End Hour

Stop Frac

0 0

Jan Mar May Jul Sep Nov 0 5 1 1 2 2 3 3

Day Day

Figure 7: Solar asset calibration. Left: maximum daily production envelope d 7→ gdmax for Blue

Bell solar farm. Right: respective solar diurnal envelopes, ŝd (top) t̂d (bottom), d = 1, . . . , 365; (also observed sd , td ), expressed as fractions of the entire 24-hour day.

smaller the data set, the larger the temporal windows for estimation, which renders the rescaling issue more significant. Thus, the goal of the meta- and calibration stages is to obtain a normalized training set that is i.i.d. To do so, separate calibration is done for each target date and the training data is taken from a window around that date to mitigate the annual seasonality.

Rescaling production is accomplished by dilation of the diurnal boundaries and volumetric

scaling by the ratio maximum production quantities. Given a year-frac threshold Θ that determines the width of the time window, set Id ≡ {d˜ : φ(d) ˜ − φ(d) ≤ Θ} to be all days of the year that are less than Θ-years away from d. The rescaling procedure is as follows—described in continuous variables: • For each d˜ ∈ Id the function ! gdmax

Hd˜(u) := gd hŝd˜ + u t̂d˜ − ŝd˜ i , u ∈ [0, 1], (4)

defines a rescaled generation profile for day d˜ with quantities normalized by the ratio of the maximum generation of the target day d to that of day d. ˜

• Applying Hd˜(·) to each of the hour intervals on the target day d yields a set of stationary hourly realizations of rescaled actual quantities g̃d˜.

• A similar procedure is performed on the forecasts fd to yield rescaled forecasts f̃d˜. h i • The maximum hourly generation quantity is estimated as gd,h max := max ˜ d∈Id g̃ d,h . ˜

• For hours h intersecting Id˜ (those with positive probability of generation) production ratio d,h g̃ ˜ variables are then computed as: αd,h ˜ = g max ; similarly βd,h ˜ are computed for the forecasts. d,h Note that the daily vectors αd and βd are in RH for H < 2 and have component values in the unit interval. Typically H ∼ 10. The production ratios shown in Figure 7 were computed in this fashion. The results presented are based on the rescaling methodology discussed above. As mentioned,

the purpose of the rescaling is to transform actual and realized production data at dates near the

target d so as to be consistent with the behavior at the given calibration date— consistent in the sense that the resulting deviates are (nearly) stationary. Additional refinements to utilize more information than the max-gen and diurnal boundaries are available but are beyond the scope of this paper and will be described elsewhere.

4.3 Calibration

For a given target date d the metacalibration associated with each asset is used to affect the rescaling discussed above yielding a set of production ratios αd˜ and βd˜ for actual and forecasted volumes respectively on each date and defined calibration date range d˜ ∈ Id of width Θ. These are the variables of interest — the independent variable being the β’s, the dependent being the α’s.

We observe that the two distributions are qualitatively different and moreover, raw forecast

errors are not zero-mean. Figure 8 shows the distribution of forecasted β·,h and realized production ratios α·,h for a representative wind and solar asset. We fix an hour of the day and consider a window of 1 = 0.3 × 3 days around April 1. Several features are apparent:

• Forecasts are biased: Ave(fd,h ) 6= Ave(gd,h ). This is both intrinsic (for example on a calm day wind forecast might be essentially zero, but realized production is non-negative, so on average will be higher than forecast; an analogous downward bias transpires when forecast production ratios are close to 100%) and data-driven. It appears that wind forecasts systematically underestimate production.

• The distribution of realized production can be bimodal, i.e. often the production ratios α are close to zero or to 100%. On the other hand, forecast production ratios β are closer to

Beta-distributed (and for some assets nearly uniform when tabulated across long periods of

• The conditional variance of actuals α·,h is much higher for forecasts in the middle than at the edges of the production range (a day forecasted to be cloudless is unlikely to witness materialization of heavy cloud cover).

In order to obtain Gaussianized forecast errors, we proceed to model the conditional mean and variance of αd,h given βd,h . The resulting residuals then act as the inputs to the correlation inference within the hierarchical cluster structure. The modeling premise is that for each active hour:

α·,h = 0 ∨ 1 ∧ [µh (β·,h ) + σh (β·,h ) · Z·,h ] (5)

with the interpretation being that the realized production ratio α·,h is distributed normally with a mean and variance that depends upon the forecasted production ratio βh,· , subject to bounds at 0 and 1. Endowing σh with dependence on β·,h is required in order to capture the observed heteroskedasticity; the inclusion of µh and its similar dependence on β·,h achieves bias correction and provides a degree of freedom in the calibration method discussed below. In the results shown in this paper, we have assumed a quadratic form for both µh (βh ) and

σh (βh ). To ensure stability of the inference for values of β outside  of the historical data used in βd,h −µβ ex each calibration, the values are first transformed: β̃d,h := ψ σβ where ψ(x) = 1+e x and the

empirical mean and standard deviations of the forecast data are denoted by µβ and σβ respectively. This stabilizes the functional forms of the estimated µh and σh on the length scale of the empirical standard deviation of the data. Therefore, the functional form used here corresponds to µh and σh being quadratic in β̃d,h .

Figure 8: Realized αd,h vs. forecasted βd,h production ratios. Left: Solar Asset (Hecate). Right:

Wind Asset (Briscoe). The sina violin plots show the 1 days centered on April 1 from 20 and

2018. The boxplots in the middle provide summary statistics, as well as the respective empirical

mean (blue dots), illustrating the bias and different variance of forecasts and actuals.

Calibration of µh , σh is via maximum likelihood estimation. The likelihood function is modified appropriately for the values of the argument outside of the unit interval (the point masses). Values of the normal deviates that are not uniquely determined (corresponding to production ratios of

0 or 1) are subsequently inferred by conditional normal calculations using those that are known.

Specifically, denoting the forecast error by d,h := αd,h − βd,h , and its normalized deviate zd,h := d,h −µh (βd,h ) σh (βd,h ) , the coefficients ϑ defining the parameterizations µ· , σ· are set to minimize:

X 1

L (ϑ) = 1{−βd,h <d,h <Ud,h } zd,h (d,h ) + log(σh (βd,h ))

d,h  Z d,h   

X 1 1 2

+ 1{d,h ≤−βd,h } fµh ,σh (x) zd,h (x) + log(σh (βd,h )) dx

Fµh ,σh (zd,h (−βd,h )) −∞ 2

d,h " Z ∞   #

X 1 1 2

+ 1{d,h ≥Ud,h } fµ ,σ (x) zd,h (x) + log(σh (βd,h )) dx (6) 1 − Fµh ,σh (zd,h (Ud,h )) Ud,h h h 2 d,h

where fµ,σ and Fµ,σ denote the normal pdf and CDF with mean µ and variance σ 2 respectively, g max and Ud,h = gdmax is the level at which asset maximum generation would be exceeded. The first term d,h is the usual form of the normal log-likelihood function and applies to values of d,h corresponding to realized production strictly above zero and below maximum generation. The second and third terms are the expected values of the same likelihood function conditioned on d,h breaching the

respective bounds. An example of such a calibration is shown in Figure 9. We utilize quadratic fits on the logit of the conditional mean and volatility. Intuitively, µh (·) is the bias correction, translating a fore- cast production ratio into the expected generation ratio (modulo truncation) and σh (·) is the heteroskedasticity correction, capturing the higher variability of realized production when forecasts are in the middle of the possible range. As mentioned, the intrinsic constraint that αh ∈ [0, 1]

implies that we expect µh (0) > 0 and µh (1) < 0 and similarly an umbrella shape on σh (·).

Figure 9: Fitted µh (·) (top row) and σh (·) across all the assets in the case study for the indicated

target date. Left: solar (across 2 assets). Middle: solar (2 assets). Right: wind (2 assets).

We observe that in our dataset, solar production is typically over-estimated, µh (β) > 0 for mid- range forecasts (i.e. less production tends to materialize on days that are neither very sunny nor very grey), while for wind, production is strongly under-estimated µh (β) < 0 for mid-range forecasts.

This could be partly linked to the concavity/convexity of the respective production curves. For

both types of assets, conditional variance σh (β) is highest for forecasts around β = 0.6. For solar, lowest uncertainty σh (β) is on very sunny days (βd,h ≃ 1), while for wind lowest uncertainty is for very calm days (βd ≃ 0) and conditional variances grows in β. The conditional variance is about the same across solar and wind assets, generally on the scale of σd,h ∈ [0.2, 0.3]. All the above are specific to this case study; the overall platform is agnostic to such particularities.

Each set of realized deviates zd,h is now “copula-ized” by application of the empirical CDF followed by the inverse of the standard normal CDF. This ensures that cluster construction works in the fully Gaussianized space where Gaussian covariance estimation is correctly specified. We will index those hours h for which αd,h ∈ (0, 1), namely not hitting the boundaries of production limits, by Id . The calculation of normal deviates used in the subsequent correlation analysis involves the

following steps for each active hour.

1. Estimate pairwise inter-hour correlations of the normalized residuals zd,h

˜ = σh (βd,h ) based upon the difference between hours. Specifically assuming that corr [Xh , Xh+k ] = ρ(k) for all hours h and h + k in the set of active hours, calculate the empirical pair-wise correlations using the values computed above. Note that values corresponding to point masses at the boundaries are excluded from these estimates, rendering each estimate for ρ(k) computed from potentially different subsets of the empirical data.

2. Ensure a positive definite correlation matrix by computing the eigenvalues and eigenvectors

for the estimated covariance matrix, setting all negative eigenvalues to zero, and rebuilding the original matrix, normalizing it by its diagonal elements.

3. For each day d˜ in the dataset, assuming joint normality of X̄d˜, compute the conditional mean

and variance for X̄Ic˜ (these are the deviates that remain unknown since production ratio d was zero or 1) given that these were at the boundaries of production limits, given X̄Id˜ which is known. Generate simulations from this joint distribution and for each h ∈ Icd˜, estimate xd,h ˜ by the conditional expectation conditioned on exceedance of the boundary of generation production. For example, if the associated αd,h ˜ = 0, the estimate is the expected value of −βd,h ˜ −µh (βd,h

˜ ) the simulations conditioned on zd,h ˜ < σh (βd,h ˜ ) , this event corresponding to αd,h

˜ = 0. A

similar condition pertains to the upper boundary.

4. With a complete set of zd,cot

˜ values in hand for the entire dataset, use a standard normal ˜ . copula by hour to compute normal deviates z̃d,·

4.4 Simulation

The last piece of the platform concerns generation of i.i.d. joint scenarios. Given any target date and a respective asset-level hourly forecast, the platform can output an arbitrary number of joint scenarios. This is done by using the correlation clusters from step (vi) and the historical calibration from step (v). The simulations are generated one by one (vectorized in our code) and start in the Gaussianized space via conditional normals. At each level of the cluster hierarchy, we have the

cluster centroid’s deviates being passed down and the cluster-level deviates being then generated from a conditional normal formula. Then we finally reverse the calibration steps to obtain quantities in terms of MWh. More precisely, we nucleate a multivariate normal sample E (0) for the top-level deviate and then recursively simulate deviates for level-` clusters conditional on their “parent” E (`−1) normal deviates. This is done by conditioning: inputting the already generated E (`−1) of the cluster

delegate and inferring the rest of the cluster deviates via a conditional Gaussian sampling. Iteration continues to the bottom-most asset level, which yields a complete set of sampled Gaussian z’s for each asset and each hour of the day. In the final step, these are inverted through the conditional mean and variance transformations and merged with the forecast fh to yield production simulations in MWh: ([µh (fh ) + σh (fh )E (L) ] · Mh + fh ) ∨ 0 ∧ gd,h max .

5 Results from a Case Study

In this section we illustrate our simulation platform with the results from the NREL Proposed testbed. As a start, we generate 10 scenarios for the fixed day of April 12, 2018. As explained, the scenarios are joint across the 4 assets, and can be understood as 10 counter-factual realizations of renewable generation on that day, conditional on the given forecast. Since our model captures the cross-asset correlations, one can examine the simulations at any level of generation granularity—

marginal at each asset, aggregated by a geographic region, aggregated at a zonal level, or across the entire grid. The latter options just require summing up across asset subsets by scenario. Similarly, since the model captures the temporal correlations across hours, we can sum up across periods to obtain scenarios for aggregated daily generation, etc.

Figure 1 shows the hourly-based view of single asset simulations, for a solar (Castro) and a

wind asset (Aguayo) respectively. The Figure shows the forecast fh , the realization gh and the mean simulated generation mh = Ave(gh1:10 ) based on 10 scenarios gh1:10 . We also show the 95%-scenario band, obtained by sorting for each hour ghi and then saving the respective 2.5% and 97.5% quantiles qhα := (gh1:10 ). We observe that mh 6= fh per the de-biasing calibration based on µh (·); we also observe the non-constant standard deviation σ̂h = StDev(gh1:10 ) that reflects the

non-constant conditional variance σh (β) and the temporal pattern of f1:2 .

As mentioned, there are often non-zero probabilities of zero or maximum generation, for example

on the right panel of Figure 1 maximum wind production has about 50-69% chance of occurrence in the late evening,

Figure 1 in the Appendix repeats the above for the aggregated generation in the Far West

zone that contains 4 solar assets and 6 wind assets. We observe some partial diversification with tighter relative uncertainty bands, but still quite a bit of variability at the zonal level. Figure 1

Figure 10: Representative asset-level scenarios. Left: Solar Asset (Agate) with 2 daily scenarios.

Right: Wind Asset (Ajax). Numbers indicate the probability of the point masses for maximum

wind generation. All simulations are calibrated to 04/12/2018.

in the Appendix shows the realized intra-day hourly correlations for representative solar and wind assets. We observe decorrelation taking hold after 4-6 hours.

5.1 Probabilistic Assessment

To assess the generated simulations, the main tool is to compare the (randomly sampled) scenario distribution Ft (·) to the realized actual gt . This implies defining a loss metric D(F, g) and then averaging it across a set of test hours or day [LDP19, WDS+ 18, RSWW20, ZS18]. The metric judges the closeness of the realized actual to the distribution; the averaging is necessary to draw statistics about the distribution of the forecast fitness. We note that the oft-mentioned nonstationarity

means that each F and g come from different underlying distributions, hence the averaging is also important to average out performance across different potential settings (i.e. months of the year, different weather patterns, etc). To that extent, one should not draw any conclusions from performance on a given test instance, and focus on aggregate performance. The Probability Integral Transform (PIT) looks at the realized percentile, Ft (gt ). Under the hypothesis that F perfectly captures the actuals, Ft (gt ) ∼ U (0, 1) should be uniform, hence one may

test the uniformity of the PIT. More locally, one may test statistical coverage, i.e. the frequency that a particular range of percentiles appear; this is especially relevant for checking the tails, for example how frequently are the actuals far from the bulk of the simulations (i.e. yield extreme percentiles close to zero or to 1). Figure 1 shows summaries of coverage for the 10% and 90% quantiles across all the assets. Since the nominal coverage level is chosen to be 10% in both cases,

under a perfect statistical fit, the exceedance probabilities would be close to 10% in all 4 panels. We observe that this is indeed so in 3 of the 4 panels; for some solar assets we observe a higher frequency of very high generation (i.e. scenarios underestimating the probability of sunny conditions).

More generally, one may consider the so-called strict scoring rules [GR07, GBR07, GK14], such

as the Continuous Ranked Probability Score (CRPS) for univariate assessment and the Energy Score (ES) for multivariate assessment.

Figure 1 shows the PIT histograms for the aggregated zonal wind production for each of the

th th solar 1 Percentile Exceedance By Asset (Hour: 12) wind 1 Percentile Exceedance By Asset (Hour: 12) 0.1 0.1

0.0 0.0

0 0 0 5 1 1 2 0 5 1 1 2 2

solar 9 th Percentile Exceedance By Asset wind 90th Percentile Exceedance By Asset 0.4 0.1 0.3 0.0

0.2 0.0

0 0 0 5 1 1 2 0 5 1 1 2 2

Figure 11: Summary of scenario coverage probabilities by asset. Left: 2 solar assets. PRight: 2

wind assets. Both panels are for noon h = 1 across the entire year 2018. We show d=1 1{ gd < q0.1 } (top) and 3 P d=1 1{ gd > q0.9 } (bottom), i.e. the frequency of the actuals being in the left/right tail of the scenario distribution.

8 ERCOT zones. We observe that the histograms are very close to uniform except for some ex-

ceedances at the lowest decile (in other words, there is more than expected frequency of days/hours where actuals are substantially below all scenarios). This is a material consistency check since the distribution of the sum is sensitive to the correlation structure, and moreover zonal production is important for SCUC and SCED.

Figure 12: PIT histograms for zonal wind generation. We show the percentiles for the 3 days in

April 20 and all 2 hours of the day (7 total hours).

6 Conclusion

The presented platform provides a novel framework for generating day-ahead scenarios for short-

term grid operational planning. The outputted simulations can be used for uncertainty quan- tification at multiple stages of the daily ISO tasks: for stochastic optimization during security- constrained unit commitment, for risk indexing to rank renewable assets (which otherwise all have zero marginal generation costs), for risk planning to anticipate reserves needs, etc. Moreover, our probabilistic framework is amenable to additional extensions. For example, we have implemented

an extension for intra-day simulations: generating hourly scenarios for 6-hour blocks conditioned on T − 6 forecast updates. Since the methodology is agnostic to many of the empirical features, it can also be modified to provide scenarios at the sub-hourly scale (e.g. 15- or 5-min intervals). The method can also be applied modulo minor adjustments to load modeling; the latter is nowa- days highly stochastic due to large amounts of behind-the-meter rooftop photovoltaic panels which

induce correlation between load and solar.

Several aspects of our model warrant further investigation, especially in terms of the correlation

modeling. The approach in Section 3 works solely with the empirical normalized deviates, and is not aware of any spatial structure. Given that generation is weather-driven, incorporation of spatial constraints, for instance to ensure high correlation of closely located assets could be con- sidered. Similarly, our approach concentrates on creating a hierarchical structure of the covariance matrix; other regularization, such as sparse precision matrices are alternatives to be analyzed. On

the calibration side, all described calibration steps are currently done asset-by-asset. Information fusion, for example to improve estimation of hourly maximum or mean generation via a Bayesian framework, could be beneficial. We continue to actively develop the platform and several of the above extensions will be addressed in subsequent articles.

Acknowledgements:

Both authors are partially supported via the ARPA-E PERFORM grant DE-AR0001289. We are

thankful to Rene Carmona, Xinshuo Yang, Arvind Shrivats and Mahashweta Patra for many useful discussions. AS and MP have also contributed some of the code for probabilistic assessment. We also thank ARPA-E and the PERFORM D