Effects of regulating the European Internal Market on the integration of variable renewable energy

The new proposal for regulating the European Internal Market for Electricity (EIME) can motivate the harmonization of the various National markets. The process of harmonizing the day‐ahead markets (DAMs) is at an advanced stage, with an efficiency in the use of interconnectors of 86%. However, the harmonization of both intraday (IDMs) and balancing markets (BMs) is still in its infancy, with an efficiency in the use of interconnectors of 50 and 19%, respectively. The new proposal brings new targets to DAMs, and European countries should make efforts to comply with them. The same is true for IDMs and BMs, but involving more ambitious targets, requiring higher efforts to be accomplished. Both the analysis of the various National markets (according to their compliance with the new proposal for regulating the EIME) and the advantages of the new proposal for key market participants (particularly, consumers, variable renewable generation, and conventional generation) are presented. The analysis indicates that the proposal contributes to a potential increase of the general welfare of market participants. However, some aspects of the proposal can negatively affect the revenue obtained from the National markets, notably for variable renewable generation and conventional generation.

of markets, and also sending relevant price signals for investment in more interconnection capacity. The efficiency of this mechanism is indicated by an increase in the price convergence between different market areas.
In the Nordic region of Europe (12 countries), the convergence of market prices is quite significant (30%), but in remaining European regions this convergence is lower (ACER, 2017a). However, for the particular case of 2011, the price convergence between Belgium and France was around 90%, and the same is true for Germany and the Netherlands, as well as Slovakia and Czech Republic, and Portugal and Spain (Böckers, Haucap, & Heimeshoff, 2013).
The level of harmonization of the European DAMs is large, but most countries are far from complying with the new EC proposal. Table 1 shows the main differences between several EU countries according to their compliance with the new proposal. The table indicates that almost all analyzed countries comply with some articles of the proposal. However, some countries should make efforts to give balance responsibility for RES in order to be nondiscriminatory, and also increase the incentives to minimize imbalances. In addition, Italy, Portugal, and Spain should allow aggregated bidding in their DAMs, since flexible aggregated demand and VRE are typically beneficial, namely by reducing forecast errors (Ayón, Gruber, Hayes, Usaola, & Prodanovic, 2017). In the short-term, all countries should make efforts to decrease the market time unit up to 15 min, to comply with the imbalance settlement period, and should also not impose price caps on markets. The minimum bid size in all selected countries is 0.1 MW, complying with Art. 6,par. 2a,and with Art. 7,par. 3. 2 All selected countries have already market coupling, in accordance with Art. 6, par. 1, 3 and thus also comply with par. 2b (EC, 2017a). We note that cooperation and market coupling are important aspects toward increasing the level of efficiency of power systems, notably the economic welfare of consumers.
Overall, the level of harmonization of the different European DAMs is currently quite satisfactory, despite the changes needed to comply with the new proposal. Also, the level of efficiency in the use of interconnectors is high (see Figure 1). Art. 4 has three key paragraphs. The first paragraph defines that all market participants have to be financially responsible for imbalances, with only a few restrictions, specified in the second and third paragraphs. d According to EC (2017a), the first three points of the second paragraph of Art. 6 considers that "Day-head and intraday markets shall: (a) be organised in such a way as to be non-discriminatory; (b) maximise the ability of market participants to contribute to avoid system imbalances; (c) maximise the opportunities for market participants to participate in cross-border trade as close as possible to real time across all bidding zones." (p. 42). The third paragraph defines that market operators are free to develop new market products to increase the participation of the demand side and VRE (individually or through aggregation). e The second paragraph of Art. 7 refers to a full harmonization of day-ahead and intraday markets in relation to the time intervals of trading and the settlement period of imbalances. The fourth paragraph refers that the imbalance settlement period shall be 15 min in all control areas on January 1, 2025. f Art. 9, paras. 1 and 2, impose no price caps on markets.

| Intraday markets
At present, IDMs across Europe present several differences and raise several problems, notably differences related to the type of trading (auction or continuous trade) and problems associated with low liquidity (but see Table 2). Almost all countries adopted a gate closure close to real-time, but the market time unit of 15 min is only considered in two countries (namely, Germany and the Netherlands). Furthermore, for the various aspects of the new EC proposal considered in this work, the harmonization of IDMs is at an equal or lower stage of development than that of the DAMs (see Table 2). This can also be observed in the efficiency level in the use of interconnectors, which is 50% for IDMs and 86% for DAMs (recall Figure 1). Although the liquidity level of IDMs did not increase in a significant way during the past few years, the level of efficiency in the use of interconnectors increased significantly.
Accordingly, we consider that the new EC proposal should be complemented with rules related to auctions and exclusive IDMs in order to increase the liquidity and harmonization of markets. Also, European countries should make efforts to integrate all cross-border capacity in their markets and harmonize IDMs in terms of the following: type (auction or continuous  trade), number of sessions (in case of auctions), gate closure, market time unit and type of products. Furthermore, we note that Neuhoff, Ritter, Salah-Abou-El-Enien, and Vassilopoulos (2016) concluded that the implementation of a 15-min auction in Germany doubled the liquidity of the IDM, in comparison with a continuous trade. The ratio between the quantity traded in IDMs and the electricity demand, during the period 2011-2016, was nearly 15% for Spain, 10% for Italy and Portugal, 7% for Great Britain, and less than 5% for the other countries analyzed. Thus, due to the existence of alternatives to IDMs, such as short-run bilateral contracts, the pan-European harmonization of IDMs is still in its infancy, and RES participation is quite reduced (ACER, 2017a). As noted for DAMs, some IDMs do not allow aggregated bidding nor give balance responsibility for RES, and thus are far from complying with the new EC proposal.
Spain, Italy, and Portugal consider auctions and do not allow aggregated bidding, and IDMs are mandatory (i.e., bilateral contracts are no alternative to IDMs), meaning that such markets have high liquidity (see Tables 1 and 2). This can be explained by the fact that IDMs are often used by market participants to correct the expected deviations from both VRE and the demand-side. Figure 2 shows the prices of DAMs and IDMs (in the first two sessions, i.e., IDM-S1 and IDM-S2) in Spain, Italy, and Germany, during the period 2014-2016. The prices of the DAMs in Spain and Italy were higher than the prices of the IDMs. However, for Germany, the price of the energy traded in the IDM using auctions was lower than the price of continuous trade, and both prices were higher than the price of the DAM. This can be explained, at least in part, by the fact that Spain and Italy do not allow aggregated bidding (thus increasing imbalances), and also by taking into account that IDMs are mandatory in these countries, increasing the RES penetration and market liquidity (ACER, 2017a).

| Balancing markets
BMs are mandatory and imposed by the European Network of transmission system operators (ENTSO-E). Operationally, each TSO has the responsibility of guaranteeing the power reserve values for BMs within the control zone, based on ENTSO-E requirements (e.g., the reserved capacity for the frequency control reserve is 3,000 MW in Continental Europe). 4 In Europe, there are three key types of load-frequency control products negotiated in BMs, namely frequency control reserve (FCR), automatic-activated frequency restoration reserve (aFRR) and manually activated FRR (mFRR).
During real-time operation, primary reserve or FCR is the first to be activated, after grid disturbances, incidents or imbalances between production and consumption that result in frequency oscillations. It must be activated up to 15 s and the disturbances need to be controlled in 30 s. In some European countries, FCR is a mandatory and nonremunerated system service for all generators connected to the grid, who have technical capability for fast response. They need to reserve 5% of their nominal power to FCR. However, in the Nordic countries, FCR is a nonmandatory remunerated system service. Secondary reserve or aFRR needs to be fully activated in 30 s, replacing FCR. It can be maintained active for a maximum of 15 min. Tertiary reserve or mFRR needs be fully activated within 15 min, and can continue active for hours, freeing up FCR and aFRR.
Art. 5 of the new EC proposal refers to BMs. 5 This article is complemented with the recent Electricity Balancing Guideline (EC, 2017b), which has the objectives of increasing the security of supply, limiting emissions, and diminishing the costs to customers (e.g., allowing the participation of demand response and renewables in BMs). The efficient exchange of balancing services is the principal component of the Electricity Balancing Guideline (EBG), but it is only observed in Norway, Sweden, Finland, and Eastern Denmark, countries that share the same balancing system (ENTSO-E, 2016). Currently, the major challenges to implement the EBG are related to a full harmonization of pricing mechanisms, techniques for procuring balancing energy and methods for defining imbalance prices. The urgency should be given to improve the elements of the design of BMs that inhibit (in some way) a free fluctuation of prices for balancing energy, such as an ineffective trade of balancing capacity and regulated prices and pricing mechanisms not founded on marginal pricing, which decrease the motivation of producers and the demand-side to answer to instantaneous balancing needs.
In EU-28, there are four different methods for procuring balancing energy from aFRR markets: • pay as bid (adopted by eight countries); • marginal pricing (eight countries); • regulated price (three countries); and • hybrid (five countries).
There are also seven different options for procuring balancing energy from mFFR markets: • mandatory offers (adopted by four countries); • mandatory provision (two countries); • precontracted offers (four countries); • precontracted offers and mandatory offers (France); • precontracted and free offer (four countries); • bilateral market (two countries); and • organized market (nine countries).
All such different methods have led to a low harmonization of BMs, including a low efficiency in the use of the crossborder capacity for exchanging balancing services and a large discrepancy between balancing costs and imbalance prices (ACER, 2017a). Also, the method to define imbalance prices is different in several countries . Accordingly, we consider that there is a need to define a general methodology for the different countries participating in the EIME.
Currently, the discrepancy between the techniques adopted by European countries for procuring balancing energy in aFRR and mFRR markets is large. This situation leads to substantial differences in the overall costs of balancing services and imbalance prices, varying between 0.5 €/MWh (in Germany) and 6 €/MWh (in Slovakia), and also contributing to a low harmonization of BMs (ACER, 2017a). This, in turn, leads to a reduced efficiency in the use of the interconnectors for balancing energy (recall Figure 1). Thus, as described by Morch et al. (2016) the lack of progress observed in the integration of BMs can be explained by the specific characteristics of the National markets in conjunction with their importance to TSOs, in order to obtain a reliable and stable operation of the power systems.
Overall, to comply with the new EC proposal and the EBG, an important aspect that should be adopted by all markets consists in allowing the participation of VRE in BMs. Spain, Germany, and Denmark already allow the participation of VRE in traditional BMs, with some restrictions to guarantee the security of power systems, thus complying with the first and second paragraphs of Art. 5.

| THE INTERNAL MARKET FOR ELECTRICITY: POTENTIAL IMPACT OF THE NEW EC PROPOSAL
The articles of the new proposal can benefit power systems and consumers by increasing market efficiency and decreasing market prices, therefore contributing to a market harmonization. However, from the point of view of VRE and conventional generation, some articles can reduce the remuneration of producers (see Table 3).
Both conventional generation units (properly equipped to manage their active power output) and VRE producers trade the energy required by the demand-side in DAMs and IDMs (as well as through physical bilateral contracts). In these markets, if the energy delivered differs from the scheduling value, the difference is counterbalanced in BMs, which results in the payment of imbalance prices. Mills and Wiser (2014) classified the economic value of wind as the sum of four components: the market value of energy, the capacity value, the forecast error and the ancillary services values. It is worth noting that the market value of energy is (usually) positive, but the forecast error and the ancillary services values are (usually) negative. The balance responsibility for RES may increase the costs associated with the forecast errors. However, such an increase may be compensated either by strategic bidding (increasing the revenue from markets) or by participating in BMs (increasing the revenue of VRE and reducing the payment of imbalances) (Algarvio, Lopes, Couto, & Estanqueiro, 2019; Martín-Martínez, Lorenzo-Bonache, Honrubia-Escribano, Cañas-Carretón, & Gómez-Lázaro, 2018; Skytte & Bobo, 2018). Also, the balance responsibility for RES is one of the proposed nondiscriminatory actions that can benefit European consumers.
The participation of VRE in BMs can reduce system imbalances, which in turn can increase VRE remuneration in the following ways: (i) by reducing penalties and (ii) by being remunerated from BMs. However, the participation of VRE in BMs can contribute to an increase of market prices, particularly when VRE is not aggregated with conventional generation, due to a potential waste of energy when VRE needs to comply with a schedule (see Algarvio  where WPPs can participate in the BMs of Denmark, obtaining an increase of 6.5% in the wind energy value. Great Britain (GB) has also allowed WPPs to participate in some balancing services that are different from traditional BMs, specifically "Manage Constraint" and "Rebalance System" (only breakdown of constraint), receiving 40% more to curtail energy than to produce it (National Grid, 2018;REF, 2018). And Belgium has studied this aspect in relation to downward aFRR, with a reliability in the service quality above 90% (WindVision et al., 2015). Furthermore, several studies were performed to investigate the technical capability of WPPs to provide balancing services, analyzing several aspects, notably security, quality of service and economic outcomes (Attya, Dominguez-Garcia, & Anaya-Lara, 2018).
Overall, the participation of VRE in BMs may increase competition and to some extent may reduce the need of the market due to a reduction of VRE imbalances, which may decrease market prices, as well as the conventional generation participation and remuneration.
The implicit allocation of the cross-border capacity may increase the efficiency of power systems, improving the harmonization of EMs and increasing the transmission capacity between borders (ACER, 2017a; Nordpool, 2018). Also, large interconnected power systems may benefit the integration of VRE, since a large geographical dispersion typically reduces the total variability and uncertainty of forecasts. Furthermore, these measures may contribute to the harmonization of market prices and increase the efficient use of conventional generation, specially by increasing the participation of the more cost-effective power plants Holttinen, Miettinen, & Sillanpää, 2013). These actions can also benefit consumers by reducing market prices and decreasing both the use of incentives to VRE and the capacity needs of conventional generation, which can be reflected in the tariffs Burgholzer et al., 2016).
Aggregated and individual bidding in spot markets (limited, due to grid constraints) is already followed by almost all European countries, with the exception of Italy, Portugal, and Spain. The spatial aggregation of different VRE sources is typically advantageous to minimize forecast errors (Miettinen, Holttinen, & Giebel, 2014). And this is mainly true for the particular case of WPPs, since the aggregation methodologies are related to the reduction of wind power fluctuations, by taking into consideration the spatial variability of wind. From the point of view of the demand-side, aggregated bidding is also advantageous, especially when using demand-side management or demand response programs (Algarvio et al., 2015;Ayón et al., 2017;Lopes & Algarvio, 2018). Overall, the implicit allocation of the cross-border capacity and the possible aggregation of VRE (and/or other sources) can contribute to the development of regional forecasts that may cover large areas of Europe, which can reduce forecast errors and increase the VRE value to the market.
Apparently, the issue of adjusting the gate closure of the DAM to a time closer to real-time was deferred by EC, despite its importance in markets with increasing levels of VRE Vilm & Botterud, 2014). This importance is related to the forecasts of WPPs, which may have large errors for larger time horizons Vasilj, Sarajcev, & Jakus, 2015). A study conducted for the particular case of Spain , taking into account the extra dispatchable costs with thermal units, indicated that the adjustment of the gate closure is only advantageous for reductions in forecast errors around 30% (for wind power) and 75% (for solar power). Also, for wind power, only forecasts around 6 hrs ahead may reduce the error in more than 30%. Another study, performed for the Finish power system (Holttinen, Miettinen, & Sillanpää, 2013), highlighted the advantages of allowing aggregated bidding and postponing the gate closure of the DAM for minimizing the forecast errors of WPPs. It also revealed that aggregated solutions are more effective to reduce forecast uncertainty than reducing the forecast horizon, which only begins to be relevant around 6 hrs ahead (or less). Also, Miettinen et al. (2014) showed that the mean absolute error of wind power forecasts can be lower than 3% when all Nordic areas are aggregated, while in a smaller area the error can be higher than 10%. Liu, Gao, Yan, Han, and Infield (2014) tested several clustering techniques to improve wind power forecasts for WPPs. The results showed that some techniques reduced errors in nearly 1.5% (when compared to forecasts based on a single wind speed or power). Giebel, Sorensen, and Holttinen (2007) showed that the correlation of forecast errors for the DAM decrease exponentially as the distance between wind parks increases. Also, some authors proposed the aggregation of WPPs in order to submit strategic bids to EMs (see, e.g., Algarvio, Couto, Lopes, Estanqueiro, & Santana, 2017;Sharma, Bhakar, & Tiwari, 2014;Zhang, Johari, & Rajagopal, 2015).
Overall, in relation to DAMs, a gate closure closer to real-time is advantageous to VRE, especially to WPPs. It seems to be, however, disadvantageous to power systems highly dependent of coal power plants that need around 6-8 hrs (of planning) ahead to schedule their commitments (if not retrofitted for flexibility). But some markets with flexible generation (and not so dependent of coal power plants), such as the Nordic power market, may clearly benefit with this measure.
The standardization of the market time unit to 15 min is essentially an issue of uniformity and clarity of operation, since it may not be considered a good procedure to operate with 1-hr bids and imbalance volumes of 15 min (EC, 2017a). This measure may be beneficial to VRE producers, by enabling them to perform more accurate forecasts (VRE is more uncertain and volatile in an 1-hr time period than in a 15-min time period). However, it may be disadvantageous to coal power plants that need to change production every 15 min, since their flexibility and ramp rate is limited.
Other aspects of the new EC proposal consider removing barriers to demand response as well as the possibility to increase the effectiveness of short-term markets by bringing them closer to real-time. The latter, particularly the case of bringing congestion and re-dispatch markets closer to real-time, may be considered an opportunity to meet flexibility needs and increase the effectiveness of BMs (EC, 2016a;EEX & WindEurope, 2018).

| CONCLUSION
The harmonization of DAMs is already at an advanced stage, with an efficient use of interconnectors. However, there are several issues that should be addressed more thoroughly. Specifically, most countries should make efforts to: • consider bids for periods of 15 min (full harmonization with the imbalance settlement period); • give balance responsibility for RES (as a nondiscriminatory proposal); • allow aggregated bidding (as an action to reduce the VRE and demand-side imbalances); and • remove (or not impose) price caps.
The harmonization of IDMs is also reasonable, but there are still significant differences between the various markets, reflected on the level of efficiency of the interconnectors (50%). A key problem for IDMs is the lack of liquidity, mainly because most of them are not exclusive. Like DAMs, there are some questions for IDMs that need to be addressed more thoroughly, particularly in relation to the balance responsibility for RES, aggregated bidding, a market time unit of 15 min and the removal of price caps. Currently, almost all IDMs consider a gate closure close to real time, but should take into account the allocation of all cross-border capacity.
In relation to BMs, the harmonization is at an early stage of development, where only a few countries exchange balancing services, resulting in an efficient use of the interconnectors of only 19%. This occurs because there are significant discrepancies between countries, mainly by considering different methodologies for procuring energy activated from both aFRR and mFRR markets, leading to substantial differences in the overall cost of balancing services, and also in imbalance prices between European countries. Accordingly, it is important to improve (in some way) the elements of the design of BMs that inhibit a free fluctuation of prices for balancing energy, such as an ineffective trade of balancing capacity, and regulated prices and pricing mechanisms not founded on marginal pricing, which decrease the incentives for producers and the demand-side to answer to instantaneous balancing requirements.
The analysis of the impacts of the new EC proposal from the point of view of VRE, conventional generation, and power systems has highlighted the following six aspects that can potentially increase the VRE integration and the general welfare of European consumers: balance responsibility for RES, participation of VRE producers in BMs, implicit allocation of crossborder capacity, aggregated bidding, a gate closure closer to real-time operation and markets with a time unit up to 15 min. For the particular case of VRE producers, we highlight that "balance responsibility" could potentially lead to a reduction of their remuneration. However, the other aspects analyzed can positively affect their social welfare, notably: • The participation in BMs can increase remuneration and decrease imbalances; • The implicit allocation of the cross-border capacity can contribute for regional forecasts of VRE, reducing forecast errors; • Aggregated bidding can reduce forecast errors by contributing for different spatial forecasts of VRE; • Markets with a gate closure closer to real-time operation can increase the forecast accuracy of VRE producers, since VREs have large errors for larger time horizons; and • Markets with a time unit up to 15 min can give the flexibility that VRE needs, contributing for a potential reduction of the imbalances of VRE producers.
Regarding the European consumers and power systems generally, we also highlight a number of aspects. The implicit allocation of the cross-border capacity, aggregated bidding and a market time unit up to 15 min are aspects that can improve the effectiveness of power systems, by increasing the use of interconnectors (which, in turn, may increase the use of the most cost-effective power plants in Europe), and decreasing forecast errors (and consequently the requirements of BMs, lowering their prices as well as the imbalance prices).
Giving balance responsibility for RES and allowing the participation of VRE in BMs are two nondiscriminatory aspects that can benefit European consumers. So, by start paying penalties (balance responsibility), the remuneration of VRE can decrease. However, the participation of VRE in BMs can reduce system imbalances and increase competition, which in turn can increase VRE remuneration and decrease the use of conventional generation. These aspects can also benefit consumers by reducing the market prices of electricity and decreasing both the use of incentives to VRE and the capacity needs of conventional generation, which can be reflected in tariffs (Algarvio, Lopes, Sousa, & Lagarto, 2017).
The adjustment of the gate closure of the DAM to a time closer to real time seems to be deferred by EC. However, while VRE can be positively affected by this aspect, by increasing forecast accuracy, the slower technologies (e.g., coal) that need around 6-8 hrs of planning (if not retrofitted) can be negatively affected. This aspect can increase the operating costs of the system and is only advantageous for a gate closure 6 hrs ahead or less (with no participation of slower technologies).
Changing the market time unit to 15 min can also be problematic to slower technologies, since they may need to change their production every 15 min, and the low ramp rates limit their flexibility. These measures can be advantageous for systems with more flexible generation and/or no dependency of slower technologies. However, taking into account the decarbonization measures, a possible increase in the carbon tax and the retirement of most coal power plants, it is expected that most countries reduce their dependency from coal power plants, and thus such measures may become favorable to all countries of EU-28.

CONFLICT OF INTEREST
The authors have declared no conflicts of interest for this article. Portugal, Spain, France, Italy, Slovenia, Great Britain, Luxembourg, Belgium, the Netherlands, Germany, Austria, the Czech Republic, Slovakia,Hungary,Denmark,Norway,Sweden,Finland,Latvia,Estonia,Lithuania,Poland,and Romania. 2 The third paragraph of Art. 7 mentions that the products provided by MOs shall have a minimum bid size of 1 MW in order to allow the effective participation of demand-side response, energy storage and small-scale renewables. 3 The first paragraph of Art. 6 of EC (2017a) states that "Transmission system operators and nominated electricity market operators shall jointly organize the management of the integrated day-ahead and intraday markets based on market coupling as set out in Regulation (EU) 2015/1222. Transmission system operators … shall cooperate at Union level or, where more appropriate, on a regional basis in order to maximise the efficiency and effectiveness of Union electricity day-ahead and intraday trading … In their functions relating to electricity trading, transmission system operators and nominated market operators shall be subject to regulatory oversight by regulators and the Agency pursuant to Article 59 of [recast of Directive 2009/72/EC as proposed by COM (2016)  The Union for the Coordination of the Transmission of Electricity synchronous area. 5 Art. 5 has six key paragraphs. The first paragraph states that all market players must have access to BMs. The second paragraph states that an effective nondiscrimination between participants should be ensured. The third paragraph specifies the need to separate procurement between balancing energy and capacity. The fourth paragraph incentives the optimal allocation of the cross-zonal capacity. The fifth paragraph considers the exclusive use of the marginal pricing methodology. The sixth paragraph specifies that the settlement of imbalance prices should reflect the real-time price of electricity.