Europe’s energy sector is undergoing an unprecedented transition. It is gearing up to meet its clean energy goal of reaching net-zero carbon emissions by 2050. Moreover, by 2030, over 50 per cent of electricity demand in the pan-European power system is targeted to be met by renewable energy sources (RES). The majority of this renewable power will be added through variable renewable sources, namely wind and solar. It is estimated that across Europe, over 350 GW of wind and 285 GW of solar power is required to meet the 2030 targets. Given the amount of distributed variable RES (VRES) generation, the need for flexibility and balancing is more than ever. Additionally, power demand is expected to increase dramatically in the coming decades from the present 20 per cent of overall European energy use to more than 40 per cent of energy needs by 2050. This increase in demand will be driven by the use of electric vehicles (EVs), along with industrial and domestic heating. This increased demand will also lead to system constraints. The simultaneous increase in variable power generation as well as electricity demand will require suitable technologies and market mechanisms to maintain a reliable, balanced and secure system.

Against this background, the recently concluded EU-SysFlex project funded by the European Union’s (EU) Research and Innovation Programme Horizon 2020 aims to address system operation and flexibility challenges associated with EU’s goal of integrating 50 per cent renewables by 2030. The EU-SysFlex consortium consisted of 34 organisations from 15 European countries, including electricity transmission system operators (TSOs), distribution system operators (DSOs), technology providers, manufacturers, universities and research centres. Launched in November 2017, the EUR26.5 million project concluded in February 2022 (after four years and four months) with the release of the European Power System Flexibility Roadmap, the final deliverable of the project. The research initiative, which received EUR20.5 million or 77.4 per cent funding from Horizon 2020, identified key flexibility issues and solutions associated with integrating large-scale renewable energy.

The basis of the project is that the transformation of the European system to incorporate a high share renewable energy can be better realised by integrating three elements, namely, a  better understanding of the technical shortfalls; adopting technologies and innovative approaches to address these system shortfalls; and finally, responding to these technical scarcities with an approach based on the provision of services supported by markets, rather than promoting a specific technology.

EU-SysFlex provided this by demonstrating different business use cases in seven field tests at all system levels and across Europe: Portugal, Germany, Italy, Finland, France, Poland and Estonia as well as a qualification trial process in Ireland and Northern Ireland.

Based on these demonstrations and trials, the final roadmap contains eight key recommendations/findings, i.e., technical scarcities, required flexibility, financial gaps, market evolution, new operating tools, TSO-DSO coordination, aggregation of decentralised resources, and data management.

Figure 1: Key recommendations of EU-SysFlex

Source: European Power System Flexibility Roadmap 2022

Technical scarcities

The high share of variable renewables has given rise to technical scarcities in the power system. While these scarcities existed in the system even with conventional power generation, the non-synchronous nature of wind and solar has accentuated these scarcities.

The project examines three different synchronous areas (Nordic, Continental and Island of Ireland System) with increasing levels of non-synchronous generation penetration. It was established that the increasing levels of non-synchronous generation, and consequently reduced numbers of synchronous machines on the system, lead to technical scarcities in the system. The project identifies inertia, frequency containment, steady state voltage, dynamic voltage, critical clearing times, rotor angle margin, oscillation damping and system congestion as some of the technical issues that need to be resolved.

Further, the EU-Sysflex project has predicted that these scarcities will increase at a rapid pace in future with the rise in demand. The project lays out two core scenarios for the pan-European system. One foresees energy transition with 52 per cent renewable energy while the other is the renewable ambition with 66 per cent renewable energy.

Evidence from the project suggests that the Continental European system is likely to experience more issues of concern and technical scarcities in key system support capabilities as it evolves with higher levels of variable non-synchronous generation.

Figure 2: Summary of the technical scarcities identified for the three synchronous areas

Source: European Power System Flexibility Roadmap 2022

Required flexibility

Another key finding of the project is that system services capabilities need a variety of technologies to effectively mitigate the technical scarcities identified above. New resources, such as variable renewable technologies, energy storage and demand-side response can offer the required system flexibility.

Network technologies, such as synchronous condensers, static synchronous compensators (STATCOM), static VAr compensators, as well as renewable generation technologies, such as wind and solar generation, along with batteries and demand-side management, are suitable technologies for mitigating a range of scarcities caused due to high levels of renewable generation.

The required mix of solutions will have to be assessed holistically considering the trade-offs and synergies. This is because some scarcities can be mitigated by a range of different technologies and strategies, while some technologies are more effective in mitigating a selection of different issues. The key will be to identify the mix of technologies needed to ensure the reliability, stability and resilience of the power system while delivering value to consumers. One solution is to develop regulation and electricity market designs that incentivise investment in suitable technologies, providing choice, and treating all relevant technologies on a level playing field

During the course of the project, several technologies such as battery energy storage systems (BESS), wind farms, and a combination of wind farms and DSO capacitor banks were implemented in the Finnish, French and Portuguese Flexibility Hub (FlexHub) demonstrations.

The field experiments showed that these technologies can maintain a safe and flexible power system and that the capabilities of system operators can be enabled through enhanced TSO-DSO coordination and platforms. Further, interconnections will be imperative in this regard so as to pool more cross-border resources for a more flexible system. A supportive regulatory environment will also be critical in implementing these measures.

Financial gaps

The project concluded that the required flexibility will not be guaranteed by the existing market structure, which was developed for conventional, centralised and high availability plants.

An analysis of the Continental, Nordic and Ireland and Northern Ireland power systems demonstrated that as the share of renewable energy increases, there is a decrease in average energy prices since, for many hours, renewable generation becomes the marginal generation. The fall in prices in an energy-only market is caused as the marginal cost of production of VRES such as wind and solar is almost zero due to their ‘free’ energy sources. A downside of the decreasing energy prices is that overall energy revenues will consequently decrease, resulting in a financial gap for many generation technologies.

Therefore, major incentives and additional revenue sources will need to be developed so that the required flexible and low-carbon technologies remain intact in the system. For instance, the case study of Ireland and Northern Ireland demonstrated that enhanced system services can provide a revenue stream to improve the financial viability of both VRES and conventional technologies.

This indicates that it is better to adopt a market-based approach rather than a regulatory approach to acquiring technology to address system scarcities.. New electricity market designs that incentivise investment in these technologies must be developed. However, EU-Sysflex does not entirely rule out the regulatory approach and suggests that a regulated approach may be required where challenges exist for implementing market-based solutions for the required system services.

Market evolution

As a solution for financial gaps and to increase revenues, the EU-Sysflex project calls for evolution of the present market. It vouches for newer market designs that incorporate a portfolio of flexibility products. The project suggests that the technical features of each system should be evaluated and a suitable market design should be framed for the same. Furthermore, it states that partial design improvements would also be worthwhile and as market designing is a dynamic process, not all changes to products, services and markets need to be introduced at the same time.

The EU-SysFlex project identified potential gaps in the products that are required to meet the future system needs. Based on this, the project has proposed new products including synchronous inertial response, fast post-fault active power recovery, dynamic reactive response, ramping products and congestion management products.

Importantly, the project recognises that there is no one-size-fits-all approach that can be applied across Europe. Hence, the technical characteristics of each system should be considered, and the design adapted accordingly. Also, it recommends close to real-time markets for better access to data and information. Such markets help flexibility service providers (FSPs) to develop more accurate forecasts of their available capacity and, as a result, they can participate in the market in a more efficient manner. That said, new and more complex tools are required to deal with the volume and frequency of data to process.

Several challenges also need to be factored in when framing new market designs. Some of these include methodologies to share cost and benefits between TSOs and between TSOs and DSOs; approach to incorporate grid limitations in the market design; and effect on prices of demand response and energy storage. Further, the market design must provide sufficient stable investment signals to ensure appropriate level of investments in generation and other flexibility sources.

Five key results from the demonstrations and trials were chosen for further assessment with their route-to-market analysed. These have been referred to as exploitable results (ERs). Based on the ERs analysed, the key emerging trends are as follows:

  • Promising opportunities will be available for technology/software developers to create value and gain profit in four of the five ERs analysed – virtual power plant (VPP), and traffic light qualification (TLQ) in Portugal, Betriebsführtungs und Energiemanagement Datenintegrationsplattform or operational and energy management data integration platform (BeeDIP) software for German congestion problems, and an interoperability solution in Europe.
  • All the five ERs indicate that it will be necessary to revise and change some existing regulations to facilitate commercial opportunities.
  • Some solutions such as the aggregation of industrial-scale BESS could be financially viable even in the current market structures.
  • In other instances, such as for TLQ in Portugal and congestion management in Germany, it is clear that value is created. However, there is a need to find stakeholders willing to manage coordination amongst all stakeholders involved and implement and provide solutions. In the case of TLQ in Portugal, it was identified that a business developer would be most likely to lead this role and solution deployment.

Overall, the demos showed that while some solutions could bring about commercial value to the project developer and other stakeholders in the business ecosystem, others focused on bringing improvements to the system as a whole and generating cost savings for some of the stakeholders. Though the demos were a success, the project calls for rolling out trials to fully evaluate their reliability and ability to provide flexibility in scenarios with high renewable energy generation.

New operating tools

Presently, majority of the system operator decisions are taken based on real-time observations of system representations in energy management systems fed by supervisory control and data acquisition (SCADA) data or state estimation. However, in future, with the coming of small, distributed, renewable generators on the power system, data volume will increase, making it difficult to take decisions. Also, system operations will become more complex and will require increased monitoring of a lot more parameters than are currently monitored, making the requirement of advanced energy management systems (EMS) critical. Further, forecasting, estimation and optimisation capability in these newer operator decision support tools will be required for future power systems.

Several demonstration projects were conducted as part of the EU-SysFlex project wherein operator decision support tools were implemented in field tests, and their performance was evaluated by offline simulation and real-time tests. These solutions have been evaluated by scalability and replicability analysis, verifying that they are scalable across large power system areas, and replicable from one country/area to the next.

Decision support tools from the system operator perspective were explored through three demonstrations – Germany, Italy and FlexHub in Portugal. In the German demonstration, two tools (IEE.NetOpt and PQ-Maps) were developed to help schedule preventive and corrective measures in congestion management and voltage control. The demonstration in Italy developed different tools, including a forecasting tool, state estimation (SE) tool, reactive power calculation tool and optimal power flow tool.  The FlexHub demonstration developed several tools to provide different services for the TSO. In particular, the TLQ tool provides a verification process to determine if activation of the physical resources corresponding to the bid could compromise secure operation of the distribution grid.

A set of aggregator or VPP tools was developed through the Finnish and Portuguese demonstrations (in the field) as well as the French VPP demonstrator (in a dedicated grid concept). In these demonstration projects, the required optimisation and forecasting tools, control and communication units, as well as the EMS were developed for the efficient performance of VPPs in real-time operation. These demonstrations will help optimise the contribution of BESS and EV charging stations to the ancillary services and energy markets.

In addition to this, works were carried out in EU-SysFlex towards a dispatcher training system and a qualification and trial process. A prototype decision support tool (DST) was developed and integrated on the integrated Polish TSO dispatcher training simulator (DTS). DST is built from a cross-border coordination module, and a dispatch and scheduling module using a security-constrained optimal power flow. DTS tools give TSOs and DSO dispatchers many opportunities to prepare for extreme operational situations when operators learn how to use new flexibility services, system-level special protection schemes, automation systems and decision support tools. Meanwhile, the qualification trial process (QTP) is the mechanism through which the TSOs in Ireland and Northern Ireland are managing the transition to a wider portfolio of system service providers. The QTP is a platform to trial services from new technologies and provides a route to an enduring flexibility products’ market.

Figure 3: Overview of different tools developed by demonstrations

Source: European Power System Flexibility Roadmap 2022

TSO-DSO coordination

Given the significant share of future generation that is expected to be connected to the distribution network, there is an increasing need for TSO-DSO coordination to utilise all solutions for congestion management and voltage control to ensure that operational security limits are not violated on both the TSO and DSO
networks. The project also acknowledged that operational challenges will arise as the use of distributed energy resources (DER) by different operators may lead to regional and national need-serving conflicts.  Therefore, stronger TSO-DSO coordination is essential for the mutual benefit of both. It investigated two coordination approaches (incremental and whole-system frameworks) to enhance TSO-DSO integration while enabling maximum use of DER.

As per the principle of the incremental coordination approach, the DSOs shall facilitate the available capacity of DER services to be offered to wholesale electricity markets and ensure that utilisation of the offered capacity does not violate distribution network constraints. Additionally, distribution network assets can be used to provide services to transmission.

The whole-system framework approach follows the principle of optimising the use of DER capacities for TSO and DSOs simultaneously and requires centralised electricity markets for all plant (including DERs) and centralised system operation. It requires a central entity that fully integrates TSO and DSO system operation to optimise the whole system. A key challenge in this approach is that it would not be compatible with the existing operational structure, whereby a TSO focuses on the national transmission system operation while DSOs operate distribution networks. The performance difference between both the approaches can be reduced if the appropriate information is shared between the parties, market operation is close to real-time and distribution network operation is quite flexible, by using smart grid technologies.

Cross-border coordination between TSOs was also investigated as part of the project and it was concluded that such coordination would reduce the overall need for balancing capacity, which would also result in significant cost savings.

Aggregation of decentralised resources

Aggregating multiple decentralised sources could help with better integration of the growing renewable energy influx in the power system. Aggregation would make it possible to balance supply and demand by relying on wind farms, solar farms, energy storage systems, EVs and heat pumps. The project gives aggregation in the form of VPPs as one of the solutions to prevent individual resource shortfalls through optimal use of multiple assets. For example, in normal circumstances, wind and solar photovoltaic (PV) resources can be used to provide downward reserves, and by pooling them with storage systems in a VPP, can be used to offer upward reserve also. Variable energy can also be used to charge storage batteries to avoid renewable curtailment or energy purchases from the grid. Therefore, the project gives VPPs as a solution to allow participation of renewables in the existing market products.

Aggregation would also lead to the adding up of a significant volume of system services from several small resources down to the residential level. It was concluded that low entry barriers in terms of setting-up costs, price, bid size, etc., and simultaneously empowering consumers to choose between different aggregators can help fully integrate these decentralised resources in the energy flexibility market. The ‘affordable tool’ for aggregators, developed through data exchange demonstration (led by Estonia), aims to bring active smaller customers to the energy market through the provision of flexibility services.

Data management

As discussed above, increased renewable penetration, aggregation of decentralised resources, new market designs and operator tools will result in a large amount of data, which will need to be managed efficiently. Interoperability is a key requirement for the future power system in which multiple players will handle and share large volumes of energy-related data.

Several demonstrators within the project proved the usability of the customer-centric approach based on the data exchange platform to exchange different types of data between stakeholders including cross-border communication between different data exchange platforms as well as across sectors.

It further suggests providing single points for data access to empower customers and give them choices between different application and energy efficiency services provided by energy service companies (ESCOs), and other emerging stakeholders in the energy market.

A key concern in the data exchange approach is that of privacy and cyber security. Although guidelines exist for data protection, more advanced technology needs to be deployed to ensure privacy protection from cyberattacks.

Figure 4: EU-SysFlex data management framework

Note: SGAM – Smart Grid Architecture Model; CIM – Common Information Model

Source: European Power System Flexibility Roadmap 2022

Way forward

Operating the power system is becoming more challenging with each passing day due to limited availability of transmission, the need for system operators to maintain the proper mix of generation resources to accommodate the variability of renewable resources, and load and resource uncertainty based on weather conditions. “The challenges are enormous, not just in terms of the scale of renewables that need to be built, but how we integrate those sources of energy onto the grid while maintaining reliability, stability, efficiency and resilience,” says John Lowry, director of the EU-SysFlex project.

The SysFlex project is a great initiative in this direction. It provides a pathway that will facilitate renewable energy integration across Europe. However, market incentives, regulatory support and efficient coordination between TSOs and DSOs will be required for its deployment. With the accurate mix of technologies in generation and storage in addition to new flexibilities in the demand side and networks, the European power system will be able to make a smooth transition towards the 2030 energy goals and the 2050 net-zero target.