Optimization Problem Formulation

Here, we describe how to load provided non time-series dependent data or your non time-series dependent data as OptDataCEP. We second describe the datatypes within the OptDataCEP and how to access it.

General

The capacity expansion problem (CEP) is designed as a linear optimization model. It is implemented in the algebraic modeling language JUMP. The implementation within JuMP allows to optimize multiple models in parallel and handle the steps from data input to result analysis and diagram export in one open source programming language. The coding of the model enables scalability based on the provided data input, single command based configuration of the setup model, result and configuration collection for further analysis and the opportunity to run design and operation in different optimizations.

Plot

The basic idea for the energy system is to have a spacial resolution of the energy system in discrete nodes. Each node has demand, non-dispatchable generation, dispatachable generation and storage capacities of varying technologies connected to itself. The different energy system nodes are interconnected with each other by transmission lines. The model is designed to minimize social costs by minimizing the following objective function:

\[min \sum_{account,tech}COST_{account,'EUR/USD',tech} + \sum LL \cdot cost_{LL} + LE \cdot cos_{LE}\]

Sets

The models scalability is relying on the usage of sets. The elements of the sets are extracted from the input data and scale the different variables. An overview of the sets is provided in the table. Depending on the models configuration the necessary sets are initialized.

The sets are setup as a dictionary and organized as set[tech_name][tech_group]=[elements...], where:

  • tech_name is the name of the dimension like e.g. tech, or node
  • tech_group is the name of a group of elements within each dimension like e.g. ["all", "generation"]. The group 'all' always contains all elements of the dimension
  • [elements...] is the Array with the different elements like ["pv", "wind", "gas"]
namedescription
linestransmission lines connecting the nodes
nodesspacial energy system nodes
techgeneration, conversion, storage, and transmission technologies
carriercarrier that an energy balance is calculated for electricity, hydrogen...
impactimpact categories like EUR or USD, CO 2 − eq., ...
accountfixed costs for installation and yearly expenses, variable costs
infrastructinfrastructure status being either new or existing
time Knumeration of the representative periods
time T periodnumeration of the time intervals within a period
time T pointnumeration of the time points within a period
time I periodnumeration of the time invervals of the full input data periods
time I pointnumeration of the time points of the full input data periods
dir transmissiondirection of the flow uniform with or opposite to the lines direction

Variables

The variables can have different types:

  • cv: cost variable - information of the costs
  • dv: design variable - information of the energy system design
  • ov: operation variable - information of the energy system operation
  • sv: slack variable - information of unmet demands or exceeded emission limits

An overview of the variables used in the CEP is provided in the table:

nametypedimensionsunitdescription
COSTcv[account,impact,tech]EUR or USD, LCA-categoriesCosts
CAPdv[tech,infrastruct,node]MWCapacity
GENov[tech,carrier,t,k,node]MWGeneration
SLACKsv[carrier,t,k,node]MWPower gap, not provided by installed CAP
LLsv[carrier]MWhLoastLoad Generation gap, not provided by installed CAP
LEsv[impact]LCA-categoriesLoastEmission Amount of emissions that installed CAP crosses the Emission constraint
INTRASTORov[tech,carrier,t,k,node]MWhStorage level within a period
INTERSTORov[tech,carrier,i,node]MWhStorage level between periods of the full time series
FLOWov[tech,carrier,dir,t,k,line]MWFlow over transmission line
TRANSov[tech,infrastruct,lines]MWmaximum capacity of transmission lines

Running the Capacity Expansion Problem

Note

The CEP model can be run with many configurations. The configurations themselves don't mess with each other though the provided input data must fulfill the ability to have e.g. lines in order for transmission to work.

An overview is provided in the following table:

descriptionunitconfigurationvaluestypedefault value
enforce an emission-limitkg-impact/MWh-carrierlimit_emissionDict{String,Number}(impact/carrier=>value)::Dict{String,Number}Dict{String,Number}()
including existing infrastructure (no extra costs) and limit infrastructure-infrastructureDict{String,Array}("existing"=>[tech-groups...], "limit"=>[tech-groups...])::Dict{String,Array}Dict{String,Array}("existing"=>["demand"])
type of storage implementation-storage_type"none", "simple" or "seasonal"::String"none"
allowing conversion between energy carriers-conversiontrue or false::Boolfalse
allowing demand-demandtrue or false::Booltrue
allowing dispatchable generation-dispatchable_generationtrue or false::Booltrue
allowing non dispatchable generation-non_dispatchable_generationtrue or false::Booltrue
allowing transmission-transmissiontrue or false::Boolfalse
fix. installed capacities to dispatch problem-fixed_design_variablesdesign variables from design run or nothing::OptVariablesnothing
allowing lost load (necessary for dispatch)price/MWh-carrierlost_load_costDict{String,Number}(carrier=>value)::Dict{String,Number}Dict{String,Number}()
allowing lost emission (necessary for dispatch)price/kg-impactlost_emission_costDict{String,Number}(impact=>value)::Dict{String,Number}Dict{String,Number}()

They can be applied in the following way:

CapacityExpansion.run_optFunction
run_opt(ts_data::ClustData,opt_data::OptDataCEP,config::Dict{String,Any},optimizer::DataType)

Organizing the actual setup and run of the CEP-Problem. This function shouldn't be called by a user, but from within the other run_opt-functions Required elements are:

  • ts_data: The time-series data.
  • opt_data: In this case the OptDataCEP that contains information on costs, nodes, techs and for transmission also on lines.
  • config: This includes all the settings for the design optimization problem formulation.
  • optimizer: The used optimizer, which could e.g. be Clp: using Clp optimizer=Clp.Optimizer or Gurobi: using Gurobi optimizer=Gurobi.Optimizer.
source
 run_opt(ts_data::ClustData,opt_data::OptDataCEP,config::Dict{String,Any},fixed_design_variables::Dict{String,Any},optimizer::DataTyple;lost_el_load_cost::Number=Inf,lost_CO2_emission_cost::Number)

This problem runs the operational optimization problem only, with fixed design variables. provide the fixed design variables and the config of the previous step (design run or another opterational run) Required elements are:

  • ts_data: The time-series data, which should be be the original time-series data for this operational run. The keys(ts_data.data) need to match the [time_series_name]-[node]
  • opt_data: In this case the OptDataCEP that contains information on costs, nodes, techs and for transmission also on lines. - Should be the same as in the design run.
  • config: This includes all the previous settings for the design optimization problem formulation and ensures that the configuration is the same.
  • fixed_design_variables: All the design variables that are determined by the previous design run.
  • optimizer: The used optimizer, which could e.g. be Clp: using Clp optimizer=Clp.Optimizer or Gurobi: using Gurobi optimizer=Gurobi.Optimizer.

What you can change in the config:

  • lost_load_cost: Dictionary with numbers indicating the lost load price per carrier (e.g. electricity in price/MWh should be greater than 1e6), give Inf for no SLACK and LL (Lost Load - a variable for unmet demand by the installed capacities)
  • lost_emission_cost: Dictionary with numbers indicating the emission price/kg-emission (Suggestion: around 700), give Inf for no LE (Lost Emissions - a variable for emissions that will exceed the limit in order to provide the demand with the installed capacities)
source
run_opt(ts_data::ClustData,
       opt_data::OptDataCEP,
       optimizer::DataType;
       descriptor::String="",
       storage_type::String="none",
       demand::Bool=true,
       dispatchable_generation::Bool=true,
       non_dispatchable_generation::Bool=true,
       conversion::Bool=false,
       transmission::Bool=false,
       lost_load_cost::Dict{String,Number}=Dict{String,Number}(),
       lost_emission_cost::Dict{String,Number}=Dict{String,Number}(),
       limit_emission::Dict{String,Number}=Dict{String,Number}(),
       infrastructure::Dict{String,Array}=Dict{String,Array}("existing"=>["demand"],"limit"=>Array{String,1}()),
       scale::Dict{Symbol,Int}=Dict{Symbol,Int}(:COST => 1e9, :CAP => 1e3, :GEN => 1e3, :SLACK => 1e3, :INTRASTOR => 1e3, :INTERSTOR => 1e6, :FLOW => 1e3, :TRANS =>1e3, :LL => 1e6, :LE => 1e9),
       print_flag::Bool=true,
       optimizer_config::Dict{Symbol,Any}=Dict{Symbol,Any}(),
       round_sigdigits::Int=9)

Wrapper function for type of optimization problem for the CEP-Problem (NOTE: identifier is the type of opt_data - in this case OptDataCEP - so identification as CEP problem). Required elements are:

  • ts_data: The time-series data, which could either be the original input data or some aggregated time-series data. The keys(ts_data.data) need to match the [time_series_name]-[node]
  • opt_data: The OptDataCEP that contains information on costs, nodes, techs and for transmission also on lines.
  • optimizer: The used optimizer, which could e.g. be Clp: using Clp optimizer=Clp.Optimizer or Gurobi: using Gurobi optimizer=Gurobi.Optimizer.

Options to tweak the model are:

  • descriptor: A name for the model
  • storage_type: String "none" for no storage, "simple" to include simple (only intra-day storage), or "seasonal" to include seasonal storage (inter-day)
  • demand: Bool true or false for technology-group
  • dispatchable_generation: Bool true or false for technology-group
  • non_dispatchable_generation: Bool true or false for technology-group
  • conversion: Bool true or false for technology-group. Allows for conversion between different energy carriers. By default false, and by default true if storage is enabled.
  • transmission:Bool true or false for technology-group. If no transmission should be modeled, a 'copperplate' is assumed with no transmission restrictions between the nodes
  • limit: Dictionary with numbers limiting the kg.-emission-eq./MWh (e.g. CO2 normally in a range from 5-1250 kg-CO2-eq/MWh), give Inf or no kw if unlimited
  • lost_load_cost: Dictionary with numbers indicating the lost load price per carrier (e.g. electricity in price/MWh should be greater than 1e6), give Inf for no SLACK and LL (Lost Load - a variable for unmet demand by the installed capacities). Example: lostloadcost=Dict{String,Number}("electricity"=>1e6)
  • lost_emission_cost: Dictionary with numbers indicating the emission price/kg-emission (Suggestion: around 700), give Inf for no LE (Lost Emissions - a variable for emissions that will exceed the limit in order to provide the demand with the installed capacities). Example: lostemissioncost=Dict{String,Number}("CO2"=>700)
  • infrastructure : Dictionary with Arrays indicating which technology groups should have existing infrastructure ("existing" => ["demand","dispatchable_generation"]) and which technology groups should have infrastructure limited ("limit" => ["non_dispatchable_generation"])
  • scale: Dict{Symbol,Int} with a number for each variable (like :COST) to scale the variables and equations to similar quantities. Try to acchieve that the numerical model only has to solve numerical variables in a scale of 0.01 and 100. The following equation is used as a relationship between the real value, which is provided in the solution (real-VAR), and the numerical variable, which is used within the model formulation (VAR): real-VAR [EUR, MW or MWh] = scale[:VAR] ⋅ VAR.
  • descriptor: String with the name of this paricular model like "kmeans-10-co2-500"
  • print_flag: Bool to decide if a summary of the Optimization result should be printed.
  • optimizer_config: Each Symbol and the corresponding value in the Dictionary is passed on to the with_optimizer function in addition to the optimizer. For Gurobi an example Dictionary could look like Dict{Symbol,Any}(:Method => 2, :OutputFlag => 0, :Threads => 2) more information can be found in the optimizer specific documentation.
  • round_sigdigits: Can be used to round the values of the result to a certain number of sigdigits.
source

Transmission

A CapacityExpansion model can be run with or without the technology transmission.

Note

If the technology transmission is not modeled (transmission=false), the transmission between nodes is not restricted, which is equivalent to a copperplate assumption.

Note

Include transmission=true and infrastructure = Dict{String,Array}("existing"=>[...,"transmission"], "limit"=>[...,"transmission"]) to model existing transmission. This sets the existing transmission TRANS to the values defined in the lines.csv file in column power_ex, and limits the transmission by the values defined in lines.csv in the column power_lim. If no new transmission should be setup, use the same values for existing transmission(column power_ex) and the limit (column power_lim).

Solver

The package provides no optimizer and a solver has to be added separately. For the linear optimization problem suggestions are:

  • Clp as an open source solver
  • Gurobi as a proprietary solver with free academic licenses. Gurobi is faster than Clp and we prefer it in the academic setting.
  • CPLEX as an alternative proprietary solver

Install the corresponding julia-package for the solver and call its optimizer like e.g.:

using Pkg
Pkg.add("Clp")
using Clp
optimizer=Clp.Optimizer

Solver Configuration

Depending on the Solver different solver configurations are possible. The information is always provided as Dict{Symbol,Any}. The keys of the dictionary are the parameters and the values of the dictionary are the values passed to the solver.

For example the Gurobi solver can be configured to have no OutputFlag and run on two threads (per julia thread) the following way:

optimizer_config=Dict{Symbol,Any}(:OutputFlag => 0, :Threads => 2)

Further information on possible keys for Gurobi can be found at Gurobi parameter description.

Scaling

The package features the scaling of variables and equations. Scaling variables, which are used in the numerical model, to 0.01 ≤ x ≤ 100 and scaling equations to 3⋅x = 1 instead of 3000⋅x = 1000 improves the shape of the optimization space and significantly reduces the computational time used to solve the numerical model.

The values are only scaled within the numerical model formulation, where we call the variable VAR, but the values are unscaled in the solution, which we call real-VAR. The following logic is used to scale the variables: real-VAR [EUR, USD, MW, or MWh] = scale[:VAR] ⋅ VAR 0.01 ≤ VAR ≤ 100 ⇔ 0.01 ≤ real-VAR / scale[:VAR] ≤ 100

The equations are scaled with the scaling parameter of the first variable, which is scale[:COST] in the following example: scale[:COST]⋅COST = 10⋅scale[:CAP]⋅CAP ⇔ COST = 10⋅(scale[:CAP]/scale[:COST])⋅CAP

Change scaling parameters

Changing the scaling parameters is useful if the data you use represents a much smaller or bigger energy system than the ones representing Germany and California provided in this package Determine the right scaling parameters by checking the real-values of COST, CAP, GEN... (real-VAR) in a solution using your data. Select the scaling parameters to match the following: 0.01 ≤ real-VAR / scale[:VAR] ≤ 100 Create a dictionary with the new scaling parameters for EACH variable and include it as the optional scale input to overwrite the default scale in run_opt:

scale=Dict{Symbol,Int}(:COST => 1e9, :CAP => 1e3, :GEN => 1e3, :SLACK => 1e3, :INTRASTOR => 1e3, :INTERSTOR => 1e6, :FLOW => 1e3, :TRANS =>1e3, :LL => 1e6, :LE => 1e9)
scale_result = run_opt(ts_clust_data,cep_data,optimizer;scale=scale)

Adding another variable

  • Extend the default scale-dictionary in the src/optim_problems/run_opt-file to include the new variable as well.
  • Include the new variable in the problem formulation in the src/optim_problems/opt_cep-file. Reformulate the equations by dividing them by the scaling parameter of the first variable, which is scale[:COST] in the following example:
  • scale[:COST]⋅COST = 10⋅scale[:CAP]⋅CAP + 100
  • ⇔ COST = 10⋅(scale[:CAP]/scale[:COST])⋅CAP + 100/scale[:COST]