Quick Start Guide

This quick start guide introduces the main concepts of using CapacityExpansion. For more detail on the different functionalities that CapacityExpansion provides, please refer to the subsequent chapters of the documentation or the examples in the examples folder.

Generally, the workflow consists of three steps:

Data preparation of ClustData
Data preparation of OptDataCEP
Optimization

Example Workflow

After CapacityExpansion and a Solver like, e.g. Clp are installed, you can use them by saying:

julia> using CapacityExpansion

julia> using Clp

julia> optimizer=Clp.Optimizer # defines the optimizer used by CapacityExpansion
Clp.Optimizer

The first step is to load the time-series input data. The following example loads hourly wind, solar, and demand data for Germany (1 region) for the year 2016. The hourly input-data is split into periods with 24 elements, which equals days.

julia> ts_input_data = load_timeseries_data_provided("GER_1"; T=24, years=[2016])
┌ Warning: `getindex(df::DataFrame, col_ind::ColumnIndex)` is deprecated, use `df[!, col_ind]` instead.
│   caller = #add_timeseries_data!#12(::Int64, ::Int64, ::Array{Int64,1}, ::Function, ::Dict{String,Array}, ::SubString{String}, ::DataFrames.DataFrame) at load_data.jl:132
└ @ TimeSeriesClustering ~/.julia/packages/TimeSeriesClustering/3T6SG/src/utils/load_data.jl:132
ClustData("GER_1", [2016], 366, 24, Dict{String,Array}("solar-germany"=>[0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0],"wind-germany"=>[0.1429 0.1453 … 0.1329 0.1832; 0.1368 0.1758 … 0.1312 0.1802; … ; 0.1098 0.4955 … 0.1904 0.3122; 0.1254 0.4875 … 0.1865 0.3187],"demand_electricity-germany"=>[41913.0 39121.0 … 45343.0 45600.0; 40331.0 38271.0 … 44402.0 44332.0; … ; 44439.0 48859.0 … 50278.0 48988.0; 41257.0 45600.0 … 47534.0 47641.0]), [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0  …  1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0], Dict{String,Array}("solar-germany"=>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],"wind-germany"=>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0],"demand_electricity-germany"=>[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]), Dict{String,Array}("solar-germany"=>[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0  …  1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],"wind-germany"=>[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0  …  1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0],"demand_electricity-germany"=>[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0  …  1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]), [1.0 1.0 … 1.0 1.0; 1.0 1.0 … 1.0 1.0; … ; 1.0 1.0 … 1.0 1.0; 1.0 1.0 … 1.0 1.0], [1, 2, 3, 4, 5, 6, 7, 8, 9, 10  …  357, 358, 359, 360, 361, 362, 363, 364, 365, 366])

The output ts_input_data is a ClustData data struct that contains the data and additional information about the data.

julia> ts_input_data.data # a dictionary with the data.
Dict{String,Array} with 3 entries:
  "solar-germany"              => [0.0 0.0 … 0.0 0.0; 0.0 0.0 … 0.0 0.0; … ; 0.…
  "wind-germany"               => [0.1429 0.1453 … 0.1329 0.1832; 0.1368 0.1758…
  "demand_electricity-germany" => [41913.0 39121.0 … 45343.0 45600.0; 40331.0 3…

julia> ts_input_data.data["wind-germany"] # the wind data (choose solar, `demand_electricity` as other options in this example)
24×366 Array{Float64,2}:
 0.1429  0.1453  0.4843  0.4279  0.248   …  0.3359  0.0793  0.1329  0.1832
 0.1368  0.1758  0.4819  0.4186  0.2574     0.318   0.0803  0.1312  0.1802
 0.1232  0.2135  0.4792  0.407   0.2682     0.2949  0.0791  0.1337  0.1779
 0.1096  0.2466  0.4838  0.3976  0.2764     0.2739  0.0775  0.1363  0.1796
 0.0964  0.2818  0.4917  0.3873  0.2784     0.2688  0.0774  0.1382  0.1872
 0.082   0.3209  0.4862  0.3776  0.2797  …  0.2638  0.0781  0.1387  0.1971
 0.0706  0.3548  0.4784  0.3655  0.2796     0.2419  0.08    0.1401  0.2072
 0.0593  0.3921  0.471   0.3528  0.2828     0.2151  0.0818  0.1416  0.2151
 0.0438  0.422   0.4786  0.3432  0.2878     0.1813  0.0777  0.1347  0.2067
 0.0317  0.4536  0.475   0.3259  0.2823     0.1494  0.0653  0.1177  0.1981
 ⋮                                       ⋱                          ⋮
 0.0116  0.5288  0.4637  0.248   0.2738  …  0.1112  0.1266  0.2099  0.2792
 0.0222  0.542   0.4832  0.262   0.2733     0.1005  0.1458  0.233   0.2855
 0.0346  0.5416  0.4879  0.2673  0.2672     0.0811  0.1578  0.2339  0.2864
 0.0497  0.5316  0.4804  0.2629  0.2585     0.0636  0.1626  0.2217  0.2926
 0.0669  0.521   0.4651  0.2549  0.2511     0.0521  0.1627  0.2086  0.2974
 0.0817  0.5103  0.4526  0.2448  0.2453  …  0.0472  0.1576  0.2014  0.2987
 0.0948  0.5017  0.446   0.2379  0.2403     0.0517  0.1493  0.1961  0.3033
 0.1098  0.4955  0.4387  0.2395  0.2356     0.0617  0.1431  0.1904  0.3122
 0.1254  0.4875  0.4349  0.2433  0.2316     0.0725  0.1376  0.1865  0.3187

julia> ts_input_data.K # number of periods
366

The second step is to include the optimization data, which is not time-series depending.

julia> cep_data = load_cep_data_provided("GER_1")
┌ Warning: `getindex(df::DataFrame, col_ind::ColumnIndex)` is deprecated, use `df[!, col_ind]` instead.
│   caller = load_cep_data_nodes(::String, ::OptVariable{OptDataCEPTech,1,Tuple{Array{Any,1}},Tuple{Dict{Any,Int64}}}) at load_data.jl:118
└ @ CapacityExpansion ~/build/YoungFaithful/CapacityExpansion.jl/src/utils/load_data.jl:118
┌ Warning: `getindex(df::DataFrame, col_ind::ColumnIndex)` is deprecated, use `df[!, col_ind]` instead.
│   caller = load_cep_data_costs(::String, ::OptVariable{OptDataCEPTech,1,Tuple{Array{Any,1}},Tuple{Dict{Any,Int64}}}, ::OptVariable{OptDataCEPNode,2,Tuple{Array{Any,1},Array{String,1}},Tuple{Dict{Any,Int64},Dict{String,Int64}}}) at load_data.jl:241
└ @ CapacityExpansion ~/build/YoungFaithful/CapacityExpansion.jl/src/utils/load_data.jl:241
OptDataCEP("GER_1", 5-dimensional OptVariable{Number,5,...} of type fv with index sets:
    Dimension 1 - tech, Any["bat_in", "bat_out", "bat_e", "h2_in", "pv", "trans", "coal", "gas", "demand", "wind", "oil", "h2_e", "h2_out"]
    Dimension 2 - node, ["germany"]
    Dimension 3 - year, [2015]
    Dimension 4 - account, ["cap_fix", "var"]
    Dimension 5 - impact, ["EUR", "CO2"]
And data, a 13×1×1×2×2 Array{Number,5}:
[:, :, 2015, "cap_fix", "EUR"] =
   8961.67479
      0.0
  25653.7085
 479782.071
 136926.376
     51.0061474
 153702.673
  29964.5882
      0.0
 168417.541
  38429.5834
     68.4962783
      0.0

[:, :, 2015, "var", "EUR"] =
   0.01
   0.0
   0.0
   1.9
   0.0
   0.0
  10.708
  86.564
   0.0
   0.0
 153.586
   0.0
   0.0

[:, :, 2015, "cap_fix", "CO2"] =
     0.0
     0.0
  1658.35472
 96666.6667
   160.233933
     3.3275
    13.8595556
     1.2334
     0.0
    36.75
     8.25
     0.0
     0.0

[:, :, 2015, "var", "CO2"] =
    0.0
    0.0
    0.0
    0.0
    0.0
    0.0
 1217.49
  728.71
    0.0
    0.0
  874.6
    0.0
    0.0 , 1-dimensional OptVariable{OptDataCEPTech,1,...} of type fv with index sets:
    Dimension 1 - tech, Any["bat_in", "bat_out", "bat_e", "h2_in", "pv", "trans", "coal", "gas", "demand", "wind", "oil", "h2_e", "h2_out"]
And data, a 13-element Array{OptDataCEPTech,1}:
 OptDataCEPTech("Battery Charge", ["conversion", "all"], "power", "node", 25, 30, 0.04, 0.0640119628, Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("carrier"=>"electricity_bat"), Dict{Any,Any}("efficiency"=>0.97))
 OptDataCEPTech("Battery Discharge", ["conversion", "all"], "power", "node", 25, 30, 0.04, 0.0640119628, Dict{Any,Any}("carrier"=>"electricity_bat"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("efficiency"=>0.97,"cap_eq"=>"bat_in"))
 OptDataCEPTech("Battery Storage", ["storage", "all"], "energy", "node", 25, 30, 0.04, 0.0640119628, Dict{Any,Any}("carrier"=>"electricity_bat"), Dict{Any,Any}("carrier"=>"electricity_bat"), Dict{Any,Any}("efficiency"=>0.93))
 OptDataCEPTech("Hydrogen Storage Charge", ["conversion", "all"], "power", "node", 15, 30, 0.04, 0.0899411004, Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("carrier"=>"hydrogen"), Dict{Any,Any}("efficiency"=>0.83))
 OptDataCEPTech("Photo Voltaic", ["non_dispatchable_generation", "generation", "all"], "power", "node", 15, 30, 0.04, 0.0899411004, Dict{Any,Any}("timeseries"=>"solar"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}())
 OptDataCEPTech("Transmission Line", ["transmission", "all"], "power", "line", 80, 30, 0.04, 0.0578300991, Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("efficiency"=>0.9995))
 OptDataCEPTech("Coal Plant", ["dispatchable_generation", "generation", "all"], "power", "node", 45, 30, 0.04, 0.0578300991, Dict{Any,Any}("fuel"=>"coal"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}())
 OptDataCEPTech("Gas Plant", ["dispatchable_generation", "generation", "all"], "power", "node", 50, 30, 0.04, 0.0578300991, Dict{Any,Any}("fuel"=>"gas"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}())
 OptDataCEPTech("Electricity demand", ["demand", "all"], "power", "node", 15, 30, 0.04, 0.0899411004, Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("timeseries"=>"demand_electricity"), Dict{Any,Any}())
 OptDataCEPTech("Onshore Wind", ["non_dispatchable_generation", "generation", "all"], "power", "node", 15, 30, 0.04, 0.0899411004, Dict{Any,Any}("timeseries"=>"wind"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}())
 OptDataCEPTech("Oil Plant", ["dispatchable_generation", "generation", "all"], "power", "node", 40, 30, 0.04, 0.0578300991, Dict{Any,Any}("fuel"=>"oil"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}())
 OptDataCEPTech("Hydrogen Storage", ["storage", "all"], "energy", "node", 25, 30, 0.04, 0.0640119628, Dict{Any,Any}("carrier"=>"hydrogen"), Dict{Any,Any}("carrier"=>"hydrogen"), Dict{Any,Any}("efficiency"=>0.99))
 OptDataCEPTech("Hydrogen Storage Discharge", ["conversion", "all"], "power", "node", 40, 30, 0.04, 0.0578300991, Dict{Any,Any}("carrier"=>"hydrogen"), Dict{Any,Any}("carrier"=>"electricity"), Dict{Any,Any}("efficiency"=>0.53,"cap_eq"=>"h2_in")), 2-dimensional OptVariable{OptDataCEPNode,2,...} of type fv with index sets:
    Dimension 1 - tech, Any["bat_in", "bat_out", "bat_e", "h2_in", "pv", "trans", "coal", "gas", "demand", "wind", "oil", "h2_e", "h2_out"]
    Dimension 2 - node, ["germany"]
And data, a 13×1 Array{OptDataCEPNode,2}:
 OptDataCEPNode("germany", 0, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 32312, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 0, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 45027, 100000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 22370, 100000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 1, 1, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 31827, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 7004, 100000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 10000000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))
 OptDataCEPNode("germany", 0, 1000000, "GER", LatLon(lat=51.167261°, lon=10.450738°))    , 2-dimensional OptVariable{OptDataCEPLine,2,...} of type fv with index sets:
    Dimension 1 - tech, String[]
    Dimension 2 - line, String[]
And data, a 0×0 Array{OptDataCEPLine,2})

The cep is a OptDataCEP data struct.

julia> cep_data.region # the region of the input-data
"GER_1"

julia> cep_data.costs # the information of costs as an `OptVariable` with 5 dimensions
5-dimensional OptVariable{Number,5,...} of type fv with index sets:
    Dimension 1 - tech, Any["bat_in", "bat_out", "bat_e", "h2_in", "pv", "trans", "coal", "gas", "demand", "wind", "oil", "h2_e", "h2_out"]
    Dimension 2 - node, ["germany"]
    Dimension 3 - year, [2015]
    Dimension 4 - account, ["cap_fix", "var"]
    Dimension 5 - impact, ["EUR", "CO2"]
And data, a 13×1×1×2×2 Array{Number,5}:
[:, :, 2015, "cap_fix", "EUR"] =
   8961.67479
      0.0
  25653.7085
 479782.071
 136926.376
     51.0061474
 153702.673
  29964.5882
      0.0
 168417.541
  38429.5834
     68.4962783
      0.0

[:, :, 2015, "var", "EUR"] =
   0.01
   0.0
   0.0
   1.9
   0.0
   0.0
  10.708
  86.564
   0.0
   0.0
 153.586
   0.0
   0.0

[:, :, 2015, "cap_fix", "CO2"] =
     0.0
     0.0
  1658.35472
 96666.6667
   160.233933
     3.3275
    13.8595556
     1.2334
     0.0
    36.75
     8.25
     0.0
     0.0

[:, :, 2015, "var", "CO2"] =
    0.0
    0.0
    0.0
    0.0
    0.0
    0.0
 1217.49
  728.71
    0.0
    0.0
  874.6
    0.0
    0.0

The third step is to setup the model and run the optimization.

julia> result = run_opt(ts_input_data,cep_data,optimizer;optimizer_config=Dict{Symbol,Any}(:LogLevel => 0));
┌ Warning: Limit is reached for techs ["demand-germany"]
└ @ CapacityExpansion ~/build/YoungFaithful/CapacityExpansion.jl/src/utils/utils.jl:249
[ Info: Solved Scenario : OPTIMAL min COST: 1.671e10 [EUR] ⇨ 33.03 [EUR per MWh] s.t.

The result is a OptResult data struct and contains the information of the optimization result.

julia> result.info["model"] # the equations of the setup model
19-element Array{String,1}:
 ""
 "Variable COST[account, impact, tech] in EUR CO2 "
 "Variable CAP[tech_n, infrastruct, nodes] ≥ 0 in MW"
 "Variable GEN[tech_power, carrier, t, k, node] in MW"
 "COST['var',impact,tech] = (1) ⋅ Σ_{t,k,node}GEN[tech, carrier_input, t, k, node]⋅ ts_weights[k] ⋅ ts_deltas[t,k]⋅ var_costs[tech,impact] ∀ impact, tech_demand"
 "COST['cap_fix',impact,tech] = Σ_{t,k}(ts_weights ⋅ ts_deltas[t,k])/8760h ⋅ Σ_{node}CAP[tech,'new',node] ⋅ cap_costs[tech,impact] ∀ impact, tech_demand"
 "GEN[tech, carrier, t, k, node] = (-1) ⋅ Σ_{infrastruct} CAP[tech,infrastruct,node] * ts[tech-node,t,k] ∀ node, tech_demand, t, k"
 "COST['var',impact,tech] = (1) ⋅ Σ_{t,k,node}GEN[tech, carrier_output, t, k, node]⋅ ts_weights[k] ⋅ ts_deltas[t,k]⋅ var_costs[tech,impact] ∀ impact, tech_non_dispatchable_generation"
 "COST['cap_fix',impact,tech] = Σ_{t,k}(ts_weights ⋅ ts_deltas[t,k])/8760h ⋅ Σ_{node}CAP[tech,'new',node] ⋅ cap_costs[tech,impact] ∀ impact, tech_non_dispatchable_generation"
 "0 ≤ GEN[tech, carrier, t, k, node] ≤ Σ_{infrastruct}CAP[tech,infrastruct,node]*ts[tech-node,t,k] ∀ node, tech_generation{non_dispatchable}, t, k"
 "COST['var',impact,tech] = (1) ⋅ Σ_{t,k,node}GEN[tech, carrier_output, t, k, node]⋅ ts_weights[k] ⋅ ts_deltas[t,k]⋅ var_costs[tech,impact] ∀ impact, tech_dispatchable_generation"
 "COST['cap_fix',impact,tech] = Σ_{t,k}(ts_weights ⋅ ts_deltas[t,k])/8760h ⋅ Σ_{node}CAP[tech,'new',node] ⋅ cap_costs[tech,impact] ∀ impact, tech_dispatchable_generation"
 "0 ≤ GEN[tech, carrier, t, k, node] ≤ Σ_{infrastruct} CAP[tech,infrastruct,node] ∀ node, tech_dispatchable_generation, t, k"
 "CAP[tech, 'ex', node] = existing infrastructure ∀ node, tech ∈ tech_group_ex"
 "CAP[tech, 'ex', node] = 0  ∀ node, tech ∉ tech_group_ex"
 "∑_{infrastuct} CAP[tech, infrastruct, node] <= limit infrastructure ∀ tech_n, node"
 "Σ_{tech,node}GEN[tech, carrier, t,k,node] + SLACK[carrier,t,k,node] = 0 ∀ t,k"
 "Σ_{tech,node}GEN[tech, carrier, t,k,node] = 0 ∀ t,k"
 "min Σ_{account,tech}COST[account,'EUR',tech] + Σ_{node,carrier_ll} LL[carrier,node] ⋅ lost_load_cost[carrier]) +  Σ_{impact_le} LE[impact] ⋅ lost_emission_cost[impact] st. above"

julia> result.status # the status of the optimization
:OPTIMAL

julia> result.objective # the value of the objective
1.6708870355646713e10

julia> result.variables["CAP"] # the newly installed and existing capacities of the different technologies along the nodes. Other options are "COST" (the costs) and "GEN" (the generation)
3-dimensional OptVariable{Float64,3,...} of type dv with index sets:
    Dimension 1 - tech, ["pv", "coal", "gas", "demand", "wind", "oil"]
    Dimension 2 - infrastruct, ["new", "ex"]
    Dimension 3 - nodes, ["germany"]
And data, a 6×2×1 Array{Float64,3}:
[:, :, "germany"] =
     0.0  0.0
 68521.0  0.0
 10966.0  0.0
     0.0  1.0
     0.0  0.0
     0.0  0.0

julia> result.sets["tech"]["generation"] # a `"tech"` (dimension) set of all `"generation"` (tech-group) within the model
5-element Array{String,1}:
 "pv"
 "coal"
 "gas"
 "wind"
 "oil"

julia> result.config["generation"] # Detailed model configuration
true