Running models from a manifest¶

This page runs several solves of one model from a YAML file and one command, with no Python written. The file is a manifest: the arguments of solve and solve_over, one entry per run. Every key is in the manifest reference.

Every command on this page runs when the site is built, on the files in examples/manifest, and what you see under it is what it printed on this commit.

Install the command¶

The cli extra installs the specsolve command and the reader for Excel workbooks:

pip install 'specsolve[cli]'

Write the manifest¶

Keep the data in files beside the manifest. This study holds the dispatch model and four places its tables come from:

examples/
├── dispatch.yaml                 the spec
└── manifest/
    ├── specsolve.yaml            the manifest
    └── data/
        ├── base.xlsx             sheets generator, p_max, cost
        ├── high_gas.xlsx         sheet cost, with dearer gas
        ├── timeseries/           snapshot.csv, load.csv
        └── scenarios/load.parquet  the load, with a scenario column

specsolve.yaml names the spec and the data once, at the top level, and lists three runs under runs::

manifest: 1
spec: ../dispatch.yaml
sources: [data/base.xlsx, data/timeseries/]
solver_name: highs
solver_options: {time_limit: 600}
archive: runs/

runs:
  base: {}
  high_gas:
    sources: [data/base.xlsx, data/high_gas.xlsx, data/timeseries/]
  scenarios:
    from: high_gas
    sources: [data/base.xlsx, data/high_gas.xlsx, data/timeseries/snapshot.csv, data/scenarios/load.parquet]
    axis: {EachCoordinate: {dim: scenario}}
    workers: 2

• A top-level key is the default of every run. All three runs solve ../dispatch.yaml with HiGHS and a ten-minute limit.
• A run's own sources replace the defaults. A run reads exactly the locations its sources list, so high_gas lists the default locations again, then its own workbook. Its cost table replaces the cost sheet of base.xlsx, because a later location in one list wins.
• from starts a run from another run. scenarios takes everything high_gas resolved to except sources, which it sets itself: the snapshot table, and a load table with a scenario column in place of the hourly one.
• A run with an axis is a sweep. scenarios solves once per scenario, two at a time in separate processes.
• archive is a directory of runs. Each run archives to runs/<run name>.

Paths are relative to the manifest, wherever the command runs from.

Check the runs¶

specsolve check reads every run's spec and data, and attaches the data without solving. It reports each run on one line:

$ specsolve check
base: ok
high_gas: ok
scenarios: ok

A run with a problem names it, and the command exits with 1.

Solve the runs¶

specsolve solve solves every run in manifest order, one after another, and reports how each one ended:

$ specsolve solve
base: optimal, objective 7500, 0.0s
high_gas: optimal, objective 9500, 0.0s
scenarios: 2 slices: 2 optimal, 0.5s

Name runs to solve only those. A run whose archive exists is skipped, never overwritten, so the same command resumes a study that stopped part-way:

$ specsolve solve base
base: skipped, runs/base exists

Delete runs/base to solve base again. specsolve list says which runs are archived:

$ specsolve list
base       solve  archived
high_gas   solve  archived
scenarios  sweep  archived

Each command reads specsolve.yaml in the working directory. To read another file, name it first: specsolve solve studies/winter.yaml base.

Read the answers¶

Each archive holds the spec, the data it was solved with and the answer. load_archive reads one back:

import specsolve as sps

archive = sps.load_archive(study / 'runs' / 'scenarios')
print(archive.sweep.record.select('scenario', 'termination_condition', 'objective'))

shape: (2, 3)
┌──────────┬───────────────────────┬───────────┐
│ scenario ┆ termination_condition ┆ objective │
│ ---      ┆ ---                   ┆ ---       │
│ str      ┆ str                   ┆ f64       │
╞══════════╪═══════════════════════╪═══════════╡
│ high     ┆ optimal               ┆ 24000.0   │
│ low      ┆ optimal               ┆ 9500.0    │
└──────────┴───────────────────────┴───────────┘

Run a manifest from Python¶

load_manifest reads the same file. Each run's arguments are the keyword arguments of its call, and solve makes the call, with any keyword replacing the manifest's:

manifest = sps.load_manifest(study / 'specsolve.yaml')
run = manifest.runs['high_gas']

result = run.solve(archive=None, solver_options={'time_limit': 60})
print(result.termination_condition, result.objective)

optimal 9500.0

Use Python for what the manifest does not hold: data computed before the solve, logic between solves, or a remote executor such as a dask Client:

sweep = manifest.runs['scenarios'].solve(executor=client)