Skip to content

Optimization

Outer loops that run many workflows.

tango_core.pymoo_api

Multi-objective optimization API for tango, built on pymoo.

A thin wrapper around pymoo that provides:

  • Typed interfaces (Minimization, ParamSpec, ObjectiveSpec, PymooConfig) in place of pymoo's *args/**kwargs-heavy API, so a misconfigured search fails at construction instead of deep inside pymoo.
  • A strong mechanism for setting decision variables and extracting objectives: each ParamSpec applies its value to a config dict (and reads it back), and each ObjectiveSpec reads one number from a finished run.
  • Parallel evaluation with minimal configuration: set PymooConfig.n_processes.
  • Plotting utilities (Pareto scatter and parallel-coordinates plots) to help downselect designs from a front.

Adapted to tango's actual seams:

  • Parameters mutate config dicts, not live objects. Workflow.to_dict()'s own class docstring already shows this exact pattern (cfg = base_wf.to_dict(); cfg[...] = ...; Workflow(cfg).run()) — a fresh Workflow is built and run from a mutated copy of the base config on every evaluation, rather than trying to mutate a live, already-solved Workflow in place. The same goes for everything sent to worker processes: only plain data (config dicts, picklable specs) crosses a process boundary, never a live Workflow.
  • Constraint violations are real pymoo constraints, not objective-space penalties. A common alternative pattern falls back to a fixed huge objective value (e.g. 1.02e12) on simulation failure — that value has to be large enough to dominate every real objective, which makes it scale-dependent and easy to get wrong. Here, a failed evaluation instead sets a real out["G"] constraint violation (> 0); pymoo's own constraint-domination ranking (feasible always beats infeasible, regardless of F) means the objective fallback value's magnitude is irrelevant, so it can just be 0.0 rather than a magic huge number.
  • Only specific, known-physical failure modes become a CV — everything else raises. _evaluate_one catches exactly INFEASIBLE_DESIGN_EXCEPTIONS (CoolPropError, FrackingViolationError, ConvergenceError, InvalidPowerStateError, CoolingLimitError, SaturationError, WellPressureCollapseError) — the tango_core errors that genuinely mean "this sampled design is infeasible," not "something is broken." A bare except Exception would also swallow ConfigError (almost always a bug in a ParamSpec.apply, not an infeasible design) and the LockedStateError family (internal contract violations), silently poisoning results across a long unattended search instead of crashing loudly where the bug actually is.

InjectionPolicy: the base config must set sim_config.injection_mode (MANUAL or MANUAL_FRACKING_CHECK, the only modes there are); the search itself chooses the operating point, with a fracking violation under MANUAL_FRACKING_CHECK counting as an infeasible design.

IDEAL_DISTANCE_COL = 'ideal_distance' module-attribute

Results column: each design's normalized distance to the front's ideal corner.

ParamToleranceDuplicateElimination(params, tol_frac)

Bases: DuplicateElimination

Two designs are duplicates if EVERY param is within tol_frac of the other's, per-param.

Unlike pymoo's own DefaultDuplicateElimination (one combined Euclidean distance against a single absolute epsilon, which silently conflates parameters of wildly different scales/units — e.g. mass_rate_kg_per_s O(10-200) vs injection_pressure_Pa O(1e6-1e7), where an epsilon large enough to catch near-duplicate pressures would swamp mass_rate entirely, and vice versa), this checks each parameter independently against its OWN bounds range — matches surrogate_api._cached_row_is_valid's per-param math.isclose convention elsewhere in this codebase, rather than one unitless combined distance.

Used as NSGA2(..., eliminate_duplicates=ParamToleranceDuplicateElimination(params, tol_frac)) — pymoo's Mating.do() calls this BEFORE evaluating offspring, so a near-duplicate design is never actually real-simulated at all, not just filtered out afterward. Only catches duplicates within the current offspring batch and against the current population (algorithm.pop) — NOT against the full multi-generation trajectory history, since pymoo's mating only ever compares against what's live right now. A design from 3 generations ago that's since been selected out of the population can still be near-duplicated by a later generation; a supplementary whole-history dedup pass would be needed to close that gap (not built — see REDESIGN_TODO.md's Q2).

tol_frac of each param's own bounds range is that param's tolerance.

PymooConfig(n_max_gen=200, ftol=_DEFAULT_FTOL, xtol=_DEFAULT_XTOL, termination_period=_DEFAULT_PERIOD, pop_size=40, n_processes=1, seed=1, crossover_prob=None, mutation_prob=None, report_archive=False) dataclass

Pymoo hyperparameters for Minimization.run().

Defaults chosen to match pymoo's own DefaultMultiObjectiveTermination() defaults (ftol/xtol) and NSGA2()'s own default population size, so "the defaults" here actually mean pymoo's own defaults, not arbitrary tango-specific numbers.

n_max_gen = 200 class-attribute instance-attribute

Maximum generations — hard cap even if ftol/xtol convergence is never reached.

ftol = _DEFAULT_FTOL class-attribute instance-attribute

Objective-space convergence tolerance.

xtol = _DEFAULT_XTOL class-attribute instance-attribute

Design-space (parameter) convergence tolerance.

termination_period = _DEFAULT_PERIOD class-attribute instance-attribute

Generations ftol/xtol/cvtol must each stay satisfied for, consecutively, before that alone ends the run (pymoo's RobustTermination window, passed straight through to DefaultMultiObjectiveTermination's own period arg — a single converged-looking generation isn't enough; convergence must be sustained). Larger = more confidence a plateau is real, at the cost of running that many more generations past wherever it actually started.

pop_size = 40 class-attribute instance-attribute

Population size per generation.

n_processes = 1 class-attribute instance-attribute

Number of worker processes for parallel evaluation. 1 = serial (LoopedElementwiseEvaluation); >1 spins up a multiprocessing.Pool and uses pymoo's StarmapParallelization.

seed = 1 class-attribute instance-attribute

Random seed passed to pymoo.optimize.minimize — fixes it for reproducibility by default (matches pymoo's own convention of a fixed default rather than a random one); pass None for a non-reproducible (system-entropy-seeded) run instead.

crossover_prob = None class-attribute instance-attribute

Probability two parents actually recombine (pymoo's default crossover operator, SBX, calls this prob — default 0.9 if left None here). Only applied when Minimization builds the DEFAULT algorithm itself (no explicit algorithm= passed) — same rule as pop_size above: a caller-supplied algorithm already has its own operators configured and this is ignored, not merged in.

mutation_prob = None class-attribute instance-attribute

Probability each variable gets mutated (pymoo's default mutation operator, PM, calls this prob — default 1/n_var if left None here). Same algorithm= caveat as crossover_prob above.

report_archive = False class-attribute instance-attribute

Report the non-dominated set of every feasible design evaluated during the run, instead of only the final population's non-dominated designs.

NSGA-II keeps a fixed-size population (pop_size) and thins its front to fit, so the final population holds at most pop_size designs and can drop good ones found earlier, especially near the ends of the front. The archive keeps them all. It only changes what's reported; the search itself is unchanged. Each reported design is re-solved once for the report columns, so a larger front costs that many extra simulations.

ParamSpec(name, apply, bounds, read=None) dataclass

One decision variable pymoo can tweak.

pymoo_api.py is deliberately generic — it has NO built-in ParamSpec instances of its own, and none should be added here. Every concrete param (mass flow rate, injection pressure, a reservoir's own inlet/outlet-mismatch handling, whatever a future coupled model needs) belongs in the calling app script (e.g. applications/opt_hydro_df_electric/opt_hydro_df_electric.py), not in this module — this module doesn't need to, and must not, know what fields any particular system exposes. Duplicating a param's definition across multiple app scripts that happen to need the same one is expected and fine; it is NOT evidence that param belongs here instead.

Parameters:

Name Type Description Default
name str

Column name in the results dataframe.

required
apply Callable[[dict[str, Any], float], dict[str, Any]]

(config_dict, value) -> config_dict — mutates (and returns) a copy of the base config with this parameter set to value. Must be a module-level, picklable callable (a closure defined inside a factory function can't be pickled — pickle/multiprocessing serialize callables by module + qualified name and can't re-import a runtime-only local closure. That only breaks in the n_processes > 1 path (multiprocessing.Pool.starmap requires every argument, including this callable, to pickle) — confirmed empirically the first time n_processes=2 was actually exercised, back when this pattern still lived in this module: it raised AttributeError: Can't pickle local object '<factory>.<locals>.apply'. A module-level dataclass with __call__ pickles fine, by class reference + simple field values — use that shape, not a closure.)

required
bounds tuple[float, float]

(lower, upper) bound pymoo searches within.

required
read Callable[[dict[str, Any]], float] | None

config_dict -> value: the value this parameter currently has in a config, in the same units as bounds. Gives Minimization's parameter check its baseline (see Minimization.check_param_sensitivity); without it, the bounds midpoint stands in. Same picklability requirement as apply.

None

ObjectiveSpec(name, evaluate) dataclass

One objective pymoo minimizes.

Same rule as ParamSpec above: no built-in ObjectiveSpec instances live in this module — every concrete objective (LCOE, mean electric power, mean thermal power, whatever a future system's own economics/output looks like) belongs in the calling app script.

Parameters:

Name Type Description Default
name str

Column name in the results dataframe.

required
evaluate Callable[[Workflow], float]

(solved_workflow) -> float — pymoo minimizes this value, so a "maximize energy" goal must be expressed as its negative. Same module-level-callable/picklability requirement as ParamSpec.apply above.

required

ConstraintSpec(name, evaluate) dataclass

One inequality constraint pymoo enforces, IN ADDITION TO physical feasibility.

Same rule as ParamSpec/ObjectiveSpec above: no built-in ConstraintSpec instances live in this module — every concrete constraint (a minimum production temperature, a maximum pressure, etc.) belongs in the calling app script.

Every Minimization already rejects a design that raises one of _cv_exceptions (a genuine physical-simulation failure) via a built-in constraint — ConstraintSpec is for a DIFFERENT kind of infeasibility: the simulation solved successfully, but the resulting design isn't acceptable on some other engineering ground (e.g. a district-heating design that solves fine but delivers water too cold to be useful). Both kinds feed the SAME pymoo constraint-domination ranking (feasible always beats infeasible, regardless of objective values) — see _Problem's n_ieq_constr and Minimization. _evaluate_one.

Parameters:

Name Type Description Default
name str

Label only — constraint G values aren't retained in Minimization's results dataframe (only params/ objectives are), same as the built-in physical-feasibility constraint.

required
evaluate Callable[[Workflow], float]

(solved_workflow) -> float, pymoo's own inequality convention: <= 0 is satisfied, > 0 is a violation. Return a genuine violation MAGNITUDE where possible (e.g. 105.0 - T_prod_C, degrees below the threshold), not just a flat +1/-1 sign — pymoo's constraint-domination ranking only needs the sign to decide feasible-vs-not, but a continuous magnitude gives the search itself more gradient to climb back toward feasibility, the same way a real objective would. Same module-level-callable/picklability requirement as ParamSpec.apply/ObjectiveSpec.evaluate above.

required

ParetoProgressOutput()

Bases: Output

Per-generation progress table for Minimization.run() (verbose=True).

n_nds and ideal_min both describe algorithm.opt (the current non-dominated front), not the whole population, so the two columns describe the same designs. ideal_min is the smallest ideal_distance on that front.

This only changes what prints during a run, not the saved results.

Set up the extra progress columns.

update(algorithm)

Refresh each column from the current generation's non-dominated front.

Minimization(base_cfg, params, objectives, pymoo_config=None, algorithm=None, verbose=False, save_results=True, save_pareto_configs=True, save_dir=DEFAULT_SAVE_DIR, seed_from=None, pad_sampling=None, constraints=list(), report_specs=list(), log_failures=False, extra_cv_exceptions=(), check_param_sensitivity=True, sensitivity_step_frac=0.05, capture_trajectory_path=None) dataclass

Multi-objective optimization over a tango Workflow, via pymoo.

Usage — ParamSpec/ObjectiveSpec factories are defined by the CALLING SCRIPT, not by this module (see their own docstrings for why)::

# Defined in the calling app script, e.g.
# applications/opt_hydro_df_electric/opt_hydro_df_electric.py —
# NOT in pymoo_api.py itself.
cfg = yaml.safe_load(Path("templates/hydro_df_electric.yml").read_text(encoding="utf-8"))
m = Minimization(
    base_cfg=cfg,
    params=[
        mass_rate_param(bounds=(10.0, 100.0)),
        injection_pressure_param(bounds=(1e6, 10e6)),
    ],
    objectives=[lcoe_objective(), negative_mean_power_objective()],
)
res, df = m.run()

Parameters:

Name Type Description Default
base_cfg dict[str, Any]

Template config dict, as written (e.g. a loaded example config) — the starting point every evaluation mutates a copy of. Validated once, up front, by building a Workflow from it in __post_init__, but kept as given rather than replaced by Workflow.to_dict(), which would pin defaults derived from inputs the params change. Only plain data is kept: Minimization is pickled to worker processes when n_processes > 1, and a live Workflow can't always be pickled (a TESPy network can't).

required
params list[ParamSpec]

Decision variables — see ParamSpec's own docstring for why the concrete factories live in the calling script, not here.

required
objectives list[ObjectiveSpec]

Objectives to minimize — see ObjectiveSpec's own docstring, same reasoning as params above.

required
pymoo_config PymooConfig | None

Hyperparameters — see PymooConfig. Defaults to PymooConfig() if omitted.

None
algorithm GeneticAlgorithm | None

Any pymoo GeneticAlgorithm instance — defaults to NSGA2(pop_size=pymoo_config.pop_size), but any other algorithm (NSGA3, etc.) can be passed instead; this class never hardcodes NSGA2-specific behavior elsewhere.

None
verbose bool

Print pymoo's per-generation progress table.

False
save_results bool

Write the results dataframe (CSV) and, if save_pareto_configs, every pareto-front design's config (YAML) to save_dir. Default True.

True
save_pareto_configs bool

Save each pareto-front design's CONFIG DICT (not a solved live Workflow) as its own YAML file — trivially re-run via Workflow.from_file(path).run(), and avoids pickling live TESPy/CoolProp state that a solved object would carry. Default True.

True
save_dir Path | str

Where to save results — default outputs/pymoo/.

DEFAULT_SAVE_DIR
seed_from Path | str | None

Path to a previous run's pareto_results.csv (or pareto_results_partial.csv — same schema) to WARM-START the initial population from, instead of pymoo's default random/LHS init. A cheap alternative to true checkpoint/resume (which would need to pickle the whole Algorithm — including a live multiprocessing.Pool when n_processes > 1, which can't be pickled at all): this only reuses the numeric param columns already sitting in a saved CSV, so it works unconditionally regardless of n_processes. NOT a true resume — the new run starts at generation 1 with a fresh RNG state, just seeded with better-than-random starting points. The non-dominated designs from seed_from are combined with pad_sampling-generated designs to fill out the rest of algorithm.pop_size (if seed_from has MORE designs than pop_size, only the first pop_size are used). Default None (no seeding, pymoo's normal init).

None
pad_sampling Sampling | None

pymoo Sampling used to fill out the population beyond what seed_from provides. Default LHS() — better design-space coverage than plain random sampling for the "fill in the rest" role. Only used if seed_from is given.

None
constraints list[ConstraintSpec]

Extra inequality constraints beyond the built-in physical-feasibility one — see ConstraintSpec. Default () (no extra constraints).

list()
report_specs list[ObjectiveSpec]

Extra per-design read-outs, computed once for the final pareto front only and merged into the results dataframe — NOT a real objective, doesn't affect the search. See its own field docstring. Default () (no extra columns).

list()
log_failures bool

Append one JSON line per constraint-violating evaluation (see INFEASIBLE_DESIGN_EXCEPTIONS) to save_dir / "failures.jsonl" — the mutated parameter values plus the exception type/message. No objective values are logged for a failure: a design either fully evaluates (all objectives computed) or raises before any of them are — there's no partial objective to save. Off by default (extra I/O on every failure); turn on when characterizing WHERE/WHY a search is failing (e.g. mapping which parameter region a given InvalidPowerStateError/CoolPropError clusters in) instead of re-deriving it from ad hoc scratch scripts each time.

False
extra_cv_exceptions tuple[type[Exception], ...]

Additional exception TYPES (beyond the built-in INFEASIBLE_DESIGN_EXCEPTIONS — CoolPropError/ FrackingViolationError/ConvergenceError/ InvalidPowerStateError) that _evaluate_one also treats as a constraint violation rather than letting crash the whole run. EXTENDS the built-in set (merged into self._cv_exceptions in __post_init__), never replaces it — a caller widens what counts as "this design is infeasible," it doesn't get to narrow the base physical-failure modes. Meant for a caller-specific exception type not (yet) common enough to belong in the shared base tuple — e.g. a surrogate-model evaluator's own "classifier predicts infeasible" signal (see tango_core.surrogate_api). Default () (no change from the base set).

()
capture_trajectory_path Path | str | None

If given, append every generation's newly-evaluated individuals (not just the final front) to this CSV path, e.g. as surrogate training data or to study how the front improves per generation — see _append_trajectory. Default None (no capture).

None

Raises:

Type Description
ValueError

If base_cfg's own sim_config.injection_mode is not one of the two allowed policies (see module docstring).

constraints = field(default_factory=list) class-attribute instance-attribute

Extra inequality constraints beyond the built-in physical-feasibility one — see ConstraintSpec. Default () (no extra constraints, identical behavior to before this field existed).

report_specs = field(default_factory=list) class-attribute instance-attribute

Extra per-design read-outs (SAME (wf) -> float shape as ObjectiveSpec) computed ONCE for the FINAL pareto front only and merged into the results dataframe/CSV/plots — NOT fed to NSGA2 as a real objective, so it never affects which designs survive or the front's shape/dimensionality. Use this for a value worth reporting (e.g. OPEX) that you deliberately don't want the search itself optimizing for or against. See _add_report_columns. Default () (no extra columns).

check_param_sensitivity = True class-attribute instance-attribute

Before searching, check that every param moves at least one objective or constraint (see _check_param_sensitivity); raises InertParameterError otherwise. Costs 1 + 2 * len(params) evaluations, parallelized like the search. Turn off for a setup already known to be good.

sensitivity_step_frac = 0.05 class-attribute instance-attribute

Size of the parameter check's up/down step, as a fraction of each param's bounds width.

capture_trajectory_path = None class-attribute instance-attribute

If given, append EVERY generation's newly-evaluated individuals (params + objectives if feasible + a success bool) to this CSV path, one row per individual — not just the final pareto front. Row schema matches tango_core.surrogate_api's training-data convention exactly, so train_surrogate() can consume the result unmodified regardless of whether it came from an LHS sweep or a real optimizer's own search trajectory. See _append_trajectory.

n_failures = field(init=False, default=0) class-attribute instance-attribute

Count of evaluations that failed the built-in feasibility constraint (a CV exception or a non-finite objective) — inspect after run(). Counted in run(), in the main process, from each generation's evaluated designs: with n_processes > 1 the evaluations run on copies of this object in worker processes, so a count kept in _evaluate_one would be lost.

__post_init__()

Validate policy, resolve base_cfg/algorithm/pad_sampling defaults.

NOTE for readers cross-checking against method bodies below: mypy does NOT treat these resolved-in-place values (pymoo_config, algorithm, save_dir, pad_sampling — every field here whose declared type admits None/str but is always resolved to a concrete value by the time __post_init__ returns) as narrowed everywhere else in the class the way a plain __init__'s inferred-per-assignment attribute types would be — a dataclass field's declared type is fixed from its class-level annotation, full stop (confirmed empirically: even re-annotating the assignment target here doesn't change what mypy sees in OTHER methods). run/_save_intermediate/ _write_pareto each carry a short assert/isinstance at the top to re-narrow -- see their docstrings/comments, not repeated here.

run()

Run the optimization and return pymoo's raw Result plus a results dataframe.

Runs generation-by-generation (algorithm.setup() + a while algorithm.has_next(): algorithm.next() loop) instead of pymoo's minimize() convenience wrapper — behaviorally identical (that's literally what minimize() does internally), but this gives a hook between generations to save the CURRENT non-dominated front after every one (_save_intermediate, if save_results) — if the run gets killed (SLURM time limit, ^C, crash) partway through, whatever the front looked like as of the LAST completed generation is already on disk, not lost. The same hook also drives _append_trajectory (if capture_trajectory_path), appending every generation's newly-evaluated individuals — not just the front — as surrogate training data.

Returns:

Type Description
Result

(res, df) — res is pymoo's own Result (res.X/

DataFrame

res.F are the pareto-front decision variables/objectives);

tuple[Result, DataFrame]

df has one row per pareto-front design, one column per

tuple[Result, DataFrame]

param, one column per objective, plus ideal_distance (see

tuple[Result, DataFrame]

_build_results_df).

ObjectiveScatter(x_col, x_label, y_col, y_label, y_scale=1.0, color_col=None) dataclass

Which two result columns an OptApp's objective-space scatter plots show.

Parameters:

Name Type Description Default
x_col str

Display column (after OptApp.display_rename) on the x axis.

required
x_label str

x-axis label.

required
y_col str

Display column on the y axis.

required
y_label str

y-axis label, in the units after y_scale.

required
y_scale float

Factor applied to y_col before plotting (e.g. 1e-6 for W -> MW).

1.0
color_col str | None

Display column the points are colored by; None for a single color.

None

OptApp(title, description, config_path, app_dir, injection_mode, default_bounds, param_factory, objective_specs, scatter, report_specs=list(), constraint_factory=None, configure_parser=None, display_rename=dict(), column_labels=dict(), smoke_args=list()) dataclass

Everything that differs between opt_* app scripts; the rest is shared here.

An app script builds one OptApp from its own ParamSpec/ ObjectiveSpec/ConstraintSpec factories (which stay in the app, see ParamSpec's docstring) and calls app.main(). This class supplies the CLI, the forced injection policy, the baseline run, the Minimization setup, and the plots, none of which depend on the system being optimized.

Each param's apply must also have a read(config) -> float method returning the value the config currently holds, in the param's own units. param_specs copies it into ParamSpec.read, which gives Minimization's parameter check its baseline and lets the smoke tests build narrow bounds around each config's own design point.

Parameters:

Name Type Description Default
title str

Short name used in plot titles, e.g. "AGS_ORC_electric".

required
description str

One-line --help description.

required
config_path Path

The template this app optimizes.

required
app_dir Path

The app's directory (holds params.yaml and outputs/).

required
injection_mode InjectionPolicy

Injection policy forced onto the base workflow, whatever the config says (manual or manual-fracking-check; see Minimization).

required
default_bounds dict[str, tuple[float, float]]

Production bounds per param name.

required
param_factory Callable[[dict[str, tuple[float, float]]], list[ParamSpec]]

bounds -> params; called with default_bounds or an override with the same keys.

required
objective_specs list[ObjectiveSpec]

Active NSGA2 objectives.

required
report_specs list[ObjectiveSpec]

Reporting-only read-outs (see Minimization).

list()
constraint_factory Callable[[Namespace], list[ConstraintSpec]] | None

parsed CLI args -> constraints, so a constraint threshold can be a CLI flag (see configure_parser).

None
configure_parser Callable[[ArgumentParser], None] | None

Adds app-specific CLI flags.

None
display_rename dict[str, str]

Negated-objective rename for plots (see prepare_plot_df).

dict()
column_labels dict[str, str]

Full axis label per display column.

dict()
scatter ObjectiveScatter

Axes of the objective-space scatter plots.

required
smoke_args list[str]

Extra CLI arguments --smoke adds, e.g. relaxing a constraint that a tiny population would otherwise miss entirely, leaving an empty Pareto front for the plots.

list()

params_file property

This app's params.yaml.

default_output_dir property

This app's outputs/.

param_specs(bounds=None)

Decision variables with default_bounds, or bounds (same keys) instead.

Raises:

Type Description
ValueError

If bounds' keys, or the factory's param names, don't match default_bounds' keys.

build_parser()

build_pymoo_parser plus --smoke and this app's own flags.

parse_args(argv=None)

Parse argv (default sys.argv[1:]); --smoke appends its settings, which win.

base_config()

The template with injection_mode forced to this app's policy.

build_baseline()

Run the config's own design point, to report the baseline objectives.

run_optimization(args, param_specs=None)

Run NSGA2 over param_specs (default: self.param_specs()).

Returns:

Type Description
tuple[Result, DataFrame, Minimization]

(res, df, m).

pretty_label(column)

column_labels[column] if there is one, else column.

make_plots(df, save_dir)

Build every plot from a pareto-front dataframe (bare column names).

No empty-df guard: an empty front means no feasible design was found, which should fail loudly here.

main(argv=None)

Run the baseline and the search, then plot — or one half, via --no-plot/--plot-only.

The split lets an app's dvc.yaml run "optimize" and "plot" as separate stages, so a plotting-only change reruns just the plots.

load_params(params_file)

Load optimization hyperparameters from a params.yaml file, if present.

Same pattern across every opt_* app script: DVC's dvc.yaml declares which of these keys it tracks for staleness, but the calling script is what actually reads the values — DVC doesn't inject them into the command line.

build_pymoo_parser(description, default_output_dir, params=None)

Build the shared CLI for opt_* pymoo-optimization app scripts.

Every opt_* script needs the exact same mechanical plumbing: exposing PymooConfig's own fields as CLI flags (an explicit CLI flag overrides params, which overrides PymooConfig's own default — one source of truth for "the default"), plus a handful of generic execution-control flags (--quiet, --log-level, --save-dir, --clean, --seed-from). None of that is domain-specific — unlike ParamSpec/ObjectiveSpec (see their own docstrings), which DO stay in the calling script because they encode assumptions about a specific physical system. This function is the shared, non-domain-specific half of an opt_* script's CLI; each script calls it, then can .add_argument(...) on the returned parser for anything script-specific it ever needs.

No --skip-minimize-style flag threading a previous-results cache: Minimization already manages its own worker pool internally via PymooConfig.n_processes, and there's no cached-results-loading feature to expose a flag for.

Parameters:

Name Type Description Default
description str

This script's own one-line description (shown in --help).

required
default_output_dir Path

Default --save-dir value.

required
params dict[str, Any] | None

Already-loaded params.yaml dict (see load_params).

None

strip_column_prefixes(df)

Undo Minimization._write_pareto's par:/obj:/rep: CSV column prefixes.

For a script's --plot-only path, which re-reads a previously-saved pareto_results.csv from disk: that file has the prefixed names (see Minimization._column_rename), but every plotting function (DISPLAY_RENAME/COLUMN_LABELS dicts, prepare_plot_df, etc.) still works in terms of bare names, matching what a freshly-run() Minimization returns in memory. Round-tripping through this function keeps that one consumer consistent with the other, rather than requiring plotting code to know about the on-disk-only prefix.

prepare_plot_df(df, display_rename)

Undo pymoo's negated-objective minimization convention, for plotting only.

A maximization objective (e.g. mean power/thermal power) is expressed to NSGA2 as negative_mean_*_W so minimizing it maximizes the real quantity — but a human reading a plot cares about power PRODUCED, not the sign flip NSGA2 needed. This renames columns per display_rename (e.g. {"negative_mean_power_W": "mean_power_W"}) and negates any renamed column whose ORIGINAL name started with "negative_" back to its natural sign — inferred from the name, not a separately-maintained list, so a script can't add/rename an objective without the negation rule automatically following it.

df/the saved CSV (pareto_results.csv) keep the objective's own (negated) sign always — only the returned copy is flipped. Use this (not a hand-rolled df.rename(...) + manual -df[col] line) for ANY plot built from pareto results, including one-off replotting from a previously-saved CSV — a hand-rolled version is exactly what silently reintroduced a raw negative_mean_thermal_power_W axis into opt_AGS_direct_heat's parallel-coordinates plot once, when a fix unrelated to this renaming/sign convention regenerated the plot straight from the CSV and skipped it.

plot_objective_vs_param_scatter(df, param_cols, objective_cols, out_path, title='Pareto front — objectives vs. parameters')

Grid of 2D scatter plots: one subplot per (objective, parameter) pair.

Complements the parallel-coordinates/objective-space plots every opt_* script already builds — those show the pareto front as a whole, but don't make it easy to read off "how does this ONE parameter move THIS ONE objective" at a glance, especially once a front has 3+ decision variables. One row per objective, one column per parameter, so a scan down any column shows that parameter's effect on every objective, and a scan across any row shows every parameter's effect on that objective. Deliberately generic (plain column-name lists, no assumption about how many params/objectives or what they're called) — same reasoning as build_pymoo_parser, not ParamSpec/ObjectiveSpec: this is mechanical plotting plumbing, not a domain-specific choice, so it belongs here rather than duplicated in every opt_* script.

plot_param_pair_vs_objective_scatter(df, param_cols, objective_cols, out_path, title='Pareto front — objective vs. parameter pairs')

Grid of 2D parameter-pair scatter plots, each point colored by an objective.

plot_objective_vs_param_scatter only shows one parameter's MARGINAL effect on an objective at a time — it can't reveal an interaction, e.g. "high LCOE only when mass rate AND injection pressure are BOTH high," where neither parameter alone would show the pattern in its own marginal plot. Here, each subplot's (x, y) axes are a PAIR of parameters and the point COLOR is one objective's value — a visual cross-correlation view an ordinary 1D scatter can't provide. One row per parameter pair (every 2-combination of param_cols, via itertools.combinations — order-independent, no duplicate/reversed pairs), one column per objective.

With only 2 parameters there is exactly one pair (one row); with N parameters there are N*(N-1)/2 pairs/rows, growing quickly — fine for the 3-4 parameter searches this module's opt_* callers currently run, but a caller with a much larger parameter count should expect a correspondingly tall figure.

Same generic, column-name-list design as plot_objective_vs_param_scatter (see that function's docstring for why this lives here rather than duplicated per opt_* script).

normalized_shortfall(f)

Each objective rescaled to 0-1 over the given designs: 0 = best, 1 = worst.

f holds minimized objectives, one row per design (as pymoo stores them; a maximized quantity is stored negated, e.g. negative_mean_power_W). An objective that's the same for every design maps to 0.

ideal_distance(f)

Each design's distance to the ideal corner of a set of designs.

The ideal corner pairs the best value of every objective found among the designs (for LCOE vs. power: the cheapest LCOE and the highest power). No single design usually reaches it. Each objective is first rescaled to 0-1 over the designs (normalized_shortfall), so objectives with different units count equally. The result runs from 0 (a design that's best in every objective) to sqrt(n_objectives). The design with the smallest distance is the default pick: the one that gives up the least in all objectives at once.

It depends only on the best and worst value of each objective, not on how the optimizer happened to space the designs along the front.

plot_pareto_parallel_coordinates(df, out_path, title, labels=None, color_col=IDEAL_DISTANCE_COL)

Static parallel-coordinates plot across every pareto-front design.

Min-max normalized axes, viridis_r coloring clipped to the 5th-95th percentile of color_col. Prepends a design_number axis (0-based, matching pareto_configs/design_NNN.yml) so a line can be traced back to its saved config. Operates on a copy of df.

plot_pareto_parallel_coordinates_interactive(df, out_path, title, labels=None, color_col=IDEAL_DISTANCE_COL)

Interactive HTML version of plot_pareto_parallel_coordinates (Plotly Parcoords).

Returns:

Type Description
bool

False (and writes nothing) if plotly isn't installed.

plot_pareto_objective_scatter(df, out_path, scatter, title, labels=None, *, shortfall=None)

2D scatter of two result columns, each point labeled with its design number.

Parameters:

Name Type Description Default
df DataFrame

Pareto results, display column names.

required
out_path Path

PNG to write.

required
scatter ObjectiveScatter

Axes and coloring.

required
title str

Plot title.

required
labels dict[str, str] | None

Colorbar label per display column.

None
shortfall NDArray[float64] | None

If given, plot this instead of the raw values: one row per design, columns for the x and y objectives, each rescaled to 0-1 (0 = best on the front; see normalized_shortfall). The ideal corner is then (0, 0), and the design closest to it (the smallest ideal_distance in these two objectives) is ringed.

None