otm#

Create and solve optimization models.

This module is responsible for creating and solving the optimization problems that try to find the optimal set-points to reduce production costs related to the pelletizing process.

This module creates one optimization problem for the following production ranges:

  • 700~750

  • 750~800

  • 800~850

  • 850~900

  • 900~950

  • 950~1000

wip.otm.build_restrictions(models_results: dict, tmp_path: str, models_coefficients: dict, datasets: dict, scalers: dict, models_features: dict, continue_limits: list, temp_limits: DataFrame, _critical_cols_dict: dict)[source]#

Write constraints for optimization models into text files.

This function builds constraints for optimization models by iterating over a predefined production range and generating constraints for each range. It also calculates lower and upper limits for each feature and applies various transformations and checks.

Flow:

  1. The function starts by extracting the production data from the datasets.

  2. It then iterates over a predefined production range using the track function.

  3. For each range, it creates a text file to write the constraints.

  4. It loops through the models results and retrieves the feature’s coefficients.

  5. It applies various checks and transformations to calculate the lower and upper limits for each feature.

  6. It writes the feature constraints into the text file.

  7. It writes the simple range terms, simple constraints, and complex constraints into the text file.

  8. It writes the special constraints, variable constraints, and targets limits into the text file.

Parameters

  • models_results (dict) – A dictionary with ridge regression models results, that are created by the wip.mltrainer module.

  • tmp_path (str) – Directory path, to save the constraint files to.

  • models_coefficients (dict) – Dictionary with the coefficients of each ridge regression model, that were created by the wip.mltrainer module.

  • datasets (dict) – Dictionary with the datasets that were used to train the ridge regression models. The keys of this dictionary are the names of the datasets and the values are the datasets themselves.

  • scalers (Dict[str, sklearn.preprocessing.MinMaxScaler]) – Dictionary with the scalers that were used to normalize the ridge data before training the ridge regression models. The keys of this dictionary are the names of the datasets and the values are the fitted scalers.

  • models_features (Dict[str, pd.Series]) – Dictionary with the features used to train the ridge regression models. The keys of this dictionary are the column names that were used during model training, and the values are pandas.Series with the column values.

  • continue_limits (List[str]) – List of tags to include in the optimization problem constraints.

  • temp_limits (pd.DataFrame) – DataFrame with the limits generated by SHAP.

  • _critical_cols_dict (dict) – Dictionary with lower and upper bounds for some of the optimization problem variables.

Notes

This function relies heavily on several external modules, classes and functions including datasets, constants, track, solver_operations, Limits, operations, and Constraints. It assumes these are available in the same context and that the data and methods they provide are in a specific format.

wip.otm.get_limits()[source]#

Get limits for the optimization problem.

Returns

A list of tags to continue.

Return type

List[str]

wip.otm.main_otm()[source]#

Define and solve the optimization problems for each production range.

wip.otm.process_critical_cols_dict(_critical_cols_dict: dict, _tags_ventiladores: List[str], scalers: Dict[str, MinMaxScaler]) → dict[source]#

Process the critical column’s dictionary.

Parameters

  • _critical_cols_dict (dict) –

    Dictionary with lower and upper bounds for the optimization problem variables. The keys of this dictionary are the tag names and the values are dictionaries, with one of the following structures:

    • Option 1: "<TAG-NAME>": {"lmin": <float or int>, "lmax": <float or int>}

    • Option 2: Specify different variable bounds for each production range:

      # ...
"<TAG-NAME>": {
    700: {"lmin": <float or int>, "lmax": <float or int>},
    750: {"lmin": <float or int>, "lmax": <float or int>},
    800: {"lmin": <float or int>, "lmax": <float or int>},
    850: {"lmin": <float or int>, "lmax": <float or int>},
    900: {"lmin": <float or int>, "lmax": <float or int>},
    950: {"lmin": <float or int>, "lmax": <float or int>},
},
# ...

  • _tags_ventiladores (List[str]) –

  • scalers (Dict[str, MinMaxScaler]) –

Return type

dict

_tags_ventiladoresList[str]

A list of tag names that represent the fan tags (“ventiladores”).

scalersDict[str, sklearn.preprocessing.MinMaxScaler]

A dictionary of tag’s scalers.

Returns

Dictionary with lower and upper bounds for the optimization problem variables.

Return type

dict

Parameters

Notes

The scalers dictionary values must contain the attributes:

  • data_min_: Contains the minimum values for each feature in the dataset that you’ve passed to the fit method. It is an array with the same number of entries as the number of features in your data.

  • data_max_: Contains the maximum values for each feature in the dataset that you’ve passed to the fit method. Similarly, it is an array with the same number of entries as the number of features in your data.

These attributes are only found inside the sklearn.preprocessing.MinMaxScaler class. If a new type of scaler is used, the roles that these attributes have must be replicated.

wip.otm.read_model_results() → tuple[source]#

Read machine learning model results from predefined file paths.

This function reads the results of machine learning models, scalers, model coefficients, model features, and datasets from their respective file paths. The file paths are predefined as constants in the code.

The function uses the read_joblib() function to read the data from the files.

Returns

A tuple containing five elements: - models_results: The results of the machine learning models. - scalers: Scikit-Learn scaler objects used to normalize or standardize data. - models_coeficients: The coefficients of the machine learning models. - models_features: The features used by the machine learning models. - datasets: The final datasets used to train or test the machine learning models. - df_sql: The tag values as a pandas.DataFrame, after applying all filters

and data transformations.

Return type

tuple

wip.otm.save_otm_results(solver: PulpSolver, scalers: Dict[str, sklearn.preprocessing.MinMaxScaler], datasets: Dict[str, pd.DataFrame], tmp_path: str | Path)[source]#

Save the optimization results of linear programming models.

This function saves the results of linear programming optimization using PuLP in various formats (lp, mps, xlsx, csv, pickle). If the optimization problem is infeasible or unbounded, it logs warnings and skips outputs generation for that problem.

Parameters

  • solver (PulpSolver) – The solver object that contains the results of linear programming optimization.

  • scalers (Dict[str, sklearn.preprocessing.MinMaxScaler]) – Dictionary containing the MinMaxScaler for each column in the dataset.

  • datasets (Dict[str, pd.DataFrame]) – Dictionary containing the input datasets for the optimization problems.

  • tmp_path (str | Path) – The path to the temporary directory where the results should be saved.

Notes

This function has dependencies on certain functions and objects that are not defined in the provided code, such as is_running_on_databricks, LpScaledConstraint, logger, lp_variables_to_excel, and define_optimization_results.