diff --git a/docs/source/reference/configuration.md b/docs/source/reference/configuration.md index a7a7e2e05..f6352cbaf 100644 --- a/docs/source/reference/configuration.md +++ b/docs/source/reference/configuration.md @@ -1,20 +1,22 @@ # Design Configuration Files + Unless the design uses the API directly, each OpenLane-compatible design must come with a configuration file. These configuration files can be written in one of two grammars: JSON or Tcl. Tcl offers more flexibility at the detriment of security, while JSON is more -straightforward at the cost of flexbility. While Tcl allows you to do all -manner of computation on your variables, JSON has a limited expression engine -that will be detailed later in this document. Nevertheless, for security (and +straightforward at the cost of flexbility. While Tcl allows you to do all manner +of computation on your variables, JSON has a limited expression engine that will +be detailed later in this document. Nevertheless, for security (and future-proofing), we recommend you use either the JSON format or write Python scripts using the API. -The folder containing your `config.tcl`/`config.json` is known as the -**Design Directory**. The design directory is special in that paths in the JSON -configuration files can be resolved relative to this directory and that in TCL -configuration files, it can be referenced via the environment variable -`DESIGN_DIR`. This will be explained in detail in later sections. +The folder containing your `config.tcl`/`config.json` is known as the **Design +Directory** -- though the design directory can also be set explicitly over the +command-line using `--design-dir`. The design directory is special in that paths +in the JSON configuration files can be resolved relative to this directory and +that in TCL configuration files, it can be referenced via the environment +variable `DESIGN_DIR`. This will be explained in detail in later sections. ```{note} When using the API, you can provide the inputs directly as a Python dictionary, @@ -23,12 +25,13 @@ JSON or Tcl files. You can still use `ref::` and such like JSON files though. ``` ## JSON + The JSON files are simple key-value pairs. The values can be scalars (strings, numbers, booleans, and `null`s), lists or -dictionaries, subject to validation. +dictionaries, subject to validation. All files must be ECMA404-compliant, i.e., pure JSON with no extensions such as comments or the new elements introduced in [JSON5](https://json5.org/). @@ -55,8 +58,8 @@ An minimal demonstrative configuration file would look as follows: ### Pre-processing The JSON files are pre-processed at runtime. Features include conditional -execution, a way to reference the design directory, other variables, -and a basic numeric expression engine. +execution, a way to reference the design directory, other variables, and a basic +numeric expression engine. #### Conditional Execution @@ -69,11 +72,10 @@ SCL matches those in the key, i.e., for `pdk::sky130A` as shown above, this particular `dict` will be evaluated and its values used if and only if the PDK is set to `sky130A`, meanwhile with say, `asap7`, it will not be evaluated. - - -The match is evaluated using [`fnmatch`](https://docs.python.org/3.6/library/fnmatch.html), -giving it limited wildcard support: meaning that `pdk::sky130*` would match both -`sky130A` and `sky130B`. +The match is evaluated using +[`fnmatch`](https://docs.python.org/3.6/library/fnmatch.html), giving it limited +wildcard support: meaning that `pdk::sky130*` would match both `sky130A` and +`sky130B`. Note that ***the order of declarations matter here***: as seen in the following example, despite a more specific value for a PDK existing, the unconditionally @@ -94,14 +96,14 @@ declared value later in the code would end up overwriting it: } } ``` + > In the first example, the final value for A would always be 4 given the order > of declarations. In the second example, it would be 40 is the PDK is sky130A > and 4 otherwise. -It is worth nothing that the final resolved configuration would have the -symbol in the parent object with no trace left of the conditionally-executed, -dict i.e., the second example with the -sky130A PDK simply becomes: +It is worth nothing that the final resolved configuration would have the symbol +in the parent object with no trace left of the conditionally-executed, dict +i.e., the second example with the sky130A PDK simply becomes: ```json { @@ -128,16 +130,18 @@ reference a variable that is declared after the current expression. "A": "ref::$B" } ``` + > In this example, the first configuration is invalid, as B is referenced before -> it is declared, but the latter is OK, where the value will be "vdd gnd" as well. +> it is declared, but the latter is OK, where the value will be "vdd gnd" as +> well. Do note that unlike Tcl config files, environment variables (other than `DESIGN_DIR`, `PDK`, `PDKPATH`, `STD_CELL_LIBRARY`) are not exposed to `config.json` by default. If the files you choose lie **inside** the design directory, a different prefix, -`refg::`, supports non-recursive globs, i.e., you can use an -asterisk as a wildcard to pick multiple files in a specific folder. +`refg::`, supports non-recursive globs, i.e., you can use an asterisk as a +wildcard to pick multiple files in a specific folder. * Outside the design directory, this is disabled for security reasons and the final path will continue to include the asterisk. @@ -156,25 +160,25 @@ in the `src` folder inside the design directory. ``` There are some shorthands for the exposed default variables: + * `dir::` is equivalent to `refg::$DESIGN_DIR/` * `pdk_dir::` is equivalent to `refg::$PDK_ROOT/$PDK` - #### Expression Engine By adding `expr::` to the beginning of a string, you can write basic infix mathematical expressions. Binary operators supported are `**`, `*`, `/`, `+`, -and `-`, while operands can be any floating-point value, and previously evaluated -numeric variables prefixed with a dollar sign. Unary operators are not supported, -though negative numbers with the - sign stuck to them are. Parentheses (`()`) -are also supported to prioritize certain operations. +and `-`, while operands can be any floating-point value, and previously +evaluated numeric variables prefixed with a dollar sign. Unary operators are not +supported, though negative numbers with the - sign stuck to them are. +Parentheses (`()`) are also supported to prioritize certain operations. -Your expressions must return exactly one value: multiple expressions in the -same `expr::`-prefixed value are considered invalid and so are empty expressions. +Your expressions must return exactly one value: multiple expressions in the same +`expr::`-prefixed value are considered invalid and so are empty expressions. -It is important to note that, like variable referencing and conditional execution, -the order of declarations matter: i.e., you cannot reference a variable that is -declared after the current expression. +It is important to note that, like variable referencing and conditional +execution, the order of declarations matter: i.e., you cannot reference a +variable that is declared after the current expression. ```json { @@ -187,8 +191,10 @@ declared after the current expression. "A": "expr::$B * 2" } ``` + > In this example, the first configuration is invalid, as B is used in a -> mathematical expression before declaration, but the latter is OK, evaluating to 8. +> mathematical expression before declaration, but the latter is OK, evaluating +> to 8. You can also simply reference another number using this prefix: @@ -198,9 +204,11 @@ You can also simply reference another number using this prefix: "B": "expr::$A" } ``` + > In this example, B will simply hold the value of A. ## Tcl + These configuration files are simple Tcl scripts with environment variables that are sourced by the OpenLane flow. Again, Tcl config files are not recommended for newer designs, but is still maintained and supported at the moment. diff --git a/openlane/__main__.py b/openlane/__main__.py index accf844fc..b00d42bb3 100644 --- a/openlane/__main__.py +++ b/openlane/__main__.py @@ -45,7 +45,7 @@ from . import common from .container import run_in_container from .plugins import discovered_plugins -from .config import Config, InvalidConfig +from .config import Config, InvalidConfig, PassedDirectoryError from .common.cli import formatter_settings from .flows import Flow, SequentialFlow, FlowException, FlowError, cloup_flow_opts @@ -66,45 +66,54 @@ def run( with_initial_state: Optional[State], config_override_strings: List[str], _force_run_dir: Optional[str], - _force_design_dir: Optional[str], -) -> int: - if len(config_files) == 0: - err("No config file has been provided.") - ctx.exit(1) - elif len(config_files) > 1: - err("OpenLane does not currently support multiple configuration files.") - ctx.exit(1) - config_file = config_files[0] - # Enforce Mutual Exclusion + design_dir: Optional[str], +): + try: + if len(config_files) == 0: + err("No config file(s) have been provided.") + ctx.exit(1) - flow_description: Union[str, List[str]] = flow_name or "Classic" + flow_description: Optional[Union[str, List[str]]] = None - if meta := Config.get_meta(config_file, flow_override=flow_name): - if flow_ids := meta.flow: - flow_description = flow_ids + for config_file in config_files: + if meta := Config.get_meta(config_file, flow_override=flow_name): + if flow_ids := meta.flow: + if flow_description is None: + flow_description = flow_ids - TargetFlow: Type[Flow] + if flow_name is not None: + flow_description = flow_name - if not isinstance(flow_description, str): - TargetFlow = SequentialFlow.make(flow_description) - else: - if FlowClass := Flow.factory.get(flow_description): - TargetFlow = FlowClass + if flow_description is None: + flow_description = "Classic" + + TargetFlow: Type[Flow] + + if not isinstance(flow_description, str): + TargetFlow = SequentialFlow.make(flow_description) else: - err( - f"Unknown flow '{flow_description}' specified in configuration file's 'meta' object." - ) - return -1 + if FlowClass := Flow.factory.get(flow_description): + TargetFlow = FlowClass + else: + err( + f"Unknown flow '{flow_description}' specified in configuration file's 'meta' object." + ) + ctx.exit(1) - try: flow = TargetFlow( - config_file, + config_files, pdk_root=pdk_root, pdk=pdk, scl=scl, config_override_strings=config_override_strings, - _force_design_dir=_force_design_dir, + design_dir=design_dir, ) + except PassedDirectoryError as e: + err(e) + info( + f"If you meant to pass this as a design directory alongside valid configuration files, pass it as '--design-dir {e.config}'." + ) + ctx.exit(1) except InvalidConfig as e: info(f"[green]Errors have occurred while loading the {e.config}.") for error in e.errors: @@ -115,12 +124,12 @@ def run( for warning in e.warnings: warn(warning) info("OpenLane will now quit. Please check your configuration.") - return 1 + ctx.exit(1) except ValueError as e: err(e) debug(traceback.format_exc()) info("OpenLane will now quit.") - return 1 + ctx.exit(1) try: flow.start( @@ -137,14 +146,12 @@ def run( err(f"The flow has encountered an unexpected error: {e}") traceback.print_exc() err("OpenLane will now quit.") - return 1 + ctx.exit(1) except FlowError as e: if "deferred" not in str(e): err(f"The following error was encountered while running the flow: {e}") err("OpenLane will now quit.") - return 2 - - return 0 + ctx.exit(2) def print_version(ctx: Context, param: Parameter, value: bool): @@ -250,7 +257,7 @@ def run_example( with_initial_state=None, config_override_strings=[], _force_run_dir=None, - _force_design_dir=None, + design_dir=None, ) if status == 0: info("Smoke test passed.") @@ -403,8 +410,8 @@ def cli(ctx, /, **kwargs): ]: if subcommand_flag in run_kwargs: del run_kwargs[subcommand_flag] - - ctx.exit(run(ctx, **run_kwargs)) + run(ctx, **run_kwargs) + ctx.exit(0) if __name__ == "__main__": diff --git a/openlane/common/__init__.py b/openlane/common/__init__.py index cd3d1ffe4..ce2bf3127 100644 --- a/openlane/common/__init__.py +++ b/openlane/common/__init__.py @@ -50,6 +50,7 @@ is_string, Number, Path, + AnyPath, ScopedFile, ) from .toolbox import Toolbox diff --git a/openlane/common/generic_dict.py b/openlane/common/generic_dict.py index 73c767cec..7fafab19f 100644 --- a/openlane/common/generic_dict.py +++ b/openlane/common/generic_dict.py @@ -211,11 +211,24 @@ def update(self, incoming: "Mapping[KT, VT]"): """ A convenience function to update multiple values in the GenericDict object at the same time. - :param + :param incoming: The values to update """ for key, value in incoming.items(): self[key] = value + def update_reorder(self, incoming: "Mapping[KT, VT]"): + """ + A convenience function to update multiple values in the GenericDict object + at the same time. Pre-existing keys are deleted first so the values in + incoming are emplaced at the end of the dictionary. + + :param incoming: The values to update + """ + for key, value in incoming.items(): + if key in self: + del self[key] + self[key] = value + class GenericImmutableDict(GenericDict[KT, VT]): __lock: bool diff --git a/openlane/common/types.py b/openlane/common/types.py index f02b969a9..c52b1d11b 100644 --- a/openlane/common/types.py +++ b/openlane/common/types.py @@ -90,6 +90,9 @@ def rel_if_child( return Path(my_abspath) +AnyPath = Union[str, os.PathLike] + + class ScopedFile(Path): """ Creates a temporary file that remains valid while this variable is in scope, diff --git a/openlane/config/__init__.py b/openlane/config/__init__.py index 590157d69..2a59f3c51 100644 --- a/openlane/config/__init__.py +++ b/openlane/config/__init__.py @@ -20,5 +20,13 @@ """ from .preprocessor import Keys from .variable import Instance, Macro, Variable -from .config import Meta, Config, InvalidConfig +from .config import ( + Meta, + Config, + InvalidConfig, + AnyConfig, + AnyConfigs, + PassedDirectoryError, + UnknownExtensionError, +) from .flow import flow_common_variables as universal_flow_config_variables diff --git a/openlane/config/__main__.py b/openlane/config/__main__.py new file mode 100644 index 000000000..2ca5ce204 --- /dev/null +++ b/openlane/config/__main__.py @@ -0,0 +1,158 @@ +# Copyright 2023 Efabless Corporation +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +import os +import sys +import json +import functools +from decimal import Decimal + +import click + +from .config import Config +from ..flows.flow import universal_flow_config_variables +from ..steps.yosys import verilog_rtl_cfg_vars +from ..flows.cli import cloup_flow_opts + + +@click.group +def cli(): + pass + + +@click.command() +@click.option( + "--file-name", + type=click.Path(exists=False, file_okay=True, dir_okay=False), + default="config.json", + prompt="Please input the file name for the configuration file", + help="The file name of the configuration file.", +) +@click.option( + "--design-dir", + type=click.Path(exists=True, dir_okay=True, file_okay=False), + default=".", + prompt="Enter the base directory for your design", + help="The top-level design directory. Typically, the configuration file goes in the design directory as well.", +) +@click.option( + "--design-name", + "--top-module", + type=str, + prompt="Enter the design name (which should be equal to the HDL name of your top module)", + help="The name of the design, i.e. the name of the top-level module of the design.", +) +@click.option( + "--clock-port", + type=str, + prompt="Enter the name of your design's clock port", + help="The identifier for the clock port.", +) +@click.option( + "--clock-period", + type=Decimal, + prompt="Enter your desired clock period in nanoseconds", + help="The clock period, in nanoseconds.", +) +@cloup_flow_opts( + config_options=False, + run_options=False, + sequential_flow_controls=False, + jobs=False, + accept_config_files=False, +) +@click.argument( + "source_rtl", + type=click.Path( + exists=True, + dir_okay=False, + file_okay=True, + ), + nargs=-1, +) +def create_config( + pdk_root, + pdk, + scl, + file_name, + design_name, + design_dir, + clock_port, + clock_period, + source_rtl, +): + """ + Generates an OpenLane JSON configuration file for a design interactively. + """ + if len(source_rtl) == 0: + source_rtl = [] + try: + while True: + file = input( + f"Input the RTL source file #{len(source_rtl)} (Ctrl+D to stop): " + ) + if not os.path.isfile(file): + print(f"Invalid file {file}.", file=sys.stderr) + exit(1) + source_rtl.append(file) + except EOFError: + print("") + if len(source_rtl) == 0: + print("At least one source RTL file is required.", file=sys.stderr) + exit(1) + source_rtl_key = "VERILOG_FILES" + if not functools.reduce( + lambda acc, x: acc and (x.endswith(".sv") or x.endswith(".v")), source_rtl, True + ): + print( + "Only Verilog/SystemVerilog files are supported by create-config.", + file=sys.stderr, + ) + exit(-1) + source_rtl_rel = [f"dir::{os.path.relpath(x, design_dir)}" for x in source_rtl] + config_dict = { + "DESIGN_NAME": design_name, + "CLOCK_PORT": clock_port, + "CLOCK_PERIOD": clock_period, + source_rtl_key: source_rtl_rel, + "meta": { + "version": 2, + }, + } + config, _ = Config.load( + config_dict, + universal_flow_config_variables + verilog_rtl_cfg_vars, + design_dir=design_dir, + pdk=pdk, + pdk_root=pdk_root, + scl=scl, + ) + with open(file_name, "w") as f: + print( + json.dumps(config_dict, cls=config.get_encoder(), indent=4), + file=f, + ) + + design_dir_opt = "" + if os.path.abspath(design_dir) != os.path.abspath(os.path.dirname(file_name)): + design_dir_opt = f"--design-dir {design_dir} " + + print(f"Wrote config to '{file_name}'.") + print("To run this design, invoke:") + print(f"\topenlane {design_dir_opt}{file_name}") + + +cli.add_command(create_config) + +if __name__ == "__main__": + cli() diff --git a/openlane/config/config.py b/openlane/config/config.py index cce991d1c..1fd08f763 100644 --- a/openlane/config/config.py +++ b/openlane/config/config.py @@ -30,7 +30,6 @@ List, Optional, Sequence, - Callable, Dict, Set, ) @@ -44,7 +43,55 @@ from .preprocessor import preprocess_dict, Keys as SpecialKeys from ..logging import info, warn from ..__version__ import __version__ -from ..common import GenericDict, GenericImmutableDict, TclUtils, Path +from ..common import ( + GenericDict, + GenericImmutableDict, + TclUtils, + Path, + AnyPath, + is_string, +) + +AnyConfig = Union[AnyPath, Mapping[str, Any]] +AnyConfigs = Union[AnyConfig, Sequence[AnyConfig]] + + +class UnknownExtensionError(ValueError): + """ + When a passed configuration file has an unrecognized extension, i.e., + not .json or .tcl. + """ + + def __init__(self, config: AnyPath) -> None: + self.config = str(config) + _, ext = os.path.splitext(config) + super().__init__( + f"Unsupported configuration file extension '{ext}' for '{config}'." + ) + + +class PassedDirectoryError(ValueError): + """ + When a passed configuration file is in fact a directory. + """ + + def __init__(self, config: AnyPath) -> None: + self.config = str(config) + super().__init__( + "Passing design directories as arguments is unsupported in OpenLane 2 or higher: please pass the configuration file(s) directly." + ) + + +def _validate_config_file(config: AnyPath) -> Literal["json", "tcl"]: + config = str(config) + if config.endswith(".tcl"): + return "tcl" + elif config.endswith(".json"): + return "json" + elif os.path.isdir(config): + raise PassedDirectoryError(config) + else: + raise UnknownExtensionError(config) class InvalidConfig(ValueError): @@ -258,22 +305,30 @@ def with_increment( @classmethod def get_meta( Self, - json_config_in: Union[str, os.PathLike], + config_in: AnyConfig, flow_override: Optional[str] = None, - ) -> Optional[Meta]: + ) -> Meta: """ - Returns the Meta object of a JSON configuration file + Returns the Meta object of a configuration dictionary or file. - :param config_in: A configuration file. - :returns: Either a Meta object, or if the file is not a JSON file, None. + :param config_in: A configuration object or file. + :returns: Either a Meta object, or if the file is invalid, None. """ - try: - obj = json.load(open(json_config_in, encoding="utf8")) - except (json.JSONDecodeError, IsADirectoryError): - return None + default_meta_version = 2 if isinstance(config_in, Mapping) else 1 + + if is_string(config_in): + config_in = str(config_in) + validated_type = _validate_config_file(config_in) + if validated_type == "tcl": + return Meta(version=1) + elif validated_type == "json": + config_in = json.load(open(config_in, encoding="utf8")) - meta = Meta() - if meta_raw := obj.get("meta"): + assert not isinstance(config_in, str) + assert not isinstance(config_in, os.PathLike) + + meta = Meta(version=default_meta_version) + if meta_raw := config_in.get("meta"): meta = Meta(**meta_raw) if flow_override is not None: @@ -354,7 +409,7 @@ def interactive( @classmethod def load( Self, - config_in: Union[str, os.PathLike, Mapping[str, Any]], + config_in: AnyConfigs, flow_config_vars: Sequence[Variable], *, config_override_strings: Optional[Sequence[str]] = None, @@ -363,7 +418,6 @@ def load( scl: Optional[str] = None, design_dir: Optional[str] = None, _load_pdk_configs: bool = True, - _force_design_dir: Optional[str] = None, ) -> Tuple["Config", str]: """ Creates a new Config object based on a Tcl file, a JSON file, or a @@ -382,8 +436,12 @@ def load( NAME=VALUE strings. These are primarily for running OpenLane from the command-line and strictly speaking should not be used in the API. - :param design_dir: The design directory for said configuration. - Supported and required *if and only if* config_in is a dictionary. + :param design_dir: The design directory for said configuration(s). + + If not explicitly provided, the design directory will be the + directory holding the last file in the list. + + If no files are provided, this argument is required. :param pdk: A process design kit to use. Required unless specified via the "PDK" key in a configuration object. @@ -398,56 +456,81 @@ def load( :returns: A tuple containing a Config object and the design directory. """ - loader: Callable = Self.__loads - raw: Union[str, Mapping] = "" - default_meta_version = 1 - if not isinstance(config_in, Mapping): - config_in = os.path.abspath(config_in) - - if design_dir is not None: - raise TypeError( - "The argument design_dir is not supported when config_in is not a dictionary." - ) - - design_dir = _force_design_dir or str(os.path.dirname(config_in)) - - config_in = str(config_in) - if config_in.endswith(".json"): - raw = open(config_in, encoding="utf8").read() - elif config_in.endswith(".tcl"): - raw = open(config_in, encoding="utf8").read() - loader = Self.__loads_tcl + if isinstance(config_in, Mapping): + config_in = [config_in] + elif is_string(config_in): + config_in = [str(config_in)] + + assert not isinstance(config_in, str) + assert not isinstance(config_in, os.PathLike) + + if len(config_in) == 0: + raise ValueError("The value for config_in must not be empty.") + + file_design_dir = None + configs_validated: List[AnyConfig] = [] + for config in config_in: + if isinstance(config, Mapping): + configs_validated.append(config) + # Path else: - if os.path.isdir(config_in): - raise ValueError( - "Passing design folders as arguments is unsupported in OpenLane 2 or higher: please pass the JSON configuration file directly." + config = str(config) + _validate_config_file(config) + config_abspath = os.path.abspath(config) + file_design_dir = os.path.dirname(config_abspath) + configs_validated.append(config_abspath) + + design_dir = design_dir or file_design_dir + if design_dir is None: + raise ValueError( + "The design_dir argument is required when configuration dictionaries are used." + ) + + config_obj = Config() + for config_validated in configs_validated: + try: + meta = Self.get_meta(config_validated) + except TypeError as e: + identifier = "configuration dict" + if is_string(config_validated): + identifier = os.path.relpath(str(config_validated)) + raise InvalidConfig(identifier, [], [f"'meta' object is invalid: {e}"]) + + assert meta is not None + + mapping = None + if isinstance(config_validated, Mapping): + mapping = config_validated + elif isinstance(config_validated, str): + if config_validated.endswith(".tcl"): + mapping = Self.__mapping_from_tcl( + config_validated, + design_dir, + pdk_root=pdk_root, + pdk=pdk, + scl=scl, + ) + else: + mapping = json.load( + open(config_validated, encoding="utf8"), parse_float=Decimal ) - _, ext = os.path.splitext(config_in) - raise ValueError( - f"Unsupported configuration file extension '{ext}' for '{config_in}'." - ) - else: - default_meta_version = 2 - if design_dir is None: - raise TypeError( - "The argument design_dir is required when using attempting to load a Config with a dictionary." - ) - raw = config_in - loader = Self.__load_dict - loaded = loader( - raw, - design_dir, - flow_config_vars=flow_config_vars, - pdk_root=pdk_root, - pdk=pdk, - scl=scl, - config_override_strings=(config_override_strings or []), - default_meta_version=default_meta_version, - _load_pdk_configs=_load_pdk_configs, - ) + assert mapping is not None, "Invalid validated config" + mutable = config_obj.copy_mut() + mutable.update_reorder(mapping) + config_obj = Self.__load_dict( + mutable, + design_dir, + flow_config_vars=flow_config_vars, + pdk_root=pdk_root, + pdk=pdk, + scl=scl, + config_override_strings=(config_override_strings or []), + meta=meta, + _load_pdk_configs=_load_pdk_configs, + ) - return (loaded, design_dir) + return (config_obj, design_dir) ## For Jupyter def _repr_markdown_(self) -> str: # pragma: no cover @@ -476,20 +559,6 @@ def _repr_markdown_(self) -> str: # pragma: no cover ) ## Private Methods - @classmethod - def __loads( - Self, - json_str: str, - *args, - **kwargs, - ): - raw = json.loads(json_str, parse_float=Decimal) - return Self.__load_dict( - raw, - *args, - **kwargs, - ) - @classmethod def __load_dict( Self, @@ -497,26 +566,18 @@ def __load_dict( design_dir: str, flow_config_vars: Sequence[Variable], *, + meta: Meta, config_override_strings: Sequence[str], # Unused, kept for API consistency pdk_root: Optional[str] = None, pdk: Optional[str] = None, scl: Optional[str] = None, full_pdk_warnings: bool = False, - default_meta_version: int = 1, _load_pdk_configs: bool = True, ) -> "Config": raw = dict(mapping_in) - meta: Optional[Meta] = None - if raw.get("meta") is not None: - meta_raw = raw["meta"] + if "meta" in raw: del raw["meta"] - try: - meta = Meta(**meta_raw) - except TypeError as e: - raise InvalidConfig( - "design configuration file", [], [f"'meta' object is invalid: {e}"] - ) flow_option_vars = [] flow_pdk_vars = [] @@ -526,9 +587,6 @@ def __load_dict( else: flow_option_vars.append(variable) - if meta is None: - meta = Meta(version=default_meta_version) - override_keys = set() for string in config_override_strings: key, value = string.split("=", 1) @@ -619,32 +677,21 @@ def __load_dict( return Config(processed, meta=meta) @classmethod - def __loads_tcl( + def __mapping_from_tcl( Self, - config: str, + config: AnyPath, design_dir: str, - flow_config_vars: Sequence[Variable], *, - config_override_strings: Sequence[str], pdk_root: Optional[str] = None, pdk: Optional[str] = None, scl: Optional[str] = None, - full_pdk_warnings: bool = False, - default_meta_version: int = 1, # Unused, kept for API consistency - _load_pdk_configs: bool = True, # Unused, kept for API consistency - ) -> "Config": + ) -> Mapping[str, Any]: + config_str = open(config, encoding="utf8").read() + warn( "Support for .tcl configuration files is deprecated. Please migrate to a .json file at your earliest convenience." ) - flow_option_vars = [] - flow_pdk_vars = [] - for variable in flow_config_vars: - if variable.pdk: - flow_pdk_vars.append(variable) - else: - flow_option_vars.append(variable) - pdk_root = Self.__resolve_pdk_root(pdk_root) tcl_vars_in = GenericDict( @@ -655,7 +702,7 @@ def __loads_tcl( ) tcl_vars_in[SpecialKeys.scl] = "" tcl_vars_in[SpecialKeys.design_dir] = design_dir - tcl_config = GenericDict(TclUtils._eval_env(tcl_vars_in, config)) + tcl_config = GenericDict(TclUtils._eval_env(tcl_vars_in, config_str)) process_info = preprocess_dict( tcl_config, @@ -670,44 +717,20 @@ def __loads_tcl( "The pdk argument is required as the configuration object lacks a 'PDK' key." ) - mutable, _, scl = Self.__get_pdk_config( + _, _, scl = Self.__get_pdk_config( pdk=pdk, scl=scl, pdk_root=pdk_root, - full_pdk_warnings=full_pdk_warnings, - flow_pdk_vars=flow_pdk_vars, + full_pdk_warnings=False, ) tcl_vars_in[SpecialKeys.pdk] = pdk tcl_vars_in[SpecialKeys.scl] = scl tcl_vars_in[SpecialKeys.design_dir] = design_dir - mutable.update(GenericDict(TclUtils._eval_env(tcl_vars_in, config))) - for string in config_override_strings: - key, value = string.split("=", 1) - mutable[key] = value - - processed, design_warnings, design_errors = Config.__process_variable_list( - mutable, - list(flow_config_vars), - [], - removed_variables, - on_unknown_key="warn", - ) - - if len(design_errors) != 0: - raise InvalidConfig( - "design configuration file", design_warnings, design_errors - ) - - if len(design_warnings) > 0: - info( - "Loading the design configuration file has generated the following warnings:" - ) - for warning in design_warnings: - warn(warning) + tcl_mapping = GenericDict(TclUtils._eval_env(tcl_vars_in, config_str)) - return Config(processed) + return tcl_mapping @classmethod def __resolve_pdk_root( diff --git a/openlane/flows/cli.py b/openlane/flows/cli.py index f64f14f44..a6f96a75b 100644 --- a/openlane/flows/cli.py +++ b/openlane/flows/cli.py @@ -267,20 +267,20 @@ def decorator(f): callback=initial_state_cb, help="Use this JSON file as an initial state. If this is not specified, the latest `state_out.json` of the run directory will be used if available.", )(f) + f = o( + "--design-dir", + "design_dir", + type=Path( + exists=True, + file_okay=False, + dir_okay=True, + ), + default=None, + help="The top-level directory for your design that configuration objects may resolve paths relative to.", + )(f) if _enable_debug_flags: f = option_group( "Debug flags", - o( - "--force-design-dir", - "_force_design_dir", - type=Path( - exists=True, - file_okay=False, - dir_okay=True, - ), - hidden=True, - default=None, - ), o( "--force-run-dir", "_force_run_dir", diff --git a/openlane/flows/flow.py b/openlane/flows/flow.py index 61fcc4941..e39f0368c 100644 --- a/openlane/flows/flow.py +++ b/openlane/flows/flow.py @@ -44,11 +44,7 @@ ) from deprecated.sphinx import deprecated -from ..config import ( - Config, - Variable, - universal_flow_config_variables, -) +from ..config import Config, Variable, universal_flow_config_variables, AnyConfigs from ..state import State from ..steps import Step from ..logging import ( @@ -287,7 +283,7 @@ class Flow(ABC): def __init__( self, - config: Union[Config, str, os.PathLike, Dict], + config: AnyConfigs, *, name: Optional[str] = None, pdk: Optional[str] = None, @@ -295,7 +291,6 @@ def __init__( scl: Optional[str] = None, design_dir: Optional[str] = None, config_override_strings: Optional[Sequence[str]] = None, - _force_design_dir: Optional[str] = None, ): if self.__class__.Steps == NotImplemented: raise NotImplementedError( @@ -321,7 +316,6 @@ def __init__( pdk_root=pdk_root, scl=scl, design_dir=design_dir, - _force_design_dir=_force_design_dir, ) self.config: Config = config diff --git a/setup.py b/setup.py index 449d100ce..8acd91746 100755 --- a/setup.py +++ b/setup.py @@ -51,6 +51,7 @@ "console_scripts": [ "openlane = openlane.__main__:cli", "openlane.steps = openlane.steps.__main__:cli", + "openlane.config = openlane.config.__main__:cli", "openlane.env_info = openlane:env_info_cli", ] }, diff --git a/test/config/test_config.py b/test/config/test_config.py index 4b7d4d7b2..29eabf520 100644 --- a/test/config/test_config.py +++ b/test/config/test_config.py @@ -165,6 +165,67 @@ def test_tcl_config(): ), "Generated configuration does not match expected value" +@pytest.mark.usefixtures("_mock_conf_fs") +@mock_variables() +def test_mixed_configs(): + from openlane.config import Meta, Config + + with open("/cwd/config.json", "w") as f: + f.write( + """ + { + "meta": { + "version": 2, + "flow": "Whatever" + }, + "DEFAULT_CORNER": "ref::$DESIGN_NAME" + } + """ + ) + + with open("/cwd/config2.tcl", "w") as f: + f.write( + """ + set ::env(EXAMPLE_PDK_VAR) "30" + """ + ) + + cfg, _ = Config.load( + [ + {"DESIGN_NAME": "whatever", "VERILOG_FILES": "dir::src/*.v"}, + "/cwd/config2.tcl", + "/cwd/config.json", + ], + config.flow_common_variables, + pdk="dummy", + scl="dummy_scl", + pdk_root="/pdk", + ) + + assert cfg == Config( + { + "DESIGN_DIR": "/cwd", + "DESIGN_NAME": "whatever", + "PDK_ROOT": "/pdk", + "PDK": "dummy", + "STD_CELL_LIBRARY": "dummy_scl", + "VERILOG_FILES": ["/cwd/src/a.v", "/cwd/src/b.v"], + "EXAMPLE_PDK_VAR": Decimal("30"), + "GRT_REPAIR_ANTENNAS": True, + "RUN_HEURISTIC_DIODE_INSERTION": False, + "DIODE_ON_PORTS": "none", + "MACROS": None, + "TECH_LEFS": { + "nom_*": Path( + "/pdk/dummy/libs.ref/techlef/dummy_scl/dummy_tech_lef.tlef" + ) + }, + "DEFAULT_CORNER": "whatever", + }, + meta=Meta(version=2, flow="Whatever"), + ), "Generated configuration does not match expected value" + + @pytest.mark.usefixtures("_mock_conf_fs") @mock_variables() def test_copy_filtered(): diff --git a/test/conftest.py b/test/conftest.py index 71686d066..5d1d4eaf7 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -65,7 +65,7 @@ def _mock_conf_fs(): if { ![info exists ::env(STD_CELL_LIBRARY)] } { set ::env(STD_CELL_LIBRARY) "dummy2_scl" } - set ::env(TECH_LEF) "/pdk/dummy2/libs.ref/techlef/dummy_scl/dummy_tech_lef.tlef" + set ::env(TECH_LEF) "/pdk/dummy2/libs.ref/techlef/dummy2_scl/dummy_tech_lef.tlef" set ::env(LIB_SYNTH) "sky130_fd_sc_hd__tt_025C_1v80.lib" """, ) @@ -73,7 +73,7 @@ def _mock_conf_fs(): "/pdk/dummy/libs.ref/techlef/dummy_scl/dummy_tech_lef.tlef", ) patcher.fs.create_file( - "/pdk/dummy2/libs.ref/techlef/dummy_scl/dummy_tech_lef.tlef", + "/pdk/dummy2/libs.ref/techlef/dummy2_scl/dummy_tech_lef.tlef", ) patcher.fs.create_file( "/pdk/dummy/libs.tech/openlane/dummy_scl/config.tcl",