Restructure SPIRESEngine to accomodate abstract model

[caufieldjh]

or rather, don't use different Engines for different sources, like GPT4All.

Also address #305 - this is a bit overloaded for a single PR but will be messy to handle concurrently.

Support for loading local models will be in a different PR.

[caufieldjh]

For #305 (pre-load template details and pass to KEs):

• Move template loading from class function in KnowledgeEngine to util io method
• Handle target_class argument - this is currently retrieved from ke.schemaview.get_class(target_class) but let's just get it from the loaded template object
• Ensure changes are propagated to all calls to KnowledgeEngine (or at least all of them involving a template, which should be all of them)

[caufieldjh]

This is fairly broken at the moment - WIP.
Running into this error again upon importing llm:

OSError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found

Thoughts in the relevant issue: #272

[caufieldjh]

Seems like this is close to working, though I'm getting a poetry install error with adeft:

(ontogpt-py3.10) harry@JHCaufield-M90:~/ontogpt$ poetry install
Installing dependencies from lock file

Package operations: 18 installs, 0 updates, 0 removals

  • Installing adeft (0.11.2): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-310
  creating build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/disambiguate.py -> build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/locations.py -> build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/util.py -> build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/discover.py -> build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/recognize.py -> build/lib.linux-x86_64-cpython-310/adeft
  copying adeft/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft
  creating build/lib.linux-x86_64-cpython-310/adeft/score
  copying adeft/score/score.py -> build/lib.linux-x86_64-cpython-310/adeft/score
  copying adeft/score/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/score
  creating build/lib.linux-x86_64-cpython-310/adeft/download
  copying adeft/download/__main__.py -> build/lib.linux-x86_64-cpython-310/adeft/download
  copying adeft/download/download.py -> build/lib.linux-x86_64-cpython-310/adeft/download
  copying adeft/download/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/download
  creating build/lib.linux-x86_64-cpython-310/adeft/gui
  copying adeft/gui/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/gui
  creating build/lib.linux-x86_64-cpython-310/adeft/modeling
  copying adeft/modeling/label.py -> build/lib.linux-x86_64-cpython-310/adeft/modeling
  copying adeft/modeling/classify.py -> build/lib.linux-x86_64-cpython-310/adeft/modeling
  copying adeft/modeling/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/modeling
  creating build/lib.linux-x86_64-cpython-310/adeft/nlp
  copying adeft/nlp/nlp.py -> build/lib.linux-x86_64-cpython-310/adeft/nlp
  copying adeft/nlp/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/nlp
  creating build/lib.linux-x86_64-cpython-310/adeft/gui/ground
  copying adeft/gui/ground/ground.py -> build/lib.linux-x86_64-cpython-310/adeft/gui/ground
  copying adeft/gui/ground/__init__.py -> build/lib.linux-x86_64-cpython-310/adeft/gui/ground
  running egg_info
  writing adeft.egg-info/PKG-INFO
  writing dependency_links to adeft.egg-info/dependency_links.txt
  writing requirements to adeft.egg-info/requires.txt
  writing top-level names to adeft.egg-info/top_level.txt
  reading manifest file 'adeft.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  adding license file 'LICENSE'
  writing manifest file 'adeft.egg-info/SOURCES.txt'
  /tmp/tmpg2peef39/.venv/lib/python3.10/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'adeft.gui.ground.static' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'adeft.gui.ground.static' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'adeft.gui.ground.static' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'adeft.gui.ground.static' to be distributed and are
          already explicitly excluding 'adeft.gui.ground.static' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  /tmp/tmpg2peef39/.venv/lib/python3.10/site-packages/setuptools/command/build_py.py:207: _Warning: Package 'adeft.gui.ground.templates' is absent from the `packages` configuration.
  !!
  
          ********************************************************************************
          ############################
          # Package would be ignored #
          ############################
          Python recognizes 'adeft.gui.ground.templates' as an importable package[^1],
          but it is absent from setuptools' `packages` configuration.
  
          This leads to an ambiguous overall configuration. If you want to distribute this
          package, please make sure that 'adeft.gui.ground.templates' is explicitly added
          to the `packages` configuration field.
  
          Alternatively, you can also rely on setuptools' discovery methods
          (for example by using `find_namespace_packages(...)`/`find_namespace:`
          instead of `find_packages(...)`/`find:`).
  
          You can read more about "package discovery" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
          If you don't want 'adeft.gui.ground.templates' to be distributed and are
          already explicitly excluding 'adeft.gui.ground.templates' via
          `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
          you can try to use `exclude_package_data`, or `include-package-data=False` in
          combination with a more fine grained `package-data` configuration.
  
          You can read more about "package data files" on setuptools documentation page:
  
          - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
          [^1]: For Python, any directory (with suitable naming) can be imported,
                even if it does not contain any `.py` files.
                On the other hand, currently there is no concept of package data
                directory, all directories are treated like packages.
          ********************************************************************************
  
  !!
    check.warn(importable)
  copying adeft/score/_score.c -> build/lib.linux-x86_64-cpython-310/adeft/score
  copying adeft/nlp/stopwords.json -> build/lib.linux-x86_64-cpython-310/adeft/nlp
  creating build/lib.linux-x86_64-cpython-310/adeft/gui/ground/static
  copying adeft/gui/ground/static/ground.css -> build/lib.linux-x86_64-cpython-310/adeft/gui/ground/static
  copying adeft/gui/ground/static/scripts.js -> build/lib.linux-x86_64-cpython-310/adeft/gui/ground/static
  creating build/lib.linux-x86_64-cpython-310/adeft/gui/ground/templates
  copying adeft/gui/ground/templates/input.jinja2 -> build/lib.linux-x86_64-cpython-310/adeft/gui/ground/templates
  running build_ext
  building 'adeft.score._score' extension
  creating build/temp.linux-x86_64-cpython-310
  creating build/temp.linux-x86_64-cpython-310/adeft
  creating build/temp.linux-x86_64-cpython-310/adeft/score
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/tmpg2peef39/.venv/include -I/usr/include/python3.10 -c adeft/score/_score.c -o build/temp.linux-x86_64-cpython-310/adeft/score/_score.o
  error: command 'x86_64-linux-gnu-gcc' failed: No such file or directory
  

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with adeft (0.11.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "adeft (==0.11.2)"'.

[caufieldjh]

That error ended up being due to having an incorrect libgcc - uninstalled that and installed an OS-appropriate version, then a poetry install worked as expected.