Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New transpile #122

Merged
merged 3 commits into from
Oct 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions docs/command_line_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,17 @@ whose minimal options were modeled after Python's most commonly used:
The Lissp Compiler
------------------

While the recommended way to compile Lissp modules is with
`transpile <hissp.reader.transpile>`
calls in the REPL, the main module, and the package ``__init__.py`` files,
there is also a command-line interface in case that is needed by an external build system.

The command usage is ``python -m hissp.reader package [module [module ...]]``,
and it takes the same arguments as ``transpile``.

The package can be an empty string for top-level modules.
Remember to use the *module* names, not the *file* names.
E.g. ``python -m hissp.reader "" spam eggs``,
not ``python -m hissp.reader "" spam.lissp eggs.lissp``.
The recommended way to compile Lissp modules is with
`transpile <hissp.reader.transpile>` calls in `__init__.py` files (or the main module).

This can be done manually in the REPL.
However, an external build system may need to use shell commands.
It is possible to run transpile commands in the shell via `python -c` or `lissp -c`.

For example,

.. code-block:: shell

$ alias lisspc="lissp -c '(hissp.reader..transpile_file None (getitem sys..argv -1))'"
$ lisspc spam.lissp

27 changes: 21 additions & 6 deletions docs/faq.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1222,12 +1222,27 @@ Or equivalently, in Python:

transpile(__package__, "sausage", "bacon")

Consider putting the above in each package's ``__init__.py`` to
auto-compile each Hissp module in the package on package import during
development. You can disable it again on release, if desired, but this
gives you fine-grained control over what gets compiled when. Note that
you usually would want to recompile the whole project rather than only
the changed files like Python does, because macros run at compile time.
Consider putting the above in each package's ``__init__.py``
to auto-compile each Hissp module in the package on package import during development.
This gives you fine-grained control over what gets compiled when.
If desired for a release,
you can remove these lines after compilation,
or disable them via some externally set parameter,
such as an environment variable, config file, or global "constant".

For example,

.. code-block:: python

if __import__('os').getenv('AUTOCOMPILING_LISSP_PROJECT_FOO'):
from hissp.reader import transpile

transpile(__package__, "sausage", "bacon")


Note that you usually *would* want to recompile the whole project
rather than only the changed files like Python does,
because macros run at compile time.
Changing a macro in one file normally doesn't affect the code that uses
it in other files until they are recompiled.

Expand Down
78 changes: 28 additions & 50 deletions src/hissp/reader.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,20 +14,17 @@

import ast
import builtins
import os
import re
import sys
from collections import namedtuple
from contextlib import contextmanager, nullcontext
from functools import reduce
from importlib import import_module, resources
from itertools import chain
from keyword import iskeyword as _iskeyword
from pathlib import Path, PurePath
from pprint import pformat, pprint
from pprint import pformat
from threading import Lock
from types import ModuleType
from typing import Any, Iterable, Iterator, NewType, Optional, Tuple, Union
from typing import Any, Iterable, Iterator, NewType, Tuple, Union

from hissp.compiler import Compiler, MAYBE, readerless
from hissp.munger import force_qz_encode, munge
Expand Down Expand Up @@ -437,54 +434,35 @@ def is_qualifiable(symbol):
)


def transpile(package: Optional[resources.Package], *modules: Union[str, PurePath]):
"""
Compiles the named modules to Python files.
If the package is None or "", it uses the current working directory without using a package.
Lissp files must know their package at compile time to resolve imports correctly.
"""
# TODO: allow pathname without + ".lissp"?
if package:
for module in modules:
transpile_module(package, module + ".lissp")
else:
for module in modules:
with open(module + ".lissp") as f:
code = f.read()
out = module + ".py"
_write_py(out, module, code)


def transpile_module(
package: resources.Package,
resource: Union[str, PurePath],
out: Union[None, str, bytes, Path] = None,
):
"""Transpile a single submodule in a package."""
code = resources.read_text(package, resource)
path: Path
with resources.path(package, resource) as path:
out = out or path.with_suffix(".py")
if isinstance(package, ModuleType):
package = package.__package__
if isinstance(package, os.PathLike):
resource = resource.stem
_write_py(out, f"{package}.{resource.split('.')[0]}", code)
def transpile(package, *modules):
"""Transpiles the named Python modules from Lissp.

A .lissp file of the same name must be present. If the package is
empty, `transpile` writes modules to the current working
directory without a package.
"""
t = transpile_package if package else transpile_file
for m in modules:
t(package, f"{m}.lissp")

def _write_py(out, qualname, code):
with open(out, "w") as f:
print(f"compiling {qualname} as", out, file=sys.stderr)
if code.startswith("#!"): # ignore shebang line
_, _, code = code.partition("\n")
f.write(Lissp(qualname, evaluate=True, filename=str(out)).compile(code))

def transpile_package(package, resource):
"""Locates & transpiles a packaged .lissp resource file to .py."""
with resources.path(package, resource) as path:
transpile_file(package, path)

def main():
"""Calls `transpile` with arguments from `sys.argv`."""
transpile(*sys.argv[1:])

def transpile_file(package, path: Union[Path, str]):
"""Transpiles a single .lissp file to .py in the same location.

if __name__ == "__main__":
# TODO: test CLI
main()
Code in .lissp files is executed upon compilation. This is necessary
because macro definitions can alter the compilation of subsequent
top-level forms. Lissp files must know their package at compile time
to resolve imports correctly. The package can be empty.
"""
path = Path(path).resolve(strict=True)
qualname = f"{package or ''}{'.' if package else ''}{PurePath(path.name).stem}"
python = Lissp(
qualname=qualname, evaluate=True, filename=str(path)
).compile(re.sub(r'^#!.*\n', '', path.read_text()))
path.with_suffix('.py').write_text(python, 'utf8')
1 change: 1 addition & 0 deletions src/hissp/repl.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ def runsource(self, source, filename="<input>", symbol="single"):
return super().runsource(source, filename, symbol)

def raw_input(self, prompt=""):
""":meta private:"""
prompt = {sys.ps2: ps2, sys.ps1: ps1}.get(prompt, prompt)
return super().raw_input(prompt)

Expand Down