make explicit the symbols that imports from the wrapper module into the friendly module #469

junkmd · 2023-01-14T14:13:23Z

THIS IS FOR `drop_py2` PLAN! related to #392.

Friendly modules such as comtypes.gen.stdole have imported the symbols of the wrapper module in the following way so far.

from comtypes.gen import _00020430_0000_0000_C000_000000000046_0_2_0
globals().update(_00020430_0000_0000_C000_000000000046_0_2_0.__dict__)
__name__ = 'comtypes.gen.stdole'

This is not redundant, however, all the type hinting information that is so important in modern Python is lost.
Therefore, all symbols specified by CodeGenerator.names, CodeGenerator.imports and CodeGenerator.declarations are explicitly imported into the friendly module.

During this work, I noticed that typelib_path and Library were public symbols but not included in __all__, so I added them.

The generated comtypes.gen.stdole code changes as follows.

from comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0 import (
    FontEvents, FONTBOLD, IUnknown, OLE_CANCELBOOL,
    OLE_YSIZE_CONTAINER, dispid, StdFont, DISPMETHOD,
    OLE_YPOS_HIMETRIC, typelib_path, OLE_HANDLE, IFont,
    OLE_XSIZE_HIMETRIC, GUID, BSTR, IFontEventsDisp, Picture, Default,
    OLE_XSIZE_CONTAINER, OLE_COLOR, HRESULT, FONTUNDERSCORE,
    _check_version, OLE_YSIZE_HIMETRIC, CoClass, Monochrome, Library,
    EXCEPINFO, FONTITALIC, _lcid, OLE_XPOS_PIXELS, OLE_XPOS_CONTAINER,
    OLE_XPOS_HIMETRIC, IEnumVARIANT, Unchecked, VgaColor, IDispatch,
    Font, Checked, StdPicture, IPicture, OLE_XSIZE_PIXELS, FONTNAME,
    OLE_TRISTATE, IPictureDisp, VARIANT_BOOL, IFontDisp, COMMETHOD,
    LoadPictureConstants, FONTSIZE, Color, Gray, FONTSTRIKETHROUGH,
    OLE_ENABLEDEFAULTBOOL, DISPPARAMS, OLE_OPTEXCLUSIVE,
    OLE_YPOS_CONTAINER, DISPPROPERTY, OLE_YSIZE_PIXELS,
    OLE_YPOS_PIXELS
)


__all__ = [
    'FontEvents', 'FONTBOLD', 'OLE_XPOS_CONTAINER', 'OLE_CANCELBOOL',
    'OLE_YSIZE_CONTAINER', 'OLE_XPOS_HIMETRIC', 'StdFont', 'VgaColor',
    'Unchecked', 'OLE_YPOS_HIMETRIC', 'typelib_path', 'Font',
    'OLE_HANDLE', 'OLE_YSIZE_PIXELS', 'Checked', 'IFont',
    'OLE_XSIZE_HIMETRIC', 'StdPicture', 'IPicture',
    'OLE_XSIZE_PIXELS', 'FONTNAME', 'IFontEventsDisp', 'Picture',
    'Default', 'OLE_XSIZE_CONTAINER', 'IPictureDisp', 'OLE_TRISTATE',
    'OLE_COLOR', 'IFontDisp', 'OLE_YSIZE_HIMETRIC',
    'LoadPictureConstants', 'Monochrome', 'FONTSIZE', 'Library',
    'Color', 'Gray', 'FONTSTRIKETHROUGH', 'FONTITALIC',
    'OLE_ENABLEDEFAULTBOOL', 'OLE_XPOS_PIXELS', 'OLE_OPTEXCLUSIVE',
    'OLE_YPOS_CONTAINER', 'FONTUNDERSCORE', 'OLE_YPOS_PIXELS'
]

Previously, the __file__ and __repr__ had unintentionally become those of the wrapper module because all namespaces had been overwritten, but with this change, the __file__ and __repr__ are now "normally"(related to #328).
_00020430_0000_0000_0000_C000_0000000000000046_0_2_0 symbol is removed from comtypes.gen.stdole.
Symbols imported into wrapper module by wildcard(from ctypes import *) would not be imported to friendly modules.

Also, the argument name of generate_wrapper_code (originally generate_code) was changed because it does not go well with type annotations.

junkmd · 2023-01-14T14:43:18Z

I noticed there are indirect references like comtypes.gen.UIAutomationClient.VARIANT in pywinauto.

I would change the codebase to import those symbols and aliases as well.

memo

https://ci.appveyor.com/project/pywinauto/pywinauto/builds/45938451/job/nxl2gh093j845eld/tests

junkmd · 2023-01-14T16:30:09Z

CIs are passed.

memo

https://github.com/enthought/comtypes/actions/runs/3919058149/jobs/6699828444
https://ci.appveyor.com/project/cfarrow/comtypes/builds/45938604
https://ci.appveyor.com/project/pywinauto/pywinauto/builds/45938630

junkmd · 2023-01-14T16:32:52Z

@vasily-v-ryabov

This change does more than just dropping backward compatibility with Py2, so I would like to make agreement.

Please review this.

junkmd · 2023-01-16T01:03:44Z

I have rebased this branch to bring in the changes in #470.

CIs are passed.

memo

https://github.com/enthought/comtypes/actions/runs/3925693725/jobs/6710781338
https://ci.appveyor.com/project/cfarrow/comtypes/builds/45944748
https://ci.appveyor.com/project/pywinauto/pywinauto/builds/45944759

because that symbol is public but not included.

that encapsulates `CodeGenerator` instance.

from `generate_code`

to `DeclaredNamespaces` and `ImportedNamespaces`

in `generate_friendly_code`

junkmd · 2023-01-17T01:08:10Z

I have rebased this branch to bring in the changes in #471, and added new commits for type annotations and f-string.

CIs are passed.

memo

https://github.com/enthought/comtypes/actions/runs/3934715711/jobs/6729706447
https://ci.appveyor.com/project/cfarrow/comtypes/builds/45956062
https://ci.appveyor.com/project/pywinauto/pywinauto/builds/45956067

jonschz · 2023-01-17T09:45:07Z

Just mentioning that I have used this new code generator both for testing / debugging and for my Windows Portable Devices code and I have had no issues so far.

junkmd · 2023-01-17T21:45:36Z

Thank you for your review!

…he friendly module (enthought#469) * add `Library` to generated modules' `__all__`. because that symbol is public but not included. * add `typelib_path` to generated modules' `__all__`. because that symbol is public but not included. * make `ModuleGenerator` class that encapsulates `CodeGenerator` instance. * rename to `generate_wrapper_code` from `generate_code` * add `generate_friendly_code` * add type annotations to `generate_wrapper_code` * add docstring * add `get_symbols` methods to `DeclaredNamespaces` and `ImportedNamespaces` * update imporing symbols * add type annotation to return value for `__init__` * change to using f-string in `generate_friendly_code`

…he friendly module (enthought#469) * add `Library` to generated modules' `__all__`. because that symbol is public but not included. * add `typelib_path` to generated modules' `__all__`. because that symbol is public but not included. * make `ModuleGenerator` class that encapsulates `CodeGenerator` instance. * rename to `generate_wrapper_code` from `generate_code` * add `generate_friendly_code` * add type annotations to `generate_wrapper_code` * add docstring * add `get_symbols` methods to `DeclaredNamespaces` and `ImportedNamespaces` * update imporing symbols * add type annotation to return value for `__init__` * change to using f-string in `generate_friendly_code` (cherry picked from commit 532c399)

@junkmd

* update `appveyor.yml` (#398) (cherry picked from commit 30f497a) * just remove trailing whitespaces (#410) (cherry picked from commit 7a14804) * adjust comments (#411) (cherry picked from commit 80ec49b) * adjust comments (#412) (cherry picked from commit 817b6b7) * adjust comments (#413) (cherry picked from commit 5b0dc8d) * adjust comments (#414) (cherry picked from commit a6fc08b) * adjust comments (#415) (cherry picked from commit 21b6ab6) * adjust comments (#416) (cherry picked from commit 086ec13) * adjust comments (#417) (cherry picked from commit 4e53792) * adjust comments and brackets (#420) (cherry picked from commit 14ae44d) * unify styles for `__all__` and `__known_symbols__` (#421) (cherry picked from commit 1398608) * Improve error message on non Win (#423) * Improve COM error message * Drop Py2 stuff * Remove dot * Remove empty line (cherry picked from commit 18cafb0) * adjust styles of `client/__init__.py` (#424) (cherry picked from commit 12d22a2) * add auto-formatter GHA triggered by PR (#427) * add `# fmt: ...` for avoiding redundant format * add GHA setting * apply automatic formatter Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> (cherry picked from commit 22bfc50) * fix GHA settings (#431) (cherry picked from commit 0664400) * fix passing keyword arguments in `client.dynamic` (#432) (cherry picked from commit 32d6621) * remove `__cmp__` methods in `client.dynamic` (#434) (cherry picked from commit b6a4181) * remove `sys.version_info` bridges (#436) (cherry picked from commit 0cadf77) * remove `hints.AnnoField` (#437) (cherry picked from commit 39d9e00) * adjust import part and references (#439) (cherry picked from commit 0fecfea) * add tests for `client.dynamic` (#440) (cherry picked from commit 7d69dce) * fix `client.dynamic._Dispatch.QueryInterface` (#442) (cherry picked from commit 6b551a4) * add type annotations other than `Dispatch` func (#444) (cherry picked from commit 11410ce) * fix importing `typing` under `if TYPE_CHECKING:` (#448) (cherry picked from commit ba1b18f) * remove `sys.version_info` bridges (#449) (cherry picked from commit 9238cc7) * fix `IPersist.GetClassID` to `TYPE_CHECKING` only method from `Callable` annotation (#451) (cherry picked from commit 446f52e) * remove skip marks (#452) (cherry picked from commit 6732c26) * fix type hints to inline annotations (#453) (cherry picked from commit 7321eaf) * change type hints from comment annotations to inline annotations in `comtypes/__init__.py`, `comtypes/automation.py` and `comtypes/typeinfo.py` (#454) * update `comtypes/__init__.py` * update `comtypes/automation.py` * update `comtypes/typeinfo.py` * fix `Array[T]` to quoted literal strings (cherry picked from commit 7b38a7a) * change type hints from comment annotations to inline annotations in `_memberspec` (#455) (cherry picked from commit 5d230aa) * change type hints from comment annotations to inline annotations in `comtypes/client/...` (#456) * change type hints from comment annotations to inline annotations in `comtypes/client/...` * fix type annotations (cherry picked from commit 18dc2cc) * Update `codegenerator` type annotations (#457) * update `codegenerator` type annotations * lcid (cherry picked from commit 9e7b900) * Update `tlbparser` type annotations (#459) (cherry picked from commit 0b4dbca) * Update type annotations in `typedesc` and `typedesc_base` (#461) * update `typedesc` type annotations * update `typedesc_base` type annotations (cherry picked from commit 95962e9) * remove raising `unittest.SkipTest` from `test_getactiveobj` (#462) (cherry picked from commit 00f2317) * add `black` style checking to GHA settings (#465) (cherry picked from commit 04037ae) * modernize `test_client` (#466) * change from udf `silence_stdout` to stdlib `contextlib.contextmanager` * remove `sys.version_info` bridges * remove excessive blank lines (cherry picked from commit d470b62) * fix `__next__()` in IEnum code generator (#467) (cherry picked from commit a76f0b1) * correcting type hint for CreateObject (#470) Co-authored-by: jonschz <[email protected]> (cherry picked from commit 1615af5) * Add unit test for generated IEnum (#468) (#471) * unit test for generated IEnum (#468) * change enum_test to a non-empty IEnum * improved checks and clarity * apply requested changes Co-authored-by: jonschz <[email protected]> (cherry picked from commit e1ee6f0) * make explicit the symbols that imports from the wrapper module into the friendly module (#469) * add `Library` to generated modules' `__all__`. because that symbol is public but not included. * add `typelib_path` to generated modules' `__all__`. because that symbol is public but not included. * make `ModuleGenerator` class that encapsulates `CodeGenerator` instance. * rename to `generate_wrapper_code` from `generate_code` * add `generate_friendly_code` * add type annotations to `generate_wrapper_code` * add docstring * add `get_symbols` methods to `DeclaredNamespaces` and `ImportedNamespaces` * update imporing symbols * add type annotation to return value for `__init__` * change to using f-string in `generate_friendly_code` (cherry picked from commit 532c399) * remove `'Programming Language :: Python :: 2.7'` (#483) from `setup.py` (cherry picked from commit 9724dc0) * change `_MemberSpec`s to `NamedTuple`s (#484) * change `_MemberSpec` to `NamedTuple` * `_ComMemberSpec` is no more `Generic` for Py<3.11 backward compatibility. * apply `black` style (cherry picked from commit 94b81af) * split `DISPPARAMS` instantiation in `IDispatch.Invoke` method with `__make_dp` method (#485) (cherry picked from commit 62ce303) * 477 Move clear_comtypes_cache to be a callable module (#478) * move clear_comtypes_cache to be a callable module This commit modifies the clear_comtypes_cache.py script so that it is inside the main comtypes module (renamed as just clear_cache) so that is can be called more easily as "py -m comtypes.clear_cache". The main function of the script is also exported using the "console_scripts" entry point so that the script also goes into the standard Python "Scripts" folder as before, but now as a .exe instead of a .py script, which makes it easier to run if systems are set to open .py files instead of running them. This version also includes a test case using the 3rd party package pyfakefs. Currently, this is not desired to avoid the requirement of 3rd party packages in comtypes, but is included here for potential use if the position changes. A subsequent commit will modify the tests to use unittest.patch instead, which is an inferior technical solution but avoids a 3rd party package. * modify clear_cache tests to not use pyfakefs This commit updates the test for comtypes.clear_cache to not use any 3rd party packages, instead relying on mocking the shutil.rmtree function which is used to do the actual cache deletion. * change quotes in print string * style changes based on review by @junkmd * Apply suggestions from code review Co-authored-by: Jun Komoda <[email protected]> --------- Co-authored-by: Ben Rowland <[email protected]> Co-authored-by: Jun Komoda <[email protected]> (cherry picked from commit 8954f61) * delint long lines (#487) (cherry picked from commit 6532d32) * remove `from ctypes import *` (#488) (cherry picked from commit 65e3c10) * remove dead optional `for_stub` arg (#489) from `ImportedNamespaces.getvalue` (cherry picked from commit da27aaf) * importing the wrapper-module into the friendly-module using an abstracted name, `__wrapper_module__` (#493) * add test * add the `__wrapper_module__` to friendly modules By doing so, it is possible to achieve a combination of friendly-modules functionalities and improved access to the wrapper-modules. (cherry picked from commit b7d5f1f) * Fix old index bug in `call_with_inout` (#473) * bugfix in `call_with_inout` * minor cleanup * handling the case of no in and no out * Test case for _fix_inout_args * additional cleanup and error handling * code formatting fixed * fix python 3.7 and 3.8 compatibility * Temporary addition of real-world test * code cleanup * intermediate commit, do not review * Refactor of unit test, removing portdevice test * fix global side-effect of other skipped test * Update comtypes/test/test_outparam.py Co-authored-by: Jun Komoda <[email protected]> * work on tests for inout_args and outparam - cleanup for test_outparam.py - improvements to test_inout_args.py - comments on a possible error in _memberspec.py * removing dead code * rename variables and add assertions * pass `MagicMock` instead of `ut.TestCase` * make tests for each argument passing patterns * remove duplicated comments * update test code for readability - remove name from mock - move line breaks to between mock preparations and assertions * split the testcases * add `Test_Error` * minor corrections, remove redundancy, migration - rewrite the permutations test - missing direction and omitted name redundant - migrate autogenerated keywords - TBD: more real life tests * Add tests covering 24 patterns - instead of using `if` statements and `permutations` * update test name * add real world tests, remove old code * formatting issue * Update comtypes/_memberspec.py dict type annotation Co-authored-by: Jun Komoda <[email protected]> * Change missing 'in' or 'out' to be treated as 'in' * Add real-world test: param without 'in' or 'out' * add `contextlib.redirect_stdout` to supress warnings * apply review feedback * update comments * add line breaks to lines longer than 88 characters * Update comtypes/test/test_inout_args.py --------- Co-authored-by: jonschz <[email protected]> Co-authored-by: Jonathan <[email protected]> Co-authored-by: Jun Komoda <[email protected]> (cherry picked from commit 876801f) * Fix rendering emojis in `CONTRIBUTING.md` and remove "Breaking troubles down" from "Table of contents" (#505) (cherry picked from commit af78614) --------- Co-authored-by: Cristi Fati <[email protected]> Co-authored-by: jonschz <[email protected]> Co-authored-by: Ben Rowland <[email protected]>

junkmd added enhancement New feature or request drop_py2 dev based on supporting only Python3, see #392 labels Jan 14, 2023

junkmd requested a review from vasily-v-ryabov January 14, 2023 14:13

junkmd added a commit to junkmd/pywinauto that referenced this pull request Jan 14, 2023

test enthought/comtypes#469

58eea4b

junkmd added a commit to junkmd/pywinauto that referenced this pull request Jan 14, 2023

test enthought/comtypes#469

6f6a9fc

junkmd marked this pull request as ready for review January 14, 2023 16:30

jonschz mentioned this pull request Jan 15, 2023

Correcting type hint for CreateObject #470

Merged

junkmd force-pushed the update_friendly_mods branch from d9e0d40 to a03d34d Compare January 15, 2023 23:00

junkmd added a commit to junkmd/pywinauto that referenced this pull request Jan 15, 2023

test enthought/comtypes#469

c3928a8

junkmd added 11 commits January 17, 2023 08:34

add Library to generated modules' __all__.

c13d668

because that symbol is public but not included.

add typelib_path to generated modules' __all__.

9171e08

because that symbol is public but not included.

make ModuleGenerator class

93e67ad

that encapsulates `CodeGenerator` instance.

rename to generate_wrapper_code

9c51711

from `generate_code`

add generate_friendly_code

2c662b8

add type annotations to generate_wrapper_code

5af032a

add docstring

9cf5305

add get_symbols methods

433e42c

to `DeclaredNamespaces` and `ImportedNamespaces`

update imporing symbols

4d01edc

add type annotation to return value for __init__

226912c

change to using f-string

38e5a9c

in `generate_friendly_code`

junkmd force-pushed the update_friendly_mods branch from a03d34d to 38e5a9c Compare January 16, 2023 23:35

junkmd added a commit to junkmd/pywinauto that referenced this pull request Jan 16, 2023

test enthought/comtypes#469

43b49b3

vasily-v-ryabov approved these changes Jan 17, 2023

View reviewed changes

junkmd added this to the 1.3.0 milestone Jan 17, 2023

junkmd linked an issue Jan 17, 2023 that may be closed by this pull request

comtypes.gen.Friendly's __file__, __repr__, ... are not what they really are #328

Closed

junkmd merged commit 532c399 into enthought:drop_py2 Jan 17, 2023

junkmd mentioned this pull request Jan 17, 2023

comtypes.gen.Friendly's __file__, __repr__, ... are not what they really are #328

Closed

junkmd deleted the update_friendly_mods branch January 17, 2023 22:47

junkmd mentioned this pull request Jul 5, 2023

importing the wrapper module into the friendly module using an abstracted name #492

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make explicit the symbols that imports from the wrapper module into the friendly module #469

make explicit the symbols that imports from the wrapper module into the friendly module #469

junkmd commented Jan 14, 2023 •

edited

Loading

junkmd commented Jan 14, 2023 •

edited

Loading

junkmd commented Jan 14, 2023

junkmd commented Jan 14, 2023

junkmd commented Jan 16, 2023

junkmd commented Jan 17, 2023

jonschz commented Jan 17, 2023

junkmd commented Jan 17, 2023

make explicit the symbols that imports from the wrapper module into the friendly module #469

make explicit the symbols that imports from the wrapper module into the friendly module #469

Conversation

junkmd commented Jan 14, 2023 • edited Loading

THIS IS FOR drop_py2 PLAN! related to #392.

junkmd commented Jan 14, 2023 • edited Loading

junkmd commented Jan 14, 2023

junkmd commented Jan 14, 2023

junkmd commented Jan 16, 2023

junkmd commented Jan 17, 2023

jonschz commented Jan 17, 2023

junkmd commented Jan 17, 2023

junkmd commented Jan 14, 2023 •

edited

Loading

THIS IS FOR `drop_py2` PLAN! related to #392.

junkmd commented Jan 14, 2023 •

edited

Loading