Improve USBProxy reliability #179

antoinevg · 2024-08-23T09:46:33Z

This PR fixes a few things:

Refactor clear_feature_endpoint_halt arguments in the USB HAL driver to match stall arguments.
Add a clear_feature_endpoint_halt verb to the Moondancer GCP class.
Compiler warnings on the latest rustc release.
Incorrect reference to a Makefile target.
Fixes a compiler error when ladybug is enabled.

Depends on: greatscottgadgets/facedancer#106

… number and direction instead of an endpoint address

mossmann · 2024-08-28T14:11:58Z

I'm seeing rubber-ducky.py fail with this:

INFO    | moondancer     | Using the Moondancer backend.
INFO    | moondancer     | Connected FULL speed device 'facedancer.devices.keyboard.USBKeyboardDevice' to target host.
INFO    | rubber-ducky   | Beginning message typing demo...
INFO    | moondancer     | Disconnecting from target host.
Traceback (most recent call last):
  File "/tmp/./rubber-ducky.py", line 25, in <module>
    main(device, type_letters())
  File "/tmp/f/lib/python3.11/site-packages/facedancer/devices/__init__.py", line 41, in default_main
    device.emulate(*coroutines)
  File "/tmp/f/lib/python3.11/site-packages/facedancer/device.py", line 232, in emulate
    self.run_with(*coroutines)
  File "/tmp/f/lib/python3.11/site-packages/facedancer/device.py", line 217, in run_with
    asyncio.run(inner())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/tmp/f/lib/python3.11/site-packages/facedancer/device.py", line 215, in inner
    await asyncio.gather(self.run(), *coroutines)
  File "/tmp/f/lib/python3.11/site-packages/facedancer/device.py", line 204, in run
    self.backend.service_irqs()
  File "/tmp/f/lib/python3.11/site-packages/facedancer/backends/moondancer.py", line 398, in service_irqs
    events: List[Tuple[int, int]] = self.api.get_interrupt_events()
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/f/lib/python3.11/site-packages/pygreat/comms.py", line 1126, in method
    return self.execute_command(verb_number, in_format, out_format, name=name, class_name=class_name,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/f/lib/python3.11/site-packages/pygreat/comms.py", line 1281, in execute_command
    return self.comms_backend.execute_command(self.CLASS_NUMBER, verb, in_format,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/f/lib/python3.11/site-packages/pygreat/comms.py", line 788, in execute_command
    raw_result = self.execute_raw_command(class_number, verb, payload, timeout,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/f/lib/python3.11/site-packages/pygreat/comms_backends/usb1.py", line 403, in execute_raw_command
    self.device_handle.controlWrite(usb1.TYPE_VENDOR | usb1.RECIPIENT_DEVICE,
  File "/home/mossmann/.local/lib/python3.11/site-packages/usb1/__init__.py", line 1349, in controlWrite
    return self._controlTransfer(request_type, request, value, index, data,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mossmann/.local/lib/python3.11/site-packages/usb1/__init__.py", line 1326, in _controlTransfer
    mayRaiseUSBError(result)
  File "/home/mossmann/.local/lib/python3.11/site-packages/usb1/__init__.py", line 127, in mayRaiseUSBError
    __raiseUSBError(value)
  File "/home/mossmann/.local/lib/python3.11/site-packages/usb1/__init__.py", line 119, in raiseUSBError
    raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorTimeout: LIBUSB_ERROR_TIMEOUT [-7]

antoinevg · 2024-08-28T14:43:34Z

I thought this might have been because of the other subtle ZLP I caught in #183 but both branches are working fine for me?

I see it died during a libgreat RPC call, it could maybe be that the firmware had crashed or gotten in a weird state?

Can you replicate it again if you do a fresh firmware build&boot?

antoinevg · 2024-08-29T08:01:20Z

Another thought occurred to me last night, did you also update your Facedancer to: greatscottgadgets/facedancer#106

mossmann · 2024-08-29T21:10:18Z

Another thought occurred to me last night, did you also update your Facedancer to: greatscottgadgets/facedancer#106

No. Should I? I test everything with released versions of dependencies. If a different version of a dependency is required, that should be specified in the PR.

antoinevg · 2024-08-30T05:59:41Z

No. Should I? I test everything with released versions of dependencies. If a different version of a dependency is required, that should be specified in the PR.

That would explain it then. 😂

The real question though is how did I manage to forget to add any details to the PR description? 🙈

KeuntaeShin · 2024-09-25T10:56:03Z

@mossmann, I'm also facing this issue exactly same. Any clue?

cynthion info
Cynthion version: 0.1.6
Apollo version: 1.1.0
Python version: 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]

Found Apollo stub interface!
Bitstream: Facedancer (Cynthion Project)
Vendor ID: 1d50
Product ID: 615b
bcdDevice: 0104
Bitstream serial number: 398162cba3ac60de

For additional device information use the --force-offline option.

python ./rubber-ducky.py
INFO | moondancer | Using the Moondancer backend.
INFO | moondancer | Connected FULL speed device 'facedancer.devices.keyboard.USBKeyboardDevice' to target host.
INFO | rubber-ducky | Beginning message typing demo...
INFO | moondancer | Disconnecting from target host.
Traceback (most recent call last):
File "/facedancer/examples/./rubber-ducky.py", line 25, in
main(device, type_letters())
File "/lib/python3.10/site-packages/facedancer/devices/init.py", line 41, in default_main
device.emulate(*coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 228, in emulate
self.run_with(*coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 213, in run_with
asyncio.run(inner())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/lib/python3.10/site-packages/facedancer/device.py", line 211, in inner
await asyncio.gather(self.run(), *coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 200, in run
self.backend.service_irqs()
File "/lib/python3.10/site-packages/facedancer/backends/moondancer.py", line 427, in service_irqs
events: List[Tuple[int, int]] = self.api.get_interrupt_events()
File "/lib/python3.10/site-packages/pygreat/comms.py", line 1126, in method
return self.execute_command(verb_number, in_format, out_format, name=name, class_name=class_name,
File "/lib/python3.10/site-packages/pygreat/comms.py", line 1281, in execute_command
return self.comms_backend.execute_command(self.CLASS_NUMBER, verb, in_format,
File "/lib/python3.10/site-packages/pygreat/comms.py", line 788, in execute_command
raw_result = self.execute_raw_command(class_number, verb, payload, timeout,
File "/lib/python3.10/site-packages/pygreat/comms_backends/usb1.py", line 403, in execute_raw_command
self.device_handle.controlWrite(usb1.TYPE_VENDOR | usb1.RECIPIENT_DEVICE,
File "/lib/python3.10/site-packages/usb1/init.py", line 1330, in controlWrite
return self._controlTransfer(request_type, request, value, index, data,
File "/lib/python3.10/site-packages/usb1/init.py", line 1307, in _controlTransfer
mayRaiseUSBError(result)
File "/lib/python3.10/site-packages/usb1/init.py", line 127, in mayRaiseUSBError
__raiseUSBError(value)
File "/lib/python3.10/site-packages/usb1/init.py", line 119, in raiseUSBError
raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorTimeout: LIBUSB_ERROR_TIMEOUT [-7]

antoinevg · 2024-09-25T11:38:05Z

Can you please try updating your cynthion & facedancer installs to the latest version and let us know if the problem still occurs:

pip install --upgrade cynthion
cynthion update

pip install --upgrade facedancer

KeuntaeShin · 2024-09-26T00:47:58Z

@antoinevg , Packetry is working, but Facedancer is failed.

pip install --upgrade cynthion
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: cynthion in /home/babo/.local/lib/python3.10/site-packages (0.1.6)
Requirement already satisfied: future in /usr/lib/python3/dist-packages (from cynthion) (0.18.2)
Requirement already satisfied: libusb1 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (3.1.0)
Requirement already satisfied: prompt-toolkit in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (3.0.41)
Requirement already satisfied: pyfwup~=0.5 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (0.5.2)
Requirement already satisfied: pyserial>=3.5 in /usr/local/lib/python3.10/dist-packages (from cynthion) (3.5)
Requirement already satisfied: pyusb in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (1.2.1)
Requirement already satisfied: tabulate in /usr/lib/python3/dist-packages (from cynthion) (0.8.9)
Requirement already satisfied: tomli in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (2.0.1)
Requirement already satisfied: tqdm in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (4.66.5)
Requirement already satisfied: pygreat~=2024.0 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (2024.0.2)
Requirement already satisfied: apollo-fpga~=1.1 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (1.1.0)
Requirement already satisfied: amaranth==0.4.1 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (0.4.1)
Requirement already satisfied: luna-usb~=0.1 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (0.1.2)
Requirement already satisfied: luna-soc~=0.2 in /home/babo/.local/lib/python3.10/site-packages (from cynthion) (0.2.0)
Requirement already satisfied: pyvcd<0.5,>=0.2.2 in /home/babo/.local/lib/python3.10/site-packages (from amaranth==0.4.1->cynthion) (0.4.0)
Requirement already satisfied: Jinja2~=3.0 in /usr/lib/python3/dist-packages (from amaranth==0.4.1->cynthion) (3.0.3)
Requirement already satisfied: pyxdg>=0.27 in /usr/lib/python3/dist-packages (from apollo-fpga~=1.1->cynthion) (0.27)
Requirement already satisfied: deprecation>=2.1.0 in /home/babo/.local/lib/python3.10/site-packages (from apollo-fpga~=1.1->cynthion) (2.1.0)
Requirement already satisfied: usb-protocol~=0.9.1 in /home/babo/.local/lib/python3.10/site-packages (from luna-usb~=0.1->cynthion) (0.9.1)
Requirement already satisfied: wcwidth in /home/babo/.local/lib/python3.10/site-packages (from prompt-toolkit->cynthion) (0.2.11)
Requirement already satisfied: backports.functools-lru-cache in /home/babo/.local/lib/python3.10/site-packages (from pygreat~=2024.0->cynthion) (2.0.0)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from deprecation>=2.1.0->apollo-fpga~=1.1->cynthion) (21.3)
Requirement already satisfied: construct~=2.10 in /home/babo/.local/lib/python3.10/site-packages (from usb-protocol~=0.9.1->luna-usb~=0.1->cynthion) (2.10.70)

cynthion update
Updating FPGA configuration flash with 247181 bytes...
Operation complete!
Updating device firmware with 13876 bytes...
Operation complete!

pip install --upgrade facedancer
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: facedancer in /home/babo/.local/lib/python3.10/site-packages (3.0.3)
Requirement already satisfied: pyusb in /home/babo/.local/lib/python3.10/site-packages (from facedancer) (1.2.1)
Requirement already satisfied: pyserial in /usr/local/lib/python3.10/dist-packages (from facedancer) (3.5)
Requirement already satisfied: prompt-toolkit in /home/babo/.local/lib/python3.10/site-packages (from facedancer) (3.0.41)
Requirement already satisfied: libusb1 in /home/babo/.local/lib/python3.10/site-packages (from facedancer) (3.1.0)
Requirement already satisfied: wcwidth in /home/babo/.local/lib/python3.10/site-packages (from prompt-toolkit->facedancer) (0.2.11)

cynthion run facedancer
Updating SoC firmware flash with 55900 bytes...
Operation complete!
Uploading target bitstream to FPGA with 441927 bytes...
Operation complete!

cynthion info
Cynthion version: 0.1.6
Apollo version: 1.1.0
Python version: 3.10.12 (main, Sep 11 2024, 15:47:36) [GCC 11.4.0]

Found Apollo stub interface!
Bitstream: Facedancer (Cynthion Project)
Vendor ID: 1d50
Product ID: 615b
bcdDevice: 0104
Bitstream serial number: 398162cba3ac60de

For additional device information use the --force-offline option.

python ./rubber-ducky.py
INFO | moondancer | Using the Moondancer backend.
INFO | moondancer | Connected FULL speed device 'facedancer.devices.keyboard.USBKeyboardDevice' to target host.
INFO | rubber-ducky | Beginning message typing demo...
INFO | moondancer | Disconnecting from target host.
Traceback (most recent call last):
File "/facedancer/examples/./rubber-ducky.py", line 25, in
main(device, type_letters())
File "/lib/python3.10/site-packages/facedancer/devices/init.py", line 41, in default_main
device.emulate(*coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 228, in emulate
self.run_with(*coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 213, in run_with
asyncio.run(inner())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/lib/python3.10/site-packages/facedancer/device.py", line 211, in inner
await asyncio.gather(self.run(), *coroutines)
File "/lib/python3.10/site-packages/facedancer/device.py", line 200, in run
self.backend.service_irqs()
File "/lib/python3.10/site-packages/facedancer/backends/moondancer.py", line 427, in service_irqs
events: List[Tuple[int, int]] = self.api.get_interrupt_events()
File "/lib/python3.10/site-packages/pygreat/comms.py", line 1126, in method
return self.execute_command(verb_number, in_format, out_format, name=name, class_name=class_name,
File "/lib/python3.10/site-packages/pygreat/comms.py", line 1281, in execute_command
return self.comms_backend.execute_command(self.CLASS_NUMBER, verb, in_format,
File "/lib/python3.10/site-packages/pygreat/comms.py", line 788, in execute_command
raw_result = self.execute_raw_command(class_number, verb, payload, timeout,
File "/lib/python3.10/site-packages/pygreat/comms_backends/usb1.py", line 403, in execute_raw_command
self.device_handle.controlWrite(usb1.TYPE_VENDOR | usb1.RECIPIENT_DEVICE,
File "/lib/python3.10/site-packages/usb1/init.py", line 1330, in controlWrite
return self._controlTransfer(request_type, request, value, index, data,
File "/lib/python3.10/site-packages/usb1/init.py", line 1307, in _controlTransfer
mayRaiseUSBError(result)
File "/lib/python3.10/site-packages/usb1/init.py", line 127, in mayRaiseUSBError
__raiseUSBError(value)
File "/lib/python3.10/site-packages/usb1/init.py", line 119, in raiseUSBError
raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorTimeout: LIBUSB_ERROR_TIMEOUT [-7]

KeuntaeShin · 2024-09-26T08:25:52Z

@antoinevg , There was only one chance after reboot:

cynthion run facedancer
Updating SoC firmware flash with 55900 bytes...
Operation complete!
Uploading target bitstream to FPGA with 441927 bytes...
Operation complete!

sudo python rubber-ducky.py
INFO | moondancer | Using the Moondancer backend.
INFO | moondancer | Connected FULL speed device 'facedancer.devices.keyboard.USBKeyboardDevice' to target host.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | rubber-ducky | Beginning message typing demo...
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | device | Host issued a bus reset; resetting our connection.
INFO | moondancer | Target host configuration complete.
INFO | rubber-ducky | Typing complete. Idly handling USB requests.

After that no more working without reboot:

sudo python rubber-ducky.py
INFO | moondancer | Using the Moondancer backend.
INFO | moondancer | Connected FULL speed device 'facedancer.devices.keyboard.USBKeyboardDevice' to target host.
INFO | rubber-ducky | Beginning message typing demo...
INFO | moondancer | Disconnecting from target host.
Traceback (most recent call last):
File "/facedancer/examples/rubber-ducky.py", line 25, in
main(device, type_letters())
File "/usr/local/lib/python3.10/dist-packages/facedancer/devices/init.py", line 41, in default_main
device.emulate(*coroutines)
File "/usr/local/lib/python3.10/dist-packages/facedancer/device.py", line 228, in emulate
self.run_with(*coroutines)
File "/usr/local/lib/python3.10/dist-packages/facedancer/device.py", line 213, in run_with
asyncio.run(inner())
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/local/lib/python3.10/dist-packages/facedancer/device.py", line 211, in inner
await asyncio.gather(self.run(), *coroutines)
File "/usr/local/lib/python3.10/dist-packages/facedancer/device.py", line 200, in run
self.backend.service_irqs()
File "/usr/local/lib/python3.10/dist-packages/facedancer/backends/moondancer.py", line 427, in service_irqs
events: List[Tuple[int, int]] = self.api.get_interrupt_events()
File "/usr/local/lib/python3.10/dist-packages/pygreat/comms.py", line 1126, in method
return self.execute_command(verb_number, in_format, out_format, name=name, class_name=class_name,
File "/usr/local/lib/python3.10/dist-packages/pygreat/comms.py", line 1281, in execute_command
return self.comms_backend.execute_command(self.CLASS_NUMBER, verb, in_format,
File "/usr/local/lib/python3.10/dist-packages/pygreat/comms.py", line 788, in execute_command
raw_result = self.execute_raw_command(class_number, verb, payload, timeout,
File "/usr/local/lib/python3.10/dist-packages/pygreat/comms_backends/usb1.py", line 403, in execute_raw_command
self.device_handle.controlWrite(usb1.TYPE_VENDOR | usb1.RECIPIENT_DEVICE,
File "/usr/local/lib/python3.10/dist-packages/usb1/init.py", line 1330, in controlWrite
return self._controlTransfer(request_type, request, value, index, data,
File "/usr/local/lib/python3.10/dist-packages/usb1/init.py", line 1307, in _controlTransfer
mayRaiseUSBError(result)
File "/usr/local/lib/python3.10/dist-packages/usb1/init.py", line 127, in mayRaiseUSBError
__raiseUSBError(value)
File "/usr/local/lib/python3.10/dist-packages/usb1/init.py", line 119, in raiseUSBError
raise __STATUS_TO_EXCEPTION_DICT.get(value, __USBError)(value)
usb1.USBErrorTimeout: LIBUSB_ERROR_TIMEOUT [-7]

antoinevg added 5 commits August 23, 2024 11:45

firmware: fix cfg warnings when compiling with rustc >= 1.80

ba1bdc9

firmware: fix gateware target in cynthion.sh error message

ea88404

hal: refactor clear_feature_endpoint_halt to take a separate endpoint…

ac867bd

… number and direction instead of an endpoint address

moondancer: fix compilation error caused by ladybug changes

6f778ec

moondancer: add new api method 'clear_feature_endpoint_halt'

146d1da

antoinevg marked this pull request as ready for review August 23, 2024 16:36

antoinevg requested a review from mossmann August 26, 2024 08:21

antoinevg mentioned this pull request Aug 30, 2024

Improve USBProxy reliability greatscottgadgets/facedancer#106

Merged

antoinevg merged commit f7724d2 into greatscottgadgets:main Sep 16, 2024
18 checks passed

antoinevg deleted the antoinevg/better-usbproxy branch September 16, 2024 08:10

KeuntaeShin mentioned this pull request Sep 27, 2024

Traceback Error: USB Timeout #188

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve USBProxy reliability #179

Improve USBProxy reliability #179

antoinevg commented Aug 23, 2024 •

edited

Loading

mossmann commented Aug 28, 2024

antoinevg commented Aug 28, 2024

antoinevg commented Aug 29, 2024

mossmann commented Aug 29, 2024

antoinevg commented Aug 30, 2024

KeuntaeShin commented Sep 25, 2024 •

edited

Loading

antoinevg commented Sep 25, 2024

KeuntaeShin commented Sep 26, 2024 •

edited

Loading

KeuntaeShin commented Sep 26, 2024 •

edited

Loading

Improve USBProxy reliability #179

Improve USBProxy reliability #179

Conversation

antoinevg commented Aug 23, 2024 • edited Loading

mossmann commented Aug 28, 2024

antoinevg commented Aug 28, 2024

antoinevg commented Aug 29, 2024

mossmann commented Aug 29, 2024

antoinevg commented Aug 30, 2024

KeuntaeShin commented Sep 25, 2024 • edited Loading

antoinevg commented Sep 25, 2024

KeuntaeShin commented Sep 26, 2024 • edited Loading

KeuntaeShin commented Sep 26, 2024 • edited Loading

antoinevg commented Aug 23, 2024 •

edited

Loading

KeuntaeShin commented Sep 25, 2024 •

edited

Loading

KeuntaeShin commented Sep 26, 2024 •

edited

Loading

KeuntaeShin commented Sep 26, 2024 •

edited

Loading