Have usable VID/PIDs for standard situations and prototype ones for examples

[chrysn]

Description

Currently, all examples use generic prototype VID/PID pairs that should really not make it into widespread use; building with them shows a big red warning to please get own IDs before handing out anything.

With USB support becoming more usable, boards like the nrf52840-dongle (#12189) whose default console is a USB serial device become available to RIOT. Those flashing USB warnings to developers who didn't even touch a line of USB code or intentionally use USB (other than for make term) is confusing, and those situations may qualify for some of the ID pair pools for Free Software.

The hard parts here are to

• define when such IDs could be used, and to
• make the build system detect whether the IDs are usable.

Applicability

The IMO clear-cut corner cases for applicability are "user is building a demo like gcoap on a board that happens to have default USB UART and networking" (that should get a VID/PID that says 'RIOT built-in serial adapter'), and "user is building an application with own USB peripherals" (clearly, the prototype pair should be used here and flash the red). Examples should probably show the latter behavior as well as to indicate what a developer should do in own code.

It would be easiest to have one code that just says "RIOT built-in peripherals", and has serial and/or Ethernet configured as applicable or configured. There may be room for a third endpoint that allows for reset into the bootloader by whichever means. If that flexibility within an ID pair is asking too much, there'd be 3 (or 7, with bootloader) IDs for whichever combination of features is active.

Detection

Detection of whether a build is eligible for the "RIOT built-in foo" IDs is another open point; relying on users to do the right thing might not cut it here.

One suggestion for detection would be to create a usbus_user (pseudo?)module(?). The core USB module headers would check for whether either that is active or a USBUS_INTERNAL define is present (outside of their include guards), and otherwise #error out with a message asking to activate usbus_user. The internal users of USB that are sanctioned to use the VID/PID pair set that define for while they're including the USB headers. All this is easy to circumvent intentionally (and there's no ambition to avoid that), but both accidental use of the RIOT IDs and "I'm lazy and will do the quickest thing that'll make things work" cases should be curbed by that.

(The build system would then set the default VID/PIDs to the current prototype ones if usbus_user is active, and to the internal ones otherwise. Either could be overridden. The red warning would show iff the prototype ones are in use, independent of the modules).

Useful links

• https://github.com/openmoko/openmoko-usb-oui is one registry that could provide ID pairs
• http://pid.codes/ the registry from which the current prototype codes are taken

Context

This was brought up by @bergzand in an IRC discussion of VID/PID pairs.

(edit: added pid.codes link)

[chrysn]

I've prepared a branch for a PR to pid.codes at https://github.com/chrysn-pull-requests/pidcodes.github.com/tree/riot-codes (chrysn-pull-requests/pidcodes.github.com@1aea5b9), but not sent it as I'm not convinced this is all correct.

To summarize that request, it's asking for one PID for when only RIOT-OS is setting up peripherals (serial, Ethernet, bootloader reset), one RIOT specific Test PID that has similar policy as the existing test PIDs but can still capture RIOT shipped peripherals, and one for RIOT's DFU bootloader (that, AFAICT, is so far just an idea).

Especially, I'm not sure we get away with a single number for the "just what the board ships" case, as that may include serial and/or Ethernet only if the application demands a stdio or a network interface, resulting in different USB behaviors. As I understand the IDs are largely relevant for Windows drivers (as in Linux and MacOS things always work out of the box, and things like udev can hook into any attribute to decide permissions/groups/names etc). Can a single Windows … driver description? … be created that would encompass, on a single VID/PID pair, all the variation there can be on that? (I've been wondering that before, and pidcodes/pidcodes.github.com#289 reaffirms my doubts that one would be enough).

If it is, can someone confirm that? If it is not, how many "bits" do we need to set aside? I suppose we'll have the (bootloader) reset all the time once it's there, but serial and Ethernet are the two variables, so I'd be asking for 4 IDs then (or 8, 4 for regular and 4 for the Test PID)? Do we anticipate other peripherals to be enabled on a board by default? (I don't really see how a mass storage device would interact there, unless it's the bootloader that gets its own ID anyway.)

Is there any reason why devices running test cases can not just use the Test PID?

[bergzand]

I've prepared a branch for a PR to pid.codes at https://github.com/chrysn-pull-requests/pidcodes.github.com/tree/riot-codes (chrysn-pull-requests/pidcodes.github.com@1aea5b9), but not sent it as I'm not convinced this is all correct.

@kaspar030 @dylad @aabadie: Any opinions on this?

[bergzand]

Especially, I'm not sure we get away with a single number for the "just what the board ships" case, as that may include serial and/or Ethernet only if the application demands a stdio or a network interface, resulting in different USB behaviors.

This is also how I've always understood VID/PID pairs, as one product with a specific and fixed feature set.

as in Linux and MacOS things always work out of the box, and things like udev can hook into any attribute to decide permissions/groups/names etc

Linux and MacOS seem to match USB interface class and subclass numbers. Not sure if it is possible to use that with Windows too.

IMHO the bootloader really requires a separate PID as it makes DFU flashing a lot more convenient. I would prefer to also have a VID/PID for examples, but I can imagine that they don't allow that use case.

[chrysn]

This is also how I've always understood VID/PID pairs, as one product with a specific and fixed feature set.

How set-in-stone do they need to be? Can the USB stack give long-term guarantees that applications that pull in ECM and ACM will always have the ACM at If=0 and the ECM at If=2? Are we sufficiently far along that we feel comfortable picking a virtual reset-pin feature that'd go into that feature set?

(If those are difficult questions to answer, I suggest we keep using the Test PIDs for some more time, let nothing block on this issue, and maybe start experimenting with internal conventions on the use of the Test PIDs before we apply for real ones; a la "Test1 is for bootloader, Test2 is for what we'd later call Generic Application with ACM+ECM+Reset etc").

the bootloader really requires a separate PID as it makes DFU flashing a lot more convenient

Yes, sketched as such.

prefer to also have a VID/PID for examples

The examples have a tendency of getting copy-pasted around as skeletons, so if they use a non-screaming-red pair, that'd easily get out of hand -- and the build system couldn't really know when the example is pristine, when someone's just tweaking it a little, and when it's a separate application. That's what I'm suggesting the RIOT test PID for, it'd scream less loudly (yellow?) and say that this is for RIOT examples and you should pick a real one at some point in time.

(Examples that are not USB examples would use the "Only using RIOT system USB-emulated peripherals" anyway, which never scream but conflict with the usbus_user pseudomodule, or some mechanism to that effect).

[rtek1000]

Hello, I was interested in using the USB port for data transfer from a datalogger that captures data on flash memory, and due to the amount of data, transfer via serial port becomes too long to complete.

It would be great to be able to use an address freely for situation like: pendrive-like device and generic serial port-like device, like FTDI/CH340 etc.

This library below allows the STM32F103 to operate as a USB stick and serial port at the same time (In conjunction with another library, flash memory can be used in place of SD card, ideal for dataloggers):
https://github.com/arpruss/USBComposite_stm32f1

I searched a bit about buying only PID (resale), but found several posts on the internet saying that USB-IF is preventing this procedure. See what I found:

The USB VID is the property of the USB Implementers Forum (“USB-IF”) and is assigned by the USB-IF for use solely by the original vendor to whom the VID is issued. The VID is provided to the assigned company to identify only its own products and neither the VID nor associated PIDs may be sublicensed, transferred or offered for resale in any manner.

The policy of the USB-IF regarding vendor ID numbers (VIDs) is as stated in the attached policy statement. In general, VIDs are not transferable.

The USB-IF has long had a VID/PID process for hobbyists.

Please immediately cease and desist raising funds to purchase a unique USB VID for the purpose of transferring, reselling or sublicensing PIDs and delete all references to the USB-IF, VIDs and PIDs for transfer, resale or sublicense from your website and other marketing materials.

Please kindly reply no later than Friday, October 25 with your written assurance that you will no longer promote the purchase of a community VID or PIDs for sale, transfer, or use by a third party.

We do have a vendor ID number designated for prototype products. This vendor ID number may not be used for a production product. Anybody who has such a need may contact us directly and we will provide them with the proto VID once they confirm that will not be making production products to be distributed and/or sold in the marketplace.

Source: http://www.arachnidlabs.com/blog/2013/10/18/usb-if-no-vid-for-open-source/

So for countries where wireless networking is allowed, it is a more attractive alternative. (In my country the wireless communication device must be government approved, and this approval is costly, so there is no alternative between WiFi / Bluetooth etc and USB, both are unfeasible for small developers, and is one of the reasons so many projects stored in the drawer that get lost over time).
"As long as there is a need to profit from the basics, we will never get off the ground!"

[chrysn]

It would be great to be able to use an address freely for situation like: pendrive-like device and generic serial port-like device, like FTDI/CH340 etc.

That's where I think the "RIOT device with system peripherals" would come in; when used on a serial-only device, it'd instruct the OS/user to treat the device like you would a generic UART adapter, with the higher-level protocol being application defined. (The application's name should probably be indicated in a uniform and locatable way in the idVendor/idProduct).

I searched a bit about buying only PID (resale), but found several posts on the internet saying that USB-IF is preventing this procedure.

Not quite, according to pid.codes. When someone first did this, the IF changed their policy to prevent that. However, the VIDs assigned to pid.codes, openmoko and probably some others were bought when transfer of individual PIDs was not forbidden yet, and the IF apparently had no clause for unilateral change of the contract in there. We're not trying to buy a VID here (for which we'd get a cease-and-desist like in the quoted text), but asking someone who got a VID block back when the old rules applied for some of their PIDs.

So for countries where wireless networking is allowed, it is a more attractive alternative.

It's probably off-topic here, but let me add two notes: * Some radio modules come pre-certified, which should make it a lot easier to go through the necessary approval steps (though it won't spare you all the trouble). Some legislations have provisions for prototyping that can be used until certification can be obtained. * If you are considering network-based access to a data logger, you may want to go that way already for pendrive-based designs; USB Ethernet (CDC-ECM) support is available in RIOT. That would allow you to keep your host-side software unmodified when you switch between USB-, WiFi or other network services. I do have some interest in and experience with network-based; if you want to follow up on that, drop me a mail to chrysn-at-fsfe.org or stop by #RIOT on freenode.org IRC.

[chrysn]

With today's tests with Windows, I'm confident that the proposed PIDs for OS-only and testing can be used as such. A code for a bootloader can still be requested later when we do have code for one.

With that, I've opened a PR for the RIOT organization to be created and two PIDs 7D00 and 7D01 (72xx are taken in the meantime, and with all those n3rds out there it's obviously impossible to start a pseudblock with anything that's 42 or 23 in BCD or hex encoding, or even the ASCII code for "r") to be assigned at pidcodes/pidcodes.github.com#468.

[chrysn]

While this is being processed at pid.codes, I remembered another "codes for free use with X" installment at v-usb (a bit-banging USB low-speed implementation for attiny & co). While not directly relevant if the pid.codes application works out, they do have an interesting policy that says to use domain names for device identification. I like that (though I'd rather go with URIs and some rule a la "the last valid substring that is a URI" to make it unambiguous), and if nothing else (because we probably don't strictly need such a policy) to use it as default values in the examples and in OS-provided setups.

[chrysn]

Now that we have the codes, I'd like to follow this plan for VID/PID handling:

Goal: Allow applications without custom USB code to Just Build, and to use 7D00 (the RIOT-builtin peripherals code) without further configuration. Fail the build if the user wants to do anything manual about USB unless a VID/PID pair is set (with a message that tells to use 7D01 (the test code) in the Makefile, or to get own codes). Warn whenever building with 7D01.

Constraints: Don't require the user to include any pseudomodules or similar if it can be avoided.

• Introduce a define a la USB_H_USER_IS_RIOT that builtin RIOT USB peripherals define while including usb.h
• usb.h checks (outside its include guards) for whether either a CONFIG_USB_PID is present, or USB_H_USER_IS_RIOT is set:
  • If CONFIG_USB_PID is present, use that -- no questions asked (the build system will show the warnings if that should be).
  • If CONFIG_USB_PID is not set and USB_H_USER_IS_RIOT, then set the 7D00 PID.
  • Otherwise, err out.

This will only result in "inconsistent" responses in the error case (in which the build will fail eventually), and have result in the desired over-all behavior. I'm starting a test implementation to see whether I missed anything, but if not, that will be my PR.