djsundog
/
circuitpython
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
circuitpython/tools/ci_check_duplicate_usb_vid_...

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

160 lines
6.4 KiB
Python
Raw Normal View History

add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
#!/usr/bin/env python3
#
# This file is part of the MicroPython project, http://micropython.org/
#
# The MIT License (MIT)
#
# Copyright (c) 2020 Michael Schroeder
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
from collections import defaultdict
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
import argparse
import pathlib
import re
import sys
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
DEFAULT_CLUSTERLIST = {
"0x04D8:0xEC44": ["pycubed", "pycubed_mram", "pycubed_mram_v05", "pycubed_v05"],
"0x1B4F:0x8D24": ["sparkfun_qwiic_micro_no_flash", "sparkfun_qwiic_micro_with_flash"],
"0x1D50:0x6153": [
"jpconstantineau_pykey18",
"jpconstantineau_pykey44",
"jpconstantineau_pykey60",
"jpconstantineau_pykey87",
],
"0x239A:0x8019": [
"circuitplayground_express",
"circuitplayground_express_crickit",
"circuitplayground_express_displayio",
],
"0x239A:0x801F": ["trinket_m0_haxpress", "trinket_m0"],
"0x239A:0x8021": ["metro_m4_express", "cp32-m4"],
"0x239A:0x8023": ["feather_m0_express", "feather_m0_supersized"],
"0x239A:0x80A6": ["espressif_esp32s2_devkitc_1_n4r2", "espressif_saola_1_wrover"],
"0x239A:0x80AC": ["unexpectedmaker_feathers2", "unexpectedmaker_feathers2_prerelease"],
"0x239A:0x80C8": ["espressif_kaluga_1", "espressif_kaluga_1.3"],
"0x303A:0x7003": [
"espressif_esp32s3_devkitc_1_n8",
"espressif_esp32s3_devkitc_1_n8r2",
"espressif_esp32s3_devkitc_1_n8r8",
ignore espressif_esp32s3_devkitc_1_n32r8 in vid:pid check
2022-11-17 23:24:40 -05:00
"espressif_esp32s3_devkitc_1_n32r8",
Add hack tablet
2023-09-07 13:59:42 -04:00
"espressif_esp32s3_devkitc_1_n8r8_hacktablet",
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
],
run black on tools/ci_check_duplicate_usb_vid_pid.py
2022-10-05 17:37:33 -04:00
"0x303A:0x7009": [
"espressif_esp32s2_devkitc_1_n4",
"espressif_esp32s2_devkitc_1_n4r2",
"espressif_esp32s2_devkitc_1_n8r2",
],
Allow duplicate VID and PID in CI check.
2022-11-15 18:06:36 -05:00
"0x239A:0x102E": ["weact_studio_pico", "weact_studio_pico_16mb"],
YD_ESP32_S3
2023-04-07 11:34:10 -04:00
"0x303A:0x8166": ["yd_esp32_s3_n8r8", "yd_esp32_s3_n16r8"],
Update settings to match micropython
2023-08-04 11:54:22 -04:00
"0x2341:0x056B": ["arduino_nano_esp32s3", "arduino_nano_esp32s3_inverted_statusled"],
Merge conflict resolved
2023-09-01 17:35:14 -04:00
"0x2E8A:0x1020": ["waveshare_rp2040_plus_4mb", "waveshare_rp2040_plus_16mb"],
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
}
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
cli_parser = argparse.ArgumentParser(
description="USB VID/PID and Creator/Creation ID Duplicate Checker"
)
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
run code formatting script
2021-03-15 09:57:36 -04:00
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
def configboard_files():
run code formatting script
2021-03-15 09:57:36 -04:00
"""A pathlib glob search for all ports/*/boards/*/mpconfigboard.mk file
paths.
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
allow micropython as top-level dir name in duplicate pid/vid check
2021-07-27 08:57:13 -04:00
:returns: A ``pathlib.Path.glob()`` generator object
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
"""
check vid/pid: find topdir relative to this script @dhalbert noticed that if a fork was called `micropython`, the existing check could fail during CI because it will be cloned into a directory also called `micropython`. Instead of hardcoding a range of strings that are OK as top directories, find the location of the top directory relative to the script. This will be conflicty after #5069 is merged, but it should be easy to fix.
2021-07-27 09:39:21 -04:00
working_dir = pathlib.Path(__file__).resolve().parent.parent
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
return working_dir.glob("ports/**/boards/**/mpconfigboard.mk")
run code formatting script
2021-03-15 09:57:36 -04:00
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
VID_PATTERN = re.compile(r"^USB_VID\s*=\s*(.*)", flags=re.M)
PID_PATTERN = re.compile(r"^USB_PID\s*=\s*(.*)", flags=re.M)
CREATOR_PATTERN = re.compile(r"^CIRCUITPY_CREATOR_ID\s*=\s*(.*)", flags=re.M)
CREATION_PATTERN = re.compile(r"^CIRCUITPY_CREATION_ID\s*=\s*(.*)", flags=re.M)
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
def check_vid_pid(files, clusterlist):
run code formatting script
2021-03-15 09:57:36 -04:00
"""Compiles a list of USB VID & PID values for all boards, and checks
for duplicates. Exits with ``sys.exit()`` (non-zero exit code)
if duplicates are found, and lists the duplicates.
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
"""
Update all target sdkconfig
2023-09-21 16:52:47 -04:00
usb_pattern = re.compile(
r"^CIRCUITPY_USB\s*=\s*0$|^IDF_TARGET = (esp32|esp32c3|esp32c6|esp32h2)$", flags=re.M
)
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
usb_ids = defaultdict(set)
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
for board_config in files:
src_text = board_config.read_text()
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
usb_vid = VID_PATTERN.search(src_text)
usb_pid = PID_PATTERN.search(src_text)
creator = CREATOR_PATTERN.search(src_text)
creation = CREATION_PATTERN.search(src_text)
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
non_usb = usb_pattern.search(src_text)
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
board_name = board_config.parts[-2]
if usb_vid and usb_pid:
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
id_group = f"0x{int(usb_vid.group(1), 16):04X}:0x{int(usb_pid.group(1), 16):04X}"
elif non_usb:
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
if creator is None or creation is None:
print(f"{board_name=} {creator=} {creation=}", file=sys.stderr)
continue
id_group = f"0x{int(creator.group(1), 16):08X}:0x{int(creation.group(1), 16):08X}"
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
else:
Update ci_check_duplicate_usb_vid_pid.py for ESP32
2022-07-08 16:11:01 -04:00
raise SystemExit(f"Could not find expected settings in {board_config}")
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
usb_ids[id_group].add(board_name)
duplicates = []
for key, boards in usb_ids.items():
if len(boards) == 1:
continue
# It is a cluster
cluster = set(clusterlist.get(key, []))
if cluster != boards:
if key == "":
duplicates.append(f"- Non-USB:\n" f" Boards: {', '.join(sorted(boards))}")
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
else:
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
duplicates.append(f"- VID/PID: {key}\n" f" Boards: {', '.join(sorted(boards))}")
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
if duplicates:
duplicates = "\n".join(duplicates)
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
duplicate_message = (
Add more PIDs for unique boards
2020-09-15 16:43:35 -04:00
f"Duplicate VID/PID usage found!\n{duplicates}\n"
f"If you are open source maker, then you can request a PID from http://pid.codes\n"
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
f"For boards without native USB, you can request a Creator ID from https://github.com/creationid/creators/\n"
add a link to the faq about this
2023-08-30 11:54:43 -04:00
f"Otherwise, companies should pay the USB-IF for a vendor ID: https://www.usb.org/getting-vendor-id\n"
f"FAQ: Why does CircuitPython require a unique VID:PID for every board definition? https://learn.adafruit.com/how-to-add-a-new-board-to-circuitpython/frequently-asked-questions#faq-3130480"
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
)
sys.exit(duplicate_message)
Update ci_check_duplicate_usb_vid_pid.py for ESP32
2022-07-08 16:11:01 -04:00
Add more PIDs for unique boards
2020-09-15 16:43:35 -04:00
else:
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
print("No unexpected ID duplicates found.")
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
if __name__ == "__main__":
arguments = cli_parser.parse_args()
Check uniqueness of creatorid:creationid Turns out we already have one duplicate, so I 'endorsed' it, but probably one should be reassigned. Closes: #8628
2022-08-27 22:32:39 -04:00
print("Running USB VID/PID and Creator/Creation ID Duplicate Checker...")
add ci_check_duplicate_usb_vid_pid.py; checks that a new board doesn't use duplicate USB VID/PID values unless explicitly whitelisted to do so
2020-05-27 17:51:57 -04:00
board_files = configboard_files()
Improve the USB vid:pid duplicate checker To me, it made more sense to track which boards go together in a cluster; when reviewing a request to actually use a duplicate vid/pid, you want to know what board(s) it is aliasing. I also revamped the detection of non-USB boards so that a board .mk file that couldn't be parsed by the code here would raise a problem instead of just being skipped for the purposes of checking. There were some lines with comments on the end, and some variation in capitalization of the IDs. These are all normalized and a (sometimes unfriendly!) error printed when it's incorrect. Before this, here were some ways to trick the duplicate vid/pid checker: ``` USB_PID = 0XABCD USB_PID = 0xAbCd USB_PID = 0xABCD # harmless comment? ``` None of these things were ever done on purpose.
2022-03-24 10:42:11 -04:00
check_vid_pid(board_files, DEFAULT_CLUSTERLIST)