Increase Argument Clinic test coverage

[erlend-aasland]

The Argument Clinic test suite consists of three parts:

• Lib/test/test_clinic.py: mostly parser tests
• Lib/test/clinic.test: generated code, checked by test_clinic.py (ClinicExternalTest)
• Modules/_testclinic.c: mostly converter tests (thanks, @colorfulappl!)

Test coverage is currently just above 80%, which is too low. I'd say we try to reach 90-something percent coverage.

Reminder: PRs for increased test coverage should be backported to all bugfix branches.

Linked PRs

• gh-106368: Increase Argument Clinic test coverage #106369
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106369) #106370
• gh-106368: Clean up Argument Clinic tests #106373
• [3.11] gh-106368: Increase Argument Clinic test coverage (#106369) #106374
• [3.12] gh-106368: Clean up Argument Clinic tests (#106373) #106379
• [3.11] gh-106368: Clean up Argument Clinic tests (#106373) #106381
• gh-106368: Harden Argument Clinic parser tests #106384
• [3.12] gh-106368: Harden Argument Clinic parser tests (GH-106384) #106387
• [3.11] gh-106368: Harden Argument Clinic parser tests (GH-106384) #106388
• gh-106368: Increase Argument Clinic test coverage #106389
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106389) #106390
• [3.11] gh-106368: Increase Argument Clinic test coverage (GH-106389) #106391
• gh-106368: Add tests for permutation helpers in Argument Clinic #106407
• [3.12] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) #106409
• [3.11] gh-106368: Add tests for permutation helpers in Argument Clinic (GH-106407) #106410
• gh-106368: Add tests for formatting helpers in Argument Clinic #106415
• [3.12] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) #106438
• [3.11] gh-106368: Add tests for formatting helpers in Argument Clinic (GH-106415) #106439
• gh-106368: Increase Argument Clinic test coverage #106728
• [3.12] gh-106368: Increase Argument Clinic test coverage (GH-106728) #106730
• [3.11] gh-106368: Increase Argument Clinic test coverage (#106728) #106731
• gh-106368: Increase Argument Clinic BlockParser test coverage #106759
• [3.12] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) #106769
• [3.11] gh-106368: Increase Argument Clinic BlockParser test coverage (GH-106759) #106770
• gh-106368: Add functional tests for Tools/clinic/cpp.py #106813
• gh-106368: Increase Argument Clinic test coverage for cpp.Monitor #106833
• [3.12] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) #106838
• [3.11] gh-106368: Increase Argument Clinic test coverage for cpp.Monitor (GH-106833) #106839
• gh-106368: Increase Argument Clinic test coverage for IndentStack #106933
• [3.12] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) #106943
• [3.11] gh-106368: Increase Argument Clinic test coverage for IndentStack (GH-106933) #106944
• gh-106368: Increase coverage for Argument Clinic output directive #106979
• [3.12] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) #106994
• [3.11] gh-106368: Increase coverage for Argument Clinic output directive (GH-106979) #107002
• gh-106368: Increase Argument Clinic CLI test coverage #107156
• [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) #107189
• [3.11] gh-106368: Increase Argument Clinic CLI test coverage (GH-107156) #107190
• gh-106368: Increase Argument Clinic CLI test coverage #107277
• [3.12] gh-106368: Increase Argument Clinic CLI test coverage (GH-107277) #107282
• [3.11] gh-106368: Increase Argument Clinic CLI test coverage (#107277) #107326
• gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail #107364
• [3.12] gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail (GH-107364) #107365
• [3.11] gh-106368: Argument clinic tests: improve failure message when tests in ClinicExternalTests fail (GH-107364) #107366
• gh-106368: Argument clinic: Fix minor bug in state_modulename_name #107387
• gh-106368: Add test for Argument Clinic misbehaving custom converter_init() #107496
• [3.12] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) #107499
• [3.11] gh-106368: Add test for Argument Clinic misbehaving custom converter_init() (GH-107496) #107500
• gh-106368: Increase test coverage for Argument Clinic #107514
• gh-106368: Increase Argument Clinic test coverage #107582
• gh-106368: Argument clinic tests: improve error message when expect_success() fails #107606
• gh-106368: Increase Argument Clinic test coverage #107611
• gh-106368: Argument clinic: improve coverage for self.valid_line() calls #107641
• gh-106368: Improve coverage reports for argument clinic #107693
• gh-106368: Argument clinic: add tests for more failure paths #107731
• gh-106368: Argument Clinic: Add tests for cloned functions with custom C base names #107977
• gh-106368: Argument Clinic: Test that keyword params are disallowed in groups #107985

[erlend-aasland]

Also: a side effect of good coverage, is that it can help identify unreachable paths in the code.

[erlend-aasland]

cc. @sobolevn, who I know enjoys increasing test coverage 📈