Argument Clinic: add support for deprecating positional use of parameters

[erlend-aasland]

Feature or enhancement

Suggesting to add syntax for producing deprecation warnings for positional use of optional parameters.

I made a proof-of-concept patch for this where I introduced a new special symbol x. Example use:

/*[clinic input]
mod.func
    stuff: object
    /
    x
    optarg: int = 128

This will then generate code that emits a DeprecationWarning if optarg is passed as a positional argument, but not if it is passed as a keyword.

We can use this feature to introduce deprecation warnings for parameters that will become keyword-only in future releases. When the deprecation period is done, we can simply replace the x with *, to really make the optional params keyword-only.

Pitch

Quoting @serhiy-storchaka, in issue #93057:

It is recommended to make optional rarely used arguments keyword-only. I do not think it will break much code, but we need a deprecation period for this.

The problem is that sqlite3.connect() was converted to Argument Clinic, and it was much easier to add deprecation warning in the old code.

Previous discussion

• Make optional parameters of sqlite3.connect() keyword-only #93057

Linked PRs

• gh-95065: Add Argument Clinic support for deprecating positional use of parameters #95151
• gh-95065: Argument Clinic: Pretty-print long C strings in generated code #107712
• gh-95065: PoC: Argument Clinic: Support multiple '* [from ...]' deprecations #107742
• gh-95065: Make Argument Clinic append deprecation warnings to docstrings #107745
• gh-95065: Argument Clinic: Add comment to preprocessor warning code #107766
• gh-95065: Argument Clinic: Add functional tests of deprecated positionals #107768
• gh-95065: Produce nicer deprecation messages in Argument Clinic #107808
• gh-95065, gh-107704: Argument Clinic: support multiple '/ [from ...]' and '* [from ...]' markers #108132

[erlend-aasland]

If there is sufficient support for this feature, I'll create a PR.

[erlend-aasland]

If there is sufficient support for this feature, I'll create a PR.

Two Three thumbs up; I'll create the PR 😎

[zware]

I'd suggest to go full warnings._deprecated on it, with an error if the specified removal version has reached beta without the change.

[erlend-aasland]

I'd suggest to go full warnings._deprecated on it, with an error if the specified removal version has reached beta without the change.

Maybe I'm misunderstanding you, but that would imply extending the new syntax with optional deprecated version and removed version fields, right?

[serhiy-storchaka]

I think it is great idea, although I would like to use something more distinguishable and searchable than just x.

We should consider more general problem, when we want to change parameters in incompatible way.

1. Turn positional-or-keyword parameters into keyword-only parameters. A directive which means "the following positional-or-keyword parameters will be keyword-only parameters in future".
2. Turn positional-or-keyword parameters into positional-only parameters. A directive which means "the preceding positional-or-keyword parameters will be positional-only parameters in future".
3. Remove parameter. For some converters we can use NULL or something like to distinguish absent argument, but for others all values after conversion are correct, so we need a special directive for this. When remove positional-only or positional-or-keyword parameter, all following parameters should become keyword-only parameters.
4. Rename positional-or-keyword or keyword-only parameters. In meantime the function should accept both old and new keyword names, but raise an exception if both are passed or if the argument is passed as positional and keyword. For example see Wrong keyword parameter name in regex pattern methods #64482.

There may be other possible incompatible changes, but these four are common.

[erlend-aasland]

I think it is great idea, although I would like to use something more distinguishable and searchable than just x.

Thanks for the support! I'm up for any other symbol; x was just the first thing that popped up in my head when I did the first proof-of-concept implementation. OTOH, * and / are not very searchable either.

We should consider more general problem, when we want to change parameters in incompatible way. [...]

There may be other possible incompatible changes, but these four are common.

+1 However, I think we should create separate issues for each feature :)

[hauntsaninja]

Thanks for suggesting this! This would be nice, e.g. for #56166

A quick bikeshed, do we need to use a new symbol? Something like the following seems pretty readable:

posonlyarg
/
futureposonlyarg
/-in-future
posorkwarg
*-in-3.12
futurekwonlyarg
*
kwonlyarg

Over here *-in-3.12 is a suggestion for how we could specify we want warnings._deprecated semantics.

[erlend-aasland]

do we need to use a new symbol?

I don't know; I just picked that path, because it was very easy to implement as a proof of concept :)

Regarding the proposed extended format: I'd replace the hyphens with single whitespace to make it more readable.

[erlend-aasland]

cc. @colorfulappl: this might be of interest to you.