Incorrect documentation for ignorechars in base64.a85decode()

[serhiy-storchaka]

The base64.a85decode() docstring documents the ignorechars parameter as:

    ignorechars should be a byte string containing characters to ignore from the
    input. This should only contain whitespace characters, and by default
    contains all whitespace characters in ASCII.

But the reStructuredText documentation documents it as:

   *ignorechars* should be a :term:`bytes-like object` or ASCII string
   containing characters to ignore
   from the input. This should only contain whitespace characters, and by
   default contains all whitespace characters in ASCII.

ASCII string does not work as ignorechars.

>>> import base64
>>> base64.a85decode(b' ', ignorechars=b' ')
b''
>>> base64.a85decode(b' ', ignorechars=' ')
TypeError: 'in <string>' requires string as left operand, not int

The change was introduced in a198645 (bpo-1753718).

Also, the "bytes-like object" term is vague and has different meaning in different places. Usually it means support of the buffer protocol, but in this case the code does not work with arbitrary objects implementing the buffer protocol. So I think it is better to return the old "byte string", as in the docstring.

cc @bitdancer.

Linked PRs

• gh-144027: Fix documentation for ignorechars in base64.a85decode() #144028
• [3.14] gh-144027: Fix documentation for ignorechars in base64.a85decode() (GH-144028) #144192
• [3.13] gh-144027: Fix documentation for ignorechars in base64.a85decode() (GH-144028) #144193

[bitdancer]

But it does work with bytearray, array, and memoryview, so I think "bytes-like object" is correct here. As I recall, this change was part of a general drive to clean up the docs that used "byte string" to use "bytes-like object" instead specifically because byte types other that b'xxx' were supported.

What object correctly supporting the buffer protocol does it not support? Do we need an additional term for things which support the above python types but not the buffer protocol in general? Because saying "byte string" here would not be correct.

That I didn't check before saying ascii strings were supported is my bad :(

[serhiy-storchaka]

I restored "byte string" because it is in the docstring. Both terms "byte string" and "bytes-like object" were vague, but the latter is now defined in the glossary as an object that supports the Buffer Protocol and can export a C-contiguous buffer. This has no relation to the current code of a85decode() which only uses __contains__() -- which is not a part of the Buffer Protocol.

The current implementation works also with list and set. They are too far from "byte string" and "bytes-like object", so I think we should ignore this. On other hand, it works with array and memoryview with non-byte format not as with a C-contiguous buffer. For example, 0 in array.array('I', [32]) evaluates to False, but when interpret it as a C-contiguous buffer, 0 in memoryview(array.array('I', [32])).cast('B') evaluates to True.

There is a PR (#102753) that replaces the Python implementation with the C implementation, and a PR (#144024) that adds ignorechars to b64decode(). The C code needs to be more concrete -- either accept a C-contiguous buffer, or only bytes objects, or bytes and bytearray objects.

[serhiy-storchaka]

I would be glad to use "bytes-like object" in 3.15, because we can make it supporting any object that supports the Buffer Protocol and can export a C-contiguous buffer in the C code. But for older versions it is not technically correct. We can get bug reports that the implementation does not match the documentation and will be forced to convert ignorechars to bytes or wrap it into memoryview(...).cast('B') like we did in many other functions. This is an unnecessary complication.

[bitdancer]

OK, that makes sense. Ideally we'd have a better term than 'byte string', but I can't think of one. ("byte string like" :)