[doc] Clarify bytes vs text with non-seeking tarfile stream

[chaica]

BPO	23453
Nosy	@vadmium, @slateny
PRs	gh-67641: Clarify documentation on bytes vs text with non-seeking tarfile stream #31610

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-02-12.11:19:58.435>
labels = ['easy', 'type-bug', '3.9', '3.10', '3.11', 'library', 'docs']
title = '[doc] Clarify bytes vs text with non-seeking tarfile stream'
updated_at = <Date 2022-02-28.08:40:18.686>
user = 'https://bugs.python.org/chaica'

bugs.python.org fields:

activity = <Date 2022-02-28.08:40:18.686>
actor = 'slateny'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Library (Lib)']
creation = <Date 2015-02-12.11:19:58.435>
creator = 'chaica_'
dependencies = []
files = []
hgrepos = []
issue_num = 23453
keywords = ['patch', 'easy']
message_count = 3.0
messages = ['235808', '235810', '264459']
nosy_count = 4.0
nosy_names = ['chaica_', 'docs@python', 'martin.panter', 'slateny']
pr_nums = ['31610']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23453'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• [3.12] gh-67641: Clarify documentation on bytes vs text with non-seeking tarfile stream (GH-31610) #113519
• [3.11] gh-67641: Clarify documentation on bytes vs text with non-seeking tarfile stream (GH-31610) #113520

[chaica]

I'm trying to use a tar stream to a Python tarfile object but each time I do have a TypeError: can't concat bytes to str error

Here is my test:
-----8<-----
#!/usr/bin/python3.4

import tarfile
import sys

tarobj = tarfile.open(mode='r|', fileobj=sys.stdin)
print(tarobj)
tarobj.close()
-----8<

---

$ tar cvf test.tar.gz tests/
tests/
tests/foo1
tests/foo/
tests/foo/bar
$ tar -O -xvf test.tar | ./tarstream.py
tests/
tests/foo1
tests/foo/
tests/foo/bar
Traceback (most recent call last):
  File "./tarstream.py", line 6, in <module>
    tarobj = tarfile.open(mode='r|', fileobj=sys.stdin)
  File "/usr/lib/python3.4/tarfile.py", line 1578, in open
    t = cls(name, filemode, stream, **kwargs)
  File "/usr/lib/python3.4/tarfile.py", line 1470, in __init__
    self.firstmember = self.next()
  File "/usr/lib/python3.4/tarfile.py", line 2249, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "/usr/lib/python3.4/tarfile.py", line 1082, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
  File "/usr/lib/python3.4/tarfile.py", line 535, in read
    buf = self._read(size)
  File "/usr/lib/python3.4/tarfile.py", line 543, in _read
    return self.__read(size)
  File "/usr/lib/python3.4/tarfile.py", line 569, in __read
    self.buf += buf
TypeError: can't concat bytes to str

Regards,
Carl Chenet

[vadmium]

Using fileobj=sys.stdin.buffer instead should do the trick. The “tarfile” module would expect a binary stream, not a text stream.

Given the documentation currently says, “Use this variant in combination with e.g. sys.stdin, . . .”, I presume that is why you were using plain stdin. The documentation should be clarified.

[vadmium]

Looks like the _Stream docstring needs a similar fix regarding stdin and stdout. Also, it wouldn’t hurt to specify that the read() and write() methods should work with bytes, not text.