Alex Rivera | Logout

Python file.tell() giving strange numbers?

Asked 2013-04-10T19:41:51.270
19

I am using Python 3.3.0, on windows 64bit.

I have a text file as shown below: (see bottom for download link at mediafire)

hello

-data1:blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah blah


-data2:blah blah blah blah blah blah blah blah blah blah blah
-data3: Empty

-data4: Empty

I'm trying to navigate around the file, and thus I use .tell() to figure out what my position is. However, when reading through the lines of the file as shown below, I get a very strange result:

f=open("test.txt")
while True:
    a = f.readline()
    print("{}    {}".format(repr(a),f.tell()))
    if a == "":
        break

The result:

'hello\n'    7
'\n'    9
'-data1:blah blah blah blah blah blah blah blah blah blah blah blah blah blah bl
ah blah\n'    18446744073709551714
'\n'    99
'\n'    101
'-data2:blah blah blah blah blah blah blah blah blah blah blah\n'    164
'-data3: Empty\n'    179
'\n'    181
'-data4: Empty'    194
''    194

What's with the 18446744073709551714 for the 3rd line? Though it looks like an impossible value, f.seek(18446744073709551714) is an acceptable value that apparently does bring me to the end of the 3rd line. Though, I can't seem to figure out why.

EDIT: Opening in binary mode gives no problems with tell():

f=open("test.txt","rb")
while True:
    a = f.readline()
    print("{}    {}".format(repr(a),f.tell()))
    if a == b"":
        break

The result:

b'hello\r\n'    7
b'\r\n'    9
b'-data1:blah blah blah blah blah blah blah blah blah blah blah blah blah blah b
lah blah\r\n'    97
b'\r\n'    99
b'\r\n'    101
b'-data2:blah blah blah blah blah blah blah blah blah blah blah\r\n'    164
b'-data3: Empty\r\n'    179
b'\r\n'    181
b'-data4: Empty'    194
b''    194

The test.txt text file is downloadable here, just

Edit
Report

1 Answer

17

It's a documented behaviour caused by UNIX-style line endings:

file.tell()

Return the file’s current position, like stdio's ftell().

Note: On Windows, tell() can return illegal values (after an fgets()) when reading files with Unix-style line-endings. Use binary mode ('rb') to circumvent this problem.


The above documentation is taken from the python2.7.4 documentation. The documentation for python3 changed a bit, since there is now a hierarchy of classes that handle I/O and I can't find this bit of information. Your test shows that the behaviour didn't change anyway. Also the source code for python3.3 has an XXX Windows support below is likely incomplete comment before the function called by tell.


There is an issue in python bug tracker related to this, and the final comment by Catalin Iacob is:

I tried to reproduce this, picked a file on my disk and indeed I got a negative number, but that file has Unix line endings. This is documented at http://docs.python.org/2/library/stdtypes.html#file.tell so probably there's nothing to do then.

As for Armin's report in msg180145, even though it's not intuitive, this matches ftell's behavior on Windows, as documented in the Remarks section of http://msdn.microsoft.com/en-us/library/0ys3hc0b%28v=vs.100%29.aspx. The tell() method on fileobjects is explicitly documented as matching ftell behavior: "Return the file’s current position, like stdio‘s

answered 2013-04-10T19:46:33.537

Your Answer