Alex Rivera | Logout

Limitations of PyTuple_SetItem

Asked 2011-05-24T14:16:51.793
12

I have a Python extension module which creates a tuple as an attribute of another object, and sets items in the tuple. Whenever I execute this module in Python, I keep getting the error SystemError: bad argument to internal function

After reading over the docs for PyTuple, and debugging my program for a few hours, I still couldn't figure out what the hell was going on. Running my program through a debugger indicated the problem was occurring within a library call inside the Python interpreter. So, finally, I looked at the Python source code, and at long last I realized the problem. The PyTuple_SetItem function has an interesting restriction which I didn't know about, and can't find explicitly documented.

Here is the important function in the Python source (edited for clarity):

int PyTuple_SetItem(register PyObject *op, register Py_ssize_t i, PyObject *newitem)
{
    .....
    if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
        Py_XDECREF(newitem);
        PyErr_BadInternalCall();
        return -1;
    }
    .....
}

The important line here is the condition op->ob_refcnt != 1. So here's the problem: you can't even call PyTuple_SetItem unless the Tuple has a ref-count of 1. It looks like the idea here is that you're never supposed to use PyTuple_SetItem except right after you create a tuple using PyTuple_New(). I guess this makes sense, since Tuples are supposed to be immutable, after all, so this restriction helps keep your C code more in line with the abstractions of the Python type system.

However, I can't find this restriction documented anywhere. Relevant docs seem to be here and here

Edit
Report

1 Answer

8

I'm quite sure that you can get around the restrictions by using PyTuple_SET_ITEM instead of PyTuple_SetItem. PyTuple_SET_ITEM is a macro defined in tupleobject.h as follows:

#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject*)(op))->ob_item[i] = v

So, if you are absolutely, definitely and utterly sure that:

  1. op is a tuple object
  2. you haven't initialized slot i in the tuple so far
  3. you own a reference to v and you want to let the tuple steal it and
  4. there is no chance of another Python object using the tuple for anything before you call PyTuple_SET_ITEM

then I guess you are safe to use PyTuple_SET_ITEM.

answered 2011-05-24T14:20:39.110

Your Answer