2.1.4 Subclassing other types

It is possible to create new extension types that are derived from existing types. It is easiest to inherit from the built in types, since an extension can easily use the PyTypeObject it needs. It can be difficult to share these PyTypeObject structures between extension modules.

In this example we will create a Shoddy type that inherits from the builtin list type. The new type will be completely compatible with regular lists, but will have an additional increment() method that increases an internal counter.

>>> import shoddy
>>> s = shoddy.Shoddy(range(3))
>>> s.extend(s)
>>> print len(s)
6
>>> print s.increment()
1
>>> print s.increment()
2

#include <Python.h>

typedef struct {
    PyListObject list;
    int state;
} Shoddy;

static PyObject *
Shoddy_increment(Shoddy *self, PyObject *unused)
{
    self->state++;
    return PyInt_FromLong(self->state);
}

static PyMethodDef Shoddy_methods[] = {
    {"increment", (PyCFunction)Shoddy_increment, METH_NOARGS,
     PyDoc_STR("increment state counter")},
    {NULL,	NULL},
};

static int
Shoddy_init(Shoddy *self, PyObject *args, PyObject *kwds)
{
    if (PyList_Type.tp_init((PyObject *)self, args, kwds) < 0)
        return -1;
    self->state = 0;
    return 0;
}

static PyTypeObject ShoddyType = {
    PyObject_HEAD_INIT(NULL)
    0,                       /* ob_size */
    "shoddy.Shoddy",         /* tp_name */
    sizeof(Shoddy),          /* tp_basicsize */
    0,                       /* tp_itemsize */
    0,                       /* tp_dealloc */
    0,                       /* tp_print */
    0,                       /* tp_getattr */
    0,                       /* tp_setattr */
    0,                       /* tp_compare */
    0,                       /* tp_repr */
    0,                       /* tp_as_number */
    0,                       /* tp_as_sequence */
    0,                       /* tp_as_mapping */
    0,                       /* tp_hash */
    0,                       /* tp_call */
    0,                       /* tp_str */
    0,                       /* tp_getattro */
    0,                       /* tp_setattro */
    0,                       /* tp_as_buffer */
    Py_TPFLAGS_DEFAULT |
      Py_TPFLAGS_BASETYPE,   /* tp_flags */
    0,                       /* tp_doc */
    0,                       /* tp_traverse */
    0,                       /* tp_clear */
    0,                       /* tp_richcompare */
    0,                       /* tp_weaklistoffset */
    0,                       /* tp_iter */
    0,                       /* tp_iternext */
    Shoddy_methods,          /* tp_methods */
    0,                       /* tp_members */
    0,                       /* tp_getset */
    0,                       /* tp_base */
    0,                       /* tp_dict */
    0,                       /* tp_descr_get */
    0,                       /* tp_descr_set */
    0,                       /* tp_dictoffset */
    (initproc)Shoddy_init,   /* tp_init */
    0,                       /* tp_alloc */
    0,                       /* tp_new */
};

PyMODINIT_FUNC
initshoddy(void)
{
    PyObject *m;

    ShoddyType.tp_base = &PyList_Type;
    if (PyType_Ready(&ShoddyType) < 0)
        return;

    m = Py_InitModule3("shoddy", NULL, "Shoddy module");
    if (m == NULL)
        return;

    Py_INCREF(&ShoddyType);
    PyModule_AddObject(m, "Shoddy", (PyObject *) &ShoddyType);
}

As you can see, the source code closely resembles the Noddy examples in previous sections. We will break down the main differences between them.

typedef struct {
	PyListObject list;
	int state;
} Shoddy;

The primary difference for derived type objects is that the base type's object structure must be the first value. The base type will already include the PyObject_HEAD at the beginning of its structure.

When a Python object is a Shoddy instance, its PyObject* pointer can be safely cast to both PyListObject* and Shoddy*.

static int
Shoddy_init(Shoddy *self, PyObject *args, PyObject *kwds)
{
	if (PyList_Type.tp_init((PyObject *)self, args, kwds) < 0)
		return -1;
	self->state = 0;
	return 0;
}

In the __init__ method for our type, we can see how to call through to the __init__ method of the base type.

This pattern is important when writing a type with custom new and dealloc methods. The new method should not actually create the memory for the object with tp_alloc, that will be handled by the base class when calling its tp_new.

When filling out the PyTypeObject for the Shoddy type, you see a slot for tp_base. Due to cross platform compiler issues, you can't fill that field directly with the PyList_Type; it can be done later in the module's init function.

PyMODINIT_FUNC
initshoddy(void)
{
	PyObject *m;

	ShoddyType.tp_base = &PyList_Type;
	if (PyType_Ready(&ShoddyType) < 0)
		return;

	m = Py_InitModule3("shoddy", NULL, "Shoddy module");
	if (m == NULL)
		return;

	Py_INCREF(&ShoddyType);
	PyModule_AddObject(m, "Shoddy", (PyObject *) &ShoddyType);
}

Before calling PyType_Ready, the type structure must have the tp_base slot filled in. When we are deriving a new type, it is not necessary to fill out the tp_alloc slot with PyType_GenericNew - the allocate function from the base type will be inherited.

After that, calling PyType_Ready and adding the type object to the module is the same as with the basic Noddy examples.