Creating a tool to make a 64k demo

In the process of picking up this webpage again, I can talk about something we did quite a while ago. I, together with a team, went through the process of making a 64 kilobyte demo. We happened to win at one of the biggest demoscene events in europe. Revision 2017. I still feel the afterglow of happiness from that.

If you’re not sure what that is, read on, else, scroll down! You program a piece of software that is only 64 kb in size, that shows an audio-visual experience generated in realtime. To stay within such size limits you have to generate everything, we chose to go for a rendering technique called ray marching, that allowed us to put all 3D modeling, texture generation, lighting, etc. as ascii (glsl sources) in the executable. On top of that we used a very minimal (yet versatile) modular synthesizer called 64klang2. Internally it stores a kind of minimal midi data and the patches and it can render amazing audio in realtime, so it doesn’t need to pre-render the song or anything. All this elementary and small size data and code compiles to something over 200kb, which is then compressed using an executable packer like UPX or kkrunchy

It was called Eidolon. You can watch a video:
https://youtu.be/rsZHBJdaz-Y
Or stress test your GPU / leave a comment here:
http://www.pouet.net/prod.php?which=69669

The technologies used were fairly basic, it’s very old school phong & lambert shading, 2 blur passes for bloom, so all in all pretty low tech and not worth discussing. What I would like to discuss is the evolution of the tool. I’ll keep it high level this time though. Maybe in the future I can talk about specific implementations of things, but just seeing the UI will probably explain a lot of the features and the way things work.

Step 1: Don’t make a tool from scratch

Our initial idea was to leverage existing software. One of our team members, who controlled the team besides modelling and eventually directing the whole creative result, had some experience with a real-time node based software called Touch Designer. It is a tool where you can do realtime visuals, and it supports exactly what we need: rendering into a 2D texture with a fragment shader.

We wanted to have the same rendering code for all scenes, and just fill in the modeling and material code that is unique per scene. We figured out how to concatenate separate pieces of text and draw them into a buffer. Multiple buffers even. At some point i packed all code and rendering logic of a pass into 1 grouped node and we could design our render pipeline entirely node based.

Here you see the text snippets (1) merged into some buffers (2) and then post processed for the bloom (3). On the right (4) you see the first problem we hit with Touch Designer. The compiler error log is drawn inside this node. There is basically no easy way to have that error visible in the main application somewhere. So the first iteration of the renderer (and coincidentally the main character of Eidolon) looked something like this:

The renderer didn’t really change after this.

In case I sound too negative about touch designer in the next few paragraphs, our use case was rather special, so take this with a grain of salt!

We have a timeline control, borrowed the UI design from Maya a little, so this became the main preview window. That’s when we hit some problems though. The software has no concept of window focus, so it’d constantly suffer hanging keys or responding to keys while typing in the text editor.

Last issue that really killed it though: everything has to be in 1 binary file. There is no native way to reference external text files for the shader code, or merge node graphs. There is a really weird utility that expands the binary to ascii, but then literally every single node is a text file so it is just unmergeable.

Step 2: Make a tool

So then this happened:

Over a week’s time in the evenings and then 1 long saturday I whipped this up using PyQt and PyOpenGL. This is the first screenshot I made, the curve editor isn’t actually an editor yet and there is no concept of camera shots (we use this to get hard cuts).

It has all the same concepts however, separate text files for the shader code, with an XML file determining what render passes use what files and in what buffer they render / what buffers they reference in turn. With the added advantage of the perfect granularity all stored in ascii files.

Some files are template-level, some were scene-level, so creating a new scene actually only copies the scene-level fies which can them be adjusted in a text editor, with a file watcher updating the picture. The CurveEditor feeds right back into the uniforms of the shader (by name) and the time slider at the bottom is the same idea as Maya / what you saw before.

Step 3: Make it better

Render pipeline
The concept was to set up a master render pipeline into which scenes would inject snippets of code. On disk this became a bunch of snippets, and an XML based template definition. This would be the most basic XML file:

<template>
    <pass buffer="0" outputs="1">
        <global path="header.glsl"/>
        <section path="scene.glsl"/>
        <global path="pass.glsl"/>
    </pass>
    <pass input0="0">
        <global path="present.glsl"/>
    </pass>
</template>

This will concatenated 3 files to 1 fragment shader, render into full-screen buffer “0” and then use present.glsl as another fragment shader, which in turn has the previous buffer “0” as input (forwarded to a sampler2D uniform).

This branched out into making static bufffers (textures), setting buffer sizes (smaller textures), multiple target buffers (render main and reflection pass at once), set buffer size to a portion of the screen (downsampling for bloom), 3D texture support (volumetric noise textures for cloud).

Creating a new scene will just copy “scene.glsl” from the template to a new folder, there you can then fill out the necessary function(s) to get a unique scene. Here’s an example from our latest Evoke demo. 6 scenes, under which you see the “section” files for each scene.

Camera control
The second important thing I wanted to tackle was camera control. Basically the demo will control the camera based on some animation data, but it is nice to fly around freely and even use the current camera position as animation keyframe. So this was just using Qt’s event system to hook up the mouse and keyboard to the viewport.

I also created a little widget that displays where the camera is, has an “animation input or user input” toggle as well as a “snap to current animation frame” button.

Animation control
So now to animate the camera, without hard coding values! Or even typing numbers, preferably. I know a lot of people use a tracker-like tool called Rocket, I never used it and it looks an odd way to control animation data to me. I come from a 3D background, so I figured I’d just want a curve editor like e.g. Maya has. In Touch Designer we also had a basic curve editor, conveniently you can name a channel the same as a uniform, then just have code evaluate the curve at the current time and send the result to that uniform location.
Some trickery was necessary to pack vec3s, I just look for channels that start with the same name and then end in .x, .y, .z, and possibly .w.

Here’s an excerpt from a long camera shot with lots of movement, showing off our cool hermite splines. At the top right you can see we have several built in tangent modes, we never got around to building custom tangent editing. In the end this is more than enough however. With flat tangents we can create easing/acceleration, with spline tangents we can get continuous paths and with linear tangents we get continuous speed. Next to that are 2 cool buttons that allow us to feed the camera position to another uniform, so you can literally fly to a place where you want to put an object. It’s not as good as actual move/rotate widgets but for the limited times we need to place 3D objects it’s great.

Hard cuts
Apart from being impossible to represent in this interface, we don’t support 2 keys at identical times. This means that we can’t really have the camera “jump” to a new position instantly. With a tiny amount of curve inbetween the previous and the next shot position, the time cursor can actually render 1 frame of a random camera position. So we had to solve this. I think it is one of the only big features that you won’t see in the initial screenshot above actually.

Introducing camera shots. A shot has its own “scene it should display” and its own set of animation data. So selecting a different shot yields different curve editor content. Shots are placed on a shared timeline, so scrolling through time will automatically show the right shot and setting a keyframe will automatically figure out the “shot local time” to put the key based on the global demo time. The curve editor has it’s own playhead that is directly linked to the global timeline as well so we can adjust the time in multiple places.

When working with lots of people we had issues with people touching other people’s (work in progress) shots. Therefore we introduced “disabling” of shots. This way anyone could just prefix their shots and disable them before submitting, and we could mix and match shots from several people to get a final camera flow we all liked.

Shots are also rendered on the timeline as colored blocks. The grey block underneath those is our “range slider”. It makes the top part apply on only a subsection of the demo, so it is easy to loop a specific time range, or just zoom in far enough to use the mouse to change the time granularly enough.

The devil is in the details
Some things I overlooked in the first implementation, and some useful things I added only recently.
1. Undo/Redo of animation changes. Not unimportant, and luckily not hard to add with Qt.
2. Ctrl click timeline to immediately start animating that shot
3. Right click a shot to find the scene
4. Right click a scene to create a shot for that scene in particular
5. Current time display in minutes:seconds instead of just beats
6. BPM stored per-project instead of globally
7. Lots of hotkeys!

These things make the tool just that much faster to use.

Finally, here’s our tool today. There’s still plenty to be done, but we made 2 demos with it so far and it gets better every time!

Part 3: Importing and drawing a custom mesh file

Part 3: Creating an importer

This is part 3 of a series and it is about getting started with visualizing triangle meshes with Python 2.7 using the libraries PyOpenGL and PyQt4.

Part 1
Part 2
Part 3

I will assume you know python, you will not need a lot of Qt or OpenGL experience, though I will also not go into the deeper details of how OpenGL works. For that I refer you to official documentation and the excellent (C++) tutorials at https://open.gl/. Although they are C++, there is a lot of explanation about OpenGL and why to do certain calls in a certain order.

On a final note: I will make generalizations and simplifications when explaining things. If you think something works different then I say it probably does, this is to try and convey ideas to beginners, not to explain low level openGL implementations.

3.1 Importing

Now with our file format resembling openGL so closely makes this step relatively easy. First I’ll declare some globals, because openGL does not have real enums but just a bunch of global constants I make some groups to do testing and data mapping against.

from OpenGL.GL import *

attributeElementTypes = (GL_BYTE,
                        GL_UNSIGNED_BYTE,
                        GL_SHORT,
                        GL_UNSIGNED_SHORT,
                        GL_INT,
                        GL_UNSIGNED_INT,
                        GL_HALF_FLOAT,
                        GL_FLOAT,
                        GL_DOUBLE,
                        GL_FIXED,
                        GL_INT_2_10_10_10_REV,
                        GL_UNSIGNED_INT_2_10_10_10_REV,
                        GL_UNSIGNED_INT_10F_11F_11F_REV)
sizeOfType = {GL_BYTE: 1,
             GL_UNSIGNED_BYTE: 1,
             GL_SHORT: 2,
             GL_UNSIGNED_SHORT: 2,
             GL_INT: 4,
             GL_UNSIGNED_INT: 4,
             GL_HALF_FLOAT: 2,
             GL_FLOAT: 4,
             GL_DOUBLE: 8,
             GL_FIXED: 4,
             GL_INT_2_10_10_10_REV: 4,
             GL_UNSIGNED_INT_2_10_10_10_REV: 4,
             GL_UNSIGNED_INT_10F_11F_11F_REV: 4}
drawModes = (GL_POINTS,
            GL_LINE_STRIP,
            GL_LINE_LOOP,
            GL_LINES,
            GL_LINE_STRIP_ADJACENCY,
            GL_LINES_ADJACENCY,
            GL_TRIANGLE_STRIP,
            GL_TRIANGLE_FAN,
            GL_TRIANGLES,
            GL_TRIANGLE_STRIP_ADJACENCY,
            GL_TRIANGLES_ADJACENCY,
            GL_PATCHES)
indexTypeFromSize = {1: GL_UNSIGNED_BYTE, 2: GL_UNSIGNED_SHORT, 4: GL_UNSIGNED_INT}

Next up is a Mesh class that stores a vertex array object (and corresponding buffers for deletion) along with all info necessary to draw the mesh once it’s on the GPU.

class Mesh(object):
    def __init__(self, vao, bufs, drawMode, indexCount, indexType):
        self.__vao = vao
        self.__bufs = bufs
        self.__drawMode = drawMode
        self.__indexCount = indexCount
        self.__indexType = indexType
 
    def __del__(self):
        glDeleteBuffers(len(self.__bufs), self.__bufs)
        glDeleteVertexArrays(1, [self.__vao])
 
    def draw(self):
        glBindVertexArray(self.__vao)
        glDrawElements(self.__drawMode, self.__indexCount, self.__indexType, None)

Now let’s, given a file path, open up the file and run the importer for the right version (if known).

def model(filePath):
    vao = glGenVertexArrays(1)
    glBindVertexArray(vao)
    bufs = glGenBuffers(2)
    glBindBuffer(GL_ARRAY_BUFFER, bufs[0])
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, bufs[1])
    with open(filePath, 'rb') as fh:
        fileVersion = struct.unpack('B', fh.read(1))[0]
        if fileVersion == 0:
            return _loadMesh_v0(fh, vao, bufs)
        raise RuntimeError('Unknown mesh file version %s in %s' % (fileVersion, filePath))

Next we can start reading the rest of the file:

    vertexCount = struct.unpack('I', fh.read(4))[0]
    vertexSize = struct.unpack('B', fh.read(1))[0]
    indexCount = struct.unpack('I', fh.read(4))[0]
    indexSize = struct.unpack('B', fh.read(1))[0]
    assert indexSize in indexTypeFromSize, 'Unknown element data type, element size must be one of %s' % indexTypeFromSize.keys()
    indexType = indexTypeFromSize[indexSize]
    drawMode = struct.unpack('I', fh.read(4))[0]
    assert drawMode in (GL_LINES, GL_TRIANGLES), 'Unknown draw mode.'  # TODO: list all render types

Read and apply the attribute layout:

# gather layout
numAttributes = struct.unpack('B', fh.read(1))[0]
offset = 0
layouts = []
for i in xrange(numAttributes):
   location = struct.unpack('B', fh.read(1))[0]
   dimensions = struct.unpack('B', fh.read(1))[0]
   assert dimensions in (1, 2, 3, 4)
   dataType = struct.unpack('I', fh.read(4))[0]
   assert dataType in attributeElementTypes, 'Invalid GLenum value for attribute element type.'
   layouts.append((location, dimensions, dataType, offset))
   offset += dimensions * sizeOfType[dataType]
# apply
for layout in layouts:
   glVertexAttribPointer(layout[0], layout[1], layout[2], GL_FALSE, offset, ctypes.c_void_p(layout[3]))  # total offset is now stride
   glEnableVertexAttribArray(layout[0])

Read and upload the raw buffer data. This step is easy because we can directly copy the bytes as the storage matches exactly with how openGL expects it due to the layout code above.

raw = fh.read(vertexSize * vertexCount)
glBufferData(GL_ARRAY_BUFFER, vertexSize * vertexCount, raw, GL_STATIC_DRAW)
raw = fh.read(indexSize * indexCount)
glBufferData(GL_ELEMENT_ARRAY_BUFFER, indexSize * indexCount, raw, GL_STATIC_DRAW)

3.2 The final code

This is the application code including all the rendering from chapter 1, only the rectangle has been replaced by the loaded mesh.

# the importer
import struct
from OpenGL.GL import *

attributeElementTypes = (GL_BYTE,
                        GL_UNSIGNED_BYTE,
                        GL_SHORT,
                        GL_UNSIGNED_SHORT,
                        GL_INT,
                        GL_UNSIGNED_INT,
                        GL_HALF_FLOAT,
                        GL_FLOAT,
                        GL_DOUBLE,
                        GL_FIXED,
                        GL_INT_2_10_10_10_REV,
                        GL_UNSIGNED_INT_2_10_10_10_REV,
                        GL_UNSIGNED_INT_10F_11F_11F_REV)
sizeOfType = {GL_BYTE: 1,
             GL_UNSIGNED_BYTE: 1,
             GL_SHORT: 2,
             GL_UNSIGNED_SHORT: 2,
             GL_INT: 4,
             GL_UNSIGNED_INT: 4,
             GL_HALF_FLOAT: 2,
             GL_FLOAT: 4,
             GL_DOUBLE: 8,
             GL_FIXED: 4,
             GL_INT_2_10_10_10_REV: 4,
             GL_UNSIGNED_INT_2_10_10_10_REV: 4,
             GL_UNSIGNED_INT_10F_11F_11F_REV: 4}
drawModes = (GL_POINTS,
            GL_LINE_STRIP,
            GL_LINE_LOOP,
            GL_LINES,
            GL_LINE_STRIP_ADJACENCY,
            GL_LINES_ADJACENCY,
            GL_TRIANGLE_STRIP,
            GL_TRIANGLE_FAN,
            GL_TRIANGLES,
            GL_TRIANGLE_STRIP_ADJACENCY,
            GL_TRIANGLES_ADJACENCY,
            GL_PATCHES)
indexTypeFromSize = {1: GL_UNSIGNED_BYTE, 2: GL_UNSIGNED_SHORT, 4: GL_UNSIGNED_INT}


def _loadMesh_v0(fh, vao, bufs):
    vertexCount = struct.unpack('I', fh.read(4))[0]
    vertexSize = struct.unpack('B', fh.read(1))[0]
    indexCount = struct.unpack('I', fh.read(4))[0]
    indexSize = struct.unpack('B', fh.read(1))[0]
    assert indexSize in indexTypeFromSize, 'Unknown element data type, element size must be one of %s' % indexTypeFromSize.keys()
    indexType = indexTypeFromSize[indexSize]
    drawMode = struct.unpack('I', fh.read(4))[0]
    assert drawMode in (GL_LINES, GL_TRIANGLES), 'Unknown draw mode.'  # TODO: list all render types
  
    # gather layout
    numAttributes = struct.unpack('B', fh.read(1))[0]
    offset = 0
    layouts = []
    for i in xrange(numAttributes):
        location = struct.unpack('B', fh.read(1))[0]
        dimensions = struct.unpack('B', fh.read(1))[0]
        assert dimensions in (1, 2, 3, 4)
        dataType = struct.unpack('I', fh.read(4))[0]
        assert dataType in attributeElementTypes, 'Invalid GLenum value for attribute element type.'
        layouts.append((location, dimensions, dataType, offset))
        offset += dimensions * sizeOfType[dataType]
  
    # apply layout
    for layout in layouts:
        glVertexAttribPointer(layout[0], layout[1], layout[2], GL_FALSE, offset, ctypes.c_void_p(layout[3]))  # total offset is now stride
        glEnableVertexAttribArray(layout[0])
  
    raw = fh.read(vertexSize * vertexCount)
    glBufferData(GL_ARRAY_BUFFER, vertexSize * vertexCount, raw, GL_STATIC_DRAW)
    raw = fh.read(indexSize * indexCount)
    glBufferData(GL_ELEMENT_ARRAY_BUFFER, indexSize * indexCount, raw, GL_STATIC_DRAW)
  
    assert len(fh.read()) == 0, 'Expected end of file, but file is longer than it indicates'
    return Mesh(vao, bufs, drawMode, indexCount, indexType)


class Mesh(object):
    def __init__(self, vao, bufs, drawMode, indexCount, indexType):
        self.__vao = vao
        self.__bufs = bufs
        self.__drawMode = drawMode
        self.__indexCount = indexCount
        self.__indexType = indexType
  
    def __del__(self):
        glDeleteBuffers(len(self.__bufs), self.__bufs)
        glDeleteVertexArrays(1, [self.__vao])
  
    def draw(self):
        glBindVertexArray(self.__vao)
        glDrawElements(self.__drawMode, self.__indexCount, self.__indexType, None)


def model(filePath):
    vao = glGenVertexArrays(1)
    glBindVertexArray(vao)
    bufs = glGenBuffers(2)
    glBindBuffer(GL_ARRAY_BUFFER, bufs[0])
    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, bufs[1])
    with open(filePath, 'rb') as fh:
        fileVersion = struct.unpack('B', fh.read(1))[0]
        if fileVersion == 0:
            return _loadMesh_v0(fh, vao, bufs)
        raise RuntimeError('Unknown mesh file version %s in %s' % (fileVersion, filePath))


# import the necessary modules
import time
from PyQt4.QtCore import *  # QTimer
from PyQt4.QtGui import *  # QApplication
from PyQt4.QtOpenGL import *  # QGLWidget
from OpenGL.GL import *  # OpenGL functionality


# this is the basic window
class OpenGLView(QGLWidget):
    def initializeGL(self):
        # set the RGBA values of the background
        glClearColor(0.1, 0.2, 0.3, 1.0)
  
        # set a timer to redraw every 1/60th of a second
        self.__timer = QTimer()
        self.__timer.timeout.connect(self.repaint)
        self.__timer.start(1000 / 60)
  
        # import a model
        self.__mesh = model(r'C:\Users\John\Python\maya\cube.bm')
  
    def resizeGL(self, width, height):
        glViewport(0, 0, width, height)
  
    def paintGL(self):
        glLoadIdentity()
        glScalef(self.height() / float(self.width()), 1.0, 1.0)
        glRotate((time.time() % 36.0) * 10, 0, 0, 1)
  
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
        self.__mesh.draw()


# this initializes Qt
app = QApplication([])
# this creates the openGL window, but it isn't initialized yet
window = OpenGLView()
# this only schedules the window to be shown on the next Qt update
window.show()
# this starts the Qt main update loop, it avoids python from continuing beyond this line
# and any Qt stuff we did above is now going to actually get executed, along with any future
# events like mouse clicks and window resizes
app.exec_()

Part 2: Creating an OpenGL friendly mesh exporter for Maya

Part 2: Creating an exporter

This is part 2 of a series and it is about getting started with visualizing triangle meshes with Python 2.7 using the libraries PyOpenGL and PyQt4.

Part 1
Part 2
Part 3

I will assume you know python, you will not need a lot of Qt or OpenGL experience, though I will also not go into the deeper details of how OpenGL works. For that I refer you to official documentation and the excellent (C++) tutorials at https://open.gl/. Although they are C++, there is a lot of explanation about OpenGL and why to do certain calls in a certain order.

On a final note: I will make generalizations and simplifications when explaining things. If you think something works different then I say it probably does, this is to try and convey ideas to beginners, not to explain low level openGL implementations.

2.1 File layout

Now that we can draw a model, it is time to define the data that we need to give OpenGL and decide upon a file format that can contain all this data.

Starting off with the elements, we have a way to draw them (GL_TRIANGLES in our case) and we have a data type (GL_UNSIGNED_INT in our case). Given this data type and the number of elements we can actually determine the buffer size regardless of the data type, allowing our file to support not only a number of element values, but allowing those values to be of all the supported types.

Similarly we can look at the vertex layout. We probably want a vertex count and a size of per-vertex-data. This size is a little more complicated because the attribute layout can be very flexible. I suppose it’s easier if we also write the vertex element size instead of trying to figure out what it should be based on the layout.

Then we can look at the attribute layout. We can assume all our data is tightly packed, so we can infer the offset (last argument of glVertexAttribPointer). That leaves us with a layout location, a number of values per vertex, and a data type. Of course first we need to write how many attributes we have.

After that all we need to do is fill in the buffer data. So for vertexCount * vertexElementSize bytes we specify binary data for the vertex buffer and for elementCount * elementDataSize we specify binary data for the elements buffer.

Our file format now looks like this:

Version nr byte So we can change the format later and not break things.
Vertex count unsigned int Because the elements_array can use at most an unsigned int we can never point to vertices beyond the maximum of this data, so no need to store more bytes.
Vertex element size byte Size in bytes of a vertex, based on all attribute sizes combined.
Element count unsigned int
Element data size byte To infer whether indices are unsigned char, unsigned short or unsigned int.
Render type GLenum OpenGL defines variables as GL_TRIANGLES as a GLenum type which is in turn just an unsigned int.
Number of attributes byte
[For each attribute]
Attribute location byte
Attribute dimensions byte Is it a single value, vec2, 3 or 4?
Attribute type GLenum Are the values float, or int, more types listed in the OpenGL documenation for glVertexAttribuPointer.
[End for]
Vertex buffer vertexCount * vertexElementSize bytes
Elements buffer elementCount * elementDataSize bytes

That brings us to the next step, gather this information in Maya.

2.2 Maya mesh exporter

To export I’ll use the maya API (OpenMaya). It provides a way to quickly iterate over a mesh’ data without allocating too much memory using the MItMeshPolygons. This will iterate over all the faces and allow us to extract the individual triangles and face vertices.

There are a few steps to do. First let’s make a script to generate a test scene:

from maya import cmds
from maya.OpenMaya import *
cmds.file(new=True, force=True)
cmds.polyCube()
meshShapeName = 'pCubeShape1'
outputFilePath = 'C:/Test.bgm'

Now with these variables in mind we have to convert the shape name to actual maya API objects that we can read data from.

# get an MDagPath from the given mesh path
p = MDagPath()
l = MSelectionList()
MGlobal.getSelectionListByName(mayaShapeName, l)
l.getDagPath(0, p)

# get the iterator
poly = MItMeshPolygon(p)

This sets us up to actually start saving data. Because openGL requires us to provide all the data of a vertex at 1 vertex index we have to remap some of Maya’s data. In Maya a vertex (actually face-vertex in Maya terms) is a list of indices that points to e.g. what vertex to use, what normal to use, etc. All with separate indices. In OpenGL all these indices must match. The way I’ll go about this is to simply take the triangulation and generate 3 unique vertices for each triangle. This means that to find the vertex count can be determined by counting the triangles in the mesh. Maya meshes don’t expose functionality to query this , so instead I’ll iterate over all the faces and count the triangles in them.

# open the file as binary
with open(outputFilePath, 'wb') as fh:
    # fixing the vertex data size to just X, Y, Z floats for the vertex position
    vertexElementSize = 3 * ctypes.sizeof(ctypes.c_float)
  
    # using unsigned integers as elements
    indexElementSize = ctypes.sizeof(ctypes.c_uint)
  
    # gather the number of vertices
    vertexCount = 0
    while not poly.isDone():
        vertices = MPointArray()
        vertexList = MIntArray()
        poly.getTriangles(vertices, vertexList, space)
        vertexCount += vertexList.length()
        poly.next()
    poly.reset()
    # start writing
    fh.write(struct.pack('B', FILE_VERSION))
    fh.write(struct.pack('I', vertexCount))
    fh.write(struct.pack('B', vertexElementSize))
    # currently I'm duplicating all vertices per triangle, so total indices matches    total vertices
    fh.write(struct.pack('I', vertexCount))
    fh.write(struct.pack('B', indexElementSize))
    fh.write(struct.pack('I', GL_TRIANGLES))  # render type

As you can see we had to make some assumptions about vertex data size and we had to gather some intel on our final vertex count, but this is a good setup. Next step is to write the attribute layout. I’ve made the assumption here to write only X Y Z position floats at location 0. We can expand the exporter later with more features, as our file format supports variable attribute layouts. We can write our position attribute next:

# attribute layout
# 1 attribute
fh.write(struct.pack('B', 1))
# at location 0
fh.write(struct.pack('B', 0))
# of 3 floats
fh.write(struct.pack('B', 3))
fh.write(struct.pack('I', GL_FLOAT))

Note that I am using a constant GL_FLOAT here, if you do not wish to install PyOpenGL for your maya, you can quite simply include this at the top of the file instead:

import ctypes
GL_TRIANGLES = 0x0004
GL_UNSIGNED_INT = 0x1405
GL_FLOAT = 0x1406

After that comes streaming the vertex buffer. For this I use the same iterator I used to count the vertex count. The code is pretty much the same only now I write the vertices instead of counting the vertex list.

# iter all faces
while not poly.isDone():
    # get triangulation of this face
    vertices = MPointArray()
    vertexList = MIntArray()
    poly.getTriangles(vertices, vertexList, space)
  
    # write the positions
    for i in xrange(vertexList.length()):
        fh.write(struct.pack('3f', vertices[i][0], vertices[i][1], vertices[i][2]))
  
    poly.next()

Last is the element buffer.

# write the elements buffer
for i in xrange(vertexCount):
   fh.write(struct.pack('I', i))

2.3 All the data

The next step naturally is to export more than just the position. Here is a more elaborate way to extract all the attributes. First we need to get some global data from the mesh. This goes right after where we create the MItMeshPolygons.

fn = MFnMesh(p)
tangents = MFloatVectorArray()
fn.getTangents(tangents, space)
colorSetNames = []
fn.getColorSetNames(colorSetNames)
uvSetNames = []
fn.getUVSetNames(uvSetNames)

Next we have to change our vertexElementSize code to the following:

# compute the vertex data size, write 4 floats for the position for more convenient transformation in shaders
# position, tangent, normal, color sets, uv sets
vertexElementSize = (4 + 3 + 3 + 4 * len(colorSetNames) + 2 * len(uvSetNames)) * ctypes.sizeof(ctypes.c_float)

The attribute layout is significantly changed. I’m also changing the point data from a vec3 to a vec4. I’m filling in the w component as 1.0, this to indicate a point instead of a vector. It will make transforming vertices in shaders a step simpler.

# attribute layout

# Since NVidia is the only driver to implement a default attribute layout I am following this as much as possible
# on other drivers using a custom shader is mandatory and modern buffers will never work with the fixed function pipeline.
# http://developer.download.nvidia.com/opengl/glsl/glsl_release_notes.pdf
# https://stackoverflow.com/questions/20573235/what-are-the-attribute-locations-for-fixed-function-pipeline-in-opengl-4-0-cor

# num attributes
fh.write(struct.pack('B', 3 + len(colorSetNames) + len(uvSetNames)))
# vec4 position at location 0
fh.write(struct.pack('B', 0))
fh.write(struct.pack('B', 4))
fh.write(struct.pack('I', GL_FLOAT))
# vec3 tangent at location 1
fh.write(struct.pack('B', 1))
fh.write(struct.pack('B', 3))
fh.write(struct.pack('I', GL_FLOAT))
# vec3 normal at location 2
fh.write(struct.pack('B', 2))
fh.write(struct.pack('B', 3))
fh.write(struct.pack('I', GL_FLOAT))
# vec4 color at locations (3,7) and 16+
used = {}
for i in xrange(len(colorSetNames)):
    idx = 3 + i
    if idx > 7:
        idx = 11 + i
        used.add(idx)
    fh.write(struct.pack('B', idx))
    fh.write(struct.pack('B', 4))
    fh.write(struct.pack('I', GL_FLOAT))
# vec2 uvs at locations 8-15 and 16+, but avoiding overlap with colors
idx = 8
for i in xrange(len(uvSetNames)):
    while idx in used:
        idx += 1
    fh.write(struct.pack('B', idx))
    fh.write(struct.pack('B', 2))
    fh.write(struct.pack('I', GL_FLOAT))
    idx += 1

Most of the MItMeshPolygon iterator functions, like getNormals(), gives us a list of the normals for all vertices in this face. The problem is that this data is not triangulated.

To extract the triangulation we used getTriangles(), which gives us a list of vertices used in the face. These vertex numbers are object-wide, so they keep getting bigger the further we get.

That means they’re useless if we want to use them to look up the normal returned by getNormals(), because that array is always very short, containing just the normals for this face.

So we have to do some mapping from the triangulated vertex indices into indices that match the data we’ve got. Either that or get all the normals from the mesh in 1 big array but that is not memory efficient. So at the top of the while loop (just inside) I’ve added the following dictionary:

# map object indices to local indices - because for some reason we can not query the triangulation as local indices
# but all getters do want us to provide local indices
objectToFaceVertexId = {}
count = poly.polygonVertexCount()
for i in xrange(count):
    objectToFaceVertexId[poly.vertexIndex(i)] = i

That allows us to extract all the data we want for these triangles like so:

# get per-vertex data
normals = MVectorArray()
poly.getNormals(normals, space)
colorSet = []
for i, colorSetName in enumerate(colorSetNames):
    colorSet.append(MColorArray())
    poly.getColors(colorSet[i], colorSetName)
uvSetU = []
uvSetV = []
for i, uvSetName in enumerate(uvSetNames):
    uvSetU.append(MFloatArray())
    uvSetV.append(MFloatArray())
    poly.getUVs(uvSetU[i], uvSetV[i], uvSetName)

Handling fairly small sets of data at a time. Last we have to write the data, replacing the loop writing 3 floats per vertex we had before with this longer loop:

# write the data
for i in xrange(vertexList.length()):
    localVertexId = objectToFaceVertexId[vertexList[i]]
    tangentId = poly.tangentIndex(localVertexId)
  
    fh.write(struct.pack('4f', vertices[i][0], vertices[i][1], vertices[i][2], 1.0))
    fh.write(struct.pack('3f', tangents[tangentId][0], tangents[tangentId][1], tangents[tangentId][2]))
    fh.write(struct.pack('3f', normals[localVertexId][0], normals[localVertexId][1], normals[localVertexId][2]))
    for j in xrange(len(colorSetNames)):
        fh.write(struct.pack('4f', colorSet[j][localVertexId][0], colorSet[j][localVertexId][1], colorSet[j][localVertexId][2], colorSet[j][localVertexId][3]))
    for j in xrange(len(uvSetNames)):
        fh.write(struct.pack('2f', uvSetU[j][localVertexId], uvSetV[j][localVertexId]))

And that completes the exporter with full functionality, extracting all possible data from a maya mesh we want. Unless you want blind data and skin clusters, but that’s a whole different story!

2.4 Code

Here is the final code as a function, with an additional function to export multiple selected meshes to multiple files, using Qt for UI. Note that if you wish to use PySide or PyQt5 instead the QFileDialog.getExistingDirectory and QSettings.value return types are different and require some work.

import os
import struct
from maya import cmds
from maya.OpenMaya import *
import ctypes

GL_TRIANGLES = 0x0004
GL_UNSIGNED_INT = 0x1405
GL_FLOAT = 0x1406
FILE_EXT = '.bm'  # binary mesh
FILE_VERSION = 0
EXPORT_SPACE = MSpace.kWorld  # export meshes in world space for now


def exportMesh(mayaShapeName, outputFilePath, space):
    # get an MDagPath from the given mesh path
    p = MDagPath()
    l = MSelectionList()
    MGlobal.getSelectionListByName(mayaShapeName, l)
    l.getDagPath(0, p)
  
    # get the mesh and iterator
    fn = MFnMesh(p)
    poly = MItMeshPolygon(p)
  
    tangents = MFloatVectorArray()
    fn.getTangents(tangents, space)
    colorSetNames = []
    fn.getColorSetNames(colorSetNames)
    uvSetNames = []
    fn.getUVSetNames(uvSetNames)
  
    # open the file as binary
    with open(outputFilePath, 'wb') as fh:
        # compute the vertex data size, write 4 floats for the position for more convenient transformation in shaders
        # position, tangent, normal, color sets, uv sets
        vertexElementSize = (4 + 3 + 3 + 4 * len(colorSetNames) + 2 * len(uvSetNames)) * ctypes.sizeof(ctypes.c_float)
  
        # using unsigned integers as elements
        indexElementSize = ctypes.sizeof(ctypes.c_uint)
  
        # gather the number of vertices
        vertexCount = 0
        while not poly.isDone():
            vertices = MPointArray()
            vertexList = MIntArray()
            poly.getTriangles(vertices, vertexList, space)
            vertexCount += vertexList.length()
            poly.next()
        poly.reset()
  
        # start writing
        fh.write(struct.pack('B', FILE_VERSION))
        fh.write(struct.pack('I', vertexCount))
        fh.write(struct.pack('B', vertexElementSize))
        # currently I'm duplicating all vertices per triangle, so total indices matches total vertices
        fh.write(struct.pack('I', vertexCount))
        fh.write(struct.pack('B', indexElementSize))
        fh.write(struct.pack('I', GL_TRIANGLES))  # render type
  
        # attribute layout
  
        # Since NVidia is the only driver to implement a default attribute layout I am following this as much as possible
        # on other drivers using a custom shader is mandatory and modern buffers will never work with the fixed function pipeline.
        # http://developer.download.nvidia.com/opengl/glsl/glsl_release_notes.pdf
        # https://stackoverflow.com/questions/20573235/what-are-the-attribute-locations-for-fixed-function-pipeline-in-opengl-4-0-cor
  
        # num attributes
        fh.write(struct.pack('B', 3 + len(colorSetNames) + len(uvSetNames)))
        # vec4 position at location 0
        fh.write(struct.pack('B', 0))
        fh.write(struct.pack('B', 4))
        fh.write(struct.pack('I', GL_FLOAT))
        # vec3 tangent at location 1
        fh.write(struct.pack('B', 1))
        fh.write(struct.pack('B', 3))
        fh.write(struct.pack('I', GL_FLOAT))
        # vec3 normal at location 2
        fh.write(struct.pack('B', 2))
        fh.write(struct.pack('B', 3))
        fh.write(struct.pack('I', GL_FLOAT))
        # vec4 color at locations (3,7) and 16+
        used = {}
        for i in xrange(len(colorSetNames)):
            idx = 3 + i
            if idx > 7:
                idx = 11 + i
                used.add(idx)
            fh.write(struct.pack('B', idx))
            fh.write(struct.pack('B', 4))
            fh.write(struct.pack('I', GL_FLOAT))
        # vec2 uvs at locations 8-15 and 16+, but avoiding overlap with colors
        idx = 8
        for i in xrange(len(uvSetNames)):
            while idx in used:
                idx += 1
            fh.write(struct.pack('B', idx))
            fh.write(struct.pack('B', 2))
            fh.write(struct.pack('I', GL_FLOAT))
            idx += 1
  
        # iter all faces
        while not poly.isDone():
            # map object indices to local indices - because for some reason we can not query the triangulation as local indices
            # but all getters do want us to provide local indices
            objectToFaceVertexId = {}
            count = poly.polygonVertexCount()
            for i in xrange(count):
                objectToFaceVertexId[poly.vertexIndex(i)] = i
  
            # get triangulation of this face
            vertices = MPointArray()
            vertexList = MIntArray()
            poly.getTriangles(vertices, vertexList, space)
  
            # get per-vertex data
            normals = MVectorArray()
            poly.getNormals(normals, space)
            colorSet = []
            for i, colorSetName in enumerate(colorSetNames):
                colorSet.append(MColorArray())
                poly.getColors(colorSet[i], colorSetName)
            uvSetU = []
            uvSetV = []
            for i, uvSetName in enumerate(uvSetNames):
                uvSetU.append(MFloatArray())
                uvSetV.append(MFloatArray())
                poly.getUVs(uvSetU[i], uvSetV[i], uvSetName)
  
            # write the data
            for i in xrange(vertexList.length()):
                localVertexId = objectToFaceVertexId[vertexList[i]]
                tangentId = poly.tangentIndex(localVertexId)
  
                fh.write(struct.pack('4f', vertices[i][0], vertices[i][1], vertices[i][2], 1.0))
                fh.write(struct.pack('3f', tangents[tangentId][0], tangents[tangentId][1], tangents[tangentId][2]))
                fh.write(struct.pack('3f', normals[localVertexId][0], normals[localVertexId][1], normals[localVertexId][2]))
                for j in xrange(len(colorSetNames)):
                    fh.write(struct.pack('4f', colorSet[j][localVertexId][0], colorSet[j][localVertexId][1], colorSet[j][localVertexId][2], colorSet[j][localVertexId][3]))
                for j in xrange(len(uvSetNames)):
                    fh.write(struct.pack('2f', uvSetU[j][localVertexId], uvSetV[j][localVertexId]))
  
            poly.next()
  
        # write the elements buffer
        for i in xrange(vertexCount):
            fh.write(struct.pack('I', i))


def exportSelected():
    selectedMeshShapes = cmds.select(ls=True, type='mesh', l=True) or []
    selectedMeshShapes += cmds.listRelatives(cmds.select(ls=True, type='transform', l=True) or [], c=True, type='mesh', f=True) or []
    from PyQt4.QtCore import QSettings
    from PyQt4.QtGui import QFileDialog
    settings = QSettings('GLMeshExport')
    mostRecentDir = str(settings.value('mostRecentDir').toPyObject())
    targetDir = QFileDialog.getExistingDirectory(None, 'Save selected meshes in directory', mostRecentDir)
    if targetDir and os.path.exists(targetDir):
        settings.setValue('mostRecentDir', targetDir)
        for i, shortName in enumerate(cmds.ls(selectedMeshShapes)):
            exportMesh(selectedMeshShapes[i],
                       os.path.join(targetDir, shortName.replace('|', '_'), FILE_EXT),
                       EXPORT_SPACE)

Part 1: Drawing with PyOpenGL using moden openGL buffers.

This is part 1 of a series and it is about getting started with visualizing triangle meshes with Python 2.7 using the libraries PyOpenGL and PyQt41.

Part 1
Part 2
Part 3

I will assume you know python, you will not need a lot of Qt or OpenGL experience, though I will also not go into the deeper details of how OpenGL works. For that I refer you to official documentation and the excellent (C++) tutorials at https://open.gl/. Although they are C++, there is a lot of explanation about OpenGL and why to do certain calls in a certain order.

On a final note: I will make generalizations and simplifications when explaining things. If you think something works different then I say it probably does, this is to try and convey ideas to beginners, not to explain low level openGL implementations.

Part 1: Drawing a mesh using buffers.

1.1. Setting up

Download & run Python with default settings:
https://www.python.org/ftp/python/2.7.12/python-2.7.12.amd64.msi

Download & run PyQt4 with default settings:
https://sourceforge.net/projects/pyqt/files/PyQt4/PyQt-4.11.4/PyQt4-4.11.4-gpl-Py2.7-Qt4.8.7-x64.exe/download

Paste the following in a windows command window (windows key + R -> type “cmd.exe” -> hit enter):
C:/Python27/Scripts/pip install setuptools
C:/Python27/Scripts/pip install PyOpenGL

1.2. Creating an OpenGL enabled window in Qt.

The first thing to know about OpenGL is that any operation requires OpenGL to be initialized. OpenGL is not something you just “import”, it has to be attached to a (possibly hidden) window. This means that any file loading or global initialization has to be postponed until OpenGL is available.

The second thing to know about OpenGL is that it is a big state machine. Any setting you change is left until you manually set it back. This means in Python we may want to create some contexts (using contextlib) to manage the safe setting and unsetting of certain states. I will however not go this far.

Similar to this Qt also requires prior initialization. So here’s some relevant code:

# import the necessary modules
from PyQt4.QtCore import * # QTimer
from PyQt4.QtGui import * # QApplication
from PyQt4.QtOpenGL import * # QGLWidget
from OpenGL.GL import * # OpenGL functionality
from OpenGL.GL import shaders # Utilities to compile shaders, we may not actually use this

# this is the basic window
class OpenGLView(QGLWidget):
def initializeGL(self):
# here openGL is initialized and we can do our real program initialization
pass

def resizeGL(self, width, height):
# openGL remembers how many pixels it should draw,
# so every resize we have to tell it what the new window size is it is supposed
# to be drawing for
pass

def paintGL(self):
# here we can start drawing, on show and on resize the window will redraw
# automatically
pass

# this initializes Qt
app = QApplication([])
# this creates the openGL window, but it isn't initialized yet
window = OpenGLView()
# this only schedules the window to be shown on the next Qt update
window.show()
# this starts the Qt main update loop, it avoids python from continuing beyond this
# line and any Qt stuff we did above is now going to actually get executed, along with
# any future events like mouse clicks and window resizes
app.exec_()

Running this should get you a black window that is OpenGL enabled. So let’s fill in the view class to draw something in real-time. This will show you how to make your window update at 60-fps-ish, how to set a background color and how to handle resizes.

class OpenGLView(QGLWidget):
    def initializeGL(self):
        # set the RGBA values of the background
        glClearColor(0.1, 0.2, 0.3, 1.0)
        # set a timer to redraw every 1/60th of a second
        self.__timer = QTimer()
        self.__timer.timeout.connect(self.repaint) # make it repaint when triggered
        self.__timer.start(1000 / 60) # make it trigger every 1000/60 milliseconds
   
    def resizeGL(self, width, height):
        # this tells openGL how many pixels it should be drawing into
        glViewport(0, 0, width, height)
   
    def paintGL(self):
        # empty the screen, setting only the background color
        # the depth_buffer_bit also clears the Z-buffer, which is used to make sure
        # objects that are behind other objects actually are not shown drawing 
        # a faraway object later than a nearby object naively implies that it will 
        # just fill in the pixels with itself, but if there is already an object there 
        # the depth buffer will handle checking if it is closer or not automatically
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT)
        # the openGL window has coordinates from (-1,-1) to (1,1), so this fills in 
        # the top right corner with a rectangle. The default color is white.
        glRecti(0, 0, 1, 1)

Note that the QTimer forces the screen to redraw, but because we are not animating any data this will not be visible right now.

1.3. Creating a Vertex Array Object (VAO)

In OpenGL there is a lot we can put into a mesh, not only positions of vertices, but also triangulation patterns, vertex colors, texture coordinates, normals etcetera. Because OpenGL is a state machine (as described at the start of 2.) this means that when drawing 2 different models a lot of settings need to be swapped before we can draw it. This is why the VAO was created, as it is a way to group settings together and be able to draw a mesh (once set up properly) in only 2 calls. It is not less code, but it allows us to move more code to the initialization stage, winning performance and reducing risk of errors resulting in easier debugging.

Our mesh however will not be very complicated. We require 2 sets of data, the vertex positions and the triangulation (3 integers per triangle pointing to what vertex to use for this triangle).
Untitled
As you can see this would result in the following data:
Positions = [0, 0, 1, 0, 0, 1, 1, 1]
Elements = [0, 1, 2, 1, 3, 2]
4 2D vertices and 2 triangles made of 3 indices each.

So let’s give this data to a VAO at the bottom of initializeGL.

# generate a model
# set up the data
positions = (0, 0, 1, 0, 0, 1, 1, 1)
elements = (0, 1, 2, 1, 3, 2)
# apply the data
# generate a vertex array object so we can easily draw the resulting mesh later
self.__vao = glGenVertexArrays(1)
# enable the vertex array before doing anything else, so anything we do is captured in the VAO context
glBindVertexArray(self.__vao)
# generate 2 buffers, 1 for positions, 1 for elements. this is memory on the GPU that our model will be saved in.
bufs = glGenBuffers(2)
# set the first buffer for the main vertex data, that GL_ARRAY_BUFFER indicates that use case
glBindBuffer(GL_ARRAY_BUFFER, bufs[0])
# upload the position data to the GPU
# some info about the arguments:
# GL_ARRAY_BUFFER: this is the buffer we are uploading into, that is why we first had to bind the created buffer, else we'd be uploading to nothing
# sizeof(ctypes.c_float) * len(positions): openGL wants our data as raw C pointer, and for that it needs to know the size in bytes.
# the ctypes module helps us figure out the size in bytes of a single number, then we just multiply that by the array length
# (ctypes.c_float * len(positions))(*positions): this is a way to convert a python list or tuple to a ctypes array of the right data type
# internally this makes that data the right binary format
# GL_STATIC_DRAW: in OpenGL you can specify what you will be doing with this buffer, static means draw it a lot but never access or alter the data once uploaded.
# I suggest changing this only when hitting performance issues at a time you are doing way more complicated things. In general usage static is the fastest.
glBufferData(GL_ARRAY_BUFFER, sizeof(ctypes.c_float) * len(positions), (ctypes.c_float * len(positions))(*positions), GL_STATIC_DRAW)
# set the second buffer for the triangulation data, GL_ELEMENT_ARRAY_BUFFER indicates the use here
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, bufs[1])
# upload the triangulation data
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(ctypes.c_uint) * len(elements), (ctypes.c_uint * len(elements))(*elements), GL_STATIC_DRAW)
# because the data is now on the GPU, our python positions & elements can be safely garbage collected hereafter
# turn on the position attribute so OpenGL starts using our array buffer to read vertex positions from
glEnableVertexAttribArray(0)
# set the dimensions of the position attribute, so it consumes 2 floats at a time (default is 4)
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, 0, None)

So that was quite some code, and it is quite simple because we only have positions to deal with right now. But first let’s try to draw it!
Replace the glRecti call with:

# enable the vertex array we initialized, it will bind the right buffers in the background again
glBindVertexArray(self.__vao)
# draw triangles based on the active GL_ELEMENT_ARRAY_BUFFER
# that 6 is the element count, we can save the len(elements) in initializeGL in the future
# that None is because openGL allows us to supply an offset for what element to start drawing at
# (we could only draw the second triangle by offsetting by 3 indices for example)
# problem is that the data type for this must be None or ctypes.c_void_p.
# In many C++ example you will see just "0" being passed in
# but in PyOpenGL this doesn't work and will result in nothing being drawn.
glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, None)

Now we should have an identical picture. Some more info about glVertexAttribPointer:

In OpenGL we can upload as many buffers as we want, but for now I’ll stick with the 2 we have. This means that if we want to (for example) add colors to our mesh, we have to set up multiple attrib pointers, that both point to different parts of the buffer. I like to keep all my vertex data concatenated, so that we could get (x,y,r,g,b,x,y,r,g,b…) etcetera in our buffer.

Now for OpenGL to render it not only wants to know what buffer to look at (the array_buffer), but it also wants to know how to interpret that data, and what data is provided. OpenGL understand this through attribute locations. Here we activate attribute location 0 (with glEnableVertexAttribArray) and then set our buffer to be 2 floats per vertex at attribute location 0.

The default openGL attribute locations are as follows:
0: position
1: tangent
2: normal
3: color
4: uv

To support multiple attributes in a single buffer we have to use the last 2 arguments of glVertexAttribPointer. The first of those is the size of all data per vertex, so imagine a 2D position and an RGB color that would be 5 * sizeof(float). The second of those is where this attribute location starts. Here’s an example to set up position and color:

vertex_data = (0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 1, 1, 1) ##
vertex_element_size = 5 ##
elements = (0, 1, 2, 1, 3, 2)
self.__vao = glGenVertexArrays(1)
glBindVertexArray(self.__vao)
bufs = glGenBuffers(2)
glBindBuffer(GL_ARRAY_BUFFER, bufs[0])
glBufferData(GL_ARRAY_BUFFER, sizeof(ctypes.c_float) * len(vertex_data), (ctypes.c_float * len(vertex_data))(*vertex_data), GL_STATIC_DRAW) ##
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, bufs[1])
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(ctypes.c_uint) * len(elements), (ctypes.c_uint * len(elements))(*elements), GL_STATIC_DRAW)
glEnableVertexAttribArray(3) ##
glVertexAttribPointer(3, 3, GL_FLOAT, GL_FALSE, sizeof(ctypes.c_float) * vertex_element_size, ctypes.c_void_p(2 * sizeof(ctypes.c_float))) ##
glEnableVertexAttribArray(0)
glVertexAttribPointer(0, 2, GL_FLOAT, GL_FALSE, sizeof(ctypes.c_float) * vertex_element_size, None) ##

This is an update to initializeGL, new / changed code ends in ## (because I don’t know how to override the syntax highlighting), and your rectangle will immediately start showing colors2!

One last thing. add this to the top of paintGL:

import time
glLoadIdentity()
glScalef(self.height() / float(self.width()), 1.0, 1.0)
glRotate((time.time() % 36.0) * 10, 0, 0, 1)

The first line (after the import) restores the transform state, the second line corrects aspect ratio (so a square is really square now), the last line rotates over time. We are using a startup time because Python’s time is a too large value, by subtracting the application start time from it we get a value OpenGL can actually work with.

That’s it for part 1!

1PyQt5, PySide, PySide2 are (apart from some class renames) also compatible with this.

2The color attribute binding only works on NVidia, there is no official default attribute location and most other drivers will ignore glVertexAttribPointer (or do something random) if you do not use a custom shader. So if you’re not seeing colors, don’t worry and try diving into shaders later!

Cubic root solving

Update: Autodesk released their interpolation code for Maya animation curves, weighted tangents on animation curves do exactly this.
Refer to (and use!) https://github.com/Autodesk/animx instead of the code below. I know it’s not python, but it does work where I found bugs in my version below.

I really need to get back to this, but I mocked up this bit of code and finally got it to work.
It is about animation curve evaluation, due to the parametric nature of curves it is very complicated to get the value of a curve on at an arbitrary time, and it involves finding the parameter T for a value X.

This demo shows how the parameter (left to right) relates to the time of the animcurve (top to bottom), so the red curve shows linear parameter increasing results in non linear time samples.
It then computes backwards from the found value to the parameter again, showing we can successfully reconstruct the linear input given the time output.

I intend to clean up this code and make a full 2D example, rendering a 2D cubic spline segment as normal and then overlaying an evaluation based on the X coordinate, but wanted to dump the result nonetheless. Knowing how bad I am at getting back to things…
Using QT purely for demonstration, code itself is pure python…

from PyQt4.QtCore import *
from PyQt4.QtGui import *


def cubicArgs(x0, x1, x2, x3):
    a = x3 + (x1 - x2) * 3.0 - x0
    b = 3.0 * (x2 - 2.0 * x1 + x0)
    c = 3.0 * (x1 - x0)
    d = x0
    return a, b, c, d


def cubicEvaluate(x0, x1, x2, x3, p):
    # convert points to a cubic function & evaluate at p
    a, b, c, d = cubicArgs(x0, x1, x2, x3)
    return a * p * p * p + b * p * p + c * p + d


class CurveDebug(QWidget):
    def __init__(self):
        super(CurveDebug, self).__init__()
        self.t = QTimer()
        self.t.timeout.connect(self.update)
        self.t.start(16)
        self.ot = time.time()

    def paintEvent(self, event):
        painter = QPainter(self)

        life = time.time() - self.ot

        padding = 100
        w = self.width() - padding * 2
        h = self.height() - padding * 2
        painter.translate(padding, padding)

        # zig zag 2D bezier
        x0, y0 = 0, 0
        x1, y1 = (sin(life) * 0.5 + 0.5) * w, 0
        x2, y2 = 0, h
        x3, y3 = w, h

        # draw hull
        # painter.setPen(QColor(100, 220, 220))
        # painter.drawLine(x0, y0, x1, y1)
        # painter.drawLine(x1, y1, x2, y2)
        # painter.drawLine(x2, y2, x3, y3)

        for i in xrange(w):
            p = i / float(w - 1)

            # draw curve
            # painter.setPen(QColor(220, 100, 220))
            # x = cubicEvaluate(x0, x1, x2, x3, p)
            # y = cubicEvaluate(y0, y1, y2, y3, p)
            # painter.drawPoint(x, y)

            # draw X as function of P
            painter.setPen(QColor(220, 100, 100))
            x = cubicEvaluate(x0, x1, x2, x3, p)
            painter.drawPoint(i, x)

            # now let's evaluate the curve at x and see if we can get the original p back
            # make cubic with offset
            a, b, c, d = cubicArgs(x0 - x, x1 - x, x2 - x, x3 - x)

            # find roots
            # http://www.1728.org/cubic2.htm
            f = ((3.0 * c / a) - ((b * b) / (a * a))) / 3.0
            g = (((2.0 * b * b * b) / (a * a * a)) - ((9.0 * b * c) / (a * a)) + ((27.0 * d) / a)) / 27.0
            _h = ((g * g) / 4.0) + ((f * f * f) / 27.0)
            root0, root1, root2 = None, None, None
            if _h <= 0.0:
                # we have 3 real roots
                if f == 0 and g == 0:
                    # all roots are real & equal
                    _i = d / a
                    root0 = -copysign(pow(abs(_i), 0.3333), _i)
                else:
                    _i = sqrt((g * g / 4.0) - _h)
                    j = pow(_i, 0.3333333)
                    k = acos(-(g / (2.0 * _i)))
                    m = cos(k / 3.0)
                    n = sqrt(3.0) * sin(k / 3.0)
                    _p = b / (3.0 * a)
                    root0 = 2.0 * j * m - _p
                    root1 = -j * (m + n) - _p
                    root2 = -j * (m - n) - _p
            else:
                # we have only 1 real root
                R = -(g / 2.0) + sqrt(_h)
                S = copysign(pow(abs(R), 0.3333333), R)
                T = -(g / 2.0) - sqrt(_h)
                U = copysign(pow(abs(T), 0.3333333), T)
                root0 = (S + U) - (b / (3.0 * a))

            painter.setPen(QColor(100, 100, 220))
            painter.drawPoint(i, root0 * h)
            if root1:
                painter.drawPoint(i, root1 * h)
                painter.drawPoint(i, root2 * h)


app = QApplication([])
cvd = CurveDebug()
cvd.show()
cvd.resize(300, 300)
app.exec_()