Detecting wire color in Maya II

Continueing from here I am going to look at override attributes. An object’s display color is affected by overrides of itself, or of it’s parents. For this I wrote a function that checks whether overrides are enabled, if not I check it for the parent, its parent, and so on. When an object has overrides enabled, I wish to get the color and later the displayType (template/reference). After having written the code I decided to create a simpler function that can get an override attribute by name instead of having multiple functions doing the same thing.

def overrideAttr(inObj, inAttr):
    target = inObj
    while target:
        if not cmds.getAttr('%s.overrideEnabled'%target):
            target = cmds.listRelatives(target, p=True, f=True)[0]
        return cmds.getAttr('%s.%s'%(target,inAttr))

The neat thing about this is that if the overrideDisplayType is set back to normal while the parent is templated, it will return 0 and display the object as normal, which it should, automatically. Then to apply this I only need to insert this code right before final line in drawColor:

    #override color
    overridecolor = overrideAttr(shape, 'overrideColor')
    if overridecolor: #not None and not 0
        return cmds.colorIndex( overridecolor, q=True )

But now we can easily expand this to templating and referencing as well by getting the overrideDisplayType. Then if the object is selected we need to return activeTemplate color, otherwise simply the templateColor or referenceColor will suffice. Now here’s a confusing bit: the displayColor is referenceLayer so that it won’t be confused with file referencing and for the template we use displayRGBColor because in the preferences this does not have a simple index, but a free RGB selection unlimited to the palette of other colors. This goes for a select list of colors, which you can also read by using

for i in cmds.displayRGBColor(list=True): print i

Now for ease of use I added a display layer, added the testing cube in it, and printed the color in every state: selected templated (orange), selected referenced (which is just the lead green), templated (gray), referenced (black), colorized layer and normal (in my case blue) and it all works as you may see by trying!

So here’s the drawColor function in full again:

def drawColor(inObj):
    shapes = maya.utils.executeInMainThreadWithResult( 'cmds.listRelatives(\'%s\', ad=True, type=\'shape\', f=True)'%inObj )
    if not shapes:
        if cmds.nodeType(inObj) != 'transform':
            shape = inObj
        else: #transform node without shapes has no color
            return None
    else:
        shape = shapes[0]

    nodetype = displayColorType( shape )
    selected = isParentSelected( shape )
    displaytype = overrideAttr(shape, 'overrideDisplayType')
    
    if selected:
        #templated
        if displaytype == 1:
            return cmds.colorIndex( cmds.displayColor('activeTemplate', q=True, active=True), q=True )
        #lead
        if selected == cmds.ls(os=True, l=True)[-1]:
            return cmds.colorIndex( cmds.displayColor('lead', q=True, active=True), q=True )
            
        #active
        return cmds.colorIndex( cmds.displayColor(nodetype, q=True, active=True), q=True )
        
    #referenced
    if displaytype == 2:
        return cmds.colorIndex( cmds.displayColor('referenceLayer', q=True), q=True )
        
    #templated
    if displaytype == 1:
        return cmds.displayRGBColor('template', q=True)
    
    #override color
    overridecolor = overrideAttr(shape, 'overrideColor')
    if overridecolor: #not None and not 0
        return cmds.colorIndex( overridecolor, q=True )

    #dormant
    return cmds.colorIndex( cmds.displayColor(nodetype, q=True, dormant=True), q=True )

Now the last thing to do is find out if an object is affected by another, selected, object. I will implement this by inserting, above the lines for referenced objects, directly after the indent for selected objects, the following:

    #affected
    if cmds.displayPref( q=True, displayAffected=True ) and isAffectedRecursively( shape ):
        #if obj is affected by something that is selected
        return cmds.colorIndex( cmds.displayColor('activeAffected', q=True, active=True), q=True )

Now the displayPrefe is a maya command and is necessary to disable the returning of this color if the user disabled it in their preferences. The isAffectedRecursively function is a long awnser to a simple question ‘is it purple?’ which I have described in detail (with code) here.

Leave a Reply

Your email address will not be published. Required fields are marked *