Nodeset: bugfix and small new feature

NodeSet is now also available in a Pro version. Read all about it in this article or check out my BlenderMarket shop if you are interested.


The nodeset add-on I talked about in a previous article had a small bug: if your texture set was something different than a collection of .png files the extra files that should be loaded automatically were in fact not loaded. This is fixed in the latest version (201707011445)

Code availability

The change is already committed in the GitHub repository. (right click the link to download).

New feature

Because of a bug in Blender a material with a normal map node will show up as all black if you use the experimental adaptive subdivision / micro polygon displacement. If you change the normal maps space from tangent to object this does not happen so I added a user preferences setting to do this automatically.

Converting Blender Filmic LUTs for use in Substance Painter

with the new Filmic Blender color management options getting a lot of attention I wanted to get the exact same looks when creating textures in Substance Painter.
Substance Painter supports LUTs in so called 3d format which are stored in .exr files. So our challenge is twofold: convert the bundled Blender Filmic LUTs to this format and of course to get results in Substance that match Blender as closely as possible.
The results of 3 of the filmic LUTs are compared side by side in the image below:

I made this comparison collage by keeping parameters like exposure (1) and gamma (also 1) the same in both Blender and Substance Painter and then took screenshots. I glued the screenshots together in GIMP. This way I could see all images on the same monitor. This is important because if the applied profiles are the same and both displays are sRGB, two monitors will still differ in their color response and it is fiendishly difficult to calibrate them (unless you have a very expensive monitor with the associated color calibration kit). My own monitors are not even the same brand so it is an easy trap to fall into if you compare images on two different monitors side by side.
Anyway, even this way when you have a close look the images are close in tone but not 100% identical and I am not sure what is causing this. There might be slight differences in camera aperture, focal blur and multiple importance sampling of the environment and of course Cycles' ray model is not the same as Iray's Substance painting mode renderer. Still, I think this is pretty close and useful to compare textures in Substance under different looks before transferring them to Blender.

How the LUTs were generated

I used the Python bindings of the OpenColorIO and OpenImageIO libraries to create a small script (code below). These are in fact the libraries Blender uses to work with color conversions.

The script takes a linear to linear transform that is encoded as an .exr image and creates a new .exr for each 'Look' in Blender's OCIO config file that is defined in the 'Filmic Log' process space.
The docs for the Python bindings for both libraries are not an easy read and the APIs have some small inconsistencies so it took some time to get it working. The code is far from beautiful but i commented the relevant parts. I am open to any critique that can help to improve the transforms.
Note that the code is not plug and play and i have no intention of improving that :-)

The Substance LUTs

They can be downloaded from my GitHub repository. They are bundled in one .zip file and should be unpacked before importing them in Substance Painter.

The code

#export LD_LIBRARY_PATH=/home/michel/ocio/lib
#export OCIO=/home/michel/Downloads/blender-2.78-b94a433ca34-linux-glibc219-x86_64/2.78/datafiles/colormanagement/config.ocio
#python 2.7
#
# run as
# python transformlook.py
#
# expects linear_to_linear.exr in the current directory and will write the transform there too

import OpenImageIO as OIIO  #  already installed using Ubuntu pkg manager
from sys import path
from array import array
path.append('/home/michel/ocio/lib/python2.7/site-packages/')
import PyOpenColorIO as OCIO

config = OCIO.GetCurrentConfig()

for look in config.getLooks():
 if look.getProcessSpace() == 'Filmic Log':
  lookname = look.getName()
  print(lookname)

  transform = OCIO.DisplayTransform()
  transform.setInputColorSpaceName(OCIO.Constants.ROLE_SCENE_LINEAR)
  transform.setView('Filmic')
  transform.setLooksOverrideEnabled(True)
  transform.setLooksOverride(lookname)
  transform.setDisplay('sRGB')
  processor = config.getProcessor(transform)

  # indentiy transform from https://support.allegorithmic.com/documentation/display/SPDOC/Color+Profile
  img = OIIO.ImageInput.open('linear_to_linear.exr')
  spec = img.spec()
  spec.set_format(OIIO.FLOAT) # for some reason this is not extracted from the image
  pixels = img.read_image()
  img.close()

  outfile = lookname + '.exr'
  transformedpixels = processor.applyRGB(pixels)
  imgout = OIIO.ImageOutput.create(outfile)
  ok=imgout.open(outfile, spec, OIIO.Create)
  if not ok:
   print(OIIO.geterror())
   break
  # ImageInput.read_image() returns a list of floats, and applyRGB(0 returns one too, but ImgOutput.write_image
  # expects an array of float and will die when passed a list. Bit inconsistent I think.
  a = array('d')
  a.fromlist(transformedpixels)
  imgout.write_image(a)
  imgout.close()


Nodeset: add a principled shader

NodeSet is now also available in a Pro version. Read all about it in this article or check out my BlenderMarket shop if you are interested.

Even though there are better paid and free PBR nodegroups/shaders available for Blender (for example from Jeffrey Hepburn or Remington Graphics) the new Principled BSDF (a.k.a. Disney shader or PBR shader) will no doubt prove popular with Blenderheads because it is so simple to use and gives decent results.

So I added an option to add this shader along with all the imported texture sets as well as a normal map node, basically giving you a one-click (almost) option to add a PBR material based on a set of textures from your favorite texturing tool. The new functionality is a available from Add -> Texture menu in the Node editor and sits alongside the original Set of images entry:

The resulting node setup (after selecting a set of textures) will look like this:
Note that this will of course only work with the new Blender 2.79 or with a recent daily build. If the Principled BSDF is not available in your version of Blender it will simply be omitted.

Code availability

The latest version of the code (201706251223) is available on GitHub (right click and select save as ... , then in Blender File -> user preferences ... -> Add-ons -> Install from file .... Don't forget to remove the previously installed version first!)

A short video demo:

Previous articles

Previous articles about the Nodeset add-on:
NODESET: IMPORT SUBSTANCE PAINTER TEXTURES INTO BLENDER
NODESET: TINY UPDATE MIGHT SAVE EVEN SOME MORE TIME
NODESET: SUPPORT FOR AMBIENT OCCLUSION MAPS
NODESET: MORE FLEXIBILITY

Substance Painter experiment



Inspired by a real life flower pot in our garden but weathered quite a lot more:


Sale: Blender Market turns 3


There is something to celebrate: Friday June 9 Blender Market turns 3.
To celebrate, many products will carry a 25% discount that day up til Sunday 11 (applied at checkout, and remember they are on Chicago time), including all products is my shop :-)

How to add a progress indicator to the Info header in Blender

There are many ways to signal progress for long running operations in Blender but the one I like best is the slider which is present when you render a scene. That kind of indicator is pretty clear and at the same time not to intrusive.



However there seems to be no way to add such a progress indicator for other purposes.

Now the menu bar at the top is actually the header of an area within the Info editor, so I tried to add a Panel to this header by specifying 'HEADER' for its bl_region_type. That didn't work: no errors but no visible header either.

So after some digging around I came up with a different approach: replacing the draw() method of the Info header. After all, everything is Python and being a truly dynamic language means we can monkey patch anything.

Basically we get the original draw() method, replace it with our own and call the original again. After this call we add a scene property to the layout of the header and use this float property to signal progress. The result looks like this:



The relevant code looks like this:

# update function to tag all info areas for redraw
def update(self, context):
    areas = context.window.screen.areas
    for area in areas:
        if area.type == 'INFO':
            area.tag_redraw()

# a variable where we can store the original draw funtion
info_header_draw = lambda s,c: None

def register():
    # a value between [0,100] will show the slider
    Scene.progress_indicator = FloatProperty(
                                    default=-1,
                                    subtype='PERCENTAGE',
                                    precision=1,
                                    min=-1,
                                    soft_min=0,
                                    soft_max=100,
                                    max=101,
                                    update=update)

    # the label in front of the slider can be configured
    Scene.progress_indicator_text = StringProperty(
                                    default="Progress",
                                    update=update)

    # save the original draw method of the Info header
    global info_header_draw
    info_header_draw = bpy.types.INFO_HT_header.draw

    # create a new draw function
    def newdraw(self, context):
        global info_header_draw
        # first call the original stuff
        info_header_draw(self, context)
        # then add the prop that acts as a progress indicator
        if (context.scene.progress_indicator >= 0 and
            context.scene.progress_indicator <= 100) :
            self.layout.separator()
            text = context.scene.progress_indicator_text
            self.layout.prop(context.scene,
                                "progress_indicator",
                                text=text,
                                slider=True)

    # replace it
    bpy.types.INFO_HT_header.draw = newdraw

The register() function defines two new scene properties: progress_indicator to hold a value in the range [0,100] which will be shown to indicate progress and progress_indicator_text to hold a configurable label. They refer to an update() function that will be called every time that the value of the property is changed. The update() function just tags any area the is an INFO editor (theoretically there could be more than one) for redraw which will cause the draw() method to be called for any of its regions, including the header region.

Line 30 stores a reference to the original draw() method of the the Info header. Next we define a new method newdraw() that will call the original draw() method (line 36) and then add the new scene property progress_indicator but only if it has a value between zero and 100.

The new function is then used to replace the existing draw function.

How to use the progress indicator

Long running operations are probably best implemented as modal operators and using the progress indicator from a modal operator is very simple. An example of such an operator is shown below (which also starts a timer that will send timer events to the modal operator. The operator will stop after 9 timer ticks and update the progress indicator on each tick. After the final tick it will set the value to 101 which will stop the progress indicator from being displayed:

class TestProgressModal(bpy.types.Operator):
    bl_idname = 'scene.testprogressmodal'
    bl_label = 'Test Progress Modal'
    bl_options = {'REGISTER'}

    def modal(self, context, event):
        if event.type == 'TIMER':
            self.ticks += 1
        if self.ticks > 9:
            context.scene.progress_indicator = 101 # done
            context.window_manager.event_timer_remove(self.timer)
            return {'CANCELLED'}

        context.scene.progress_indicator = self.ticks*10

        return {'RUNNING_MODAL'}

    def invoke(self, context, event):
        self.ticks = 0
        context.scene.progress_indicator_text = "Heavy modal job"
        context.scene.progress_indicator = 0
        wm = context.window_manager
        self.timer = wm.event_timer_add(1.0, context.window)
        wm.modal_handler_add(self)
        return {'RUNNING_MODAL'}

It is possible to update the progress indicator from a long running non-modal's execute() method as well but although the update functions associated with the scene properties will be called and hence the area tagged for redraw, an actual redraw is only initiated after the operator finishes. There is a way around with a documented but unsupported hack as shown in the code below (line 13):

class TestProgress(bpy.types.Operator):
    bl_idname = 'scene.testprogress'
    bl_label = 'Test Progress'
    bl_options = {'REGISTER'}

    def execute(self, context):
        context.scene.progress_indicator_text = "Heavy job"
        context.scene.progress_indicator = 0
        for tick in range(10):
            sleep(1) # placeholder for heavy work
            context.scene.progress_indicator = tick*10
            # see https://docs.blender.org/api/current/info_gotcha.html
            bpy.ops.wm.redraw_timer(type='DRAW_WIN_SWAP', iterations=1)

        context.scene.progress_indicator = 101 # done
        return {"FINISHED"}

Code availability

The full code which includes the two sample operators that illustrate how to use the progress indicator is available on GitHub.

How to remove user installed add-ons in bulk

I admit that this is probably not a problem many people have but as an add-on developer I find myself every now an then in the situation that I want remove a whole bunch of user installed add-ons in one go.

What I often do when I develop a collection of add-ons is define a common category for all of them that is not one of the predefined categories. That way I can at least easily find them and see them grouped together in the user preferences.

Removing a single add-on is simple but to remove a bunch of user installed add-ons is less so because you have to locate the Blender user config directory (which is different on various operating systems) and you'll have to open each add-on file (or __init__.py file in a subdirectory if it's a multi-file add-on) to see if it defines the relevant category.

Tedious, but fortunately Blender can help. The code below shows you how. It is not an add-on itself, it is meant to be run from the command line inside Blender or from Blender's text editor (clicking Run Script). Removing stuff always carries the risk of accidental deletion so be careful (and use this snippet at your own risk. And keep back-ups, but careful people always do that, right? ). And yes, this code removes add-ons, not just disables them!


import bpy
from bpy.utils import script_path_user
from addon_utils import modules, module_bl_info

import os.path

userdir = script_path_user()

def remove_user_installed_addons(cat='Experimental development', dry_run=True):
    for mod in modules():
        if module_bl_info(mod)['category'] == cat:
            if os.path.dirname(mod.__file__).startswith(userdir):
                print("removing " + mod.__name__)
                if not dry_run:
                    bpy.ops.wm.addon_remove(module=mod.__name__)

remove_user_installed_addons(cat='Experimental development', dry_run=False)

As you can see, Blender provides us with an addon_utils module that has both a function modules() to produce a list of all add-ons (both enabled and not-enabled)  and a function module_bl_info() that returns the bl_info block of an add-on as a dictionary.
So all we have to do is loop over all installed modules, check if the module is part of the specified category and if so, use the script_path_user() function to determine if the directory that the add-on sits in, is in the user path (so we don't accidentally remove bundled Blender add-ons).
If it checks out, we user the addon_remove() operator to do the actual removal.