Difference between revisions of "HoudiniVolumes"

From cgwiki
Line 25: Line 25:
 
Houdini's own volume format had been around for a while before VDB arrived, so there's a large amount of nodes in there for working with them (you can see this in sops by typing 'volume' in the tab menu). VDB has its own collection of nodes (either prefixed or suffixed with VDB).
 
Houdini's own volume format had been around for a while before VDB arrived, so there's a large amount of nodes in there for working with them (you can see this in sops by typing 'volume' in the tab menu). VDB has its own collection of nodes (either prefixed or suffixed with VDB).
  
While I haven't played with all the volume and VDB nodes, an area that VDB definitely wins is size on disk. A VDB when cached out can be easily half or less the size of its Houdini volume cousin.  
+
VDBs can be substantially smaller than native Houdini volumes (savings of 50% are common), and often the VDB sops are faster and have more interesting features than the native Houdini volume sops.  
  
 
There's a .vdb file format (which you'd use to export to other apps), but you don't need to write out  a .vdb file to get the space savings, just having a VDB primitive in Houdini is enough. A Houdini volume primitive can be converted to a VDB primitive at any time, just put down a convertvdb node, set its mode to VDB. If you middle click on the node, you'll see the Volume primitive has been replaced with VDB. You can now cache this out to .bgeo like any other piece of Houdini geo, but it'll be much smaller than if you didn't convert.
 
There's a .vdb file format (which you'd use to export to other apps), but you don't need to write out  a .vdb file to get the space savings, just having a VDB primitive in Houdini is enough. A Houdini volume primitive can be converted to a VDB primitive at any time, just put down a convertvdb node, set its mode to VDB. If you middle click on the node, you'll see the Volume primitive has been replaced with VDB. You can now cache this out to .bgeo like any other piece of Houdini geo, but it'll be much smaller than if you didn't convert.
  
Also, most of the core Houdini volume tools work with VDB primitives anyway. Volume VOPS and Volume Wrangles both work with VDB, I've been told a good number of the other Volume tools do too. There's a lot of overlap between the Houdini tools and the VDB tools, having seen a few production scenes, artists currently favour the VDB tools.
+
What's nice is that a lot of the core Houdini volume tools have been updated to work with VDB primitives too. Volume VOPS and Volume Wrangles for example both work with VDB. This means its pretty safe to use VDB at all times, as you can easily convert a VDB temporarily to a native volume if required, and then convert back again.
  
 
Further reading: http://www.sidefx.com/docs/houdini15.0/model/volumes
 
Further reading: http://www.sidefx.com/docs/houdini15.0/model/volumes

Revision as of 03:55, 14 January 2016

Volumes are primitives

Confused me slightly at first, so worth mentioning; A volume is a primitive, the same way that a point is a primitive, or a primitive sphere is a primitive. This means that unlike a polygon mesh for example, there's no drilling down to its sub-components; you can't open the geometry spreadsheet to see values in the individual voxels.

There are other tools to help you see whats going on within a volume (volume slice sop, volume visualisation sop, volume trails sop etc), but ultimately, if you merge a 400x400x400 voxel volume with a primitive sphere, if you inspect the merge it will show that you have 2 primitives; a volume and a sphere.

Houdini supports 2 volume types, its own volume format, and VDB. These are treated as primitives like polys or nurbs, if you middle click and hold on a node, you'll see it say '1 VDB' or '1 Volume' or similar.

VDB?

The most simple way to describe it is as Alembic, for volumes. It's an open source, standardised format so you could export a VDB from Houdini, load it into maya, and render in Vray. It has several companies helping drive it (Dreamworks, Double Negative, SideFx being the most notable), and is generally a good thing.

It's more than Alembic though, in that Alembic is mainly a file format, and a handful of example tools to manipulate and view Alembic files. The VDB toolkit is both a way to store a volume, and a large suite of algorithms to manipulate volumes.

One of VDB's most interesting qualities is that it doesn't waste storage space on empty voxels. VDB's can be substantially smaller on disk compared to other formats.

The VDB storage format is exposed in Houdini as a primitive (a 'VDB primitive' funnily enough), and the manipulation tools are exposed as sops, mostly with the VDB prefix or suffix.

The name VDB as stated in the original paper comes from "...Volumetric, Dynamic grid that shares several characteristics with B+ trees.".

Further reading: http://www.openvdb.org/about/

VDB vs Houdini volumes

Houdini's own volume format had been around for a while before VDB arrived, so there's a large amount of nodes in there for working with them (you can see this in sops by typing 'volume' in the tab menu). VDB has its own collection of nodes (either prefixed or suffixed with VDB).

VDBs can be substantially smaller than native Houdini volumes (savings of 50% are common), and often the VDB sops are faster and have more interesting features than the native Houdini volume sops.

There's a .vdb file format (which you'd use to export to other apps), but you don't need to write out a .vdb file to get the space savings, just having a VDB primitive in Houdini is enough. A Houdini volume primitive can be converted to a VDB primitive at any time, just put down a convertvdb node, set its mode to VDB. If you middle click on the node, you'll see the Volume primitive has been replaced with VDB. You can now cache this out to .bgeo like any other piece of Houdini geo, but it'll be much smaller than if you didn't convert.

What's nice is that a lot of the core Houdini volume tools have been updated to work with VDB primitives too. Volume VOPS and Volume Wrangles for example both work with VDB. This means its pretty safe to use VDB at all times, as you can easily convert a VDB temporarily to a native volume if required, and then convert back again.

Further reading: http://www.sidefx.com/docs/houdini15.0/model/volumes

Scalar and vector volumes

A volume used to represent a cloud for example only needs to store a single value per voxel, density. If density is 0 at a voxel it is completely transparent, if density is at 1 at another voxel it is opaque. The cloud sop for example uses noise functions under the hood to sculpt density into pleasing cloud shapes.

If you wanted to store colour that varies through the cloud, then you need to store the red/green/blue components. This means you require a vector volume (or field, you'll find the docs and forums refer to volumes as fields a lot), to store this information.

A more common use of vector fields is for velocity. A convention in Houdini is that a velocity volume is named 'vel'. Each voxel in the volume stores a direction. A pyro solver will use this velocity field to transfer density around, giving the impression of movement. Other properties of pyro sims is to modify the velocity field itself, so you might get a mushroom cloud that rolls upwards, or noise that swirls the density around.

Something to note is how native Houdini vector volumes and vdb vector volumes differ. Internally Houdini doesn't really have a vector volume format, but it creates 3 scalar volumes, and knows to treat them as a single entity. A standard smoke setup that uses density and vel will have 4 volume primitives; density, vel.x, vel.y, vel.z.

VDB does support vector volumes, so the same smoke setup will be 2 vdb primitives, density and vel.

Note here that its almost the reverse of how you work with standard geometry and attributes; its fine to have a polygon grid where each point stores multiple attributes; N, Cd, id, foo, myattr, name etc.

Volumes are almost treated in reverse; the primitive can only store a single attribute, so you make a volume primitive for each attribute you need. So a more complex pyro setup might contain maybe 20 or more volume primitives; one for density, 3 for velocity, 3 for Cd, 1 for temperature, 1 for fuel etc...

Luckily most of this is hidden away from the user, so you often only have to worry about density and vel (and maybe Cd).

SDF

Volume compare.jpg

A volume (or vdb) can represent an amorphous shape like fog or fire, or it can represent a solid surface. When doing the latter, each voxel stores the distance to the closest point on the surface, and if its on the inside or outside of that surface. Because these values can be fractions (eg 1.42 units from the surface rather than just 1), they can give a pretty good approximation of the input geometry. To tell if a voxel is on the inside or outside of the surface, it becomes positive or negative; positive is outside, negative is inside. If you visualise it, it would look like a gradient that is 0 on the silhouette of the shape, and a smooth gradient that extends inside and outside the shape. This volume is called a Signed Distance Field, or SDF.

Viz sdf.gif

Positive values are red to yellow, negative values are dark to light blue.


This can be handy for many things. If you have a robust way to generate a SDF (which of course we do in Houdini), it's a great way to generate collision geometry for simulations. Particles or RBD objects can very quickly sample the SDF, if their sign is negative, they're inside the shape. They can then measure the gradient of the SDF, and determine exactly how far and how much to be pushed to stop intersecting.

If you take the SDF and convert it back to polys, you get an evenly meshed, watertight surface. You can smoothly expand or dilate the surface, you can combine SDFs and get very clean booleans, they're handy in lots of situations.

Poly mesh vdb.jpg

Houdini gives you 2 ways to generate an SDF from poly geo; the native Houdini way ( IsoOffset SOP in SDF volume mode), and the VDB way ( VDB-from-polygons sop). My 30 second tests imply VDB is much faster for more detailed geometry.

Houdini visualises SDF volumes with a sprite per-voxel where the SDF field is 0. It looks like it uses the SDF gradient to derive a normal for the sprites, so they respond to lighting.

Viewing volume data

Sdf visualise.jpg

Download scene: File:vis_volume.hip

Because a volume or vdb is treated as its own primitive type, the voxel data can't be inspected from the geometry spreadsheet as you'd initially expect. If you try, all you'll see is a primitive number for each volume in the shape, no data is displayed.

Instead, you're supposed to use the visualise tools within the viewport. A Volume slice lets you see a 2d slice of voxel data mapped into a colour ramp, a volume trails node lets you follow lines of direction within a volume, say graident, through the volume.

The example hip is an example of the two different ways of working with volumes; it calculates the gradient in 2 ways, one with a vex 'volumegradient' call, the other with the 'VDB analysis' sop. The VDB nodes are presented a little differently than regular Houdini nodes, but are very powerful and very fast.

Volumes, SDFs, Trails

Vol trails.gif

Download scene: File:volume_trails_V01.hip

Saw an interesting vimeo tutorial on generating swirly lines ( https://vimeo.com/134057856 ), and an odforce post asking how to generate tracer lines from noise ( http://forums.odforce.net/topic/23535-noise-how-to-convert-into-line/ ), this is my attempt to combine the two. The volume trail sop is used to visualise fields in volumes, but you can use it for your own purposes if you get volumes setup the right way.

Here, the idea is to take a shape and create an SDF, or Signed Distance Field out of it. This is a volume where each voxel stores the distance to the closest point on the surface. Voxels on the surface have value 0, voxels outside the surface have positive values, voxels inside the surface have negative values. This makes them very handy for checking collisions, among other things (here's a nice shadertoy demo to visualise SDFs in 2d: https://www.shadertoy.com/view/ltBGzK )

I then take another volume, where each voxel can store a vector value, called 'vel' (the houdini standard for storing velocity in volumes). I then clip the vel field using the SDF, so only voxels within 0.1 of the surface remain, the rest are clipped.

I then run that volume through curl noise, so the vel field now has groovy lumpy swirls through it. Feeding that to a volume trails sop generates the sweet lines.

While this sort-of follows the pig surface, the lines tend to wobble above and below the surface quite a bit. To make them stick directly on, I just ray the curves onto a slightly fatter copy of the pig (to avoid clippy intersection issues).

There's probably a way to generate the curl noise directly from the SDF, so that the resulting lines stick dead on the surface. A ray is quick n lazy tho. :)

Volume magnetic lines

Vdb maglines.gif

Download scene: File:magnetic_lines_vdb.hipnc

Found this post ( http://pepefx.blogspot.com.au/2015/08/how-to-create-magnetic-vector-field-in.html ) explaining how to setup magnetic lines. Tried to follow it once when very tired, then tried it again a month later without looking at his notes to see if I could get a similar effect, and without using his clever vex.

Two things that might be interesting in this setup:

  • Creating vdbs from polys, specifically creating density and vel vdb's simultaneously from the one poly. The vdb-from-polygons sop has a multilister similar to creating aovs/image planes on a rop. Click to add a new one, tell it what attribute is the source (eg, v for velocity), and what kind of vdb it will map to (vector).
  • Combining vdb's. I setup a positive and negative vdb, then combine them with the 'add' mode. It just occurred to me I could have merged the poly shapes first, then made the vdb in one step, which would be much more efficient. Still, handy learning exercise eh?


The end result I'm sure is a scientific abomination, but looks pretty. To be honest I was sure the setup wouldn't work; just creating a negative and positive velocity field seemed too simple for the volume trails to do its thing, but here it is, working. The lines and colours are my usual abuse of volume trails and point wrangles.

Swirly lines, or worms with volumes and curl noise

Pig worms.gif

Download scene: File:vol_worms_curlnoise.hipnc

Another attempt at the swirly lines thing, but I think much more effective. I'd seen Raph's very cool experiment on vimeo, and was curious about how it was achieved. Naturally being too proud to ask him even though he sits 2 metres away from me, I'd been stewing on it for a while.

My experiments in pyro got me familiar with volumes and creating velocity fields, and the volume trail sop kept giving cool results. Was using curl noise in a volume vop when I noticed the input for a sdf. Started playing with it, to my surprise it does exactly the effect I was after, and the help docs confirmed it; like putting rocks in a stream, it forces the curl noise to flow around the sdf's you give it.

As such, the workflow is this:

  1. Take a shape (the trusty pig in this case)
  2. Get its bounds, make a vel fog vdb from it
  3. Also make a SDF of the pig
  4. In a volume vop, use curl noise, attach the SDF to the input of the curl noise to 'disturb it', adjust sliders to taste
  5. Test the vdb for negative values (ie, inside the pig shape), make those values 1, outside values 0
  6. Multiply the curl noise against this to set all vel outside the pig to 0. Now the curl noise flows along the surface and inside the pig volume
  7. Scatter points on the pig surface, volume trail sop to generate cool lines
  8. Either use @width to render the curves directly or polywire sop to get poly tubes


Curves with @width are more efficient of course, but there's a bug with them and opengl that causes odd spikes and pops, so I need to fall back to polywire for my flipbook captures. The dancing lines are from adding time to the inputs of the curl noise.

Here's the same effect applied to the squab, with the addition of colour being driven by the SDF values in turn driving a ramp; values near the surface are white, then red, then pink, then blue. Pleasingly gross.

Squab viscera small.jpg

Download scene: File:vol_worms_curlnoise_v02.hipnc