HoudiniHDA

From cgwiki
Revision as of 18:22, 16 September 2020 by MattEstela (talk | contribs) (→‎Handy links)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Quick blat of ideas while this is fresh in my mind...

I've avoided HDAs for a long time, I'm still avoiding them actually, but latest job requires I get my head into it.

Like a lot of Houdini things its a good idea hidden by a slightly obscure interface, this'll try and clear it all up. I compare this a lot to maya references as that's the only comparison point I have, and most people in 3d understand them. If you don't, well.. tough. :)


What is an HDA anyway?

A Houdini Digital Asset is like a maya reference. You export a chunk of a hip to an external file (eg c:/projects/library/coolthing.hda), now those nodes live somewhere else on disk, and you can bring them into your hip like a reference.

They have a few features that make them a little more complicated than maya references:

  • You can't just go File -> Export on a bunch of nodes, you need to use the HDA workflows to export.
  • Saving and updating is kind of magical and in-place, needs a mild leap of faith to trust its working.
  • As well as dumping a bunch of nodes to an external location, you can also put parameters on the top of the hda to control nodes within it, which is a trick you can't do with maya references
  • There's no 'maya reference editor' style thing to get a central overview of all HDAs used in the hip. When you edit HDAs, you do it per node, one at a time.
  • HDAs are supposed to be saved in a shared central location if you're in a big studio, meaning pipeline environment variables and all that. There's workarounds and tricks, but thats the ideal scenario.
  • HDAs when defined will appear in your tab menu rather than expecting you to go File -> Load reference, which is neat.

Once you realise that all an HDA is a bunch of nodes saved together on disk, it all becomes a lot easier to understand. It's also worth noting how many native Houdini nodes are HDAs. The clue is to look for the padlock icon next to the node. If you see it, that's an HDA, and you can dive inside to see how the sausage gets made.

Creating an HDA

Again several ways, but here's how its most often done:

  • Select the nodes you want to HDA-ify
  • Put them in a subnet
  • R.click on the subnet, Create Digital Asset
  • Set the name and location to save it. The default is to prefix with your username, I usually remove that. Also be clear about the save location, if its just for you, fine, put it in your home folder, but you probably want to put it somewhere more public if its to share with others.
  • Houdini shows you the Edit operator type dialog. This is for you to customise the interface. The simple trick here is to put the dialog to one side, dive into your hda subnet, select nodes, and drag and drop parameters you want to control directly into the 'existing parameters' region. They'll be automatically channel referenced onto the top of the hda, super handy.
  • Hit accept, you're done.

Kiran points out an even easier method than drag and dropping parameters is to hold down alt + middle click the parameter and it will automatically add to the type properties. Handy!

Lock states, editing and saving and reverting hdas

Now you have an HDA, you can do a few things with it:

  • make modifications to the hda in the hip (the equivalent of maya reference edits)
  • push those changes out of your hip and into the hda on disk (the equivalent of opening a second maya, opening the reference, editing and saving it)
  • reverting any changes (the maya equivalent of deleting all reference edits).

Next to the hda is a padlock icon. Red means its unlocked and you might've made a local change, gray means its locked and matches the definition on disk.

In maya reference terms red means you might've made reference edits, gray means you've made no changes. If you dive inside the hda when the padlock is gray you'll see the nodes inside are dimmed; you're literally locked out of making edits.

If you want to make changes you can go back to the top of the hda, r.click and choose Allow editing of contents. The padlock goes red, dive inside and it no longer dimmed, you can edit away. Any changes you make are stored in the hip, the equivalent of making reference edits in maya. Like maya, the original hda (the reference) is untouched. If you were to bring in another copy of the hda, it wouldn't have those edits (the equivalent of referencing the same file twice, the second won't have the reference edits of the first).

To save your changes to the hda on disk, right click on the hda and choose Save node type. That's it. Those changes are now in the hda on disk. That's the equivalent in Maya of opening a second maya, loading the reference, copying your changes into there and saving.

Your hda will still be in the red and unlocked state. You don't want to risk editing and saving changes to the hip do you?

To do the equivalent of a 'delete reference edits', ie make it match exactly to the state of the hda on disk, r.click again and choose Match current definition. The padlock goes gray and locked, if you go inside you can see its all dimmed, but should now show that the changes you saved are now part of the hda rather than of the hip.

This becomes a bit of a pattern. You'll make an hda, give it a nice UI, share it around. Someone suggests an improvement. You'll:

  • 'Allow editing of contents' to unlock it, dive inside to fix it, make sure it works in that edited state.
  • 'Save node type' to push those changes to the HDA on disk
  • 'Match current definition' to lock it, see how the hda will look to others.

Load $HIP/hda on hip load via hou.session

Download hip: File:hda_from_hip.zip

Hacky. But I like hacky.

My current job has a couple of interesting contraints. Hip files need to be run In The Cloud, and ideally run as standalone as possible. That means minimal environment variables, minimal fixed locations for libraries or assets, just the hip and what the hip might need in subfolders.

We also need multiple people working on a single file at once, using github to manage our files. That meant monolithic hip files were becoming an issue, because really only one person can work on it at once. We've tried using the diff tools, but they fell apart on big files, especially if you have things like stash nodes and frozen nodes.

Splitting work up into HDA's would be ideal, but the assumption is you load all your HDAs at houdini startup, and use environment variables to say where those HDAs will be found, which breaks our first requirement.

Enter this trick.

The amazing Chris Gardner shared a python script he wrote to hot load HDAs. Give it a folder, it'll scan the folder and load any HDAs it finds. That script lives here:

https://gist.github.com/chris-gardner/c9daf34a668c5dddda94b9f6276d8cb8

But how can we call that from a hip if we're trying to not load any external scripts or dependencies? Hou.session, that's how!

Hou.session is a python module that is saved as part of a hip, and is loaded (and therefore executed) on hip load. It can be accessed via Window -> Python Source Editor.

As such I copied Chris's script into hou.session, and I run

loadHdaLibrary(hou.getenv('HIP')+'/hda')

At the bottom of the script. Now I can make a hda folder next to the hip, save HDAs in there, and they'll be loaded when the hip is loaded.

Obviously you'd want to move stuff to a more solid pipeline infrastructure over time, but this'll do in the short term.

Best practices

It's easy to get carried away and start HDA-ing all the things. But it's good to have a think first, and as always the comparison to maya references come in handy.

  • Don't use HDAs to be tidy. - If you're just wanting to collapse a few nodes together to clean up the network, then just use a subnet, thats what it's for. Or the new collapsable network boxes. Or just leave all the nodes in place and colour them to group them, and give nice names.
  • Will a controller null do? - If you're after a clean central 'control panel' for your setup, you can do the drag and drop thing with a null, just be sure to uncheck the 'forbid links from this same network level' toggle that sidefx annoyingly insist on. HDAs are more for reusable assets than tidy UIs or tidy nodes.
  • Don't make HDAs too small - If you're just wrapping up 2 nodes, why? You're really not gaining much, maybe drag nodes onto the shelf or into a gallery instead.
  • Don't make HDAs too big. - If you're wrapping up 200 nodes, and every time you use it you have to unlock it to dive inside to add nodes and tweak parameters, then maybe you can break that up into smaller, more modular units.
  • Think of the collaborative workflows. - Once a HDA is on disk and away from hip files, it can be worked on in isolation by one artist, while its being used in many hips. Have many HDAs in many hips, then multiple people can be editing different HDAs in parallel, which can be a huge productivity boost. 'Will this be useful to many people in several contexts?' is a good reason to make an HDA.
  • OTL and HDA are the same thing. - For all practical purposes anyway. OTL was the old name, short for 'Operator Type Library'. Nodes in Houdini are SOPs, DOPs, POPs etc, ie Surface Operators, Dynamics Operators, Particle Operators.... Ops, or Operators in summary. You have different kinds of Operators, ie different types, and you could save many operators into a single file, making a library of operator types. OTL. Get it? Wisely someone at sidefx realised one day 'thats a terrible name', and they were changed to Houdini Digital Assets instead. A few of the tools still use the old name, like the hotl command, so don't get confused, they're the same thing.
  • An HDA file can contain many assets. - As I said the old name OTL better implied these files could contain multiple assets. These days people general save one asset per HDA, as its easier to find stuff when the name of the node in the houdini graph matches to the name of a HDA file on disk, but you can absolutely store as many assets as you want in one hda file.
  • You can save HDAs in your hip - If you want to be super portable and have an all-in-one solution, you can choose to save hdas 'embedded', ie, as part of the hip. Be careful with this though, as it kind of defeats the purpose of using HDAs in the first place. Would you save a maya reference into the working file? You'd need a very good reason, and people would look at you funny.

HDA updates

Say I have a TD modifying an HDA, and an artist using that HDA in a shot. How are those updates pushed from TD to artist?

To simulate this I have 2 houdini's open; the one on the left is editing a hda, the one on the right is using it in a shot. You can see that I've changed some of the parameters in the right, but its still a locked hda, so the innards are the same as the hda on disk.

Hda edit vs shot.jpg

Now if I unlock and change nodes in the left, say swapping the box for a sphere, it doesn't affect the right one at all, not until I save it.

Edit hda presave.gif

Once I r.click on the hda and choose 'Save node type', what happens to the right side?

Hda save.gif

As soon as I give the right a nudge, it detects a change and updates.

But what if the artist has unlocked the HDA and made changes? If the TD updates the HDA and saves, will that change be inherited?

Hda shot unlock and edit.gif

No, the right no longer updates. Once you unlock an HDA in a shot it loses its connection to the HDA on disk. If you want to get back in sync you r.click on the hda and choose 'match current definition'.

Hda match definitionsmall.gif

But what if the artist wants to update the HDA and keep the changes they've made? Unfortunately you can't. There's no built in diff/merge style workflows presented to the end user, so if you go off and do your own thing to an hda, the only way to inherits to the hde on disk is to reset any change you have made.

HDAs in HDAs

Once you start getting tricky you might start making bigger HDAs out of other HDAs. Be careful! If you update those smaller HDAs on disk, they won't be automatically updated in the bigger HDAs. You need to manually open the bigger HDA, update those components inside, save.

HDA editor

Hda manager.jpg

I mentioned before that there's no equivalent to the Maya Reference Editor window. Well that's a lie, of course there's something, but it serves a different purpose, and helps to point out an important difference between Maya references and HDAs.

Window -> Asset Manager will bring up a list of HDAs, but note that its not just HDAs in the current hip, its all the HDAs Houdini knows about. All the ones as part of the sidefx install, ones in your home folder, ones embedded in the hip, ones in your pipeline, EVERYTHING.

This implies that HDAs aren't really just 'external .ma's linked to the current .ma' like references are. A better analogy is Maya's plugin editor; deformers and tools that are part of your session to get stuff done.

Another interesting feature of the HDA editor is that you can have multiple versions of the same HDA in different places, you can locate them and swap between them. If you want to swap between them more directly you can expose a dropdown on HDAs to choose between definitions. From that Asset Manager move to the 'Configuration' tab, and at the bottom for Asset Bar choose Display Menu of All Definitions. This can be really handy when you're trying to track down versions of files, or which HDA you're using.

Handy links

Todo

  • Talk about versioning. Both the hda version and the hda identified version namespace. Covered well here: https://www.sidefx.com/docs/houdini/assets/namespaces.html
  • python callbacks. Limit them to certain contexts. Custom menus
  • That odd save mode where you can say a hda will be loaded in unlocked and ready to modify, it'll be tinted blue or something
  • hda errors
  • hotl expand?