Difference between revisions of "HoudiniHDA"

From cgwiki
Line 89: Line 89:
  
 
Obviously you'd want to move stuff to a more solid pipeline infrastructure over time, but this'll do in the short term.
 
Obviously you'd want to move stuff to a more solid pipeline infrastructure over time, but this'll do in the short term.
 +
 +
=== Best practices ===
 +
 +
It's easy to get carried away and start HDA-ing all the things. But it's good to have a think first, and as always the comparison to maya references come in handy.
 +
 +
* '''Don't use HDAs to be tidy.''' - If you're just wanting to collapse a few nodes together to clean up the network, then just use a subnet, thats what it's for. Or the new collapsable network boxes. Or just leave all the nodes in place and colour them to group them, and give nice names.
 +
* '''Will a controller null do?''' - If you're after a clean central 'control panel' for your setup, you can do the drag and drop thing with a null, just be sure to uncheck the 'forbid links from this same network level' toggle that sidefx annoyingly insist on. HDAs are more for reusable assets than tidy UIs or tidy nodes.
 +
* '''Don't make HDAs too small''' - If you're just wrapping up 2 nodes, why? You're really not gaining much, maybe drag nodes onto the shelf or into a gallery instead.
 +
* '''Don't make HDAs too big.''' - If you're wrapping up 200 nodes, and every time you use it you have to unlock it to dive inside and tweak parameters, then maybe you can break that up into smaller, more modular units.
 +
* '''Think of the collaborative workflows.''' - Once a HDA is on disk and away from hip files, it can be worked on in isolation by one artist, while its being used in many hips. Have many HDAs in many hips, then multiple people can be editing different HDAs in parallel, which can be a huge productivity boost. 'Will this be useful to many people in several contexts?' is a good reason to make an HDA.
 +
* '''OTL and HDA are the same thing.''' - For all practical purposes anyway. OTL was the old name, short for 'Operator Type Library'. Nodes in Houdini are SOPs, DOPs, POPs etc, ie Surface Operators, Dynamics Operators, Particle Operators.... Ops, or Operators in summary. You have different kinds of Operators, ie different types, and you could save many operators into a single file, making a library of operator types. OTL. Get it? Wisely someone at sidefx realised one day 'thats a terrible name', and they were changed to Houdini Digital Assets instead. A few of the tools still use the old name, like the hotl command, so don't get confused, they're the same thing.
 +
* '''An HDA file can contain many assets.''' - As I said the old name OTL better implied these files could contain multiple assets. These days people general save one asset per HDA, as its easier to find stuff when the name of the node in the houdini graph matches to the name of a HDA file on disk, but you can absolutely store as many assets as you want in one hda file.
 +
* '''You can save HDAs in your hip''' - If you want to be super portable and have an all-in-one solution, you can choose to save hdas 'embedded', ie, as part of the hip. Be careful with this though, as it kind of defeats the purpose of using HDAs in the first place. Would you save a maya reference into the working file? You'd need a very good reason, and people would look at you funny.
  
 
=== Todo ===
 
=== Todo ===
* Section on when to make hdas
 
* How small to go. When to use them
 
* How they can split up work so people can work together on a bigger setup
 
 
* Talk about versioning. Both the hda version and the hda identified version namespace
 
* Talk about versioning. Both the hda version and the hda identified version namespace
 
* Talk about what happens to updates when you update the hda. (They are lost)
 
* Talk about what happens to updates when you update the hda. (They are lost)
 
* python callbacks. Limit them to certain contexts. Custom menus
 
* python callbacks. Limit them to certain contexts. Custom menus
* on disk one hda file can contain many digital asset node definitions
 
* otl and hda are synonyms
 
 
* nested hda's can be tricky. " if a sidefx labs node was used inside the HDA we are creating, then it needs to be unlocked as well or the overall HDA needs to be unlocked in order for it to work in the game engines. this wasn't addressed anywhere and maybe you can add that."
 
* nested hda's can be tricky. " if a sidefx labs node was used inside the HDA we are creating, then it needs to be unlocked as well or the overall HDA needs to be unlocked in order for it to work in the game engines. this wasn't addressed anywhere and maybe you can add that."
 
* That odd save mode where you can say a hda will be loaded in unlocked and ready to modify, it'll be tinted blue or something
 
* That odd save mode where you can say a hda will be loaded in unlocked and ready to modify, it'll be tinted blue or something
* embedded hdas
 
 
* hda errors
 
* hda errors
 
* hotl expand?
 
* hotl expand?

Revision as of 04:56, 11 September 2020

Quick blat of ideas while this is fresh in my mind...

I've avoided HDAs for a long time, I'm still avoiding them actually, but latest job requires I get my head into it.

Like a lot of Houdini things its a good idea hidden by a slightly obscure interface, this'll try and clear it all up. I compare this a lot to maya references as that's the only comparison point I have, and most people in 3d understand them. If you don't, well.. tough. :)


What is an HDA anyway?

A Houdini Digital Asset is like a maya reference. You export a chunk of a hip to an external file (eg c:/projects/library/coolthing.hda), now those nodes live somewhere else on disk, and you can bring them into your hip like a reference.

They have a few features that make them a little more complicated than maya references:

  • You can't just go File -> Export on a bunch of nodes, you need to use the HDA workflows to export.
  • Saving and updating is kind of magical and in-place, needs a mild leap of faith to trust its working.
  • As well as dumping a bunch of nodes to an external location, you can also put parameters on the top of the hda to control nodes within it, which is a trick you can't do with maya references
  • There's no 'maya reference editor' style thing to get a central overview of all HDAs used in the hip. When you edit HDAs, you do it per node, one at a time.
  • HDAs are supposed to be saved in a shared central location if you're in a big studio, meaning pipeline environment variables and all that. There's workarounds and tricks, but thats the ideal scenario.
  • HDAs when defined will appear in your tab menu rather than expecting you to go File -> Load reference, which is neat.

Once you realise that all an HDA is a bunch of nodes saved together on disk, it all becomes a lot easier to understand. It's also worth noting how many native Houdini nodes are HDAs. The clue is to look for the padlock icon next to the node. If you see it, that's an HDA, and you can dive inside to see how the sausage gets made.

Creating an HDA

Again several ways, but here's how its most often done:

  • Select the nodes you want to HDA-ify
  • Put them in a subnet
  • R.click on the subnet, Create Digital Asset
  • Set the name and location to save it. The default is to prefix with your username, I usually remove that. Also be clear about the save location, if its just for you, fine, put it in your home folder, but you probably want to put it somewhere more public if its to share with others.
  • Houdini shows you the Edit operator type dialog. This is for you to customise the interface. The simple trick here is to put the dialog to one side, dive into your hda subnet, select nodes, and drag and drop parameters you want to control directly into the 'existing parameters' region. They'll be automatically channel referenced onto the top of the hda, super handy.
  • Hit accept, you're done.

Kiran points out an even easier method than drag and dropping parameters is to hold down alt + middle click the parameter and it will automatically add to the type properties. Handy!

Lock states, editing and saving and reverting hdas

Now you have an HDA, you can do a few things with it:

  • make modifications to the hda in the hip (the equivalent of maya reference edits)
  • push those changes out of your hip and into the hda on disk (the equivalent of opening a second maya, opening the reference, editing and saving it)
  • reverting any changes (the maya equivalent of deleting all reference edits).

Next to the hda is a padlock icon. Red means its unlocked and you might've made a local change, gray means its locked and matches the definition on disk.

In maya reference terms red means you might've made reference edits, gray means you've made no changes. If you dive inside the hda when the padlock is gray you'll see the nodes inside are dimmed; you're literally locked out of making edits.

If you want to make changes you can go back to the top of the hda, r.click and choose Allow editing of contents. The padlock goes red, dive inside and it no longer dimmed, you can edit away. Any changes you make are stored in the hip, the equivalent of making reference edits in maya. Like maya, the original hda (the reference) is untouched. If you were to bring in another copy of the hda, it wouldn't have those edits (the equivalent of referencing the same file twice, the second won't have the reference edits of the first).

To save your changes to the hda on disk, right click on the hda and choose Save node type. That's it. Those changes are now in the hda on disk. That's the equivalent in Maya of opening a second maya, loading the reference, copying your changes into there and saving.

Your hda will still be in the red and unlocked state. You don't want to risk editing and saving changes to the hip do you?

To do the equivalent of a 'delete reference edits', ie make it match exactly to the state of the hda on disk, r.click again and choose Match current definition. The padlock goes gray and locked, if you go inside you can see its all dimmed, but should now show that the changes you saved are now part of the hda rather than of the hip.

This becomes a bit of a pattern. You'll make an hda, give it a nice UI, share it around. Someone suggests an improvement. You'll:

  • 'Allow editing of contents' to unlock it, dive inside to fix it, make sure it works in that edited state.
  • 'Save node type' to push those changes to the HDA on disk
  • 'Match current definition' to lock it, see how the hda will look to others.

Load $HIP/hda on hip load via hou.session

Download hip: File:hda_from_hip.zip

Hacky. But I like hacky.

My current job has a couple of interesting contraints. Hip files need to be run In The Cloud, and ideally run as standalone as possible. That means minimal environment variables, minimal fixed locations for libraries or assets, just the hip and what the hip might need in subfolders.

We also need multiple people working on a single file at once, using github to manage our files. That meant monolithic hip files were becoming an issue, because really only one person can work on it at once. We've tried using the diff tools, but they fell apart on big files, especially if you have things like stash nodes and frozen nodes.

Splitting work up into HDA's would be ideal, but the assumption is you load all your HDAs at houdini startup, and use environment variables to say where those HDAs will be found, which breaks our first requirement.

Enter this trick.

The amazing Chris Gardner shared a python script he wrote to hot load HDAs. Give it a folder, it'll scan the folder and load any HDAs it finds. That script lives here:

https://gist.github.com/chris-gardner/c9daf34a668c5dddda94b9f6276d8cb8

But how can we call that from a hip if we're trying to not load any external scripts or dependencies? Hou.session, that's how!

Hou.session is a python module that is saved as part of a hip, and is loaded (and therefore executed) on hip load. It can be accessed via Window -> Python Source Editor.

As such I copied Chris's script into hou.session, and I run

loadHdaLibrary(hou.getenv('HIP')+'/hda')

At the bottom of the script. Now I can make a hda folder next to the hip, save HDAs in there, and they'll be loaded when the hip is loaded.

Obviously you'd want to move stuff to a more solid pipeline infrastructure over time, but this'll do in the short term.

Best practices

It's easy to get carried away and start HDA-ing all the things. But it's good to have a think first, and as always the comparison to maya references come in handy.

  • Don't use HDAs to be tidy. - If you're just wanting to collapse a few nodes together to clean up the network, then just use a subnet, thats what it's for. Or the new collapsable network boxes. Or just leave all the nodes in place and colour them to group them, and give nice names.
  • Will a controller null do? - If you're after a clean central 'control panel' for your setup, you can do the drag and drop thing with a null, just be sure to uncheck the 'forbid links from this same network level' toggle that sidefx annoyingly insist on. HDAs are more for reusable assets than tidy UIs or tidy nodes.
  • Don't make HDAs too small - If you're just wrapping up 2 nodes, why? You're really not gaining much, maybe drag nodes onto the shelf or into a gallery instead.
  • Don't make HDAs too big. - If you're wrapping up 200 nodes, and every time you use it you have to unlock it to dive inside and tweak parameters, then maybe you can break that up into smaller, more modular units.
  • Think of the collaborative workflows. - Once a HDA is on disk and away from hip files, it can be worked on in isolation by one artist, while its being used in many hips. Have many HDAs in many hips, then multiple people can be editing different HDAs in parallel, which can be a huge productivity boost. 'Will this be useful to many people in several contexts?' is a good reason to make an HDA.
  • OTL and HDA are the same thing. - For all practical purposes anyway. OTL was the old name, short for 'Operator Type Library'. Nodes in Houdini are SOPs, DOPs, POPs etc, ie Surface Operators, Dynamics Operators, Particle Operators.... Ops, or Operators in summary. You have different kinds of Operators, ie different types, and you could save many operators into a single file, making a library of operator types. OTL. Get it? Wisely someone at sidefx realised one day 'thats a terrible name', and they were changed to Houdini Digital Assets instead. A few of the tools still use the old name, like the hotl command, so don't get confused, they're the same thing.
  • An HDA file can contain many assets. - As I said the old name OTL better implied these files could contain multiple assets. These days people general save one asset per HDA, as its easier to find stuff when the name of the node in the houdini graph matches to the name of a HDA file on disk, but you can absolutely store as many assets as you want in one hda file.
  • You can save HDAs in your hip - If you want to be super portable and have an all-in-one solution, you can choose to save hdas 'embedded', ie, as part of the hip. Be careful with this though, as it kind of defeats the purpose of using HDAs in the first place. Would you save a maya reference into the working file? You'd need a very good reason, and people would look at you funny.

Todo

  • Talk about versioning. Both the hda version and the hda identified version namespace
  • Talk about what happens to updates when you update the hda. (They are lost)
  • python callbacks. Limit them to certain contexts. Custom menus
  • nested hda's can be tricky. " if a sidefx labs node was used inside the HDA we are creating, then it needs to be unlocked as well or the overall HDA needs to be unlocked in order for it to work in the game engines. this wasn't addressed anywhere and maybe you can add that."
  • That odd save mode where you can say a hda will be loaded in unlocked and ready to modify, it'll be tinted blue or something
  • hda errors
  • hotl expand?