Titolo della tesi: Enhancing Controllability in Procedural and Non-Procedural Asset Editing
The design of 2D and 3D assets is one of the key components of Computer Graphics applications. Recently, the demand for high-quality assets, like textures and materials, or 3D shapes, has seen impressive growth due to their widespread adoption in industrial design workflows as well as in the movie and video game industries. However, designing an asset is still a consuming operation in terms of time and human effort, in particular when a precise target must be matched. During the years, various techniques, such as real-world material and model captures or inverse modeling, have been developed to ease this process and to support artists in matching a desired target.
This thesis, entitled “Enhancing Controllability in Procedural and Non-Procedural Asset Editing”, explores the common techniques adopted in asset design and proposes novel approaches to reduce the time and human effort involved in this process, aiming to reach an easier and yet more controllable pipeline for asset editing in both for 2D and 3D environments.
We firstly investigate the procedural asset fields, proposing inverse procedural modeling solutions for 2D vector patterns and 3d implicit shapes defined in as differentiable programs and graphs respectively. As regards the 2D content, we propose an example-based parameter estimation tool, called pOp, and a direct manipulation tool named pEt. The first one proposes a parameter estimation method based on a user-provided sketch or render using an inverse Signed Distance Field optimization problem as its backbone. By minimizing the differences between the target pattern SDF and the procedurally generated one, it estimates the procedural parameter assignment that better matches the target design. Still in the parameter estimation field, the second approach enables users to directly manipulate the procedural vector pattern content by performing edits in the viewport, by selecting and dragging around sets of points or preventing others from moving. In this scenario, the best parameter assignments is determined by minimizing the distance between sets of points, allowing users to edit multiple parameters all at once without relying on counterintuitive GUI slider tweaking operations.
Similarly, in the 3D setting this thesis proposes a direct manipulation tool for procedurally-defined implicit surfaces, assuming end-to-end differentiability with respect to the procedural parameters. By defining a method for tracking points on the implicit surface, we set up a gradient descent based optimization loop that solves for the procedural parameters as the user performs edits directly in the viewport, such as dragging surface patches or constraining other ones. This tool allows for the editing of procedural implicit surfaces at interactive rate, supporting all the operations that make implicit modeling a robust and easy-to-use alternative to regular 3D modeling. In particular, it supports standard CSG operations and their smooth counterparts, which are widely used in creation of organic assets, while remaining resilient to topology changes.
Finally, we concentrate on non-procedural asset synthesis, specifically focusing on structured texture generation with widespread diffusion models as a backbone. We propose pAff, a method for the expansion of a small user-designed sketch to a largescale, high-quality and, moreover, tileable content. To do so, we enhance previously assessed diffusion pipelines by injecting structured pattern domain knowledge through a LoRA finetuning process, exploiting the Noise Rolling technique to improve quality and ensure tileability.
In conclusion, this thesis makes a significant contribution in improving user control in procedural and non-procedural asset editing. It proposes novel methods that could be either integrated into common 2D or 3D modeling softwares and furtherly expanded to various asset design workflows, making the design process easier for both novice and experienced users.