Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM

• Published in: arXiv.org 2024-11
• Main Authors: Mews, Maximilian, Aynetdinov, Ansar, Schiller, Vivian, Eisert, Peter, Akbik, Alan
• Format: Article
• Language: English
• Subjects: Annotations; Boundary representation; Computer & video games; Constructive solid geometry; Datasets; Geometry; Large language models; Machine learning; Mechanical engineering; Mesh generation; Natural language; Python; Scripts; Software development; Speech recognition
• ISSN: 2331-8422

While recent advancements in machine learning, such as LLMs, are revolutionizing software development and creative industries, they have had minimal impact on engineers designing mechanical parts, which remains largely a manual process. Existing approaches to generate 3D geometry most commonly use meshes as a 3D representation. While meshes are suitable for assets in video games or animations, they lack sufficient precision and adaptability for mechanical engineering purposes. This paper introduces a novel approach for the generation of 3D geometry that generates surface-based Constructive Solid Geometry (CSG) by leveraging a code-generation LLM. First, we create a dataset of 3D mechanical parts represented as code scripts by converting Boundary Representation geometry (BREP) into CSG-based Python scripts. Second, we create annotations in natural language using GPT-4. The resulting dataset is used to fine-tune a code-generation LLM. The fine-tuned LLM can complete geometries based on positional input and natural language in a plausible way, demonstrating geometric understanding.