Fine-Tuning Text-to-Image Diffusion Models with LoRA for Generative Synthesis of Balinese Endek Textile Patterns

This study investigates the adaptation of a text-to-image diffusion model to generate patterns inspired by traditional Balinese Endek textiles. The objective is to evaluate whether Low-Rank Adaptation (LoRA) can improve the model’s ability to capture structural characteristics of Endek motifs, including repetitive geometric forms, ornamental symmetry, and stylized flora and fauna elements. A curated dataset of 687 Endek textile images was compiled and categorized into four motif classes: geometric, flora, fauna, and decorative. Each image was paired with a concise Indonesian textual description to guide prompt-conditioned image generation. The diffusion model was fine-tuned using LoRA applied to selected attention layers, enabling efficient domain adaptation with a limited number of trainable parameters. Training was conducted for 4,300 steps, showing stable convergence with a minimum loss of 0.00218, maximum loss of 0.75465, and mean loss of 0.12674 ± 0.10169. Quantitative evaluation using Fréchet Inception Distance (FID), Learned Perceptual Image Patch Similarity (LPIPS), Inception Score (IS), and CLIP-based similarity demonstrates consistent improvements after LoRA fine-tuning. The average FID decreased by 29.68%, improving from 394.21 (baseline) to 276.05 (LoRA). Category-level improvements include geometric (423.32 to 278.99), flora (432.37 to 296.23), fauna (379.00 to 273.13), and decorative motifs (342.14 to 255.84). LPIPS scores also decreased across categories, indicating higher perceptual similarity to real textile patterns. CLIP similarity scores ranged between 26–30, confirming strong alignment between generated images and textual prompts. These results demonstrate that LoRA provides an effective and parameter-efficient approach for adapting diffusion models to domain-specific textile pattern generation, particularly for culturally specific datasets such as Balinese Endek textiles.