Gardenia yellow, an edible pigment derived from Gardenia jasminoides, has wide food applications; however, its safety profile is insufficiently defined. In this study, we characterized the composition of gardenia yellow and simulated its gastrointestinal digestion, absorption, and metabolism using an in vitro gastrointestinal model. Its hepatotoxicity and nephrotoxicity were assessed using the HepaRG and RPTEC/TERT1 cell lines via cellular viability, oxidative stress, and mitochondrial dysfunction assays. Its lowest effective concentration (LEC), determined after 14 days of repeated exposure, was used to estimate its oral equivalent dose (OED) via the in vitro-to-in vivo extrapolation (IVIVE) method. Crocin, the primary bioactive component of gardenia yellow (44.96%), exhibited 4.68 ± 1.00% oral bioavailability and was biotransformed to crocetin (30.47 ± 1.95%) in the colon. Crocetin induced oxidative stress and mitochondrial dysfunction in hepatic and renal cells. The predicted oral equivalent dose for gardenia yellow (1.43 g/(kg·bw·day)) exceeded the typical Chinese dietary exposure, indicating a favorable safety margin.