Bayesian Gaussian Graphical Models (BGGMs) are tools of growing popularity and interest in network psychometrics and probabilistic graphical modeling. However, some of the existing models are derived from different modeling principles that do not easily allow for extensions and combinations into new models. More specifically, the implementation of some models may not be flexible enough to test different priors or likelihoods. In this paper, we present a new approach to BGGMs that overcomes this limitation by allowing for the estimation of regularized partial correlations between any type of variables while also having an intuitive approach on how to decide about the priors. Our approach is based on using a transformation of the lower diagonal values of the Cholesky (or LDL) decomposition matrix as the parameters of the models, which can receive any zero-centered symmetric distribution as a prior, as well as to include moderators. We have developed the gbggm R package to implement some models based on this approach, and the potentials of the approach are demonstrated with a toy simulation and an empirical example. This new approach expands the range of applications and enhances the flexibility of BGGMs, making them more useful in a variety of contexts.