Questions surrounding the effects of chronic marijuana use on brain structure continue to increase. To date, however, findings remain inconclusive. In this comprehensive study that aimed to characterize brain alterations associated with chronic marijuana use, we measured gray matter (GM) volume via structural MRI across the whole brain by using voxel-based morphology, synchrony among abnormal GM regions during resting state via functional connectivity MRI, and white matter integrity (i.e., structural connectivity) between the abnormal GM regions via diffusion tensor imaging in 48 marijuana users and 62 age- and sex-matched nonusing controls. The results showed that compared with controls, marijuana users had significantly less bilateral orbitofrontal gyri volume, higher functional connectivity in the orbitofrontal cortex (OFC) network, and higher structural connectivity in tracts that innervate the OFC (forceps minor) as measured by fractional anisotropy (FA). Increased OFC functional connectivity in marijuana users was associated with earlier age of onset. Lastly, a quadratic trend was observed suggesting that the FA of the forceps minor tract initially increased following regular marijuana use but decreased with protracted regular use. This pattern may indicate differential effects of initial and chronic marijuana use that may reflect complex neuroadaptive processes in response to marijuana use. Despite the observed age of onset effects, longitudinal studies are needed to determine causality of these effects.