The past decade has seen a growing interest in evolutionary models that relax the assumption of site-independent evolution for non-coding sequences. While phylogenetic inference using such so-called context-dependent models is currently computationally prohibitive, these models have been shown to yield significant increases in model fit compared to site-independent evolutionary models, which remain the most widely used evolutionary models to study substitution patterns and perform phylogenetic inference. Context-dependent models have been shown to be suited to study the spontaneous deamination of cytosine in mammalian sequences. In this paper, I discuss various approaches presented in recent years to model context-dependent evolution. I start with discussing the empirical research and results that have led to the development of these models. To accurately estimate the context-dependent substitution patterns that arise from these models, accurate sampling of substitution histories under such models is required. Further, appropriate model selection techniques to assess model performance has become more important than ever, given the drastic increase in parameters of context-dependent models and the tendency of older model selection techniques to prefer parameter-rich models. I also present new results on two mammalian datasets (Primate and Laurasiatheria data) to shed a light on so-called lineage-dependent context-dependent evolution. I conclude this paper with a discussion on current challenges in the development of context-dependent modeling approaches.