Morphological features of the tumor microenvironment are emerging as powerful prognostic indicators for colorectal cancer (CRC). The presence of peritumoral budding (PTB), defined as the presence of single tumor cells or small clusters of up to five cells in the tumor stroma ahead of the invasive front, is a hallmark of aggressive disease biology. Presence of PTB strongly correlates with adverse clinicopathological features and is recognized as an additional adverse prognostic factor by the Union for International Cancer Control. Recent studies have also characterized intratumoral budding (ITB) in biopsy material as a prognostic indicator in the preoperative setting. This paper provides a comprehensive overview on the role of PTB and ITB in the clinical management of colon and rectal cancer.