Venous invasion has been considered a poor prognostic factor in rectal cancer for over half a century. This term has evolved in recent years and now applies specifically to tumor invasion into extramural veins – extramural venous invasion. This distinction from intramural venous invasion is important as it is more clinically relevant. Extramural venous invasion can be identified by histopathology and MRI but until recently there has been a lack of consistency in definitions and detection techniques. This paper reviews the historical evidence for the prognostic importance and detection of venous invasion in rectal cancer.