Number processing interacts with space encoding in a wide variety of experimental paradigms. Most intriguingly, the passive viewing of uninformative number symbols can shift visuo-spatial attention to different target locations according to the number magnitude, i.e., small/large numbers facilitate processing of left/right targets, respectively. The brain architecture dedicated to these attention shifts associated with numbers remains unknown. Evoked potential recordings indicate that both early and late stages are involved in this spatio-numerical interaction, but the neuro-functional anatomy needs to be specified. Here we use, for the first time, functional magnetic resonance imaging (fMRI) to investigate attentional orienting following uninformative Arabic digits. We show that BOLD response in occipital visual regions is modulated by the congruency between digit magnitude (small/large) and target side (left/right). Additionally, we report higher BOLD responses following large (8, 9) compared to small (1, 2) digits in two bilateral parietal regions, yielding a significant effect of digit magnitude. We propose and discuss the view that encoding of semantic representations related to number symbols in parietal cortex leads to shifts in visuo-spatial attention and enhances visual processing in the occipital cortex according to number-space congruency rules.