Twin studies with objective measurements suggest habitual physical activity (HPA) are modestly to highly heritable, depending on age. We aimed to confirm or refute this finding and identify relevant genetic variants using a candidate gene approach. HPA was measured for 14 days with a validated triaxial accelerometer (Tracmor) in two populations: (1) 28 monozygotic and 24 dizygotic same-sex twin pairs (aged 22 ± 5 years, BMI 21.8 ± 3.4 kg/m(2), 21 male, 31 female pairs); (2) 52 and 65 unrelated men and women (aged 21 ± 2 years, BMI 22.0 ± 2.5 kg/m(2)). Single nucleotide polymorphisms (SNPs) in PPARD, PPARGC1A, NRF1 and MTOR were considered candidates. Association analyses were performed for both groups separately followed by meta-analysis. Structural equation modeling shows significant familiality for HPA, consistent with a role for additive genetic factors (heritability 57 %, 95 % CI 32-74 %, AE model) or common environmental factors (47 %, 95 % CI 23-65 %, CE model). A moderate heritability was observed for the time spent on low- and high-intensity physical activity (P ≤ 0.05), but could not be confirmed for the time spent on moderate-intensity physical activity. For PPARD, each additional effect allele was inversely associated with HPA (P ≤ 0.01; rs2076168 allele C) or tended to be associated with HPA (P ≤ 0.05; rs2267668 allele G). Linkage disequilibrium existed between those two SNPs (alleles A/G and A/C, respectively) and meta-analysis showed that carriers of the AA GC haplotype were less physically active than carriers of the AA AA and AA AC haplotypes combined (P = 0.017). For PPARGC1A, carriers of AA in rs8192678 spent more time on high-intensity physical activity than GG carriers (P = 0.001). No associations were observed with SNPs in NRF1 and MTOR. In conclusion, HPA may be modestly heritable, which is confirmed by an association with variants in PPARD.