In this study, a novel kind of organic–inorganic core–shell SiO2-poly(p-vinylbenzyl) trimethylammonium tetrafluoroborate (SiO2–P[VBTMA][BF4]) nanoparticle was well designed and successfully synthesized via surface-initiated atom transfer radical polymerization (SI-ATRP). Fourier transform infrared spectroscopy (FT-IR), 1H nuclear magnetic resonance (1H NMR), X-ray photoelectron spectroscopy (XPS), dynamic light scattering (DLS) and scanning electron microscopy (SEM) were used to confirm the formation of the core–shell nanoparticles and the surface modification. In order to overcome the challenge of the characterization of the number average molecular weight of poly(ionic liquid)s, “sacrificial initiator” method was used here employing a trimethylsilyl (TMS)-labeled initiator as the NMR marker for integration. In addition, good thermal stability of the new hybrid polyelectrolyte was proved by thermogravimetric analysis. The electrochemical impedance measurements revealed that the room temperature conductivity reached 10−4 S cm−1, which is much higher than that of the pure poly(ionic liquid)s and varies with the amount of the grafted polymer and the test temperature. The X-ray diffraction (XRD) tests further investigated the crystal structure of the nanocomposite and pure P[VBTMA][BF4]. The temperature dependence of ionic conductivity conforms to Arrhenius behavior for both of the nanocomposites and the pure polymer. The results indicated that the SI-ATRP approach provided a simple and versatile route to tune the ionic conductivity of the hybrid nanoparticles by changing the chain length of the grafted polymer, which can be potentially used in a variety of electrochemical devices.