As Mercury orbits the Sun, gravitational torques on its ellipsoidal figure give rise to a longitudinal libration. Earth-based radar observations indicate that the outermost part of the core is fluid and only the mantle participates in the libration. If a solid inner core is present, it can alter the libration of the mantle through gravitational coupling. Previous studies have shown that the influence of the inner core on libration remains small, though not negligible if its radius is larger than 1000 km. However, these studies considered a simplified model of the libration dynamics, where the density in both the inner core and fluid core were assumed uniform. Here, we build a model of Mercury's libration that takes into account the radially varying density profile within the core. Our results indicate that the amplitude of the 88-day mantle libration is slightly larger than that predicted on the basis of the uniform density model, though the difference is well below the current error bars in observations. We also show that the period of the free libration of the combined inner core and mantle is largely unaffected by the details of the core density structure. However, the period of the second free mode of libration - broadly described by the gravitational oscillation between the mantle and inner core - is significantly longer when a more realistic core density profile is used. We discuss the implication of this result, and we also show how the role of the inner core can be taken into account when observations of gravity and rotation are combined to form constraints on the interior structure of Mercury.