We present optical light curves of the gravitationally lensed components A (=A1+A2+A3) and B of the quadruple quasar RX J0911.4+0551 (z = 2.80). The observations were primarily obtained at the Nordic Optical Telescope between 1997 March and 2001 April and consist of 74 I-band data points for each component. The data allow the measurement of a time delay of 146 +/- 8 days (2 sigma) between A and B, with B as the leading component. This value is significantly shorter than that predicted from simple models and indicates a very large external shear. Mass models including the main lens galaxy and the surrounding massive cluster of galaxies at z = 0.77, responsible for the external shear, yield H-o = 71 +/- 4 (random, 2 sigma) +/- 8 (systematic) km s(-1) Mpc(-1). The systematic model uncertainty is governed by the surface-mass density (convergence) at the location of the multiple images.