In this paper, we generalize Majid's bicrossproduct construction. We start with a pair (A, B) of two regular multiplier Hopf algebras. We assume that B is a right A-module algebra and that A is a left B-comodule coalgebra. The right action of A on B gives rise to the smash product A # B. The left coaction of B on A gives a possible coproduct Delta(#) on A # B. We discuss in detail the necessary compatibility conditions between the action and the coaction for Delta(#) to be a proper coproduct on A # B. The result is again a regular multiplier Hopf algebra. Majid's construction is obtained when we have Hopf algebras.