The capsizing of vessels in random beam seas is investigated using a single degree-of-freedom model which is limited to the roll motion. Several factors, including sea wave spectrum, nonlinear righting moment characteristics, and nonlinear damping, are taken into account in the analysis. A nonlinear probabilistic method is developed by combining ideas from modern nonlinear dynamics (specifically, the Melnikov function and the phase space area flux) and random vibrations. Conditions for the onset of vessel capsizing are obtained in terms of the sea state characteristics (significant wave height and characteristic wave period) and the vessel parameters (damping and stiffness coefficients). Extensive numerical simulations are carried out to demonstrate the validity of the analytical results. It is found that there exists an excellent correlation between the rate of phase space flux and the probability of capsizing.