In magneto- and electroencephalography (M/EEG), spatial modelling of sensor data is necessary to make inferences about underlying brain activity. Most source reconstruction techniques belong to one of two approaches: point source models, which explain the data with a small number of equivalent current dipoles and distributed source or imaging models, which use thousands of dipoles. Much methodological research has been devoted to developing sophisticated Bayesian source imaging inversion schemes, while dipoles have received less such attention. Dipole models have their advantages; they are often appropriate summaries of evoked responses or helpful first approximations. Here, we propose a variational Bayesian algorithm that enables the fast Bayesian inversion of dipole models. The approach allows for specification of priors on all the model parameters. The posterior distributions can be used to form Bayesian confidence intervals for interesting parameters, like dipole locations. Furthermore, competing models (e.g., models with different numbers of dipoles) can be compared using their evidence or marginal likelihood. Using synthetic data, we found the scheme provides accurate dipole localizations. We illustrate the advantage of our Bayesian scheme, using a multi-subject EEG auditory study, where we compare competing models for the generation of the N100 component.