In most arid areas, due to scarce hydrogeological data, it is a challenge to locate groundwater sources and to meet water demand for residential, irrigation, and mining uses. In this study, an innovative method is presented, using magnetic resonance sounding (MRS), to detect areas suitable as groundwater sources on the Mongolian Plateau. First, a target investigation area was identified with a small number of MRS surveys of potential areas by determining whether aquifers exist, whether the aquifers have relatively large water contents and relaxation times, and whether there are hydraulic connections among the aquifers. Next, an intensive MRS survey (158 points in total) was conducted in the target investigation area, and eight boreholes were drilled. A comparison of the borehole data and MRS data showed that when the MRS data had a high signal-to-noise ratio, the aquifer depth and transmissivity estimated by MRS were associated with a deviation of only 4.85 m from the measured depth, and an uncertainty in transmissivity of 15.53%, respectively. These values indicated that the proposed method is highly accurate. Finally, a kriging interpolation method was used to construct distribution maps of groundwater levels, aquifer thickness, transmissivity, and water yield, based on the borehole and MRS data. The reliability of the results was assessed from several perspectives. The findings showed that this step-by-step approach is an effective method of groundwater source detection in arid areas with scarce hydrogeological data.