A case study of evaluating the GRAPES_Meso V5.0 forecasting performance utilizing observations from South China Sea Experiment 2020 of the “Petrel Project”

The evaluations of numerical weather prediction (NWP) models are pivotal for their development. However, there is a big challenge to carry out over oceans because of the rarity of in-situ observations. With a unique suite of marine observations from South China Sea Experiment 2020 of the “Petrel Project” that undertook a mission to observe the peripheral cloud system of Typhoon Sinlaku (2020), this study evaluates the performance of GRAPES_Meso v5.0 forecasts operated by the China Meteorological Administration. Compared with the observations by dropsondes from the unmanned aerial vehicles (UAV), nine forecast cycles all well forecast the atmospheric vertical profiles with a basically consistent bias structure over ocean, which reveals that the model has a roughly accurate and stable forecast performance. The temperature bias demonstrates a pattern of warm-cool-warm distribution from model bottom level to near 300 hPa, corresponding to a wet-dry-wet pattern in humidity fields with considerable relative errors. The positive (negative) temperature (humidity) biases at heights below 925 hPa are related to the underestimation of total hydrometeor content. The pressure field has been systematically overestimated with a maximum value of ∼20 hPa at 6 km height. The vertical velocities have been underestimated, yet, they are closer to the observation in its spatial distribution when the leading time is shorter. The forecasted sea level pressure and temperature at 2-m have consistent temporal variation with two buoy observations, with biases varying from −0.82 hPa to 1.38 hPa and − 1.74 °C to 2.1 °C, respectively. These results reveal that GRAPES_Meso V5.0 has stable forecast performance with relatively small biases except for humidity field. Although the conclusions may not be universal since they are only based on the evaluation for one single typhoon event, this study highlights the potential of UAVs observations in model evaluation, especially over oceans. More observational experiments conducted by UAVs at different weather events are expected to obtain more robust results for the performance of NWP models.