A large-scale simulation of the environmental dynamics of radioactive substances is very important from the viewpoint of nuclear security. Recently, GPU has been emerging as one of high performance devices to realize a large-scale simulation with less power consumption. We design a plume dispersion simulation to achieve high performance on the GPU. Lattice Boltzmann method (LBM) is a class of CFD method which is based on a weak compressible formulation. Since the LBM has a simple algorithm with continuous memory accesses, it is suitable for large-scale computation. We apply an adaptive mesh refinement (AMR) method to the LBM. The AMR method arranges fine grids in a necessary region, so that we can realize a multi-scale analysis at high-resolution.
We measure the performance of the LBM code on the GPU-rich supercomputer TSUBAME 3.0 at Tokyo Tech. Good weak scaling from 4 GPUs to 144 GPUs, and 30 times higher node performance is achieved compared with CPUs. The code is validated against a wind tunnel test which was released from the National Institute of Advanced Industrial Science and Technology (AIST). The computational grids are subdivided by the AMR method, and the total number of grid points is reduced to less than 10% compared to the finest meshes. In spite of the fewer grid points, the turbulent statistics and plume dispersion are in good agreement with the experiment data. From these results, it is shown that the present scheme is promising for large-scale local air flow simulations on future exascale supercomputers.