Large-scale nonlinear ultrasound simulations using the open-source k-Wave toolbox are now routinely performed using the MPI version of k-Wave running on traditional CPU-based clusters. However, the all-to-all communications required by the 3D fast Fourier transform (FFT) severely impact performance when scaling to large numbers of compute cores. This can be overcome by using a domain decomposition strategy based on a local Fourier basis. In this work, we analyze the performance and accuracy of using local domain decomposition for running a high-intensity focused ultrasound (HIFU) simulation in the kidney on a single server containing eight NVIDIA P40 graphical processing units (GPUs). Different decompositions and overlap sizes are investigated and compared to a global MPI simulation running on a CPU-based supercomputer using 1280 cores. For a grid size of 960 by 960 by 1280 grid points and an overlap size of 4 grid points, the error in the simulation using local domain decomposition is on the order of 0.1$ compared to the global simulation, which is sufficient for most applications. The financial cost for running the simulation is also reduced by more than an order of magnitude.
Skip Nav Destination
Article navigation
9 July 2018
21st International Symposium on Nonlinear Acoustics
July 9–13, 2018, 50th Anniversary
Santa Fe, New Mexico, USA
Computational Acoustics: S3-5
October 22 2018
Performance and accuracy analysis of nonlinear k-Wave simulations using local domain decomposition with an 8-GPU server
Bradley Treeby
;
Bradley Treeby
1Department of Medical Physics and Biomedical Engineering,
University College London
, London, WC1E6BT, UK
; [email protected]
Search for other works by this author on:
Filip Vaverka;
Filip Vaverka
2Centre of Excellence IT4Innovation, Faculty of Information Technology,
Brno University of Technology
, Brno, South Moravian Region, CZECH REPUBLIC
; [email protected], [email protected]
Search for other works by this author on:
Jiri Jaros
Jiri Jaros
2Centre of Excellence IT4Innovation, Faculty of Information Technology,
Brno University of Technology
, Brno, South Moravian Region, CZECH REPUBLIC
; [email protected], [email protected]
Search for other works by this author on:
Proc. Mtgs. Acoust. 34, 022002 (2018)
Article history
Received:
July 17 2018
Accepted:
September 17 2018
Citation
Bradley Treeby, Filip Vaverka, Jiri Jaros; Performance and accuracy analysis of nonlinear k-Wave simulations using local domain decomposition with an 8-GPU server. Proc. Mtgs. Acoust. 9 July 2018; 34 (1): 022002. https://doi.org/10.1121/2.0000883
Download citation file:
Citing articles via
Enabling support for GNU Octave within the FOCUS software package
Jacob S. Honer, Robert J. McGough
Related Content
Parallelized optimization for substructured finite element structural acoustic problems
Proc. Mtgs. Acoust. (January 2025)
Representing arbitrary acoustic source and sensor distributions in Fourier collocation methods
J. Acoust. Soc. Am. (July 2019)