Towards Distributed Semi-speculative Adaptive Anisotropic Parallel Mesh Generation (2312.13433v1)
Abstract: This paper presents the foundational elements of a distributed memory method for mesh generation that is designed to leverage concurrency offered by large-scale computing. To achieve this goal, meshing functionality is separated from performance aspects by utilizing a separate entity for each - a shared memory mesh generation code called CDT3D and PREMA for parallel runtime support. Although CDT3D is designed for scalability, lessons are presented regarding additional measures that were taken to enable the code's integration into the distributed memory method as a black box. In the presented method, an initial mesh is data decomposed and subdomains are distributed amongst the nodes of a high-performance computing (HPC) cluster. Meshing operations within CDT3D utilize a speculative execution model, enabling the strict adaptation of subdomains' interior elements. Interface elements undergo several iterations of shifting so that they are adapted when their data dependencies are resolved. PREMA aids in this endeavor by providing asynchronous message passing between encapsulations of data, work load balancing, and migration capabilities all within a globally addressable namespace. PREMA also assists in establishing data dependencies between subdomains, thus enabling "neighborhoods" of subdomains to work independently of each other in performing interface shifts and adaptation. Preliminary results show that the presented method is able to produce meshes of comparable quality to those generated by the original shared memory CDT3D code. Given the costly overhead of collective communication seen by existing state-of-the-art software, relative communication performance of the presented distributed memory method also shows that its emphasis on avoiding global synchronization presents a potentially viable solution in achieving scalability when targeting large configurations of cores.
- Drakopoulos, F., Tsolakis, C., and Chrisochoides, N., “Fine-grained Speculative Topological Transformation Scheme for Local Reconnection Methods,” AIAA Journal, Vol. 57, 2019, pp. 4007–4018. Https://doi.org/10.2514/1.J057657.
- Tsolakis, C., and Chrisochoides, N., “Parallel Metric-based Anisotropic Mesh Adaptation using Speculative Execution,” Engineering with Computers, to be submitted.
- Barker, K., Chernikov, A., Chrisochoides, N., and Pingali, K., “A load balancing framework for adaptive and asynchronous applications,” IEEE Transactions on Parallel and Distributed Systems, Vol. 15, No. 2, 2004, pp. 183–192. 10.1109/TPDS.2004.1264800.
- Thomadakis, P., Tsolakis, C., Vogiatzis, K., Kot, A., and Chrisochoides, N., “Parallel Software Framework for Large-Scale Parallel Mesh Generation and Adaptation for CFD Solvers,” AIAA Aviation Forum 2018, Atlanta, Georgia, 2018.
- Thomadakis, P., Tsolakis, C., and Chrisochoides, N., “Multithreaded Runtime Framework for Parallel and Adaptive Applications,” Engineering with Computers, Vol. 38, 2022, p. 4675–4695. https://doi.org/10.1007/s00366-022-01713-7.
- Chrisochoides, N., “Telescopic Approach for Extreme-Scale Parallel Mesh Generation for CFD Applications,” 46th AIAA Fluid Dynamics Conference, Washington D.C., USA, 2016. 10.2514/6.2016-3181, aIAA 2016-3181.
- Loseille, A., Menier, V., and Alauzet, F., “Parallel Generation of Large-size Adapted Meshes,” Procedia Engineering, Vol. 124, 2015, pp. 57–69. https://doi.org/10.1016/j.proeng.2015.10.122, URL https://www.sciencedirect.com/science/article/pii/S1877705815032233, 24th International Meshing Roundtable.
- Park, M., and Darmofal, D., “Parallel Anisotropic Tetrahedral Adaption,” 46th AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada, USA, 2008. 10.2514/6.2008-917, aIAA 2008-917.
- Sultana, N., Ruefenacht, M., Skjellum, A., Bangalore, P., Laguna, I., and Mohror, K., “Understanding the use of message passing interface in exascale proxy applications,” Concurrency and Computation: Practice and Experience, Vol. 33, 2020. 10.1002/cpe.5901.
- Klenk, B., and Fröning, H., “An Overview of MPI Characteristics of Exascale Proxy Applications,” International Conference on High Performance Computing, 2017, pp. 217–236. https://doi.org/10.1007/978-3-319-58667-0_12.
- Tsolakis, C., Chrisochoides, N., Park, M. A., Loseille, A., and Michal, T., “Parallel Anisotropic Unstructured Grid Adaptation,” AIAA Journal, Vol. 59, 2021, pp. 4764–4776.
- Tsolakis, C., Thomadakis, P., and Chrisochoides, N., “Tasking Framework for Adaptive Speculative Parallel Mesh Generation,” The Journal of Supercomputing, Vol. 78, 2022, pp. 1–32. https://doi.org/10.1007/s11227-021-04158-9.
- Tsolakis, C., “A Unified Framework for Parallel Anisotropic Mesh Adaptation,” Ph.D. thesis, Old Dominion University, 2021. 10.25777/ask4-r595.
- Ibanez, D., Barral, N., Krakos, J., Loseille, A., Michal, T., and Park, M., “First benchmark of the Unstructured Grid Adaptation Working Group,” Procedia Engineering, Vol. 203, 2017, pp. 154–166. https://doi.org/10.1016/j.proeng.2017.09.800, URL https://www.sciencedirect.com/science/article/pii/S1877705817343618, 26th International Meshing Roundtable, IMR26, 18-21 September 2017, Barcelona, Spain.
- Thomadakis, P., and Chrisochoides, N., “Toward runtime support for unstructured and dynamic exascale-era applications,” The Journal of Supercomputing, Vol. 79, 2023a, p. 9245–9272. https://doi.org/10.1007/s11227-022-05023-z.
- Zagaris, G., Pirzadeh, S., and Chrisochoides, N., “A Framework for Parallel Unstructured Grid Generation for Practical Aerodynamic Simulations,” 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition, Orlando, Florida, USA, 2013. 10.2514/6.2009-980.
- Chrisochoides, N., Chernikov, A., Kennedy, T., Tsolakis, C., and Garner, K., “Parallel Data Refinement Layer of a Telescopic Approach for Extreme-scale Parallel Mesh Generation for CFD Applications,” 2018 Aviation Technology, Integration, and Operations Conference, Atlanta, Georgia, 2018. 10.2514/6.2018-2887, aIAA 2018-2887.
- Garner, K., “Parallelization of the Advancing Front Local Reconnection Mesh Generation Software Using a Pseudo-Constrained Parallel Data Refinement Method,” Ph.D. thesis, Old Dominion University, 2020. 10.25777/appr-3169.
- Foteinos, P. A., and Chrisochoides, N. P., “High quality real-time Image-to-Mesh conversion for finite element simulations,” Journal of Parallel and Distributed Computing, Vol. 74, No. 2, 2014, pp. 2123–2140. https://doi.org/10.1016/j.jpdc.2013.11.002, URL https://www.sciencedirect.com/science/article/pii/S0743731513002232.
- Thomadakis, P., and Chrisochoides, N., “Experience with Distributed Memory Delaunay-based Image-to-Mesh Conversion Implementation,” arXiv, 2023b.
- Hoefler, T., and Träff, J., “Sparse collective operations for MPI,” 23rd IEEE International Symposium on Parallel and Distributed Processing, 2009, pp. 1–8. 10.1109/IPDPS.2009.5160935.
- Michal, T., and Krakos, J., “Anisotropic Mesh Adaptation Through Edge Primitive Operations,” 50th AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, 2012. 10.2514/6.2012-159, aIAA 2012-0159.
- Chrisochoides, N., Houstis, E., and Rice, J., “Mapping Algorithms and Software Environment for Data Parallel PDE Iterative Solvers,” Journal of Parallel and Distributed Computing, Vol. 21, No. 1, 1994, pp. 75–95. https://doi.org/10.1006/jpdc.1994.1043, URL https://www.sciencedirect.com/science/article/pii/S0743731584710434.
- Nave, D., Chrisochoides, N., and Chew, L., “Guaranteed-quality parallel Delaunay refinement for restricted polyhedral domains,” Computational Geometry, Vol. 28, No. 2, 2004, pp. 191–215. https://doi.org/10.1016/j.comgeo.2004.03.009, URL https://www.sciencedirect.com/science/article/pii/S0925772104000240, special Issue on the 18th Annual Symposium on Computational Geometry - SoCG2002.
- Loseille, A., and Alauzet, F., “Continuous Mesh Framework Part I: Well-Posed Continuous Interpolation Error,” SIAM Journal on Numerical Analysis, Vol. 49, No. 1, 2011a, pp. 38–60. 10.1137/090754078, URL https://doi.org/10.1137/090754078.
- Loseille, A., and Alauzet, F., “Continuous Mesh Framework Part II: Validations and Applications,” SIAM Journal on Numerical Analysis, Vol. 49, No. 1, 2011b, pp. 61–86. 10.1137/10078654X, URL https://doi.org/10.1137/10078654X.
- Park, M. A., Loseille, A., Krakos, J. A., and Michal, T. R., “Comparing Anisotropic Output-Based Grid Adaptation Methods by Decomposition,” 2015. AIAA 2015-2292.
- Alauzet, F., “Size gradation control of anisotropic meshes,” Finite Elements in Analysis and Design, Vol. 46, No. 1, 2010, pp. 181–202. https://doi.org/10.1016/j.finel.2009.06.028, URL https://www.sciencedirect.com/science/article/pii/S0168874X09000912, mesh Generation - Applications and Adaptation.
- Alauzet, F., and Loseille, A., “High-order sonic boom modeling based on adaptive methods,” Journal of Computational Physics, Vol. 229, No. 3, 2010, pp. 561–593. https://doi.org/10.1016/j.jcp.2009.09.020, URL https://www.sciencedirect.com/science/article/pii/S0021999109005129.
- Park, M., Balan, A., Anderson, K., Galbraith, M., Caplan, P., Carson, H., Michal, T., Krakos, J., Kamenetskiy, D., Loseille, A., Alauzet, F., Frazza, L., and Barral, N., “Verification of Unstructured Grid Adaptation Components,” AIAA Scitech 2019 Forum, 2019. 10.2514/6.2019-1723, aIAA-1723.
- Luby, M., “A Simple Parallel Algorithm for the Maximal Independent Set Problem,” Proceedings of the Seventeenth Annual ACM Symposium on Theory of Computing, Association for Computing Machinery, New York, NY, USA, 1985, p. 1–10. 10.1145/22145.22146, URL https://doi.org/10.1145/22145.22146.