The CECAM electronic structure library and the modular software development paradigm
First-principles electronic structure calculations are now accessible to a very large community of users across many disciplines, thanks to many successful software packages, some of which are described in this special issue. The traditional coding paradigm for such packages is monolithic, i.e., regardless of how modular its internal structure may be, the code is built independently from others, essentially from the compiler up, possibly with the exception of linear-algebra and message-passing libraries. This model has endured and been quite successful for decades. The successful evolution of the electronic structure methodology itself, however, has resulted in an increasing complexity and an ever longer list of features expected within all software packages, which implies a growing amount of replication between different packages, not only in the initial coding but, more importantly, every time a code needs to be re-engineered to adapt to the evolution of computer hardware architecture. The Electronic Structure Library (ESL) was initiated by CECAM (the European Centre for Atomic and Molecular Calculations) to catalyze a paradigm shift away from the monolithic model and promote modularization, with the ambition to extract common tasks from electronic structure codes and redesign them as open-source libraries available to everybody. Such libraries include "heavy-duty" ones that have the potential for a high degree of parallelization and adaptation to novel hardware within them, thereby separating the sophisticated computer science aspects of performance optimization and re-engineering from the computational science done by, e.g., physicists and chemists when implementing new ideas. We envisage that this modular paradigm will improve overall coding efficiency and enable specialists (whether they be computer scientists or computational scientists) to use their skills more effectively and will lead to a more dynamic evolution of software in the community as well as lower barriers to entry for new developers. The model comes with new challenges, though. The building and compilation of a code based on many interdependent libraries (and their versions) is a much more complex task than that of a code delivered in a single self-contained package. Here, we describe the state of the ESL, the different libraries it now contains, the short- and mid-term plans for further libraries, and the way the new challenges are faced. The ESL is a community initiative into which several pre-existing codes and their developers have contributed with their software and efforts, from which several codes are already benefiting, and which remains open to the community ; The authors would like to thank CECAM for launching and pushing the ESL, as well as hosting part of its infrastructure, and partly funding the extended workshops where most of the coding was done, both in the Lausanne headquarters and in the Dublin, Trieste, and Zaragoza nodes. Within CECAM, the authors particularly thank Sara Bonella, Bogdan Nichita, and Ignacio Pagonabarraga. The authors also acknowledge all the people who have supported and contributed to the ESL in different ways, including Luis Agapito, Xavier Andrade, Balint Aradi, Emanuele Bosoni, Lori A. Burns, Christian Carbogno, Ivan Carnimeo, Abel Carreras Conill, Alberto Castro, Michele Ceriotti, Anoop Chandran, Wibe de Jong, Pietro Delugas, Thierry Deutsch, Hubert Ebert, Aleksandr Fonari, Luca Ghiringhelli, Paolo Giannozzi, Matteo Giantomassi, Judit Gimenez, Ivan Girotto, Xavier Gonze, Benjamin Hourahine, Jürg Hutter, Thomas Keal, Jan Kloppenburg, Hyungjun Lee, Liang Liang, Lin Lin, Jianfeng Lu, Nicola Marzari, Donal MacKernan, Layla Martin-Samos, Paolo Medeiros, Fawzi Mohamed, Jens Jørgen Mortensen, Sebastian Ohlmann, David O'Regan, Charles Patterson, Etienne Plésiat, Markus Rampp, Laura Ratcliff, Stefano Sanvito, Paul Saxe, Matthias Scheffler, Didier Sebilleau, Søren Smidstrup, James Spencer, Atsushi Togo, Joost Vandevondele, Matthieu Verstraete, and Brian Wylie. The authors would also like to thank the Psi-k network for having partially funded several of the ESL workshops. A.O., E.A., D.L.-D., S.G., E.K., A.A.M., and M.C.P. received funding from the European Union's Horizon 2020 research and innovation program under Grant Agreement No. 676531 (Centre of Excellence project E-CAM). The same project has partly funded the extended software development workshops in which most of the ESL coding effort has happened. A.G., S.M., and E.A. acknowledge support from the European Union's Horizon 2020 research and innovation program under Grant Agreement No. 824143 (Centre of Excellence project MaX). M.A.L.M. acknowledges partial support from the DFG through Project No. MA-6786/1. D.G.A.S. was supported by the U.S. National Science Foundation (NSF) (Grant No. ACI-1547580). M.C.P. acknowledges support from the EPSRC under Grant No. EP/P034616/1. A.A.M. acknowledges support from the Thomas Young Centre under Grant No. TYC-101, the Wannier Developers Group, and all of the authors and contributors of the wannier90 code (see Ref. 115 for a complete list). A.M.E. acknowledges support from CoSeC, the Computational Science Centre for Research Communities, through CCP5: The Computer Simulation of Condensed Phases (EPSRC Grant Nos. EP/M022617/1 and EP/P022308/1). A.G. and J.M.S. acknowledge Spain's Ministry of Science (Grant No. PGC2018-096955-B-C42). E.A., A.G., and J.M.S. acknowledge Spain's Ministry of Science (Grant No. FIS2015-64886-C5). Y.P., D.L.-D., and E.A. acknowledge support from the Spanish MINECO and EU Structural Investment Funds (Grant No. RTC-2016-5681-7). M.L. acknowledges support from the EPRSC under Grant No. EP/M022668/1. M.L., M.J.T.O., and Y.P. acknowledge support from the EU COST action (Grant No. MP1306). J.M. was supported by the European Regional Development Fund (ERDF), project CEDAMNF (Reg. No. CZ.02.1.01/0.0/0.0/15-003/0000358). V.W.-Z.Y., W.P.H., Y.L., and V.B. acknowledge support from the National Science Foundation under Award No. ACI-1450280 (the ELSI project). V.W.-Z.Y. also acknowledges a MolSSI fellowship (NSF Award No. ACI-1547580). Simune Atomistics S.L. is thanked for allowing A.H.L. and Y.P. to contribute to the ESL, as is Synopsys, Inc., for the partial availability of F.C ; Peer Reviewed ; Postprint (author's final draft)