Tag Archives: Einstein toolkit

The Sixteenth Release of the Einstein Toolkit

Posted on by

We are pleased to announce the sixteenth release (code name “Tesla”) of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. The highlights of this release are:

* A new thorn, Hydro_RNSID which models a rotating neutron star.

* Tutorials have been updated and the install process for new users has been simplified.

In addition, bug fixes accumulated since the previous release in June 2017 have been included.

The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic magneto-hydrodynamics code GRHydro. For parts of the toolkit, the Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic magneto-hydrodynamics.

The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from six different institutions, and host weekly meetings that are open for anyone to join in.

Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers.

For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org.

The Einstein Toolkit is primarily supported by NSF 1550551/1550461/1550436/1550514 (Einstein Toolkit Community Integration and
Data Exploration).

The Einstein Toolkit contains about 200 regression test cases. On a large portion of the tested machines, almost all of these tests pass, using both MPI and OpenMP parallelization.

The changes between this and the previous release include:

=== Larger changes since last release ===

* The support for generic machines is more robust, and the ET should compile, run, and pass the test suites out of the box on new Linux machines.

* A Jupyter-based Tutorial (https://einsteintoolkit.org/documentation/new-user-tutorial) is now available.

* The AVX512 instruction set used on the Intel “Knight’s Landing” platform is now supported.

* PITTNullCode now has test outputs

* EOS_Omni polytrope supports hybrid equations of date with up to 10 pieces

=== New thorns or tools ===

* The Hydro_RNSID thorn which provides initial data for a rotating neutron star.

=== Upcoming changes for the next releases ===

* New thorns:

* GiRaFFE, which models plasma flows in a dynamic spacetime

* Changes to WVUThorns_Diagnostics

* Seed_Magnetic_Fields-modified: Extended Seed_Magnetic_Fields thorn for binary neutron stars. Supercedes Seed_Magnetic_Fields thorn.
* Meudon_Bin_NS-modified: Modifications to Meudon BNS initial data thorn to disable the overwriting of initial lapse/shift, which acts to significantly reduce coordinate eccentricity. Supercedes Meudon_Bin_NS thorn.
* VolumeIntegrals_GRMHD-new: Performs volume integrals on arbitrary “Swiss-cheese”-like topologies, and even interoperates with Carpet to track NS centers of mass.
* VolumeIntegrals_vacuum-new: Ensures that VI_vacuum can be used without enabling a GRMHD code.
* particle_tracerET-new: Solves the ODE D_t xi = vi for typically thousands of tracer particles, using an RK4 integration atop the current timestepping.
* smallbPoynET-new: Computes b^i, b^2, and three spatial components of Poynting flux. It also computes (-1-u0), which is useful for tracking unbound matter

=== How to upgrade from Hack (ET_2017_06) ===

To upgrade from the previous release, use GetComponents with the new thornlist to check out the new version.

See the Download page (http://einsteintoolkit.org/download/) on the Einstein Toolkit website for download instructions.

=== Machine notes ===

Supported (tested) machines include:

– Default Debian, Ubuntu, Fedora, CentOS, Mint, OpenSUSE and MacOS (Homebrew and MacPorts) installations
– Bluewaters
– Comet
– Cori
– Draco
– Edison
– Golub
– Hydra
– Marconi
– Minerva
– Queenbee 2
– Stampede 2
– SuperMIC
– Wheeler

* TACC machines: defs.local.ini needs to have sourcebasedir = $WORK and basedir = $SCRATCH/simulations configured for this machine. You need to determine $WORK and $SCRATCH by logging in to the machine.

* A new configuration for KNL nodes is being worked on, but not yet included in the release (but compilation works and tests mostly pass).

All repositories participating in this release carry a branch ET_2018_02 marking this release. These release branches will be updated if severe errors are found.

The “Tesla” Release Team on behalf of the Einstein Toolkit Consortium (2018-02-15)

Steven R. Brandt
Peter Diener
Roland Haas
Ian Hinder

Feb, 2018

North American Einstein Toolkit School and Workshop at NCSA, Urbana, Illinois

Posted on by

The North American Einstein Toolkit School and Workshop will be hosted this year that NCSA, at the University of Illinois at Urbana-Champaign from July 31 to August 4, 2017 (http://www.ncsa.illinois.edu/Conferences/ETK17/).

The Einstein Toolkit is a publicly available framework used by several numerical relativity groups in the world, with applications ranging from high-energy astrophysics to cosmology.

This meeting is open to anyone interested in numerical relativity and computational astrophysics and cosmology and in particular to Einstein toolkit users.

There will be a 3 day school from July 31 to August 2, 2017 that will introduce students and postdocs to the Einstein Toolkit and numerical methods related to it.

After that, on August 3 to August 4, 2017, the Einstein Toolkit workshop will cover the most recent developments of the toolkit, offer the possibility for collaboration and discussions about future plans.

Individual registrations for each of the school and workshop are now open on http://www.ncsa.illinois.edu/Conferences/ETK17. There, you will also find information on available hotels, hotel sharing and a tentative program. Information on financial support is available on the registration page. The initial deadline for applying for support is June 1, 2017.

Separate registration is required for each of the school and the workshop. When registering for the school you have the option of listing topics of interest for the school and your level of expertise. For the workshop you can suggest discussion topics as well as register a title and abstract for a 5 minute presentation you would like to give.

The Einstein Toolkit community in Europe will host a two day meeting in Palma de Mallorca, Spain October 11-13.

For further information please do not hesitate to contact the organizers at etk2017@ncsa.illinois.edu.

SimulationTools for Mathematica

Posted on by

15th August 2013

We present “SimulationTools for Mathematica” (http://simulationtools.org/), available as free software under the GNU General Public License. SimulationTools is a Mathematica application for analysing data from numerical simulations. It has a modular design applicable to general grid-based numerical simulations, and contains specific support for the Cactus code, with a focus on the field of Numerical Relativity and the Einstein Toolkit.

SimulationTools provides a functional, programmable interface to simulation data. A highly-optimised HDF5 module can be used for reading HDF5 data from production simulations, including 1D, 2D and 3D grid data produced by the Carpet code. Simulation details such as filenames, file formats, and details of parallel I/O are hidden from the user.

Numeric data with attached coordinate information is manipulated using new data types. Many useful new functions are defined on these types, and most built-in numerical Mathematica functions such as +, -, *, /, Abs, Sin, Log and Max can be used transparently. There is also support for testing numerical convergence, with automatic resampling onto a common grid if desired.

SimulationTools has generic functionality useful for analysis of many types of data, as well as explicit support for codes including Cactus, Carpet, Llama, SimFactory and many other components of the Einstein Toolkit. It provides an overview of the state of a simulation, including speed, memory usage, and physics (e.g. trajectories and waveforms from a binary system). The design is modular, and support for output from other codes can be added.

Specific functionality for Numerical Relativity is available. Gravitational waveforms can be read from simulations using natural function semantics, and the waveforms can be manipulated, for example converting between Psi4 and strain and extrapolation to infinity. An abstraction for “binary systems” provides a convenient interface to the trajectories of members of a binary system tracked with codes from the Einstein Toolkit. Support for reading black hole masses and spins is also included. Data in the Numerical Relativity Data Format (as used in the NINJA and NR-AR projects) can be read using the same functions that are used for normal simulation data.

More details are available on the SimulationTools website (http://simulationtools.org), including an extensive feature summary, a list of capabilities and online documentation (http://simulationtools.org/Documentation/English/Tutorials/SimulationTools.html). Tutorials and reference documentation are also available within the standard Mathematica documentation system. Code quality is maintained to a high standard with ~400 unit tests.

SimulationTools has been in production use for over 5 years and has been used at several research institutions worldwide. We invite you to try out the code (http://simulationtools.org/download), join the mailing list (http://simulationtools.org/mailman/listinfo/users) and freely use SimulationTools for your research.

—
Ian Hinder and Barry Wardell
http://numrel.aei.mpg.de/people/hinder
http://barrywardell.net/

SimulationTools for Mathematica

Posted on by

15th August 2013

We present “SimulationTools for Mathematica” <http://simulationtools.org/>, available as free software under the GNU General Public License. SimulationTools is a Mathematica application for analysing data from numerical simulations. It has a modular design applicable to general grid-based numerical simulations, and contains specific support for the Cactus code, with a focus on the field of Numerical Relativity and the Einstein Toolkit.

SimulationTools provides a functional, programmable interface to simulation data. A highly-optimised HDF5 module can be used for reading HDF5 data from production simulations, including 1D, 2D and 3D grid data produced by the Carpet code. Simulation details such as filenames, file formats, and details of parallel I/O are hidden from the user.

Numeric data with attached coordinate information is manipulated using new data types. Many useful new functions are defined on these types, and most built-in numerical Mathematica functions such as +, -, *, /, Abs, Sin, Log and Max can be used transparently. There is also support for testing numerical convergence, with automatic resampling onto a common grid if desired.

SimulationTools has generic functionality useful for analysis of many types of data, as well as explicit support for codes including Cactus, Carpet, Llama, SimFactory and many other components of the Einstein Toolkit. It provides an overview of the state of a simulation, including speed, memory usage, and physics (e.g. trajectories and waveforms from a binary system). The design is modular, and support for output from other codes can be added.

Specific functionality for Numerical Relativity is available. Gravitational waveforms can be read from simulations using natural function semantics, and the waveforms can be manipulated, for example converting between Psi4 and strain and extrapolation to infinity. An abstraction for “binary systems” provides a convenient interface to the trajectories of members of a binary system tracked with codes from the Einstein Toolkit. Support for reading black hole masses and spins is also included. Data in the Numerical Relativity Data Format (as used in the NINJA and NR-AR projects) can be read using the same functions that are used for normal simulation data.

More details are available on the SimulationTools website <http://simulationtools.org>, including an extensive feature summary, a list of capabilities and online documentation <http://simulationtools.org/Documentation/English/Tutorials/SimulationTools.html>. Tutorials and reference documentation are also available within the standard Mathematica documentation system. Code quality is maintained to a high standard with ~400 unit tests.

SimulationTools has been in production use for over 5 years and has been used at several research institutions worldwide. We invite you to try out the code  <http://simulationtools.org/download>, join the mailing list <http://simulationtools.org/mailman/listinfo/users> and freely use SimulationTools for your research.

—
Ian Hinder and Barry Wardell
http://numrel.aei.mpg.de/people/hinder
http://barrywardell.net/

Llama Multi-Block Infrastructure publicly available

Posted on by

http://llamacode.org/

The Llama Multi-Block Infrastructure for Cactus is now publicly available under the GNU General Public License. Llama provides three-dimensional multi-block capability for Cactus-based simulations that can be combined with Carpet’s adaptive mesh refinement functionality. Llama decomposes the domain into multiple (potentially overlapping) blocks with different local coordinate systems. This allows e.g. spherical domains, spherical excision, adaptive radial/angular resolution, etc., without incurring coordinate singularities.

Llama provides several patch systems suitable for single and binary objects in relativistic astrophysics, and is well integrated with the Einstein Toolkit . Llama was already used for several publications , and we believe the code is ready to be used in other projects. We are seeking volunteers to help us add tutorials and documentation, improve error messages, and generally shake down and brush up the code for a future inclusion in the Einstein Toolkit.

To aid others in getting started using Llama, we will be hosting a virtual workshop where we provide an overview of the code and answer questions. Details will be announced shortly.

Llama constitutes the fruit of a significant effort of several people over several years. We make Llama public to help modernize the computational tools used in our community, and in the hope to boost Llama itself by inviting contributions from everybody. We ask you to acknowledge our effort by following the citation guidelines described on .

The Llama groomers:

R. Haas, I. Hinder, D. Pollney, C. Reisswig, E. Schnetter, B. Wardell

Einstein Toolkit Release

Posted on by

We are pleased to announce the fifth release (code name Lovelace (http://en.wikipedia.org/wiki/Ada_Lovelace) of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release includes beginning support for OpenCL (disabled by default). In addition, bug fixes accumulated since the previous release in October 2011 have been included.

The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and the relativistic hydrodynamics code GRHydro (an updated and extended version of the public release of the Whisky code). The Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic hydrodynamics.

The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from five different institutions, and host weekly meetings that are open for anyone to join in.

Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers.

For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages at http://einsteintoolkit.org.

The Einstein Toolkit is primarily supported by NSF 0903973/0903782/0904015 (CIGR), and also by NSF 0701566/0855892 (XiRel), 0721915 (Alpaca), 0905046/0941653 (PetaCactus), and 0710874 (LONI Grid).

The Einstein Toolkit contain over 170 regression test cases. On a large portion of the tested machines, all of these testsuites pass, using both MPI and OpenMP parallelization.

The changes between this and the previous release include:

– Accelerator Support

This release of the Einstein Toolkit adds support for GPUs and other accelerators. This support comprises three levels of abstraction, ranging from merely building and running both CUDA and OpenCL code, to automated code generation targeting GPUs instead of CPUs. As with any other programming paradigm (such as MPI or OpenMP), the performance benefits depend on the particular algorithms used and optimizations that are applied. In addition, the Simulation Factory greatly aids portability to a wide range of computing systems.

At the lowest level, Cactus now supports compiling, building, and running with either CUDA or OpenCL. CUDA is supported as new language in addition to C, C++, and Fortran; OpenCL is supported as an external library, and builds and executes compute kernels via run-time calls. Details are described in the user’s guide (for CUDA) and in thorn ExternalLibraries/OpenCL (for OpenCL).

Many accelerator platforms today separate between host memory and device memory, and require explicit copy or map operations to transfer data. An intermediate level of abstraction aids transferring grid variables between host and device, using schedule declarations to keep track of which data are needed where, and minimizing expensive data transfers. For OpenCL, there is a compact API to build and execute compute kernels at run time. Details are described in thorns CactusUtils/Accelerator and CactusUtils/OpenCLRunTime (with example parameter file).

Finally, the code generation system Kranc has been extended to be able to produce either C++ or OpenCL code, based on the infrastructure described above. This allows writing GPU code in a very high-level manner. However, it needs to be stated that the efficiency of the generated code depends on many variables, including e.g. the finite differencing stencil radius and the number of operations in the generated compute kernels. Non-trivial kernels typically require system-dependent tuning to execute efficiently, as GPUs and other accelerators generally show a rather unforgiving performance behavior. The thorns McLachlan/ML_WaveToy and McLachlan/ML_WaveToy_CL are examples, generated from the same Kranc script, showing the generated C++ and OpenCL code.

– SimFactory
– Machine database and optionlists updated due to system changes on HPC resources
– Simfactory’s capability of running the testsuites is properly tested on a lot of systems.
– IOUtil: checkpoint_dir is now steerable
– SphericalSurface: added functionality to name spherical surfaces
– Formaline: Support a “local repository” that collects all machine-local repositories
– TimerReport: Allow different timers on different processes
– WeylScal4: Enable use of LoopControl, and hence OpenMP
– EOS_Omni: use C interface for HDF5 to avoid needing Fortran HDF5 bindings
– EOSG_*: Support for the so-called ‘general EOS interface’ has been dropped from the Einstein Toolkit
– A new arrangement EinsteinExact has been added to the toolkit, providing a wide range of exact initial data, which will eventually replace the ‘Exact’ thorn.
– The *_O2 versions of McLachlan have been removed from the toolkit. This functionality is already provided by the regular McLachlan thorns now.
– A new thorn ADMMass has been added to the Einstein Toolkit, which can calculate approximations of the ADM mass using a finite surface or volume integral.
– The old library mechanism in Cactus (e.g. HDF5=yes) is now deprecated. Expect it to be removed in one of the next releases.
– The thorns ADM and LegoExcision are deprecated and will be removed in one of the next releases.
– GRHydro:
– use atmosphere integer mask instead of bitmask
– remove (now) unused old Tmunu interface
– Implemented enhanced PPM scheme by Colella & Sekora 2008, McCorquodale & Colella 2011. Can be activated by setting
use_enhanced_ppm = yes
– External Libraries: several updates and configuration improvements
– Cactus
– implement per-variable tolerances for Cactus testsuites, for long discussion, see ET ticket #114
– Allow arithmetic expression in ParameterSet: parameter files can now contain a limited set of expressions
– Handles requirements recursively
– A lot of smaller bug fixes
– McLachlan: Implement CCZ4 formulation
– CarpetMask: Keep track of the volume that is masked out
– CarpetLib: Define MPI reduction operators for complex numbers
– CarpetIOASCII: Add new “compact” output format
– Csrpet: Support accelerator data transfer
– CarpetRegrid2: Add periodic boundary conditions
– Simfactory
– Use OpenMP by default
– Make running testsuites using Simfactory possible
– Updated a lot of configurations

All repositories participating in this release carry a branch ET_2012_05 marking this release. These release branches will be updated if severe errors are found.

For more detailed information about the “Lovelace” release please read the long release announcement on the Einstein Toolkit web pages: http://einsteintoolkit.org/about/releases/ET_2012_05_announcement.php.

On behalf of the Einstein Toolkit Consortium: the “Lovelace” Release Team

Eloisa Bentivegna
Tanja Bode
Peter Diener
Roland Haas
Ian Hinder
Frank Löffler
Bruno Mundim
Christian D. Ott
Erik Schnetter

May 28, 2012

Einstein Toolkit New Users Workshop in Atlanta

Posted on by

The Einstein Toolkit (http://einsteintoolkit.org) will host its spring workshop 2012 following the April APS meeting in Atlanta, GA, from Tuesday, April 3rd, 4pm to Friday April 6th, noon at the Georgia Institute of Technology.

This workshop is targeted at new and potential new users of the relativity infrastructure. It will provide a general introductions into numerical relativity (although some previous knowledge would be beneficial) and in code development within large collaborations. Hands-on sessions will help to familiarize attendees with the Einstein Toolkit. Participants are asked to bring their own laptops. We would like to invite especially students from physics and computer science to participate.

The number of attendees is limited, and while registration is free, it is required. We anticipate to be able to support a small number of participants financially, by covering parts/all of their travel, hotel and meal cost. Preference will be given to students.

In order register, write an email to workshop[AT]einsteintoolkit.org and specify:

* your name, affiliation and title
* your estimated arrival and departure time/date
* whether you apply for support (and if so, state if you are undergraduate / graduate student / postdoc
* your special needs

Detailed workshop information can be found at
https://docs.einsteintoolkit.org/et-docs/ET_Workshop_Spring_2012.

The Einstein Toolkit Consortium.

Einstein Toolkit Release

Posted on by

We are pleased to announce the second release (code name “Chandrasekhar”) of the Einstein Toolkit, an open, community developed software infrastructure for relativistic astrophysics. This release is mainly a maintenance release incorporating fixes accumulated since the previous release in June 2010, as well as additional test suites.

The Einstein Toolkit is a collection of software components and tools for simulating and analyzing general relativistic astrophysical systems that builds on numerous software efforts in the numerical relativity community including CactusEinstein, the Carpet AMR infrastructure and on the public version of the Whisky hydrodynamics code (now modified and called GRHydro). The Cactus Framework is used as the underlying computational infrastructure providing large-scale parallelization, general computational components, and a model for collaborative, portable code development. The toolkit includes modules to build complete codes for simulating black hole spacetimes as well as systems governed by relativistic hydrodynamics. Current development in the consortium is targeted at providing additional infrastructure for general relativistic magnetohydrodynamics.

The Einstein Toolkit uses a distributed software model and its different modules are developed, distributed, and supported either by the core team of Einstein Toolkit Maintainers, or by individual groups. Where modules are provided by external groups, the Einstein Toolkit Maintainers provide quality control for modules for inclusion in the toolkit and help coordinate support. The Einstein Toolkit Maintainers currently involve postdocs and faculty from five different institutions, and host weekly meetings that are open for anyone to join in.

Guiding principles for the design and implementation of the toolkit include: open, community-driven software development; well thought out and stable interfaces; separation of physics software from computational science infrastructure; provision of complete working production code; training and education for a new generation of researchers.

For more information about using or contributing to the Einstein Toolkit, or to join the Einstein Toolkit Consortium, please visit our web pages http://einsteintoolkit.org.

The Einstein Toolkit is primarily supported by NSF 0903973/0903782/0904015 (CIGR), and also by NSF 0701566/0855892 (XiRel), 0721915 (Alpaca), 0905046/0941653 (PetaCactus) and 0710874 (LONI Grid).

The “Chandrasekhar” Release Team on behalf of the Einstein Toolkit Consortium (2010-11-23)