RAJA Performance Suite Structure: Adding Kernels, Variants, and Tunings¶

This section describes how to add kernels, kernel groups, variants, variant sets, and tunings to the Suite. The discussion aims to make clear the organization of the code and how it works, which is useful to understand when making a contribution.

All files containing RAJA Performance Suite infrastructure and kernels reside in the src directory of the project. If you list the contents of that directory, you will see the following:

$ ls -c1 -F
lcals/
stream/
rajaperf_config.hpp.in
polybench/
common/
basic/
apps/
algorithm/
RAJAPerfSuiteDriver.cpp
CMakeLists.txt

Each directory contains files for kernels in the group associated with the directory name. For example, the lcals directory contains kernels from the LCALS benchmark suite (a precursor to RAJA Performance Suite), the stream directory contains kernels from a stream benchmark suite, and so on. The one exception is the common directory, which contains files that implement the Suite infrastructure and utilities, such as data management routines, used throughout the Suite.

The following discussion describes how to modify and add files with new content to the Suite, such as new kernels, variants, etc.

Adding a Kernel¶

Adding a kernel to the Suite involves five main steps:

Add a unique kernel ID and a unique kernel name to the Suite.
If the kernel is part of a new kernel group, add a unique kernel group ID and kernel group name.
If the kernel exercises a RAJA feature that is not currently used in the Suite, add a unique feature ID and feature name.
Implement a kernel class that defines all operations needed to integrate the kernel into the Suite. This includes adding the kernel class header file and source files that contain kernel variant implementations.
Add appropriate targets to CMakeLists.txt files, where needed, so that the new kernel code will be compiled when the Suite is built.

We describe these steps in the following sections.

Adding a kernel ID and name¶

Two key pieces of information are used to identify each kernel in the Suite: the group in which it resides and the name of the kernel. Kernel IDs and names are maintained in the files RAJAPerfSuite.hpp and RAJAPerfSuite.cpp, respectively, which reside in the src/common directory.

For concreteness, we describe how one would add the ADD kernel, which already exists in the Suite in the Stream kernel group.

First, add an enumeration value identifier for the kernel, that is unique among all Suite kernels, in the enum KernelID in the src/common/RAJAPerfSuite.hpp header file:

enum KernelID {

  ...

  //
  // Stream kernels...
  //
  Stream_ADD,
  ...

};

Second, add the kernel name to the array of strings KernelNames in the src/common/RAJAPerfSuite.cpp source file:

static const std::string KernelNames [] =
{

  ...

  //
  // Stream kernels...
  //
  std::string("Stream_ADD"),
  ...

};

Several conventions are important to note for a kernel ID and name. Following them will ensure that the kernel integrates properly into the RAJA Performance Suite machinery.

Note

The enumeration value label for a kernel is the kernel group name followed by the kernel name separated by an underscore.
Kernel ID enumeration values for kernels in the same kernel group must appear consecutively in the enumeration.
Kernel ID enumeration labels must in alphabetical order, with respect to the base kernel name in each kernel group.
The kernel string name is just a string version of the kernel ID.
The values in the KernelID enum must match the strings in the KernelNames array one-to-one and in the same order.

Typically, adding a new kernel group or Feature is not needed when adding a kernel. One or both of these needs to be added only if the kernel is not part of an existing group of kernels, or exercises a RAJA Feature that is not used in an existing kernel. For completeness, we describe the addition of a new kernel group and feature in case either is needed.

Adding a kernel group¶

If a kernel is added as part of a new group of kernels in the Suite, a new value must be added to the KernelGroupID enum in the RAJAPerfSuite.hpp header file and an associated kernel group string name must be added to the KernelGroupNames string array in the RAJAPerfSuite.cpp source file. The process is similar to adding a new kernel ID and name described above.

Note

Enumeration values and string array entries for kernel groups must be kept consistent, in the same order and matching one-to-one.

Adding a feature¶

If a kernel is added that exercises a RAJA Feature that is not used in an existing kernel, a new value must be added to the FeatureID enum in the RAJAPerfSuite.hpp header file and an associated feature string name must be added to the FeatureNames string array in the RAJAPerfSuite.cpp source file. The process is similar to adding a new kernel ID and name described above.

Note

Enumeration values and string array entries for Features must be kept consistent, in the same order and matching one-to-one.

Adding a Variant¶

Similar to a kernel, each variant in the Suite is is identified by an enumeration value and a string name. Adding a new variant requires adding these two items in a similar fashion to adding a kernel name and ID as described above.

Adding a variant to the Suite involves four main steps:

Add a unique variant ID and a unique variant name to the Suite.
If the variant is part of any new variant sets, add unique variant set ID(s) and variant set name(s).
Add the virtual method to define the variant tunings to the KernelBase class header file. For example:
```
virtual void define<variant-name>VariantTunings() {}
```
For the kernel(s) to which the variant applies, provide kernel variant implementations in associated <kernel-name>-<variant-name>.cpp files.
Add appropriate targets to CMakeLists.txt files, where needed, so that the new kernel variant code will be compiled when the Suite is built.

We describe the steps in the following sections.

Adding a variant ID and name¶

Variant IDs and names are maintained in the files RAJAPerfSuite.hpp and RAJAPerfSuite.cpp, respectively, which reside in the src/common directory. Adding a new variant ID and name is essentially the same as adding a kernel ID and name, which is described in Adding a kernel ID and name.

Note

A variant string name is just a string version of the variant ID enum value label. This convention must be followed so that each variant works properly within the RAJA Performance Suite machinery. Also, the values in the VariantID enum and the strings in the VariantNames array must be kept consistent (i.e., same order and matching one-to-one).

Adding a variant set¶

If a variant is added as part of a new set or new sets of variants in the Suite, new value(s) must be added to the VariantSetID enum in the RAJAPerfSuite.hpp header file and an associated variant set string name(s) must be added to the VariantSetNames string array in the RAJAPerfSuite.cpp source file. Adding a new variant set ID and name is essentially the same as adding a kernel set ID and name, which is described in structure_addkernel_set-label.

Note

Enumeration values and string array entries for variant sets must be kept consistent, in the same order and matching one-to-one.

Adding kernel variant implementations¶

In the classes containing kernels to which a new variant applies, add implementations for the variant in kernel execution methods in files named <kernel-name>-<variant-name>.cpp. This is described in detail in Kernel execution methods.

Adding a Tuning¶

When adding a new tuning to a kernel follow these steps.

Add implementations for the tuning in the appropriate back-end file(s). Adding them to a new tuning specific run<variant-name>Variant<tuning-name> method or in an existing run<variant-name>Variant<tune_idx> method.
Define the new tuning in the define<variant-name>VariantTunings method for the appropriate variant(s) as needed.

Note

If a tuning is added to a kernel back-end currently using one of the tuning definition boilerplate macros, then you will have to add a custom define<variant-name>VariantTunings method.