3. Basic Syntax

3.1 Entry Methods

Member functions in the user program which function as entry methods have to be defined in public scope within the class definition. Entry methods typically do not return data and have a ``void'' return type. An entry method with the same name as its enclosing class is a constructor entry method and is used to create or spawn chare objects during execution. Class member functions are annotated as entry methods by declaring them in the interface file as:

entry void Entry1(parameters);

Parameters is either a list of serializable parameters, (e.g., ``int i, double x''), or a message type (e.g., ``MyMessage *msg''). Since parameters get marshalled into a message before being sent across the network, in this manual we use ``message'' to mean either a message type or a set of marshalled parameters.Messages are lower level, more efficient, more flexible to use than parameter marshalling.

For example, a chare could have this entry method declaration in the interface (.ci) file:

  entry void foo(int i,int k);

Then invoking foo(2,3) on the chare proxy will eventually invoke foo(2,3) on the chare object.

Since Charm++ runs on distributed memory machines, we cannot pass an array via a pointer in the usual C++ way. Instead, we must specify the length of the array in the interface file, as:

  entry void bar(int n,double arr[n]);

Since C++ does not recognize this syntax, the array data must be passed to the chare proxy as a simple pointer. The array data will be copied and sent to the destination processor, where the chare will receive the copy via a simple pointer again. The remote copy of the data will be kept until the remote method returns, when it will be freed. This means any modifications made locally after the call will not be seen by the remote chare; and the remote chare's modifications will be lost after the remote method returns- Charm++ always uses call-by-value, even for arrays and structures.

This also means the data must be copied on the sending side, and to be kept must be copied again at the receive side. Especially for large arrays, this is less efficient than messages, as described in the next section.

Array parameters and other parameters can be combined in arbitrary ways, as:

  entry void doLine(float data[n],int n);
  entry void doPlane(float data[n*n],int n);
  entry void doSpace(int n,int m,int o,float data[n*m*o]);
  entry void doGeneral(int nd,int dims[nd],float data[product(dims,nd)]);

The array length expression between the square brackets can be any valid C++ expression, including a fixed constant, and may depend in any manner on any of the passed parameters or even on global functions or global data. The array length expression is evaluated exactly once per invocation, on the sending side only. Thus executing the doGeneral method above will invoke the (user-defined) product function exactly once on the sending processor. Marshalling User-Defined Structures and Classes

The marshalling system uses the pup framework to copy data, meaning every user class that is marshalled needs either a pup routine, a ``PUPbytes'' declaration, or a working operator|. See the PUP description in Section 6 for more details on these routines.

Any user-defined types in the argument list must be declared before including the ``.decl.h'' file. Any user-defined types must be fully defined before the entry method declaration that consumes it. This is typically done by including the header defining the type in the .ci file. Alternatively, one may define it before including the .decl.h file. As usual in C++, it is often dramatically more efficient to pass a large structure by reference than by value.

As an example, refer to the following code from examples/charm++/PUP/HeapPUP:

// In HeapObject.h:

class HeapObject {
  int publicInt;

  // ... other methods ...

  void pup(PUP::er &p) {
    // remember to pup your superclass if there is one
    if (p.isUnpacking())
      data = new float[publicInt];
    PUParray(p, data, publicInt);

  bool privateBool;
  float *data;

// In

mainmodule SimplePUP {
  include "HeapObject.h";

  // ... other Chare declarations ...

  array [1D] SimpleArray{
    entry SimpleArray();
    entry void acceptData(HeapObject &inData);

// In SimplePup.h:

#include "SimplePUP.decl.h"

// ... other definitions ...

class SimpleArray : public CBase_SimpleArray {
  void acceptData(HeapObject &inData) {
    // ... code using marshalled parameter ...

// In SimplePup.C:

#include "SimplePUP.h"

main::main(CkArgMsg *m)
  // normal object construction
  HeapObject exampleObject(... parameters ...);

  // normal chare array construction
  CProxy_SimpleArray simpleProxy = CProxy_SimpleArray::ckNew(30);

  // pass object to remote method invocation on the chare array

#include "SimplePUP.def.h"

3.2 Chare Objects

Chares are concurrent objects with methods that can be invoked remotely. These methods are known as entry methods. All chares must have a constructor that is an entry method, and may have any number of other entry methods. All chare classes and their entry methods are declared in the interface (.ci) file:

    chare ChareType
        entry ChareType(parameters1);
        entry void EntryMethodName(parameters2);

Although it is declared in an interface file, a chare is a C++ object and must have a normal C++ implementation (definition) in addition. A chare class ChareType must inherit from the class CBase_ChareType, which is a special class that is generated by the Charm++ translator from the interface file. Note that C++ namespace constructs can be used in the interface file, as demonstrated in examples/charm++/namespace. To be concrete, the C++ definition of the chare above might have the following definition in a .h file:

   class ChareType : public CBase_ChareType {
       // Data and member functions as in C++
           void EntryMethodName2(parameters2);

Each chare encapsulates data associated with medium-grained units of work in a parallel application. Chares can be dynamically created on any processor; there may be thousands of chares on a processor. The location of a chare is usually determined by the dynamic load balancing strategy. However, once a chare commences execution on a processor, it does not migrate to other processors3.1. Chares do not have a default ``thread of control'': the entry methods in a chare execute in a message driven fashion upon the arrival of a message3.2.

The entry method definition specifies a function that is executed without interruption when a message is received and scheduled for processing. Only one message per chare is processed at a time. Entry methods are defined exactly as normal C++ function members, except that they must have the return value void (except for the constructor entry method which may not have a return value, and for a synchronous entry method, which is invoked by a threaded method in a remote chare). Each entry method can either take no arguments, take a list of arguments that the runtime system can automatically pack into a message and send (see section 3.1), or take a single argument that is a pointer to a Charm++ message (see section 10.1).

A chare's entry methods can be invoked via proxies (see section 1.3). Proxies to a chare of type chareType have type CProxy_chareType. By inheriting from the CBase parent class, each chare gets a thisProxy member variable, which holds a proxy to itself. This proxy can be sent to other chares, allowing them to invoke entry methods on this chare.

3.2.1 Chare Creation

Once you have declared and defined a chare class, you will want to create some chare objects to use. Chares are created by the ckNew method, which is a static method of the chare's proxy class:

   CProxy_chareType::ckNew(parameters, int destPE);

The parameters correspond to the parameters of the chare's constructor. Even if the constructor takes several arguments, all of the arguments should be passed in order to ckNew. If the constructor takes no arguments, the parameters are omitted. By default, the new chare's location is determined by the runtime system. However, this can be overridden by passing a value for destPE, which specifies the PE where the chare will be created.

The chare creation method deposits the seed for a chare in a pool of seeds and returns immediately. The chare will be created later on some processor, as determined by the dynamic load balancing strategy (or by destPE). When a chare is created, it is initialized by calling its constructor entry method with the parameters specified by ckNew.

Suppose we have declared a chare class C with a constructor that takes two arguments, an int and a double.

  1. This will create a new chare of type C on any processor and return a proxy to that chare:

       CProxy_C chareProxy = CProxy_C::ckNew(1, 10.0);

  2. This will create a new chare of type C on processor destPE and return a proxy to that chare:

       CProxy_C chareProxy = CProxy_C::ckNew(1, 10.0, destPE);

For an example of chare creation in a full application, see examples/charm++/fib in the Charm++ software distribution, which calculates Fibonacci numbers in parallel.

3.2.2 Method Invocation on Chares

A message may be sent to a chare through a proxy object using the notation:


This invokes the entry method EntryMethod on the chare referred to by the proxy chareProxy. This call is asynchronous and non-blocking; it returns immediately after sending the message.

3.2.3 Local Access

You can get direct access to a local chare using the proxy's ckLocal method, which returns an ordinary C++ pointer to the chare if it exists on the local processor, and NULL otherwise.

    C *c=chareProxy.ckLocal();
    if (c==NULL) 
        // object is remote; send message
        // object is local; directly use members and methods of c

3.3 Read-only Data

Since Charm++ does not allow global variables, it provides a special mechanism for sharing data amongst all objects. Read-only variables of simple data types or compound data types including messages and arrays are used to share information that is obtained only after the program begins execution and does not change after they are initialized in the dynamic scope of the main function of the mainchare. They are broadcast to every Charm++ Node (process) by the Charm++ runtime, and can be accessed in the same way as C++ ``global'' variables on any process. Compound data structures containing pointers can be made available as read-only variables using read-only messages(see section 10.1) or read-only arrays(see section 4. Note that memory has to be allocated for read-only messages by using new to create the message in the main function of the mainchare. Read-only variables are declared by using the type modifier readonly, which is similar to const in C++. Read-only data is specified in the .ci file (the interface file) as:

 readonly Type ReadonlyVarName;

The variable ReadonlyVarName is declared to be a read-only variable of type Type. Type must be a single token and not a type expression.

 readonly message MessageType *ReadonlyMsgName;

The variable ReadonlyMsgName is declared to be a read-only message of type MessageType. Pointers are not allowed to be readonly variables unless they are pointers to message types. In this case, the message will be initialized on every PE.

 readonly Type ReadonlyArrayName [arraysize];

The variable ReadonlyArrayName is declared to be a read-only array of type Type with arraysize elements. Type must be a single token and not a type expression. The value of arraysize must be known at compile time.

Read-only variables must be declared either as global or as public class static data in the C/C++ implementation files, and these declarations have the usual form:

 Type ReadonlyVarName;
 MessageType *ReadonlyMsgName;
 Type ReadonlyArrayName [arraysize];

Similar declarations preceded by extern would appear in the .h file.

Note: The current Charm++ translator cannot prevent assignments to read-only variables. The user must make sure that no assignments occur in the program outside of the mainchare constructor.

For concrete examples for using read-only variables, please refer to examples such as examples/charm++/array and examples/charm++/gaussSeidel3D.

Users can get the same functionality of readonly variables by making such variables members of Charm++ Node Group objects and constructing the Node Group in the mainchare's main routine.