Avoid placing extra PME load on first node 47/4147/1
authorJim Phillips <jim@ks.uiuc.edu>
Tue, 8 May 2018 20:27:15 +0000 (15:27 -0500)
committerJim Phillips <jim@ks.uiuc.edu>
Tue, 8 May 2018 20:27:15 +0000 (15:27 -0500)
At larger PE counts patches are excluded from both PE 0 and PE 1
to avoid slowing down any global integrator bottlenecks.  Since
PME work is preferentially placed on PEs without patches, extra
PME work could be placed on the first node.  This patch treats
PE 1 as having patches for the purpose of assigning PME work.

Change-Id: Ib8eac51d65f5ffe83b60a47ffbc4c97a03f8be54

src/ComputePme.C

index f4e58e3..bd24465 100644 (file)
@@ -1175,7 +1175,8 @@ void ComputePmeMgr::initialize(CkQdMsg *msg) {
                for ( int icpu=0; icpu<ncpus; ++icpu ) {
                        int ri = WorkDistrib::peDiffuseOrdering[icpu];
                        if ( ri ) { // keep 0 for special case
-                               if ( pmap->numPatchesOnNode(ri) ) patches.add(ri);
+                               // pretend pe 1 has patches to avoid placing extra PME load on node
+                               if ( ri == 1 || pmap->numPatchesOnNode(ri) ) patches.add(ri);
                                else nopatches.add(ri);
                        }
                }