Project

General

Profile

Feature #985

Make AMPI_Gather and its variants use Tuple/Set Reductions

Added by Sam White over 3 years ago. Updated about 3 years ago.

Status:
Merged
Priority:
Normal
Assignee:
Category:
AMPI
Target version:
Start date:
02/17/2016
Due date:
% Done:

100%


Description

AMPI_Gather, AMPI_Scatter, and all of their variants currently use point-to-point messages internally.
This will be a bottleneck at scale, and needs to be changed to use reductions and broadcasts instead.


Related issues

Related to Charm++ - Feature #982: Gather reduction operation using Tuple/Set reducers Merged 07/17/2016
Related to Charm++ - Bug #1084: Eliminate extra copies in AMPI reduce/gather(v) receive paths Merged 06/19/2016

History

#1 Updated by Sam White about 3 years ago

  • Tracker changed from Bug to Feature

#2 Updated by Sam White about 3 years ago

  • Status changed from New to In Progress

AMPI_{Gather, Gatherv, Allgather, Allgatherv, Igather, Igatherv, Iallgather, Iallgatherv} can all be implemented using tuple+set reducers now, since feature #982.

AMPI_{Scatter, Scatterv, Iscatter, Iscatterv, Reduce_scatter, Reduce_scatter_block, Ireduce_scatter, Ireduce_scatter_block} could use broadcasts, but that will not scale in terms of memory. Charm should have a scatter operation that is Node/PE-aware, so we can probably wait for that to change AMPI's implementation.

#3 Updated by Sam White about 3 years ago

  • % Done changed from 0 to 50

Implemented all gather operations using tuple/set reductions here: https://charm.cs.illinois.edu/gerrit/#/c/1220/

An added benefit of this I didn't think about earlier is that besides using a spanning tree, using the reduction also allows us to only have to wait on a single non-blocking request, as opposed to one for every contributor in the gather, decreasing message matching overhead.

The only uses of IATAReq's are now in AMPI_Ialltoall, AMPI_Ialltoallv, AMPI_Ialltoallw, and the nonblocking neighborhood collectives.

#4 Updated by Sam White about 3 years ago

  • Subject changed from AMPI Gather/Scatter should use Reduction/Broadcast to Make AMPI_Gather and its variants use Tuple/Set Reductions
  • Status changed from In Progress to Implemented
  • % Done changed from 50 to 100

Since the scatter case really needs an underlying Charm++ 'scatter' method, which currently doesn't exist, I'm narrowing the scope of this issue. Another one can be opened when scatter is seen to be a bottleneck for some application or when Charm++ gets a scatter operation that AMPI can use.

#5 Updated by Sam White about 3 years ago

  • Status changed from Implemented to Merged
  • translation missing: en.field_closed_date set to 2016-06-07 15:57:16.460909

Also available in: Atom PDF