Project

General

Profile

Feature #69

Get large messages sent via the ZC API to be received directly using a GET from the new home. (rather than a GET from the old home and forwarding the large message)

Added by Phil Miller about 6 years ago. Updated 14 days ago.

Status:
New
Priority:
Normal
Assignee:
Category:
-
Start date:
03/01/2013
Due date:
% Done:

0%

Tags:

Description

On systems that communicate large messages via RDMA GET operations, it takes the latency of one fewer packet across the network (4 vs 5) to convey a message to a migrated object if the sender passes a GET handle to the home PE, which then informs the recipient PE directly about where to get the message in addition to telling the sender where that object lives for future sends. Where available, pass a GET handle along with the location request to enable this.


Related issues

Blocked by Charm++ - Feature #68: LRTS support for setting up a message to send and transmitting a GET handle Closed 03/01/2013
Blocked by Charm++ - Feature #67: Reduce the impact of large messages sent to unlocated or migrated chare array elements Merged 03/01/2013

History

#1 Updated by Phil Miller about 6 years ago

  • Target version set to 6.6.0

#2 Updated by Phil Miller over 5 years ago

  • Target version changed from 6.6.0 to 6.7.0

#3 Updated by Nikhil Jain over 3 years ago

  • Target version changed from 6.7.0 to 6.8.0

#4 Updated by Phil Miller over 2 years ago

Nitin, Vipul: Your recent work on RDMA support probably makes this and #68 easy to implement as well.

#5 Updated by Eric Bohm about 2 years ago

  • Target version changed from 6.8.0 to 6.9.0

#6 Updated by Sam White about 2 years ago

  • Tags set to #rdma

#7 Updated by Eric Bohm over 1 year ago

  • Target version changed from 6.9.0 to Unscheduled

#8 Updated by Sam White about 1 year ago

  • Target version changed from Unscheduled to 6.9.1
  • Assignee set to Nitin Bhat

#9 Updated by Sam White 3 months ago

  • Target version changed from 6.9.1 to 6.10.0

#10 Updated by Nitin Bhat about 1 month ago

I think Phil is trying to basically describe a “zero-copy” based transfer scheme for a large message (whose target is a migrated object) from the original object location (PE) to the new object location (PE). Is that right?
I'm guessing that we currently use a copy-based scheme for forwarding the message from the original PE to the new PE?

#11 Updated by Nitin Bhat 21 days ago

Juan and I had a discussion about this feature on slack.

We finally understood this feature to some extent.

Let's say a chare array element 'c' migrates from PE 'y' to PE 'z'. Currently, if another chare array element 'b' on PE 'x' wants to send a large message to 'c', it sends the message to PE 'y' and then PE 'y' forwards it to PE 'z'. With this feature, chare array element 'b' on PE 'x' will not send the entire large message to PE 'y', it will only send the buffer information (GET handle) to PE 'y' and then PE 'y' will forward the buffer information (GET handle) to PE 'z'. Following that, PE 'z' will perform a GET operation to get the data directly from PE 'x'. This ends up saving the latency of one hop for the large message in addition to avoiding the memory allocation on PE 'y'.

#12 Updated by Nitin Bhat 21 days ago

  • Subject changed from Get large messages directed at migrated objects directly from the sender to Get large messages sent via the ZC API to be received directly using a GET from the new home. (rather than a GET from the old home and forwarding the large message)

This feature is only relevant for use cases of the Zero copy Entry Method API and not the regular API.

This API basically impacts use cases where a sender (PE 'x') sends a GET handle to the original home (PE 'y') and instead of the original home rgetting the entire message onto PE 'y' and then forwarding the large message to PE 'z' (the new home of the migrated chare), with this feature, PE 'y' just forwards the GET handle and PE 'z' performs the Rget.

#13 Updated by Nitin Bhat 14 days ago

  • Target version changed from 6.10.0 to 7 (Next Generation Charm++)

Since this impacts a very small use case (where ZC API is used in conjunction with load balancing) and the impact is potentially small (only the iteration after load balancing), I am not including this in 6.10.

Also available in: Atom PDF