Project

General

Profile

Bug #1440

smp pes sending messages still block due to other send activity

Added by Jim Phillips 3 months ago. Updated 3 months ago.

Status:
New
Priority:
High
Category:
Machine Layers
Target version:
-
Start date:
02/20/2017
Due date:
% Done:

0%


Description

Despite #642 being merged I still see pes that are sending messages blocked waiting on some shared resource in smp builds.
Attached is a communication torture test based on PME in NAMD that illustrates the problem quite clearly.
It is desirable for the sending pe to do anything that the communication thread does not absolutely need to do itself, but it shouldn't get stuck doing nothing.
Note that in the attached timeline screenshot the pink entries all send the same number and size of messages.

trans.tar.gz - transpose communication benchmark (1.31 KB) Jim Phillips, 02/20/2017 10:13 AM

TimelineScreenshot.png View - timeline screenshot (146 KB) Jim Phillips, 02/20/2017 10:23 AM

History

#1 Updated by Jim Phillips 3 months ago

On further research it appears that verbs/net-ibverbs only uses the comm thread for receiving so there is clearly some sort of send contention but possibly not in the Charm++ code.

#2 Updated by Eric Bohm 3 months ago

  • Assignee set to Bilge Acun

#3 Updated by Bilge Acun 3 months ago

  • Assignee changed from Bilge Acun to Karthik Senthil

Re-assigning it to Karthik since he's going to do the projections/performance tests.

Also available in: Atom PDF