Project

General

Profile

Bug #1852

verbs-linux-x86_64 failure in tests/charm++/pingpong with +p2 after direct API merge

Added by Sam White over 1 year ago. Updated over 1 year ago.

Status:
Merged
Priority:
High
Assignee:
Category:
Machine Layers
Target version:
Start date:
04/08/2018
Due date:
% Done:

0%


Description

It seems to be failing autobuild consistently here since the direct API verbs patch was merged. The next case in this pingpong test if for NodeGroup's using RDMA.

Inter-processor Pingpong..
../../../bin/testrun  ./pgm +p2  ++timeout 180 +isomalloc_sync ++nodelist /home/jchoi157/autobuild/verbs/7061582.cc-mgmt1.campuscluster.illinois.edu.nodelist
Charmrun> scalable start enabled. 
Charmrun> IBVERBS version of charmrun
Charmrun> started all node programs in 1.314 seconds.
Charm++> Running in non-SMP mode: numPes 2
Converse/Charm++ Commit ID: 2a5b279
Charm++> Synchronizing isomalloc memory region...
Charm++> Consolidated Isomalloc memory region: 0x2b9d80000000 - 0x7ff8d0000000 (88454400 MB).
Charm++> scheduler running in netpoll mode.
CharmLB> Load balancer assumes all CPUs are same.
Charm++> Running on 2 unique compute nodes (24-way SMP).
Charm++> cpu topology info is gathered in 0.002 seconds.
Pingpong with payload: 100 iterations: 1000
Roundtrip time for 1D Arrays is 6.278038 us
Roundtrip time for 1D threaded Arrays is 6.905079 us
Roundtrip time for 1D Arrays (zero copy message send api) is 213.277817 us
Roundtrip time for 1D Arrays Marshalled is 6.499052 us
Roundtrip time for 2D Arrays is 6.218910 us
Roundtrip time for 3D Arrays is 6.203175 us
Roundtrip time for Fancy Arrays is 6.232977 us
Roundtrip time for Chares (reuse msgs) is 6.186008 us
Roundtrip time for Chares (new/del msgs) is 6.193876 us
Roundtrip time for threaded Chares (reuse) is 6.616116 us
Roundtrip time for Chares (zero copy message send api) is 213.307142 us
Roundtrip time for Groups is 6.163120 us
Roundtrip time for Groups (zero copy message send api) is 213.340044 us
Roundtrip time for Groups (1 KB pipe, no memcpy, no allocs) is 6.154060 us
Roundtrip time for Groups (1 KB pipe, no memcpy, w/ allocs) is 6.188869 us
Roundtrip time for Groups (1 KB pipe, w/ memcpy, w/ allocs) is 6.245136 us
[1] Stack Traceback:
  [1:0]   [0x568bf3]
  [1:1] +0x32510  [0x2b993df71510]
  [1:2]   [0x56bc84]
  [1:3] CmiGetNonLocal+0xa1  [0x56cdf1]
[0] Stack Traceback:
  [0:0]   [0x568bf3]
  [0:1] +0x32510  [0x2b2059750510]
  [0:2] gsignal+0x35  [0x2b2059750495]
  [0:3] abort+0x175  [0x2b2059751c75]
  [0:4] +0x703a7  [0x2b205978e3a7]
  [0:5] +0x75dee  [0x2b2059793dee]
  [0:6] +0x78c80  [0x2b2059796c80]
  [0:7] CmiGetNonLocal+0x21b  [0x56cf6b]
  [0:8] CsdNextMessage+0x5d  [0x5732ad]
  [0:9] CsdScheduleForever+0xaa  [0x5733aa]
  [0:10] CsdScheduler+0x2d  [0x57365d]
  [0:11] ConverseInit+0x522  [0x571b62]
  [0:12] main+0x27  [0x49ac47]
  [0:13] __libc_start_main+0xfd  [0x2b205973cd1d]
  [0:14]   [0x484715]

History

#1 Updated by Sam White over 1 year ago

  • Priority changed from Normal to High

This is a release blocker

#2 Updated by Nitin Bhat over 1 year ago

  • Status changed from New to Implemented

#3 Updated by Sam White over 1 year ago

  • Status changed from Implemented to Merged

Also available in: Atom PDF