Project

General

Profile

Feature #934

Isomalloc: refactor to associate migratable allocations with entities other than persistent threads

Added by Phil Miller over 2 years ago. Updated 7 days ago.

Status:
Implemented
Priority:
Normal
Assignee:
Category:
Migration
Target version:
-
Start date:
12/26/2015
Due date:
% Done:

0%

Tags:

Description

Isomalloc is currently very specialized to supporting the needs of AMPI/TCharm and BigSim emulation. These use cases have user-level threads that are created at startup, and persist for the duration of the run. However, there's nothing inherent in the migratable allocation functionality that limits it to this usage. In porting Chombo to run on Charm++, I've found that it would be convenient to have migratable allocations associated with an object instead. This could prove to be useful for porting in any kind of legacy code to Charm++, where recursively implementing PUP routines through the entire code base would be extraordinarily tedious and error-prone.

I intend to refactor Isomalloc so that it simply takes a 'context' argument that says which arena of co-migrating memory an allocation should come from. The TCharm/AMPI and BigSim cases would then associate a context with their threads, while the Chombo case would associate a context with each object.

History

#1 Updated by Phil Miller over 2 years ago

  • Status changed from New to In Progress

#2 Updated by Phil Miller over 2 years ago

Took a lighter touch approach, of turning the existing CmiIsomallocBlockList that was almost the necessary encapsulation of an allocation context into that context. Each block list now stores its associated memory pool, rather than using a Ctv. Allocation is all fine, and external call sites now explicitly call for allocation from a particular block list.

I think the next step is to turn the active blocklist pointer from a Cpv to a Ctv so that it doesn't need to be explicitly en/disabled at thread context switches.

#3 Updated by Phil Miller over 2 years ago

  • Status changed from In Progress to Implemented

#4 Updated by Phil Miller almost 2 years ago

  • Status changed from Implemented to Merged
  • translation missing: en.field_closed_date set to 2016-10-05 15:49:03.629602

#5 Updated by Phil Miller almost 2 years ago

  • Status changed from Merged to In Progress
  • translation missing: en.field_closed_date deleted (2016-10-05 15:49:03.629602)

Change got reverted due to broken builds.

#6 Updated by Sam White over 1 year ago

  • Target version changed from 6.8.0 to 6.8.1

#7 Updated by Sam White 12 months ago

  • Target version changed from 6.8.1 to 6.9.0

#8 Updated by Eric Bohm 9 months ago

  • Target version changed from 6.9.0 to 7 (Next Generation Charm++)

#9 Updated by Phil Miller 8 months ago

  • Assignee changed from Phil Miller to Eric Bohm

#10 Updated by Sam White 3 months ago

  • Tags set to isomalloc

#11 Updated by Evan Ramos 7 days ago

Patch revival: https://charm.cs.illinois.edu/gerrit/4472

The remaining issue from the initial autobuild failures, which now shows up in Jenkins verification as well, is an abort from maxReduce in examples/bigsim/emulator.

The problem is the addition of a call to CmiIsomallocBlockListNew() in CthThreadBaseInit. Without the patch, CmiIsomallocBlockListNew is never called during maxReduce. With it, the debug prints in isomalloc say things like:

[6] tried to mmap 0x816dbf800000, but encountered error
[7] tried to mmap 0x96ff5f400000, but encountered error
Charm++> Isomalloc map failed to allocate 1048576 bytes at 0x816dbf800000, errno: 12.
------------- Processor 6 Exiting: Called CmiAbort ------------
Reason: Exiting

Charm++> Isomalloc map failed to allocate 1048576 bytes at 0x96ff5f400000, errno: 12.
------------- Processor 7 Exiting: Called CmiAbort ------------
Reason: Exiting

errno 12 is ENOMEM.

#12 Updated by Evan Ramos 7 days ago

If I break on mempool_init in maxReduce, the crashing PE hits the breakpoint 16 times, and the other three PEs hit it exactly 169 times each by the time the crash occurs (measured with info b in GDB). CthThreadBaseFree is never called.

Is the only solution here to make use lazy allocation on first use for isomallocBlockList, similar to what happens before the patch? Could another solution be to only allocate and use isomallocBlockList if the thread is migratable?

#13 Updated by Evan Ramos 7 days ago

  • Target version deleted (7 (Next Generation Charm++))
  • Assignee changed from Eric Bohm to Evan Ramos
  • Status changed from In Progress to Implemented

https://charm.cs.illinois.edu/gerrit/4472

With my latest push implementing lazy allocation (which only takes place during the creation of a migratable stack, or setup of tlsglobals or swapglobals), all tests pass under netlrts-linux-x86_64-smp and netlrts-linux-smp.

Also available in: Atom PDF