Re: [DynInst_API:] Abort while building Dyninst with Intel compiler - tbb related

[thaines.astro@xxxxxxxxx]

Hi, Jim.

Currently, Dyninst does not build with any compiler other than gcc. I'm working on getting Clang working, but that work has been delayed to get some bugs fixed for the upcoming release. Once that's done, I want to get back to compiler support. If you could file an issue on our Github site to help track Intel build problems, that would be great.

>internal error: bad pointer

This appears to be an internal compiler error. Have you encountered this in building other TBB projects on that machine?

Thanks.

Â- Tim

On Fri, May 8, 2020 at 4:23 PM Jim Galarowicz <jeg@xxxxxxxxxxxxxxxxx> wrote:

Hi all,

We have a user that wants to build OpenSpeedShop with Intel compilers.ÂÂ The compile/compiler is aborting when building Dyninst where it is referencing TBB includes.

I wonder if anyone can see any issues with the compiler version, tbb version or another thing that might be a problem.

[jeg@login01 include]$ icc -v
icc version 19.0.5.281 (gcc version 4.8.5 compatibility)

Listing of abort area:

 96%] Building C object CMakeFiles/dyninstAPI_RT.dir/src/RTstatic_ctors_dtors-x86.c.o
[100%] Linking C shared library libdyninstAPI_RT.so
[100%] Built target dyninstAPI_RT
[  0%] Built target DyninstRT
Scanning dependencies of target common
[  1%] Building CXX object common/CMakeFiles/common.dir/src/mcs-lock.C.o
[  1%] Building CXX object common/CMakeFiles/common.dir/src/pfq-rwlock.C.o
[  1%] Building CXX object common/CMakeFiles/common.dir/src/race-detector-annotations.C.o
[  1%] Building CXX object common/CMakeFiles/common.dir/src/string-regex.C.o
[  2%] Building CXX object common/CMakeFiles/common.dir/src/Timer.C.o
[  2%] Building CXX object common/CMakeFiles/common.dir/src/Types.C.o
[  2%] Building CXX object common/CMakeFiles/common.dir/src/lprintf.C.o
[  2%] Building CXX object common/CMakeFiles/common.dir/src/pathName.C.o
[  3%] Building CXX object common/CMakeFiles/common.dir/src/Time.C.o
[  3%] Building CXX object common/CMakeFiles/common.dir/src/fraction.C.o
[  3%] Building CXX object common/CMakeFiles/common.dir/src/timing.C.o
[  4%] Building CXX object common/CMakeFiles/common.dir/src/stats.C.o
[  4%] Building CXX object common/CMakeFiles/common.dir/src/Annotatable.C.o
[  4%] Building CXX object common/CMakeFiles/common.dir/src/MappedFile.C.o
[  4%] Building CXX object common/CMakeFiles/common.dir/src/sha1.C.o
[  5%] Building CXX object common/CMakeFiles/common.dir/src/util.C.o
[  5%] Building CXX object common/CMakeFiles/common.dir/src/Node.C.o
[  5%] Building CXX object common/CMakeFiles/common.dir/src/Graph.C.o
[  5%] Building CXX object common/CMakeFiles/common.dir/src/Edge.C.o
[  6%] Building CXX object common/CMakeFiles/common.dir/src/DOT.C.o
[  6%] Building CXX object common/CMakeFiles/common.dir/src/dyn_regs.C.o
[  6%] Building CXX object common/CMakeFiles/common.dir/src/AST.C.o
[  6%] Building CXX object common/CMakeFiles/common.dir/src/addrtranslate.C.o
[  7%] Building CXX object common/CMakeFiles/common.dir/src/arch-x86.C.o
[  7%] Building CXX object common/CMakeFiles/common.dir/src/arch-power.C.o
[  7%] Building CXX object common/CMakeFiles/common.dir/src/arch-aarch64.C.o
[  8%] Building CXX object common/CMakeFiles/common.dir/src/debug_common.C.o
[  8%] Building CXX object common/CMakeFiles/common.dir/src/VariableLocation.C.o
[  8%] Building CXX object common/CMakeFiles/common.dir/src/Buffer.C.o
[  8%] Building CXX object common/CMakeFiles/common.dir/src/MachSyscall.C.o
[  9%] Building CXX object common/CMakeFiles/common.dir/src/linuxKludges.C.o
[  9%] Building CXX object common/CMakeFiles/common.dir/src/timing-linux.C.o
[  9%] Building CXX object common/CMakeFiles/common.dir/src/parseauxv.C.o
[  9%] Building CXX object common/CMakeFiles/common.dir/src/addrtranslate-sysv.C.o
[ 10%] Building CXX object common/CMakeFiles/common.dir/src/addrtranslate-auxv.C.o
[ 10%] Building CXX object common/CMakeFiles/common.dir/src/addrtranslate-linux.C.o
[ 10%] Linking CXX shared library libcommon.so
[ 10%] Built target common
Scanning dependencies of target dynElf
[ 10%] Building CXX object elf/CMakeFiles/dynElf.dir/src/Elf_X.C.o
[ 10%] Linking CXX shared library libdynElf.so
[ 10%] Built target dynElf
Scanning dependencies of target dynDwarf
[ 10%] Building CXX object dwarf/CMakeFiles/dynDwarf.dir/src/dwarfResult.C.o
[ 10%] Building CXX object dwarf/CMakeFiles/dynDwarf.dir/src/dwarfExprParser.C.o
[ 10%] Building CXX object dwarf/CMakeFiles/dynDwarf.dir/src/dwarfFrameParser.C.o
[ 11%] Building CXX object dwarf/CMakeFiles/dynDwarf.dir/src/dwarfHandle.C.o
[ 11%] Linking CXX shared library libdynDwarf.so
[ 11%] Built target dynDwarf
Scanning dependencies of target symLite
[ 11%] Building CXX object symlite/CMakeFiles/symLite.dir/src/SymLite-elf.C.o
[ 11%] Linking CXX shared library libsymLite.so
[ 11%] Built target symLite
Scanning dependencies of target symtabAPI
[ 12%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Object.C.o
[ 12%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Aggregate.C.o
[ 12%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Function.C.o
[ 12%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Variable.C.o
[ 13%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Symbol.C.o
[ 13%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/LineInformation.C.o
[ 13%] Building CXX object symtabAPI/CMakeFiles/symtabAPI.dir/src/Symtab.C.o
In file included from /global/software/centos-7/modules/langs/intel/2019.5.281/compilers_and_libraries_2019.5.281/linux/tbb/include/tbb/tbb_machine.h(245),
                 from /global/software/centos-7/modules/langs/intel/2019.5.281/compilers_and_libraries_2019.5.281/linux/tbb/include/tbb/spin_rw_mutex.h(21),
                 from /global/software/centos-7/modules/langs/intel/2019.5.281/compilers_and_libraries_2019.5.281/linux/tbb/include/tbb/concurrent_hash_map.h(27),
                 from /global/home/users/jeg/openspeedshop-externals/BUILD/login01.hpcadvisorycouncil.com/dyninst-10.1.0/symtabAPI/h/Collections.h(34),
                 from /global/home/users/jeg/openspeedshop-externals/BUILD/login01.hpcadvisorycouncil.com/dyninst-10.1.0/symtabAPI/src/Symtab.C(49):
/global/software/centos-7/modules/langs/intel/2019.5.281/compilers_and_libraries_2019.5.281/linux/tbb/include/tbb/machine/linux_common.h(43): internal error: bad pointer
  #if defined(__linux__) || __TBB_has_include(<linux/futex.h>)
                                                             ^

compilation aborted for /global/home/users/jeg/openspeedshop-externals/BUILD/login01.hpcadvisorycouncil.com/dyninst-10.1.0/symtabAPI/src/Symtab.C (code 4)
make[2]: *** [symtabAPI/CMakeFiles/symtabAPI.dir/src/Symtab.C.o] Error 4
make[1]: *** [symtabAPI/CMakeFiles/symtabAPI.dir/all] Error 2
make: *** [all] Error 2


The Futex include:


[jeg@login01 openspeedshop-externals]$ cd /usr/include
[jeg@login01 include]$ find . -name futex.h
./linux/futex.h
[jeg@login01 include]$ cat linux/futex.h
#ifndef _LINUX_FUTEX_H
#define _LINUX_FUTEX_H


#include <linux/types.h>

/* Second argument to futex syscall */


#define FUTEX_WAIT		0
#define FUTEX_WAKE		1
#define FUTEX_FD		2
#define FUTEX_REQUEUE		3
#define FUTEX_CMP_REQUEUE	4
#define FUTEX_WAKE_OP		5
#define FUTEX_LOCK_PI		6
#define FUTEX_UNLOCK_PI		7
#define FUTEX_TRYLOCK_PI	8
#define FUTEX_WAIT_BITSET	9
#define FUTEX_WAKE_BITSET	10
#define FUTEX_WAIT_REQUEUE_PI	11
#define FUTEX_CMP_REQUEUE_PI	12

#define FUTEX_PRIVATE_FLAG	128
#define FUTEX_CLOCK_REALTIME	256
#define FUTEX_CMD_MASK		~(FUTEX_PRIVATE_FLAG | FUTEX_CLOCK_REALTIME)

#define FUTEX_WAIT_PRIVATE	(FUTEX_WAIT | FUTEX_PRIVATE_FLAG)
#define FUTEX_WAKE_PRIVATE	(FUTEX_WAKE | FUTEX_PRIVATE_FLAG)
#define FUTEX_REQUEUE_PRIVATE	(FUTEX_REQUEUE | FUTEX_PRIVATE_FLAG)
#define FUTEX_CMP_REQUEUE_PRIVATE (FUTEX_CMP_REQUEUE | FUTEX_PRIVATE_FLAG)
#define FUTEX_WAKE_OP_PRIVATE	(FUTEX_WAKE_OP | FUTEX_PRIVATE_FLAG)
#define FUTEX_LOCK_PI_PRIVATE	(FUTEX_LOCK_PI | FUTEX_PRIVATE_FLAG)
#define FUTEX_UNLOCK_PI_PRIVATE	(FUTEX_UNLOCK_PI | FUTEX_PRIVATE_FLAG)
#define FUTEX_TRYLOCK_PI_PRIVATE (FUTEX_TRYLOCK_PI | FUTEX_PRIVATE_FLAG)
#define FUTEX_WAIT_BITSET_PRIVATE	(FUTEX_WAIT_BITSET | FUTEX_PRIVATE_FLAG)
#define FUTEX_WAKE_BITSET_PRIVATE	(FUTEX_WAKE_BITSET | FUTEX_PRIVATE_FLAG)
#define FUTEX_WAIT_REQUEUE_PI_PRIVATE	(FUTEX_WAIT_REQUEUE_PI | \
					 FUTEX_PRIVATE_FLAG)
#define FUTEX_CMP_REQUEUE_PI_PRIVATE	(FUTEX_CMP_REQUEUE_PI | \
					 FUTEX_PRIVATE_FLAG)

/*
 * Support for robust futexes: the kernel cleans up held futexes at
 * thread exit time.
 */

/*
 * Per-lock list entry - embedded in user-space locks, somewhere close
 * to the futex field. (Note: user-space uses a double-linked list to
 * achieve O(1) list add and remove, but the kernel only needs to know
 * about the forward link)
 *
 * NOTE: this structure is part of the syscall ABI, and must not be
 * changed.
 */
struct robust_list {
	struct robust_list *next;
};

/*
 * Per-thread list head:
 *
 * NOTE: this structure is part of the syscall ABI, and must only be
 * changed if the change is first communicated with the glibc folks.
 * (When an incompatible change is done, we'll increase the structure
 *  size, which glibc will detect)
 */
struct robust_list_head {
	/*
	 * The head of the list. Points back to itself if empty:
	 */
	struct robust_list list;

	/*
	 * This relative offset is set by user-space, it gives the kernel
	 * the relative position of the futex field to examine. This way
	 * we keep userspace flexible, to freely shape its data-structure,
	 * without hardcoding any particular offset into the kernel:
	 */
	long futex_offset;

	/*
	 * The death of the thread may race with userspace setting
	 * up a lock's links. So to handle this race, userspace first
	 * sets this field to the address of the to-be-taken lock,
	 * then does the lock acquire, and then adds itself to the
	 * list, and then clears this field. Hence the kernel will
	 * always have full knowledge of all locks that the thread
	 * _might_ have taken. We check the owner TID in any case,
	 * so only truly owned locks will be handled.
	 */
	struct robust_list *list_op_pending;
};

/*
 * Are there any waiters for this robust futex:
 */
#define FUTEX_WAITERS		0x80000000

/*
 * The kernel signals via this bit that a thread holding a futex
 * has exited without unlocking the futex. The kernel also does
 * a FUTEX_WAKE on such futexes, after setting the bit, to wake
 * up any possible waiters:
 */
#define FUTEX_OWNER_DIED	0x40000000

/*
 * The rest of the robust-futex field is for the TID:
 */
#define FUTEX_TID_MASK		0x3fffffff

/*
 * This limit protects against a deliberately circular list.
 * (Not worth introducing an rlimit for it)
 */
#define ROBUST_LIST_LIMIT	2048

/*
 * bitset with all bits set for the FUTEX_xxx_BITSET OPs to request a
 * match of any bit.
 */
#define FUTEX_BITSET_MATCH_ANY	0xffffffff


#define FUTEX_OP_SET		0	/* *(int *)UADDR2 = OPARG; */
#define FUTEX_OP_ADD		1	/* *(int *)UADDR2 += OPARG; */
#define FUTEX_OP_OR		2	/* *(int *)UADDR2 |= OPARG; */
#define FUTEX_OP_ANDN		3	/* *(int *)UADDR2 &= ~OPARG; */
#define FUTEX_OP_XOR		4	/* *(int *)UADDR2 ^= OPARG; */

#define FUTEX_OP_OPARG_SHIFT	8	/* Use (1 << OPARG) instead of OPARG.  */

#define FUTEX_OP_CMP_EQ		0	/* if (oldval == CMPARG) wake */
#define FUTEX_OP_CMP_NE		1	/* if (oldval != CMPARG) wake */
#define FUTEX_OP_CMP_LT		2	/* if (oldval < CMPARG) wake */
#define FUTEX_OP_CMP_LE		3	/* if (oldval <= CMPARG) wake */
#define FUTEX_OP_CMP_GT		4	/* if (oldval > CMPARG) wake */
#define FUTEX_OP_CMP_GE		5	/* if (oldval >= CMPARG) wake */

/* FUTEX_WAKE_OP will perform atomically
   int oldval = *(int *)UADDR2;
   *(int *)UADDR2 = oldval OP OPARG;
   if (oldval CMP CMPARG)
     wake UADDR2;  */

#define FUTEX_OP(op, oparg, cmp, cmparg) \
  (((op & 0xf) << 28) | ((cmp & 0xf) << 24)		\
   | ((oparg & 0xfff) << 12) | (cmparg & 0xfff))

#endif /* _LINUX_FUTEX_H */
[jeg@login01 include]$ 

Thanks,
Jim G 

_______________________________________________
Dyninst-api mailing list
Dyninst-api@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/dyninst-api