Re: [DynInst_API:] Several Dyninst build issues

[Engelbert Tijskens <engelbert.tijskens@xxxxxxxxxxxxx>]

Hi all,

i tried to install from the tarball  below and things looked very promising for a long time - my machine was installing for at least an hour - but eventually gave up with the error below, but which i could easily fix with:
$ sudo apt-get install zlib1g-dev
I will start to test it tomorrow.

many thanks to you all!
bert

the error:
...
/bin/sh ../libtool --tag=CXX   --mode=link g++ -I../libopenss-guibase -I../libopenss-message -I../libopenss-framework -I../libopenss-queries -I../libopenss-guiobjects -DLIBRARY_DIR=\"/home/etijskens/software/oss-2.1/lib\" -DPLUGIN_DIR=\"/home/etijskens/software/oss-2.1/lib/openspeedshop\" -I../libltdl -I/usr/include/python2.7 -std=c++0x -L../libopenss-message -L../libopenss-framework -L../libopenss-queries -L/usr/lib//x86_64-linux-gnu -lpython2.7 -export-dynamic -version-info 0:0:0 -DLIB_DIR=lib  -o libopenss-cli.la -rpath /home/etijskens/software/oss-2.1/lib libopenss_cli_la-Commander.lo libopenss_cli_la-CommandObject.lo libopenss_cli_la-Load_Messages.lo libopenss_cli_la-PY_Input.lo libopenss_cli_la-SS_Cmd_Control.lo libopenss_cli_la-SS_Cmd_Execution.lo libopenss_cli_la-SS_CommandResult.lo libopenss_cli_la-SS_Compare.lo libopenss_cli_la-SS_Configure.lo libopenss_cli_la-SS_Exp.lo libopenss_cli_la-SS_Parse_Lex.lo libopenss_cli_la-SS_Parse_Param.lo libopenss_cli_la-SS_Parse_Interval.lo libopenss_cli_la-SS_Parse_Range.lo libopenss_cli_la-SS_Parse_Result.lo libopenss_cli_la-SS_Parse_Target.lo libopenss_cli_la-SS_Parse_Yacc.lo libopenss_cli_la-SS_ThreadInfo.lo libopenss_cli_la-SS_View.lo libopenss_cli_la-SS_View_init.lo libopenss_cli_la-SS_View_multi.lo libopenss_cli_la-SS_View_output.lo libopenss_cli_la-SS_View_stats.lo libopenss_cli_la-SS_View_util.lo libopenss_cli_la-SS_Watcher.lo libopenss_cli_la-SS_Start.lo libopenss_cli_la-SS_Settings.lo libopenss_cli_la-Start_Modes.lo  -lopenss-message -lopenss-framework -lopenss-queries ../libltdl/libltdl.la -L/usr/lib -lz -lpthread -ldl  -lutil -lgomp -lpthread -lutil -ldl
libtool: link: g++  -fPIC -DPIC -shared -nostdlib /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbeginS.o  .libs/libopenss_cli_la-Commander.o .libs/libopenss_cli_la-CommandObject.o .libs/libopenss_cli_la-Load_Messages.o .libs/libopenss_cli_la-PY_Input.o .libs/libopenss_cli_la-SS_Cmd_Control.o .libs/libopenss_cli_la-SS_Cmd_Execution.o .libs/libopenss_cli_la-SS_CommandResult.o .libs/libopenss_cli_la-SS_Compare.o .libs/libopenss_cli_la-SS_Configure.o .libs/libopenss_cli_la-SS_Exp.o .libs/libopenss_cli_la-SS_Parse_Lex.o .libs/libopenss_cli_la-SS_Parse_Param.o .libs/libopenss_cli_la-SS_Parse_Interval.o .libs/libopenss_cli_la-SS_Parse_Range.o .libs/libopenss_cli_la-SS_Parse_Result.o .libs/libopenss_cli_la-SS_Parse_Target.o .libs/libopenss_cli_la-SS_Parse_Yacc.o .libs/libopenss_cli_la-SS_ThreadInfo.o .libs/libopenss_cli_la-SS_View.o .libs/libopenss_cli_la-SS_View_init.o .libs/libopenss_cli_la-SS_View_multi.o .libs/libopenss_cli_la-SS_View_output.o .libs/libopenss_cli_la-SS_View_stats.o .libs/libopenss_cli_la-SS_View_util.o .libs/libopenss_cli_la-SS_Watcher.o .libs/libopenss_cli_la-SS_Start.o .libs/libopenss_cli_la-SS_Settings.o .libs/libopenss_cli_la-Start_Modes.o   -Wl,-rpath -Wl,/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-message/.libs -Wl,-rpath -Wl,/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-queries/.libs -Wl,-rpath -Wl,/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-framework/.libs -Wl,-rpath -Wl,/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libltdl/.libs -Wl,-rpath -Wl,/home/etijskens/software/oss-2.1/lib -Wl,-rpath -Wl,/home/etijskens/software/oss-2.1/lib -L/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-framework/.libs -L/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libltdl/.libs -L../libopenss-message -L../libopenss-framework -L../libopenss-queries -L/usr/lib//x86_64-linux-gnu -lpython2.7 /home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-message/.libs/libopenss-message.so -L/home/etijskens/software/oss-2.1/lib /home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-queries/.libs/libopenss-queries.so /home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-framework/.libs/libopenss-framework.so /home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libltdl/.libs/libltdl.so /home/etijskens/software/oss-2.1/lib/libsqlite3.so -lrt -liberty_pic ../libltdl/.libs/libltdl.so -L/usr/lib -lz -lgomp -lpthread -lutil -ldl -L/usr/lib/gcc/x86_64-linux-gnu/4.8 -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/intel64 -L/opt/intel/composer_xe_2013_sp1.2.144/ipp/../compiler/lib/intel64 -L/opt/intel/composer_xe_2013_sp1.2.144/ipp/lib/intel64 -L/opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64 -L/opt/intel/composer_xe_2013_sp1.2.144/tbb/lib/intel64/gcc4.4 -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/4.8/crtendS.o /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o    -fopenmp -Wl,-soname -Wl,libopenss-cli.so.0 -o .libs/libopenss-cli.so.0.0.0
/usr/bin/ld: cannot find -lz
collect2: error: ld returned 1 exit status
make[3]: *** [libopenss-cli.la] Error 1
make[3]: Leaving directory `/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-cli'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1/libopenss-cli'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/etijskens/software/openspeedshop-release-2.1/BUILD/bert-MacBookPro-ubuntu/openspeedshop-2.1'
make: *** [all] Error 2
error: Bad exit status from /home/etijskens/software/openspeedshop-release-2.1/INSTALL/bert-MacBookPro-ubuntu/rpm-tmp.cDbJ56 (%build)


RPM build errors:
    Bad exit status from /home/etijskens/software/openspeedshop-release-2.1/INSTALL/bert-MacBookPro-ubuntu/rpm-tmp.cDbJ56 (%build)
OpenSpeedShop FAILED TO BUILD - TERMINATING BUILD SCRIPT.  Please check for errors.

On 17/04/14 20:32, Jim Galarowicz wrote:

Hi Bill,

I think I have all the issues addressed based on your suggestions.   I ran into another similar (to issue 2) problem today on Pleiades at NASA where the binutils as assembler was getting invoked and cause xerces-c to file to configure.  So, I've changed our binutils build to not install the bin tools (ld, ar, as, etc.). 

Also, I'm not seeing any checks that our OSS build is doing that would let libdwarf.a be recognised as an acceptable dwarf installation.  We seem to be only checking for libdwarf.so and will build a shared version of libdwarf if we don't find libdwarf.so.   I thought we were checking for libdwarf.a and allowing that to satisfy the requirement.  So, I may need more information from Engelbert on the libdwarf installed on his platform to see why the issue 3 is happening.

For BG/Q, I put in a check to see if we are on a BG machine and then switching the version of dyninst to 8.1.2.  This bloats the source directory a bit, but should be ok for a while.

I have put out a new tarball ( rc2-openspeedshop-release-2.1-u3.tar.gz ) on sourceforge for Engelbert and others to try.

Thanks for your help.

Jim G

On 04/14/2014 09:09 AM, Bill Williams wrote:

Jim--

Trimming things down to issues that remain open w.r.t. 8.2:

>> 2) Does /opt/krellroot_v2.1u3/bin/ld work for linking object files

built with gcc? Is this the linker you intend to use? You're mixing
toolchains here (that linker with /usr/bin/gcc) and I have no idea
whether that's deliberate or not, and no idea whether the toolchains
in question are compatible.

I don't know exactly what to do with this.  If I need to install
binutils, then it is in the krellroot which is used to reference other
needed components that might not be installed.  ld gets installed into
the krellroot when binutils is built.   You are correct, though.
Manually moving ld to ld-back and rerunning the dyninst build allows it
to succeed.

I am presuming that you need various binutils components as direct O|SS dependencies and not just as Dyninst depedencies here. IIRC it's possible to coerce binutils into building each of its components separately, though "coerce" is absolutely the right verb--unless my memory is slipping, we do this for libiberty. I can see three general approaches here:

1) Only build the binutils components you need
2) Prune the binutils install to the set of components you need
3) Add the krell auxiliary directories at the end, rather than the beginning, of the assorted path variables (assuming that the goal is to augment what's available and not to replace something that's already present).

3) libdwarf must be built PIC or shared (we generally go for shared)
in order to link--we do not support deferred linking of static
libraries into a dyninst-based executable. Grab source and build with
--enable-shared, or let Dyninst's CMake do it for you automatically.

We do build libdwarf shared when it is determined we need to build it.
So, I check for libdwarf.so and then for libdwarf.a, but do not check
for whether or not libdwarf is compiled with -fPIC.  I found a
discussion on the internet that said you can check for that by doing
something like this:
 > for i in /usr/lib/*.a /usr/local/lib/*.a; do
 > objdump --reloc $i 2>/dev/null | grep R_X86_64_32S > /dev/null &&
echo $i;
 > done

So, I tried it on my ubuntu system:

objdump --reloc /usr/lib/libdwarf.a  2>/dev/null | grep R_X86_64_32S
00000000000000a4 R_X86_64_32S      .rodata+0x0000000000000180
000000000000011f R_X86_64_32S      .rodata+0x0000000000000198
00000000000004ba R_X86_64_32S      .rodata+0x0000000000000180
00000000000004c0 R_X86_64_32S      .rodata+0x0000000000000180


jeg@jeg-OptiPlex-GX620:~/OpenSpeedShop_ROOT$ objdump --reloc
/usr/lib/libiberty_pic.a 2>/dev/null | grep R_X86_64_32S
jeg@jeg-OptiPlex-GX620:~/OpenSpeedShop_ROOT$ objdump --reloc
/usr/lib/libiberty.a 2>/dev/null | grep R_X86_64_32S
00000000000000f5 R_X86_64_32S      .rodata
00000000000004b4 R_X86_64_32S      .rodata+0x00000000000000f0
000000000000071a R_X86_64_32S      .bss+0x0000000000000080
000000000000074f R_X86_64_32S      .bss+0x0000000000000080

To your knowledge is that the best way to check for fPIC libraries?

That works; the arguably more robust approach is to build a sample shared library that calls a method in libdwarf/libiberty/whatever else and test compilation/linking. PIC is a side question to the main one of "can we link against this from a shared library?". Pretty straightforward but I haven't gotten to it yet.

And the BG/Q issues are not forgotten--they're in the queue and will be resolved before we release, of course.

--bw

Bill Williams
Paradyn Project
bill@xxxxxxxxxxx

-- 
Dr Engelbert Tijskens
HPC Analyst/Consultant
Flemish Supercomputer Center vscentrum.be 
CalcUA Core Facility uantwerp.be/calcua
University of Antwerp www.uantwerp.be
Middelheimlaan1 G.309, B-2020 Antwerp, Belgium
engelbert.tijskens@xxxxxxxxxxxxx
+32 3 265 3879