|
| From: | Konstantinos Poulios |
| Subject: | Re: [EXT] Re: getfem installation |
| Date: | Fri, 10 Dec 2021 08:34:09 +0100 |
Dear Dr. Pulios,
Thank you so much for all your information and helps.
I troubleshot the issue. Removed the getfem installed inside the python then setup PYTHONPATH to the location where getfem python file and library located. With these changes. It no longer core dump but still return the error. Do we miss anything? Attached file is the ldd output.
$ echo $PYTHONPATH
/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem
$ echo $LD_LIBRARY_PATH
/risapps/rhel7/qhull/2020.2/lib:/risapps/rhel7/gcc/9.3.0/lib64:/risapps/rhel7/openmpi/4.1.1/lib:/risapps/rhel7/mumps/5.4.1/lib:/risapps/rhel7/scotch/6.1.1/lib:/risapps/rhel7/metis/5.1.0/lib:/risapps/rhel7/parmetis/4.0.3/lib:/risapps/rhel7/scalapack/2.1.0/lib:/risapps/rhel7/openblas/0.3.18/lib:/risapps/rhel7/python/3.9.7/lib:/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem:/risapps/rhel7/getfem-mpi/5.4.1-new/lib:/rissched/lsf/10.1/linux3.10-glibc2.17-x86_64/lib
$ python ./demo_plasticity.py
Traceback (most recent call last):
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 46, in <module>
from ._getfem import *
ImportError: attempted relative import with no known parent package
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/rsrch3/home/itops/jchen24/testmpi/./demo_plasticity.py", line 39, in <module>
import getfem as gf
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 48, in <module>
from _getfem import *
ImportError: /risapps/rhel7/getfem-mpi/5.4.1-new/lib/libgetfem.so.5: undefined symbol: __mumps_ooc_common_MOD_low_level_strat_io
Regards
Jenny
From: Konstantinos Poulios <logari81@googlemail.com>
Date: Thursday, December 9, 2021 at 1:42 PM
To: "Chen,Jinzhen" <JChen24@mdanderson.org>
Cc: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
WARNING: This email originated from outside of MD Anderson. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
Dear Jenny,
I attach a file with all dependencies on my own build of the python getfem library on our cluster (which I have tested and it works fine with demo_laplacian.py)
Best regards
Kostas
On Thu, Dec 9, 2021 at 12:22 AM Chen,Jinzhen <JChen24@mdanderson.org> wrote:
Dear Dr. Poulios,
It could be that I didn’t clean well the previous mumps libraries or other reason. After sending out the previous email, I double checked and made sure everything is clean, then I recompiled mumps and getfem again. It passed the compilation and built successfully finally. That is a great progress. However, it got core dump in my preliminary testing. Even python (without mpirun) command got core dumped too. In addition to getfem, python, mpi and metis libraries, what other libraries are required at run time? Thanks a lot.
[jchen24@tdragon4 testmpi]$ mpirun -n 4 python ./demo_parallel_laplacian.py
===================================================================================
= BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
= PID 200557 RUNNING AT tdragon4
= EXIT CODE: 139
= CLEANING UP REMAINING PROCESSES
= YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions
[jchen24@tdragon4 testmpi]$ python ./demo_plasticity.py
Segmentation fault (core dumped)
Best Regards
Jenny
From: "Chen,Jinzhen" <JChen24@mdanderson.org>
Date: Wednesday, December 8, 2021 at 4:01 PM
To: Konstantinos Poulios <logari81@googlemail.com>
Cc: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
Dear Dr. Poulios,
I really appreciate your great help. The -fallow-argument-mismatch option can’t use in our environment. It returned below on mumps complication. I tried to change it to -Wno-argument-mismatch or remove it. There is no difference , Hence I removed it in the mumps build yesterday.
…
mpif90 -fallow-argument-mismatch -O3 -fPIC -I. -I../include -I/risapps/rhel7/scotch/6.1.1/include -Dscotch -Dmetis -Dpord -Dptscotch -Dparmetis -c double_linked_list.F -o double_linked_list.o
gfortran: error: unrecognized command line option ‘-fallow-argument-mismatch’; did you mean ‘-Wno-argument-mismatch’?
make[2]: *** [double_linked_list.o] Error 1
make[2]: Leaving directory `/risapps/build7/MUMPS_5.4.1/src'
make[1]: *** [libcommon] Error 2
make[1]: Leaving directory `/risapps/build7/MUMPS_5.4.1/src'
make: *** [all] Error 2
Today, I updated the mumps Makefile.inc as your suggested but without -fallow-argument-mismatch option, then recompiled mumps and getfem. Now most errors in the getfem compilation are gone but there is still one left. I tried to add the -fPIC option to getfem configuration with-optimization parameter as well. I.e. changed --with-optimization=-O3 to --with-optimization=”-O3 -fPIC”, this didn’t help and it still returned the same error.
Mumps Makefile.inc
sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1 -O3 -fPIC\2/' Makefile.inc
sed -i '/^\(OPT[CL].*=.*\)-O\(\>.*\)/s//\1-O3 -fPIC\2/' Makefile.inc
getfem make error:
…
*** Warning: Linking the shared library libgetfem.la against the
*** static library /risapps/rhel7/openblas/0.3.18/lib/libopenblas.a is not portable!
libtool: link: mpic++ -fPIC -DPIC -shared -nostdlib /lib/../lib64/crti.o /risapps/rhel7/gcc/9.3.0/lib/gcc/x86_64-pc-linux-gnu/9.3.0/crtbeginS.o .libs/dal_backtrace.o .libs/dal_bit_vector.o .libs/dal_singleton.o .libs/dal_static_stored_objects.o .libs/bgeot_convex_structure.o .libs/bgeot_convex_ref_simplexified.o .libs/bgeot_convex_ref.o .libs/bgeot_geometric_trans.o .libs/bgeot_geotrans_inv.o .libs/bgeot_kdtree.o .libs/bgeot_mesh_structure.o .libs/bgeot_rtree.o .libs/bgeot_node_tab.o .libs/bgeot_small_vector.o .libs/bgeot_sparse_tensors.o .libs/bgeot_torus.o .libs/bgeot_poly.o .libs/bgeot_poly_composite.o .libs/bgeot_ftool.o .libs/getfem_models.o .libs/getfem_model_solvers.o .libs/getfem_superlu.o .libs/getfem_mesh.o .libs/getfem_mesh_region.o .libs/getfem_context.o .libs/getfem_mesh_fem.o .libs/getfem_mesh_im.o .libs/getfem_integration.o .libs/getfem_integration_composite.o .libs/getfem_global_function.o .libs/getfem_fem.o .libs/getfem_interpolated_fem.o .libs/getfem_projected_fem.o .libs/getfem_fem_global_function.o .libs/getfem_mesh_fem_global_function.o .libs/getfem_fem_composite.o .libs/getfem_mat_elem.o .libs/getfem_mat_elem_type.o .libs/getfem_level_set.o .libs/getfem_locale.o .libs/getfem_mesh_level_set.o .libs/getfem_mesh_im_level_set.o .libs/getfem_mesh_fem_level_set.o .libs/getfem_mesh_fem_product.o .libs/getfem_mesh_fem_sum.o .libs/getfem_fem_level_set.o .libs/getfem_partial_mesh_fem.o .libs/getfem_torus.o .libs/getfem_mesh_slicers.o .libs/getfem_mesh_slice.o .libs/getfem_regular_meshes.o .libs/getfem_import.o .libs/getfem_interpolation.o .libs/getfem_error_estimate.o .libs/getfem_export.o .libs/getfem_assembling_tensors.o .libs/getfem_generic_assembly_tree.o .libs/getfem_generic_assembly_functions_and_operators.o .libs/getfem_generic_assembly_semantic.o .libs/getfem_generic_assembly_workspace.o .libs/getfem_generic_assembly_compile_and_exec.o .libs/getfem_generic_assembly_interpolation.o .libs/getfem_mesher.o .libs/getfem_fourth_order.o .libs/getfem_nonlinear_elasticity.o .libs/getfem_linearized_plates.o .libs/getfem_HHO.o .libs/getfem_contact_and_friction_common.o .libs/getfem_contact_and_friction_nodal.o .libs/getfem_contact_and_friction_integral.o .libs/getfem_contact_and_friction_large_sliding.o .libs/getfem_plasticity.o .libs/getfem_omp.o .libs/getfem_level_set_contact.o .libs/getfem_im_data.o .libs/getfem_continuation.o -Wl,--whole-archive /risapps/build7/getfem-5.4.1/superlu/.libs/libsuperlu.a -Wl,--no-whole-archive -Wl,-rpath -Wl,/risapps/rhel7/openmpi/4.1.1/lib -Wl,-rpath -Wl,/risapps/rhel7/gcc/9.3.0/lib/../lib64 -Wl,-rpath -Wl,/risapps/rhel7/openmpi/4.1.1/lib -Wl,-rpath -Wl,/risapps/rhel7/gcc/9.3.0/lib/../lib64 -L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib -L/risapps/rhel7/qhull/2020.2/lib -lsmumps -ldmumps -lcmumps -lzmumps -lqhull -ldl /risapps/rhel7/openblas/0.3.18/lib/libopenblas.a -lmetis -L/risapps/rhel7/gcc/9.3.0/lib/gcc/x86_64-pc-linux-gnu/9.3.0 -L/risapps/rhel7/gcc/9.3.0/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/risapps/rhel7/gcc/9.3.0/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../.. /risapps/rhel7/openmpi/4.1.1/lib/libmpi.so /risapps/rhel7/gcc/9.3.0/lib/../lib64/libstdc++.so -lm -lpthread -lc -lgcc_s /risapps/rhel7/gcc/9.3.0/lib/gcc/x86_64-pc-linux-gnu/9.3.0/crtendS.o /lib/../lib64/crtn.o -O3 -pthread -Wl,-soname -Wl,libgetfem.so.5 -o .libs/libgetfem.so.5.0.4
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libdmumps.a(dmumps_c.o): relocation R_X86_64_PC32 against symbol `strlen@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [libgetfem.la] Error 1
make[2]: Leaving directory `/risapps/build7/getfem-5.4.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/risapps/build7/getfem-5.4.1'
make: *** [all] Error 2
Best Regards
Jenny
From: Konstantinos Poulios <logari81@googlemail.com>
Date: Wednesday, December 8, 2021 at 1:39 PM
To: "Chen,Jinzhen" <JChen24@mdanderson.org>
Cc: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
WARNING: This email originated from outside of MD Anderson. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
Dear Jenny,
There was a mistake in the list of my instructions, the last 2 sed commands in the MUMPS compilation should be
sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1-fallow-argument-mismatch -O3 -fPIC\2/' Makefile.inc
sed -i '/^\(OPT[CL].*=.*\)-O\(\>.*\)/s//\1-O3 -fPIC\2/' Makefile.inc
Which means, you need to add the -fPIC option in the compilation of MUMPS. An updated version of the entire recipe is attached. I also attach the editted Makefile.inc file for the compilation of MUMPS.
Best regards
Kostas
On Wed, Dec 8, 2021 at 1:32 AM Chen,Jinzhen <JChen24@mdanderson.org> wrote:
Dear. Dr. Poulios,
I investigated the issue and found that libsmumps-5.3.so is a system level mumps library that I installed from the rpm packages before. The compiler used system level mumps libraries rather than what I built mumps/5.4.1 in the previous built somehow. Hence, I rebuilt mumps/5.4.1 from a node where there are no mumps or metis rpm packages installed. Then I changed the getfem configuration to set LIBS having the same value as LDFLAGS.
./configure --prefix=/risapps/rhel7/getfem-mpi/5.4.1-new \
LDFLAGS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib -L/risapps/rhel7/qhull/2020.2/lib" \
CPPFLAGS="-I/risapps/rhel7/gcc/9.3.0/include -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/mumps/5.4.1/include -I/risapps/rhel7/parmetis/4.0.3/include -I/risapps/rhel7/metis/5.1.0/include -I/risapps/rhel7/qhull/2020.2/include" \
LIBS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib -L/risapps/rhel7/qhull/2020.2/lib" \
--with-blas="/risapps/rhel7/openblas/0.3.18/lib/libopenblas.a" \
--with-pic --with-optimization=-O3 --disable-metlab --enable-python --enable-paralevel=2 --enable-shared
With these changes, getfem can’t be built any more. Here is the last part of compiling message. Please let me know if you need more information.
…..
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_matvec.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_root_parallel.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_aux.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_aux_par.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_par_m.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zmumps_sol_es.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_LDLT_preprocess.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_ELT.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_lastrtnelind.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_mem_stack.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_root_parallel.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_type3_symmetrize.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zana_mtrans.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_bwd.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_fwd_aux.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_master_ELT_m.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_asm_master_m.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LU_type1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LU_type2.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LDLT_type1.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_LDLT_type2.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zfac_front_type2_aux.o): relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_lr.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libzmumps.a(zsol_bwd_aux.o): relocation R_X86_64_32S against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
/bin/ld: /risapps/rhel7/mumps/5.4.1/lib/libdmumps.a(dmumps_c.o): relocation R_X86_64_PC32 against symbol `strlen@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC
/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [libgetfem.la] Error 1
make[2]: Leaving directory `/risapps/build7/getfem-5.4.1/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/risapps/build7/getfem-5.4.1'
make: *** [all] Error 2
Best Regards
Jenny
From: "Chen,Jinzhen" <JChen24@mdanderson.org>
Date: Tuesday, December 7, 2021 at 11:34 AM
To: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
Dear Dr. Poulios,
I was able to follow your steps and built all dependencies. In addition, I also built metis/5.1.0, as parmetis I built has no metis.h and metis library but they are required for getfem. I passed the getfem compiling with this configuration. I could not include some libraries on LIBS somehow. However, I defined LD_LIBRARY_PATH environment variable which contains all dependency libraries. So no problem on building getfem.
./configure --prefix=/risapps/rhel7/getfem-mpi/5.4.1-new \
LDFLAGS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/mumps/5.4.1/lib -L/risapps/rhel7/parmetis/4.0.3/lib -L/risapps/rhel7/metis/5.1.0/lib -L/risapps/rhel7/scalapack/2.1.0/lib -L/risapps/rhel7/scotch/6.1.1/lib -L/risapps/rhel7/qhull/2020.2/lib" \
CPPFLAGS="-I/risapps/rhel7/gcc/9.3.0/include -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/mumps/5.4.1/include -I/risapps/rhel7/parmetis/4.0.3/include -I/risapps/rhel7/metis/5.1.0/include -I/risapps/rhel7/qhull/2020.2/include" \
LIBS="-lmetis -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common -lpord -lgfortran" \
--with-blas="/risapps/rhel7/openblas/0.3.18/lib/libopenblas.a" \
--with-pic --with-optimization=-O3 --disable-metlab --enable-python --enable-paralevel=2 --enable-shared
However, I got below error when I run a basic testing. I noticed that most built dependency libraries are static libraries. I.e. *.a files. How can I build shared libraries that getfem python needs at run time? Thank you very much !
[jchen24@tdragon4 testmpi]$ mpirun -n 4 ./parallel.py
Traceback (most recent call last):
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 46, in <module>
Traceback (most recent call last):
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 46, in <module>
Traceback (most recent call last):
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 46, in <module>
Traceback (most recent call last):
File "/risapps/rhel7/getfem-mpi/5.4.1-new/lib/python3.9/site-packages/getfem/getfem.py", line 46, in <module>
from ._getfem import *
ImportError: libsmumps-5.3.so: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
from ._getfem import *
File "/rsrch3/home/itops/jchen24/testmpi/./parallel.py", line 39, in <module>
ImportError: libsmumps-5.3.so: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/rsrch3/home/itops/jchen24/testmpi/./parallel.py", line 39, in <module>
from ._getfem import *
ImportError: libsmumps-5.3.so: cannot open shared object file: No such file or directory
….
Actually, I was able to compile getfem using system level mumps and metis with some code changes before I received your previous email. But I got below errors in testing. I could be that the system level mumps was built using different mpi as we are using or it missed other dependencies. This is just FYI.. as I forgot to mention it in my previous email.
[jchen24@tdragon4 testmpi]$ mpirun -n 4 ./parallel.py
initializing ...
numthread = 1
initializing ...
numthread = 1
initializing ...
numthread = 1
initializing ...
numthread = 1
*** The MPI_Comm_rank() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[cdragon003:240842] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** The MPI_Comm_rank() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[cdragon003:240843] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** The MPI_Comm_rank() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
[cdragon003:240844] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** The MPI_Comm_rank() function was called before MPI_INIT was invoked.
*** This is disallowed by the MPI standard.
*** Your MPI job will now abort.
Regards
Jinzhen (Jenny) Chen
From: "Chen,Jinzhen" <JChen24@mdanderson.org>
Date: Monday, December 6, 2021 at 9:45 AM
To: Konstantinos Poulios <logari81@googlemail.com>
Cc: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
Dear Dr. Poulios,
Thank you so much for your so detailed information. It is definitely very helpful. I will follow your steps and try again.
Best Regards
Jinzhen Chen
From: Konstantinos Poulios <logari81@googlemail.com>
Date: Saturday, December 4, 2021 at 3:03 AM
To: "Chen,Jinzhen" <JChen24@mdanderson.org>
Cc: "getfem-users@nongnu.org" <getfem-users@nongnu.org>
Subject: Re: [EXT] Re: getfem installation
WARNING: This email originated from outside of MD Anderson. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
Dear Chen,
Below you can see how I have compiled the parallel version of GetFEM with MUMPS on our cluster (Scientific linux 7). Depending on the specific versions of compilers and mpi, some parts of these steps need to be modified. The basic idea that worked for me is to use the 3 environment variables, LDFLAGS, CPPFLAGS and LIBS to pass any external dependencies not detected by the configure script (header files, library directories, libraries respectively). I also used the --with-blas= configure option to link to my compiled openblas (which also includes lapack). (The use of the PYTHONPATH variable is specific to a bug in our cluster's mpi4py module.)
I hope this helps. You will probably need one more fix not included in my instructions, when you come to the final step of building the getfem python interface, but let's take that once you get there.
Best regards
Kostas
module load python3/3.8.11
module load mpi/4.1.1-gcc-10.3.0-binutils-2.36.1
module load mpi4py/3.0.3-python-3.8.11-openmpi-4.1.1
wget https://github.com/xianyi/OpenBLAS/releases/download/v0.3.18/OpenBLAS-0.3.18.tar.gz
wget http://deb.debian.org/debian/pool/main/s/scalapack/scalapack_2.1.0.orig.tar.gz
wget http://deb.debian.org/debian/pool/main/s/scotch/scotch_6.1.1.orig.tar.xz
wget http://deb.debian.org/debian/pool/non-free/p/parmetis/parmetis_4.0.3.orig.tar.gz
wget http://deb.debian.org/debian/pool/main/m/mumps/mumps_5.4.1.orig.tar.gz
wget http://deb.debian.org/debian/pool/main/q/qhull/qhull_2020.2.orig.tar.gz
wget http://download-mirror.savannah.gnu.org/releases/getfem/stable/getfem-5.4.1.tar.gz
tar -zxf OpenBLAS-0.3.18.tar.gz
tar -zxf scalapack_2.1.0.orig.tar.gz
tar -xf scotch_6.1.1.orig.tar.xz
tar -zxf parmetis_4.0.3.orig.tar.gz
tar -zxf mumps_5.4.1.orig.tar.gz
tar -zxf qhull_2020.2.orig.tar.gz
tar -zxf getfem-5.4.1.tar.gz
cd OpenBLAS-0.3.18/
CC=gcc make -j8
CC=gcc make PREFIX=$HOME/tmp_opt/openblas install
cd ../scalapack-2.1.0/
sed '/^\(BLASLIB.*=\).*/ s//\1 $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' SLmake.inc.example | sed '/^\(LAPACKLIB.*=\).*/ s//\1 /' > SLmake.inc
sed -i '/^\(NOOPT.*=.*-O0\)\(\>.*\)/s//\1 -fallow-argument-mismatch -fPIC\2/' SLmake.inc
sed -i '/^\(FCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fallow-argument-mismatch -fPIC\2/' SLmake.inc
sed -i '/^\(CCFLAGS.*=.*-O3\)\(\>.*\)/s//\1 -fPIC\2/' SLmake.inc
sed -i '/^\(RANLIB.*=\).*/s//\1 echo/' SLmake.inc
make -j4
mkdir -p $HOME/tmp_opt/scalapack/lib
cp libscalapack.a $HOME/tmp_opt/scalapack/lib/
cd ../parmetis-4.0.3/
make config prefix=$HOME/tmp_opt/parmetis
make -j4
make install
cp build/Linux-x86_64/libmetis/libmetis.a $HOME/tmp_opt/parmetis/lib/
cp metis/include/metis.h $HOME/tmp_opt/parmetis/include/
cd ../scotch-6.1.1/src
cp Make.inc/Makefile.inc.x86-64_pc_linux2 Makefile.inc
sed -i '/^\(CC[SD].*=\).*/s//\1 mpicc/' Makefile.inc
make -j4 scotch
make -j4 esmumps
make -j4 ptscotch
make -j4 ptesmumps
mkdir -p $HOME/tmp_opt/scotch
cp -r ../lib $HOME/tmp_opt/scotch/
cp -r ../include $HOME/tmp_opt/scotch/
cd ../../MUMPS_5.4.1/
cp Make.inc/Makefile.inc.generic Makefile.inc
sed -i '/^#\(SCOTCHDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/scotch/' Makefile.inc
sed -i '/^#\(ISCOTCH.*\)=.*/s//\1 = -I$(SCOTCHDIR)\/include/' Makefile.inc
sed -i '/^#\(LSCOTCH.*-lpt.*\)/s//\1 -lscotch/' Makefile.inc
sed -i '/^#\(LMETISDIR.*\)=.*/s//\1 = $${HOME}\/tmp_opt\/parmetis\/lib/' Makefile.inc
sed -i '/^#\(IMETIS.*\)=.*/s//\1 = -I$${HOME}\/tmp_opt\/parmetis\/include/' Makefile.inc
sed -i '/^#\(LMETIS .*-lparmetis.*\)/s//\1/' Makefile.inc
sed -i '/^#\(ORDERINGSF .*-Dparmetis.*\)/s//\1/' Makefile.inc
sed -i '/^\(ORDERINGSF .*-Dpord$\)/s//#\1/' Makefile.inc
sed -i '/^\(CC.*= \)\(cc\)/s//\1mpi\2/' Makefile.inc
sed -i '/^\(FC.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc
sed -i '/^\(FL.*= \)\(f90\)/s//\1mpi\2/' Makefile.inc
sed -i '/^\(RANLIB .*ranlib.*\)/s//#\1/' Makefile.inc
sed -i '/^#\(RANLIB .*echo.*\)/s//\1/' Makefile.inc
sed -i '/^\(LAPACK.*=\).*/s//\1/' Makefile.inc
sed -i '/^\(SCALAP.*=\).*/s//\1 $${HOME}\/tmp_opt\/scalapack\/lib\/libscalapack.a/' Makefile.inc
sed -i '/^\(INCPAR.*=\).*/s//\1/' Makefile.inc
sed -i '/^\(\LIBPAR.*=\).*/s//\1 $(SCALAP)/' Makefile.inc
sed -i '/^\(LIBBLAS.*=\).*/s//\1 $${HOME}\/tmp_opt\/openblas\/lib\/libopenblas.a/' Makefile.inc
sed -i '/^\(OPTF.*=.*\)-O\(\>.*\)/s//\1-fallow-argument-mismatch -O3\2/' Makefile.inc
sed -i '/^\(OPT[CL].*=.*-O\)\(\>.*\)/s//\13\2/' Makefile.inc
make all
mkdir -p $HOME/tmp_opt/mumps
cp -r lib $HOME/tmp_opt/mumps/
cp -r include $HOME/tmp_opt/mumps/
cd ../qhull_2020.2/
make -j4
mkdir -p $HOME/tmp_opt/qhull
mkdir -p $HOME/tmp_opt/qhull/lib
mkdir -p $HOME/tmp_opt/qhull/include
mkdir -p $HOME/tmp_opt/qhull/include/libqhull
cp lib/libqhullstatic.a $HOME/tmp_opt/qhull/lib/libqhull.a
cp src/libqhull/*.h $HOME/tmp_opt/qhull/include/libqhull/
cd ../getfem-5.4.1
sed -i '/^\(.*\/env \)python\($\)/s//\1python3\2/' bin/extract_doc
PYTHONPATH=/appl/mpi4py/3.0.3-openmpi-4.1.1-gcc-10.3.0-binutils-2.36.1-python-3.8.11/lib/python3.8/site-packages/:$PYTHONPATH \
./configure --prefix="$HOME/tmp_opt" \
LDFLAGS="-L$HOME/tmp_opt/mumps/lib -L$HOME/tmp_opt/parmetis/lib -L$HOME/tmp_opt/scalapack/lib -L$HOME/tmp_opt/scotch/lib -L$HOME/tmp_opt/qhull/lib" \
CPPFLAGS="-I$HOME/tmp_opt/mumps/include -I$HOME/tmp_opt/parmetis/include -I$HOME/tmp_opt/qhull/include" \
LIBS="-lmumps_common -lpord -lgfortran -lscalapack -lptesmumps -lptscotch -lptscotcherr -lscotch -lparmetis -lmetis -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh" \
--with-blas="$HOME/tmp_opt/openblas/lib/libopenblas.a" \
--with-pic --with-optimization=-O3 --disable-matlab --enable-python --enable-paralevel=2
make -j4
make install
On Tue, Nov 30, 2021 at 8:51 PM Chen,Jinzhen <JChen24@mdanderson.org> wrote:
Dear Dr. Poulios,
Thank you so much for your prompt response and information. They are very helpful. I cc’ed this email to getfem-users@nongnu.org so that I don’t need to send to your personal email in next one. After reading your reference of the configure command, I started from scratch but still got compiling errors.
I installed metis and mumps in the system from EPEL reporistory.
[root@ ~]# rpm -qa |grep metis
metis-devel-5.1.0-12.el7.x86_64
metis-5.1.0-12.el7.x86_64
[root@ ~]# rpm -qa |grep metis
metis-devel-5.1.0-12.el7.x86_64
metis-5.1.0-12.el7.x86_64
[root@ ~]# rpm -qa |grep MUMPS
MUMPS-srpm-macros-5.3.5-1.el7.noarch
MUMPS-5.3.5-1.el7.x86_64
MUMPS-devel-5.3.5-1.el7.x86_64
MUMPS-common-5.3.5-1.el7.noarch
I used the openmpi/4.1.1, gcc/3.9.0, blas/3.8.0 and zlib/1.2.11 modules for this build. Below is the module list command output.
[ris_hpc_apps@r1drpswdev3 getfem-5.4.1]$ module list
Currently Loaded Modules:
1) openmpi/4.1.1 2) python/3.7.3-anaconda 3) blas/3.8.0 4) zlib/1.2.11 5) gcc/9.3.0
Here is my configuration command running on getfem-5.4.1 directory
./configure CXX="/risapps/rhel7/openmpi/4.1.1/bin/mpic++" CC="/risapps/rhel7/openmpi/4.1.1/bin/mpicc" FC="/risapps/rhel7/openmpi/4.1.1/bin/mpifort" LIBS="-L/risapps/rhel7/gcc/9.3.0/lib64 -L/risapps/rhel7/openmpi/4.1.1/lib -L/risapps/rhel7/blas/3.8.0 -L/risapps/rhel7/zlib/1.2.11/lib -L/usr/lib64 -lmetis -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common" CXXFLAGES="-I/risapps/rhel7/gcc/9.3.0/include -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include -I/usr/include -I/usr/include/MUMPS" CPPFLAGES="-I/risapps/rhel7/gcc/9.3.0/include -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include -I/usr/include -I/usr/include/MUMPS" CFLAGS="-I/risapps/rhel7/gcc/9.3.0/include -I/risapps/rhel7/openmpi/4.1.1/include -I/risapps/rhel7/zlib/1.2.11/include -I/usr/include -I/usr/include/MUMPS" PYTHON="/risapps/rhel7/python/3.7.3/bin/python" PYTHON_VERSION=3.7.3 --with-mumps="-L/usr/lib64" --with-mumps-include-dir="-I/usr/include/MUMPS" --with-blas="-L/risapps/rhel7/blas/3.8.0" --prefix=/risapps/rhel7/getfem-mpi/5.4.1 --enable-shared --enable-metis --enable-par-mumps -enable-paralevel=2
The current errors on the make command are the following:
….
libtool: compile: /risapps/rhel7/openmpi/4.1.1/bin/mpic++ -DHAVE_CONFIG_H -I. -I.. -I../src -I../src -I.. -I/usr/local/include -DGETFEM_PARA_LEVEL=2 -DGMM_USES_MPI -DGMM_USES_BLAS -DGMM_U SES_BLAS_INTERFACE -I/usr/include/MUMPS -O3 -std=c++14 -MT dal_bit_vector.lo -MD -MP -MF .deps/dal_bit_vector.Tpo -c dal_bit_vector.cc -fPIC -DPIC -o .libs/dal_bit_vector.o
In file included from ./gmm/gmm_kernel.h:49,
from getfem/bgeot_config.h:50,
from getfem/getfem_omp.h:46,
from getfem/dal_basic.h:42,
from getfem/dal_bit_vector.h:51,
from dal_bit_vector.cc:23:
./gmm/gmm_matrix.h:956:32: error: ‘MPI_Datatype’ does not name a type
956 | template <typename T> inline MPI_Datatype mpi_type(T)
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:958:10: error: ‘MPI_Datatype’ does not name a type
958 | inline MPI_Datatype mpi_type(double) { return MPI_DOUBLE; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:959:10: error: ‘MPI_Datatype’ does not name a type
959 | inline MPI_Datatype mpi_type(float) { return MPI_FLOAT; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:960:10: error: ‘MPI_Datatype’ does not name a type
960 | inline MPI_Datatype mpi_type(long double) { return MPI_LONG_DOUBLE; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:962:10: error: ‘MPI_Datatype’ does not name a type
962 | inline MPI_Datatype mpi_type(std::complex<float>) { return MPI_COMPLEX; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:963:10: error: ‘MPI_Datatype’ does not name a type
963 | inline MPI_Datatype mpi_type(std::complex<double>) { return MPI_DOUBLE_COMPLEX; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:965:10: error: ‘MPI_Datatype’ does not name a type
965 | inline MPI_Datatype mpi_type(int) { return MPI_INT; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:966:10: error: ‘MPI_Datatype’ does not name a type
966 | inline MPI_Datatype mpi_type(unsigned int) { return MPI_UNSIGNED; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:967:10: error: ‘MPI_Datatype’ does not name a type
967 | inline MPI_Datatype mpi_type(long) { return MPI_LONG; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h:968:10: error: ‘MPI_Datatype’ does not name a type
968 | inline MPI_Datatype mpi_type(unsigned long) { return MPI_UNSIGNED_LONG; }
| ^~~~~~~~~~~~
./gmm/gmm_matrix.h: In function ‘typename gmm::strongest_value_type3<V1, V2, MATSP>::value_type gmm::vect_sp(const gmm::mpi_distributed_matrix<MAT>&, const V1&, const V2&)’:
./gmm/gmm_matrix.h:1012:50: error: ‘MPI_SUM’ was not declared in this scope
1012 | MPI_Allreduce(&res, &rest, 1, mpi_type(T()), MPI_SUM,MPI_COMM_WORLD);
| ^~~~~~~
./gmm/gmm_matrix.h: In function ‘void gmm::mult_add(const gmm::mpi_distributed_matrix<MAT>&, const V1&, V2&)’:
./gmm/gmm_matrix.h:1023:20: error: there are no arguments to ‘MPI_Wtime’ that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must be available [-fpermissive]
1023 | double t_ref = MPI_Wtime();
| ^~~~~~~~~
./gmm/gmm_matrix.h:1023:20: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated)
./gmm/gmm_matrix.h:1026:21: error: there are no arguments to ‘MPI_Wtime’ that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must be available [-fpermissive]
1026 | double t_ref2 = MPI_Wtime();
| ^~~~~~~~~
./gmm/gmm_matrix.h:1028:19: error: ‘MPI_SUM’ was not declared in this scope
1028 | MPI_SUM,MPI_COMM_WORLD);
| ^~~~~~~
./gmm/gmm_matrix.h:1029:18: error: there are no arguments to ‘MPI_Wtime’ that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must be available [-fpermissive]
1029 | tmult_tot2 = MPI_Wtime()-t_ref2;
| ^~~~~~~~~
./gmm/gmm_matrix.h:1032:17: error: there are no arguments to ‘MPI_Wtime’ that depend on a template parameter, so a declaration of ‘MPI_Wtime’ must be available [-fpermissive]
1032 | tmult_tot = MPI_Wtime()-t_ref;
| ^~~~~~~~~
make[2]: *** [Makefile:941: dal_bit_vector.lo] Error 1
make[2]: Leaving directory '/risapps/build7/getfem-5.4.1/src'
make[1]: *** [Makefile:577: all-recursive] Error 1
make[1]: Leaving directory '/risapps/build7/getfem-5.4.1'
make: *** [Makefile:466: all] Error 2
I really appreciate your help. Thank you again !
Best Regards
Jinzhen Chen
From: Konstantinos Poulios <kopo@mek.dtu.dk>
Date: Tuesday, November 30, 2021 at 1:53 AM
To: "Chen,Jinzhen" <JChen24@mdanderson.org>
Subject: [EXT] Re: getfem installation
WARNING: This email originated from outside of MD Anderson. Please validate the sender's email address before clicking on links or attachments as they may not be safe.
Dear Jinzhen Chen,
Thanks for your question. Yes you should be able to compile GetFEM, also the parallel version of it on Redhat. I haven't tried it but there isn't anything distribution specific in the GetFEM code.
Having said that, the parallel version has not been tested very recently, and it might need some performance fixes from our side, to get a good scaling for Anne Ceciles problem. Having compiled the parallel version of GetFEM on your cluster is a good starting point in any case to detect bottlenecks.
The tricky parts for building GetFEM are normally how to link to mumps, metis and other dependencies. If you could send me the compilation errors that you get, either on this address or on the getfem mailing list getfem-users@nongnu.org I can try to help you to resolve the issues.
As an additional reference, some time ago, when I asked our cluster administrators to compile GetFEM they used the following configure options:
$ ../configure CXX=mpicxx CC=mpicc FC=mpifort LIBS= -L/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib -L/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib64 -L/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib64 -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/zlib/1.2.11/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/generic/numactl/2.0.11/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/libxml2/2.9.7/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/hwloc/1.11.9/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openmpi/3.0.0/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/parmetis/4.0.3/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/scalapack/204/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/scotch/6.0.4/gnu-7.3.0/lib -L/zdata/g
roups/common/nicpa/2018-feb/XeonX5550/mumps/5.1.2/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/build-tools/1.0/lib64 -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/gcc/7.3.0/lib64 -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/zlib/1.2.11/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/generic/numactl/2.0.11/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/libxml2/2.9.7/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/hwloc/1.11.9/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openmpi/3.0.0/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/parmetis/4.0.3/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/scalapack/204/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/Xeon
X5550/openblas/0.2.20/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/scotch/6.0.4/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/mumps/5.1.2/gnu-7.3.0/lib -lzmumps -ldmumps -lcmumps -lsmumps -lmumps_common -lpord -lesmumps -lscotch -lscotcherr -lparmetis -lmetis -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib -lopenblas --disable-openmp --enable-paralevel --enable-metis --enable-par-mumps --enable-python --disable-boost --disable-matlab --disable-scilab --with-blas= -L/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib -Wl,-rpath=/zdata/groups/common/nicpa/2018-feb/XeonX5550/openblas/0.2.20/gnu-7.3.0/lib -lopenblas --prefix=<change-here-for-your-installation-directory>
Best regards
Konstantinos
On Tue, 2021-11-30 at 00:24 +0000, Chen,Jinzhen wrote:
Dear Dr Konstantinos Poulios,
My name is Jinzhen Chen, a HPC administrator on MD Anderson Cancer Center, Houston, USA. I am helping a user (Dr. Lesage,Anne Cecile J) to install getfem on our HPC cluster, which OS is rhel7.9. I got its basic installed. However, When I tried to use option –enable-paralevel=2 of the configure based onhttps://getfem.org/userdoc/parallel.html, I kept getting compiling errors. Looks like some libraries are missing, not visible or not compatible. I have metis and MUMPS installed on the system and used openmpi/4.1.1 module.
My question is it is possible to install the parallel version of getfem on rhel7 ? if so, how can I contact developer or someone for the issues? I really appreciate your help and am looking forward to hearing from you.
Thank you very much !
Regards
Jinzhen Chen – HPC Team
inside:Information Services
inside:HPC Request
Email: jchen24@mdanderson.org | Tel: 713-745-6226
The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.
The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.
The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.
The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.
The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems.
| [Prev in Thread] | Current Thread | [Next in Thread] |