Notes about running Wien97 code on IBM SP machine in parallel mode

What is it ?

    Wien97 is "A Full-Potential Linearized Augmented Plane Wave Package for Calculating Crystal Properties", written in FORTRAN language. I helped one of my friends running this code on gseaborg.nersc.gov supercomputer. I'm programmer, he's scientist. I don't know FORTRAN at all, but i know UNIX and compiler technology. Here's my short notes about compiling and running this package on IBM SP supercomputer using AIX and IBM Fortran XL compiler. These notes are written for people with good knowledge of UNIX and lapw manual, they are not for begginers or people with zero knowledge of UNIX. We provide this information with hope, that it will help people avoid mistakes we made. Please consult manual and gseaborg.nersc.gov documentation first. Please note, that on gseaborg we have XL IBM Fortran compiler and all needed libraries already instaled.

If you have some questions please email me

Brief instalation guide

1. get wien97 tar files and place them in some directory in your home. Run supplied script, which will unpack .tar files.

2. run siteconfig_lapw and choose AIX OS, then f77 compiler and
   "-O -bmaxdata:0x80000000 -bmaxstack:0x8000000" options for compiler.

3. save options.

4. compile package - all programs.

5. run userconfig_lapw which will add paths to .cshrc startup file for csh.
   On gseaborg, you cannot modify .cshrc, you have to modify userconfig_lapw script. Output must be set to .cshrc.ext file in your home directory.

6. increase ulimits for user - add these lines to .cshrc (.cshrc.ext on gseaborg)
   
   limit datasize 262144000000000
   limit stacksize 262144000000000
   limit memoryuse 262144000000000
   
   because on gseaborg user limits are set too low.

7. now you can test lapw1 program by running it ./lapw1, it should work now

8. modify lapw1para and lapw2para scripts, set all marked configuration parameters to "0", we don't run rsh or delay for NFS on SP cluster.

9. now copy TIO2 (titanium dioxide) example files to scratch directory.

10. inicialize files according to manual. We had some problems running inicialization programs, they refused to work in interactive mode, this command in csh helped
    
    unsetenv mp_task_per_node
    
    

11. make LoadLeveler job file - this is important, because we will run wien in parallel mode, and we don't know nodes, that will be allocated by loadleveler, .machines file will be created on fly, when allocate nodes. This is our "GO" job file. We allocate 16 nodes (16 CPUs).
    
    #!/bin/ksh
    #@ initialdir = YOUR_SCRATCH_WITH_TIO2_FILES
    #@ class = regular
    #@ job_name = GO
    #@ output = GO.out
    #@ error = GO.err
    #@ shell = /usr/bin/csh
    #@ node = 16
    #@ tasks_per_node = 1
    #@ network.MPI = css0,not_shared,us
    #@ wall_clock_limit = 8:00:00
    #@ notification = always
    #@ job_type = parallel
    #@ queue
    # Copy each entry in the list to a new line in a file so
    # that duplicate entries can be removed.
    rm -rf YOUR_SCRATCH_WITH_TIO2_FILES/.machines
    for node in $LOADL_PROCESSOR_LIST
            do
                    echo $node|awk -F. '{print "1:"$1}' >> YOUR_SCRATCH_WITH_TIO2_FILES/.machines
            done
    
    YOUR_DIRECRY_WITH_WIEN97/run_lapw -p
    
    exit
    

12. make LoadLeveler script executable
    chmod +x

13. submit job
    llsubmit ./JOBFILE_NAME

14. you can view status using llqs command

15. wait and check *.dayfile file.

16. everything should work now

Back