04/30/2007
Most updates by John C. Vernaleo (vernaleo@astro.umd.edu)
Directed by Dr. Chris Reynolds

Center for Theory and Computation
Department of Astronomy
University of Maryland, College Park


Original code (v1.0b) Developed by:
Laboratory of Computational Astrophysics
University of Illinois at Urbana-Champaign

There is a new branch of Zeus-MP under development at UCSD (by John
Hayes) refereed to as Zeus-MP Version 2. It is a Fortran 90 rewrite of
Zeus-MP. Our version is a modified fork of the FORTRAN 77 Version 1 of
Zeus-MP.

We do ask that any research that uses our modified code acknowledges
us by including a reference to our paper: Vernaleo and Reynolds 2006
(astro-ph/0511501, ApJ 645, 83)

It would also be nice (but is in no way required) if you let us know
that you are using that code.

This version has been tested on GNU/Linux using the intel compilers, 
around 3.2 or greater versions of the GNU compilers (g77) and recent
(4.1 or greater) versions of the GNU compilers (gfortran).  
It is known NOT to work with the portland group (PG) compiler
on GNU/Linux.  It has also been run on Mac OS X 10.4 (darwin) using
the current Apple version of the GNU compilers (v4.0.0) and g77 or gfortran.
Other operating systems have not been tested although as long 
as they have the GNU compilers, lam, and the hdf4 libraries porting should 
not be a problem.  It only works consistently with the lam mpi libraries.

In src directory, configure.pl script may be altered and used to set library 
paths differently.  Current compiler options appear to give best performance.
An attempt is made to pick settings based on hostname, but only for a
few known systems (although it is easy to add your own systems to this).

Problem files and boundary update routines must be put in makefile manually,
and should follow examples in makefile (so they can go on their own lines).

Additional physics (eg, radiative cooling or kinematic viscosity) are included
in Makefile by default, and are enabled or disabled in zeusmp.def file.

Once problem files are set, and zeusmp.def is correct, compile with:
	make clean (only needed if this is not the first time compiling in 
		    this directory).
	make compile
	   or
	make compile > /dev/null or to a file (so that only errors and warnings
are shown on the screen)
           or
	make all (this make the executable and the postprocessor)

The choose.pl script in the problems directory can help by putting properly 
named zmp_inp and zeusmp.def files where they belong.  See script for more 
details.

Wrapper script (mpiwrap.pl) may be used to run executable.  It will take care
of starting and stopping lam for you.  Just edit lamhosts file and on first 
node you are running on, type:
	 ./mpiwrap.pl &
It is important to have the & because mpi programs do not always go into the 
background well after being started.  The screen output goes to 
zeusmp.x_output.txt and my be viewed by tail -f zeusmp.x_output.txt
This is better than directly running the executable since you may now log off
the node you are running on.

Wrapper automatically chooses the most recent restart dump to run from, so
for a new run, make sure no restart files are in the exe directory.

For use on cluster with the PBS batch system, the script mpisubmit.sh has been
included.  You must set the number of processors, run time, and you email 
address to use this.

---------------------------------------------------------------------------
ZMP_INP namelist options

In ggen2 you may set x2units='pi' or x2units='dg' for degrees.  Similar for x3.

Currently, you still cannot set x2min=0 in RTP, but you can set it to 
something close.

For compatability with gfortran (this also work with g77, ifort, SUNWspro f77,
f90, and f95) all namelists must end with /$.  A $ alone will not work.

-----------------------------------------------------------------------------
ZEUSMP.DEF options
problems/zeusmp.def.listing contains a nearly complete listing of options.
I am still in the process of putting notes in that to indicate which
options do not work or are incompatible with each other.

To use GNU compiler you must have no options set under operating system.  So
do not define LINUX as the Operating System.  Also, the first character in
the zmp_inp file cannot be a space with using the GNU compiler.

To use the Intel compiler, you must set the operating system as LINUX.  This 
is done automatically by the configure script if you use it.  So as long as 
you uses configure.pl and chose intel in it, you may leave Operating system
blank in the zeusmp.def file.

Define PNEWT for pseudo-newtonian force law in ZRP coordinates (provided by 
Barry McKernan).

For MHD torus (provided by Barry McKernan) , define PROBLEM torus and SPECIAL 
toruswrite.

To use Alfven speed limiter (provided by Barry McKernan for MHD torus problem)
define ALFVENLIMIT

Note, there are some options that must be defined.  If you do not want
to use the given option, it must be set to empty.  These include:
VISCOSITY, PROBLEM, SPECIAL, RESTART, and BNDYUPDATE.

NOTE:
For MHD runs with inflow boundaries, the code does NOT set the ghost
zones.  These must be set by the user in the problem file.

NOTE 2:
zeusmpv2.1.1 has a fix where the loop unrolling is remove from momx2
due to a bug that shows up in reflecting x2 bounaries in spherical
geometries.  I was not able to find the bug, so I have not fixed it.
It may not be present in this version, but it is something to look
out for.

NO_CPU_TIMER seems very similar to ARCH_CRAY from zeusmpv2.  Since I
don't expect to ever test on a Cray, I do not plan on testing this.

------------------------------------------------------------------------------
This work was made possible by financial support from NASA under the
Chandra Theory Program grant TM45007X and the National Science
Foundation under grant AST0205990.
