Jump to content
Sign in to follow this  
yao

What do the timing statistics mean at the end of the AcuSolve Log file, and why is there a difference between the CPU and Elpased time?

Recommended Posts

AcuSolve performs profiling of each run to determine where the majority of the CPU/elapsed time is spent during the simulation. The results of this profiling are printed to the .Log file at the conclusion of each run, and can be reviewed by users to assess the performance of the code on their computing system. The results from a representative .Log file will be reviewed to explain the output.

Following the completion of the last time step, AcuSolve will print information regarding the input/output time, message passing time (for distributed memory parallel simulations), interface search time (if non-conformal interfaces are present), time associated with the solution of each stagger, and total time to complete the simulation. The memory usage of the simulation is also reported in the log file. For distributed memory parallel simulations, the average and maximum CPU Times and Memory usage are printed to indicate if reasonable load balancing was achieved by the domain decomposition. The following output shows a summary at the end of a typical distributed memory parallel AcuSolve simulation that solved for flow, turbulence, temperature, and enclosure radiation. No INTERFACE_SURFACES were defined for the simulation.


acuSolve-hpmpi: ----------------------- End Time Step ----------------------- 
acuSolve-hpmpi: Input CPU/Elapse time= 5.873000e+01 2.292031e+02 Sec 
acuSolve-hpmpi: Output CPU/Elapse time= 1.872900e+02 2.104051e+02 Sec 
acuSolve-hpmpi: Exchange CPU/Elapse time= 1.425620e+03 2.088389e+03 Sec 
acuSolve-hpmpi: Intf.search CPU/Elapse time= 0.000000e+00 0.000000e+00 Sec 
acuSolve-hpmpi: Flow stagger "flow": 
acuSolve-hpmpi: No of res/lhs formations = 100 100 
acuSolve-hpmpi: Elem.Form CPU/Elapse time= 4.605900e+02 6.453795e+02 Sec 
acuSolve-hpmpi: Solution CPU/Elapse time= 4.722590e+03 4.884696e+03 Sec 
acuSolve-hpmpi: Turbulence stagger "turbulence": 
acuSolve-hpmpi: No of res/lhs formations = 100 100 
acuSolve-hpmpi: Elem.Form CPU/Elapse time= 2.556100e+02 2.722640e+02 Sec 
acuSolve-hpmpi: Solution CPU/Elapse time= 3.474900e+02 3.841555e+02 Sec 
acuSolve-hpmpi: Temperature stagger "temperature": 
acuSolve-hpmpi: No of res/lhs formations = 100 100 
acuSolve-hpmpi: Elem.Form CPU/Elapse time= 2.524900e+02 2.791516e+02 Sec 
acuSolve-hpmpi: Solution CPU/Elapse time= 3.178700e+03 3.378577e+03 Sec 
acuSolve-hpmpi: Radiation stagger "radiation": 
acuSolve-hpmpi: No of res/lhs formations = 100 100 
acuSolve-hpmpi: Elem.Form CPU/Elapse time= 1.380000e+00 9.432842e+00 Sec 
acuSolve-hpmpi: Solution CPU/Elapse time= 8.090000e+00 2.437304e+01 Sec 
acuSolve-hpmpi: Total CPU/Elapse time = 9.569710e+03 1.048043e+04 Sec 
acuSolve-hpmpi: Total Memory Usage = 1.474883e+02 Mbytes 
acuSolve-hpmpi: Ave Processor CPU time = 8.891656e+03 Sec 
acuSolve-hpmpi: Max Processor CPU time = 1.028060e+04 Sec 
acuSolve-hpmpi: Ave Processor Memory Usage = 1.477044e+02 Mbytes 
acuSolve-hpmpi: Max Processor Memory Usage = 1.579727e+02 Mbytes 
acuRun: -*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-

  1. The Input and Output times report the amount of time that AcuSolve spent reading and writing files to disk. The differences between the input and output CPU/Elapsed times are caused by file system delays. When running AcuSolve in parallel, many processes may be trying to read/write to the same NFS mounted disk. When this happens, delays can occur, forcing AcuSolve to wait for file I/O. This causes the Elapsed time to increase while no CPU time is being used. Differences in CPU/Elapsed time can also be caused by processes competing for run time on an overloaded machine.

  2. The Exchange time indicates how much time AcuSolve spent passing messages between processes. For most simulations, the value of the Exchange time is 20% or less of the total time required to complete the simulation. When there is a significant difference between the CPU/Elapsed time, the implication is that there is a message passing bottle neck somewhere. This can be caused by a poorly performing network, or a switch that is unable to handle all of the traffic that it is receiving.

  3. The Intf. Search time reports the amount of time that AcuSolve spends searching for matching faces for nonconformal interfaces (i.e. INTERFACE_SURFACE command). When a simulation containing INTERFACE_SURFACES is performed in a distributed memory parallel configuration, the search algorithm must communicate with other processes to find matching faces. Because of this, a significant amount of message passing must occur. When significant differences in CPU/Elapsed time are seen for this metric, it is typically due to delays in exchanging messages across the network/switch while searching for interface matches.

  4. For each stagger that is solved, AcuSolve reports a total of 3 metrics: No. of res/lhs formations, Elem. Form time, and Solution time.

    1. The No. of res/lhs formations is a tally of how many times the residual of the equation was computed, as well as how many times the left-hand-side matrix was formed. By default, these values should match each other, and reflect the total number of times the equation was solved. It should be noted that the LHS matrix formation frequency can be adjusted, causing these two values to be different. However, this is an advanced option, and is rarely changed.

    2. The Elem. Form time refers to the amount of time required to form the LHS matrix and loop over all of the elements to compute the residual of the equations. Any significant deviations between the CPU/Elapsed time for this metric are primarily due to delays in exchanging messages across the network/switch or competing processes on a machine that is overloaded.

    3. The Solution time that is reported for each stagger refers to the amount of time required by the linear solver routines to converge to the desired tolerance. The total elapsed time of the simulation is typically dominated by the solution time for the various staggers. Any significant deviations between the CPU/Elapsed time for this metric are primarily due to delays in exchanging messages across the network/switch or competing processes on a machine that is overloaded.

  5. The Total CPU/Elapsed time represents the accounting of all steps that AcuSolve performs in completing the simulation. Any significant deviations between the CPU/Elapsed time for this metric are primarily due to delays in exchanging messages across the network/switch or competing processes on a machine that is overloaded. One other possible cause of large differences is running the machine out of memory, forcing it to write data to swap files. Note that simply summing all other timing metrics in the summary of the log file will not yield the same value as the total CPU/Elapsed time. There are additional operations in the solver that are not profiled due to their insignificance.

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
You are posting as a guest. If you have an account, please sign in.
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this  

×
×
  • Create New...