The Ohio State University   |   Department of Physics

StormRT (http://stormrt.sourceforge.net)

StormRT is a real-time multi-resolution analysis and compression package for data compression and temporal feature detection over multiple timescales.  It was originally developed to provide compressed storage of molecular dynamics trajectories in long-time moleclular dynamics simulations for research performed at the Ohio State University.  The technique, designated Real-Time Multi-Resolution Analysis (RTMRA), employed a real-time Haar wavelet transform using an O(n) algorithm suitable for simulations running on high-performance computing platforms.  The capability of StormRT was extended to provide real-time temporal feature detection during simulations.  Using redundant-staggerd transforms, stable particle structures and transition events could be detected over multiple time-scales with good reliability by confirmed using more costly structural minimization techniques.  The feature detection capability reqired a modified transform, designated Real-Time Redundant-Stagger Multi-Resolution Analysis (RTRSMRA) in order to remove the "detection blind spot" of a standart Haar transform.  StormRT evolved into a C++ class library providing engines for performing RTMRA and RTRSMRA suitable for integration with the out engines of simulation codes (currently implemented in OHMMS and LAMMPS).  StormRT was developed by David A. Richie.

 

 

Real-Time Multi-Resolution Analysis

  • Atomic coordinates are interpreted as time-series and projected onto wavelet bases (Haar wavelet).
  • Analysis is carried out over multiple time & frequency scales (m-level).
  • Data compression is accomplished by thresholding of expansion coefficients.
  • Expasion coefficients are analyzed to detect features (meta-stable and transition states) in dynamics.

 

 

StormRT library (version 3.2.2) performance

StormRT library greatly reduces output time of simulations.  Tests were performed on the Glenn cluster at OSC (single processor : AMD Opteron 2.6 GHz) with LAMMPS.

Simulation for 40960 steps on 1 procs with 8000 Si atoms

Simple XYZ, output every step, no stride

RTMRA output (compressed mcf files)

StormRT compiled with Intel

StormRT compiled with gcc

Threshold = 0.1 Threshold = 3.2 Threshold = 0.1 Threshold = 3.2

Total time (sec)

2447        

1608        

1568        

1714        

1598        

Output time (sec)

1063        

224        

181        

320        

206        

%

43        

14        

12        

19        

13        

  • The average output time with RTMRA is about 14 % of total simulation time, which is significantly lower than the conventional dumping.
  • The performance slightly depends on the compiler that is used to build StormRT library, but with higher threshold values the difference is negligible.

 

Similar comparison between Intel and gcc compiler with different simulation size (1000, 8000, and 64000 Si atoms).

Simulation for 40960 steps on 1 procs

StormRT compiled with Intel

StormRT compiled with gcc

# of Si atoms

1000

8000

64000

1000

8000

64000

RTMRA threshold

0.1

3.2

0.1

3.2

0.1

3.2

0.1

3.2

0.1

3.2

0.1

3.2

Total time (s)

192

177

1642

1609

13648

13172

193.5

179

1729

1626

14429

13293

Output time (s)

25

11

245

215

2089

1710

26.0

13

335

228

2818

1818

%

13

6

15

13

15

13

13.4

7

19

14

20

14