Profiler for GXP make
From Intrigger
Contents |
logfuse
Logfuse is a very simple software that takes logs for GXP make.
repository:
# svn co svn+ssh://tsukuba000.intrigger.omni.hpcc.jp/data/1/shibata/logfuse_rep logfuse
build:
# make all
usage:
mount:
# ./logfuse <source_dir> <mount_dir> <log_dir>
unmount:
#　fusermount -u <mount_dir>
output:
<log_dir>
With 'GXP make' on a single node:
Important: Fisrt of all, ensure that your GXP is the most recent version.
Lunch gxpc and do not explore any node.
# cd <logfuse_dir> # ./logfuse <source_dir> <mount_dir> <log_dir> # cd <mount_dir> # gxpc make -j xx -- --state_dir = <log_dir> # killall logfuse
With 'GXP make' on multi nodes and a single file system:
Important: Fisrt of all, ensure that your GXP is the most recent version.
Lunch gxpc and do explore some nodes.
# gxpc cd <logfuse_dir> # gxpc e './logfuse <source_dir> <mount_dir> <log_dir>' # cd <mount_dir> # gxpc make -j xx -- --state_dir = <log_dir> # gxpc e 'fusermount -u <mount_dir>'
make_dependancy.rb
make_dependancy.rb makes data required for simulation of the logged workflow.
usage:
# ruby make_dependancy.rb <log_dir> <dat_dir>
output:
4 files 'dependancy.dat' and 'cost.dat', 'names.dat', 'workflow.dot' will be created at the directory <dat_dir>.
'dependancy.dat':
enumerates dependency relations between files and jobs. e.g.,
F1 J1 F2 J1
means F1 & F2 -> J1, where F1 and F2 are files and J1 is a job.
'cost.dat':
Describing computational cost of tasks and size of files. e.g.,
F1 1024 J1 0.07
means that the file F1 is 1024 byte and J1 costs 0.07 sec.
'names.dat':
Real logged names,
F1 hoge.txt J1 sort hoge.txt
means that F1 represents 'hoge.txt' and J1 represents 'sort hoge.txt'.
'workflow.dot':
A "dot" file for Graphviz. If Graphviz is installed, you can generate 'workflow.html' which is a simple task graph viewer written in java-script.
How to generate is:
# ruby make_html.rb <dat_dir>
