hat.profiler

Profilers widely used for perf in HAT.

Profilers

PassThroughProfiler(dirpath, …)	This class should be used when you don’t want the (small) overhead of profiling.
SimpleProfiler(dirpath, …)	This profiler simply records the duration of actions (in seconds) and reports the mean duration of each action and the total time spent over the entire training run.
PythonProfiler(dirpath, …)	This profiler uses Python’s cProfiler to record more detailed information about time spent in each function call recorded during a given action.

API Reference

class hat.profiler.BaseProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)

If you wish to write a custom profiler, you should inherit from this class.

describe() → None

Log a profile report after the conclusion of run.

profile(action_name: str) → None

Yield a context manager to encapsulate the scope of a profiled action.

Example:

with self.profile('load training data'):
    # load training data code

The profiler will start once you’ve entered the context and will automatically stop once you exit the code block.

setup(stage: Optional[str] = None, local_rank: Optional[int] = None) → None

Execute arbitrary pre-profiling set-up steps.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

teardown(stage: Optional[str] = None) → None

Execute arbitrary post-profiling tear-down steps.

Closes the currently open file and stream.

class hat.profiler.PassThroughProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None)

This class should be used when you don’t want the (small) overhead of profiling. The Trainer uses this class by default.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

class hat.profiler.PythonProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None, line_count_restriction: float = 1.0, output_filename: Optional[str] = None)

This profiler uses Python’s cProfiler to record more detailed information about time spent in each function call recorded during a given action. The output is quite verbose and you should only use this if you want very detailed reports.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.

teardown(stage: Optional[str] = None) → None

Execute arbitrary post-profiling tear-down steps.

Closes the currently open file and stream.

class hat.profiler.SimpleProfiler(dirpath: Optional[Union[str, pathlib.Path]] = None, filename: Optional[str] = None, extended: bool = True)

This profiler simply records the duration of actions (in seconds) and reports the mean duration of each action and the total time spent over the entire training run.

start(action_name: str) → None

Define how to start recording an action.

stop(action_name: str) → None

Define how to record the duration once an action is complete.

summary() → str

Create profiler summary in text format.