PaddlePaddle/VisualDL

PaddlePaddle/VisualDL

Description: Deep Learning Visualization Toolkit（『飞桨』深度学习可视化工具 ）

Language: HTML

License: Apache-2.0

Stars: 4883

Forks: 634

Open issues: 157

Created: 2017-12-20T12:34:31Z

Pushed: 2025-01-22T06:02:52Z

Default branch: develop

Fork: no

Archived: no

README: [中文](./README_CN.md)

Introduction

VisualDL, a visualization analysis tool of PaddlePaddle, provides a variety of charts to show the trends of parameters, and visualizes model structures, data samples, histograms of tensors, PR curves , ROC curves and high-dimensional data distributions. It enables users to understand the training process and the model structure more clearly and intuitively so as to optimize models efficiently.

VisualDL provides various visualization functions, including tracking metrics in real-time, visualizing the model structure, displaying the data sample, visualizing the relationship between hyperparameters and model metrics, presenting the changes of distributions of tensors, showing the pr curves, projecting high-dimensional data to a lower dimensional space and more. Additionally, VisualDL provides VDL.service, which enables developers easily to save, track and share visualization results of experiments. For specific guidelines of each function, please refer to [VisualDL User Guide](./docs/components/README.md). For up-to-date experience, please feel free to try our **Online Demo**. Currently, VisualDL iterates rapidly and new functions will be continuously added.

Browsers supported by VisualDL are:

• Google Chrome ≥ 79
• Firefox ≥ 67
• Microsoft Edge ≥ 79
• Safari ≥ 11.1

VisualDL natively supports the use of Python. Developers can retrieve plentiful visualization results by simply adding a few lines of Python code into the model before training.

Contents

• [Key Highlights](#Key-Highlights)
• [Installation](#Installation)
• [Usage Guideline](#Usage-Guideline)
• [Function Preview](#Function-Preview)
• [Frequently Asked Questions](#Frequently-Asked-Questions)
• [Contribution](#Contribution)
• [More Details](#More-Details)
• [Technical Communication](#Technical-Communication)

Key Highlights

Easy to Use

The high-level design of API makes it easy to use. Only one click can initiate the visualization of model structures.

Various Functions

The function contains the visualization of training parameters, data samples, graph structures, histograms of tensors, PR curves and high-dimensional data distributions.

High Compatibility

VisualDL provides the visualization of the mainstream model structures such as Paddle, ONNX, Caffe, widely supporting visual analysis for diverse users.

Fully Support

By Integrating into PaddlePaddle and related modules, VisualDL allows developers to use different components without obstructions, and thus to have the best experience in the PaddlePaddle ecosystem.

Installation

Install by PiP

python -m pip install visualdl -i https://mirror.baidu.com/pypi/simple

Install by Code

git clone https://github.com/PaddlePaddle/VisualDL.git
cd VisualDL

python setup.py bdist_wheel
pip install --upgrade dist/visualdl-*.whl

Please note that Python 2 is no longer maintained officially since January 1, 2020. VisualDL now only supports Python 3 in order to ensure the usability of codes.

Usage Guideline

VisualDL stores the data, parameters and other information of the training process in a log file. Users can launch the panel to observe the visualization results.

1. Log

The Python SDK is provided at the back end of VisualDL, and a logger can be customized through LogWriter. The interface description is shown as follows:

class LogWriter(logdir=None,
max_queue=10,
flush_secs=120,
filename_suffix='',
**kwargs)

Interface Parameters

| parameters | type | meaning | | --------------- | ------- | ------------------------------------------------------------ | | logdir | string | The path location of log file. VisualDL will create a log file under this path to record information generated by the training process. If not specified, the path will be runs/${CURRENT_TIME}as default. | | max_queue | int | The maximum capacity of the data generated before recording in a log file. Default value is 10. If the capacity is reached, the data are immediately written into the log file. | | flush_secs | int | The maximum cache time of the data generated before recording in a log file. Default value is 120. When this time is reached, the data are immediately written to the log file. (When the log message queue reaches the maximum cache time or maximum capacity, it will be written to the log file immediately)| | filename_suffix | string | Add a suffix to the default log file name. | | display_name | string | This parameter is displayed in the location of Select Data Stream in the panel. If not set, the default name is logdir.(When logdir is too long or needed to be hidden). | | file_name | string | Set the name of the log file. If the file_name already exists, setting the file_name will be new records in the same log file, which will continue to be used. Note that the name should include 'vdlrecords'. |

Example

Create a log file and record scalar values:

from visualdl import LogWriter

# create a log file under `./log/scalar_test/train`
with LogWriter(logdir="./log/scalar_test/train") as writer:
# use `add_scalar` to record scalar values
writer.add_scalar(tag="acc", step=1, value=0.5678)
writer.add_scalar(tag="acc", step=2, value=0.6878)
writer.add_scalar(tag="acc", step=3, value=0.9878)
# you can also use the following method without using context manager `with`:
"""
writer = LogWriter(logdir="./log/scalar_test/train")

writer.add_scalar(tag="acc", step=1, value=0.5678)
writer.add_scalar(tag="acc", step=2, value=0.6878)
writer.add_scalar(tag="acc", step=3, value=0.9878)

writer.close()
"""

2. Launch Panel

In the above example, the log has recorded three sets of scalar values. Developers can view the visualization results of the log file through launching the visualDL panel. There are two ways to launch the log file:

Launch by Command Line

Use the command line to launch the VisualDL panel：

visualdl --logdir --model --host --port --cache-timeout --language --public-path --api-only --component_tabs

Parameter details:

| parameters | meaning | | --------------- |…