PaddlePaddle/paddle-ce-latest-kpis

PaddlePaddle/paddle-ce-latest-kpis

Description: Paddle Continuous Evaluation, keep updating.

Language: Python

Stars: 25

Forks: 39

Open issues: 7

Created: 2018-04-02T02:40:42Z

Pushed: 2021-12-13T11:27:18Z

Default branch: develop

Fork: no

Archived: yes

README:

Paddle Continuous Evaluation Baselines

Howtos

Add New Evaluation Task

Reference mnist task, the following files are required by CE framework:

• run.xsh , a script to start this evaluation execution
• this script can be any bash script, just place #!/bin/bash or

#/bin/xonsh to the head if it is written in the bash or xonsh language

• continuous_evaluation.py to include all the KPIs this task tracks
• latest_kpis directory, include all the baseline files

PR and Add to Service

• PR to fast branch, and run ce-kpi-fast-test test on teamcity,
• if passed, PR from fast to master branch.

Add new KPI to track

Reference the interface kpi.py, there are two basic KPIs:

• LessWorseKpi
• GreaterWorseKpi