The basic procedure is transforming the model into a mlir file with model_transform.py, and then transforming the mlir into the corresponding model with model_deploy.py.
Calibration is required if you need to get the INT8 model.
The general process is shown in the figure (User interface 1).
Other complex cases such as image input with preprocessing and multiple inputs are also supported, as shown in the figure (User interface 2).
TFLite model conversion is also supported, with the following command:
Export all the results, including intermediate ones, when specified
npz_tool.py
npz will be widely used in TPU-MLIR project for saving input and output results, etc. npz_tool.py is used to process npz files.
Example:
# Check the output data in sample_out.npz
$ npz_tool.py dump sample_out.npz output
Supported functions:
npz_tool functions
Function
Description
dump
Get all tensor information of npz
compare
Compare difference of two npz files
to_dat
Export npz as dat file, contiguous binary storage
visual.py
visual.py is an visualized network/tensor compare application with interface in web browser, if accuracy of quantized network is not
as good as expected, this tool can be used to investigate the accuracy in every layer.
Example:
# use TCP port 9999 in this example
$ visual.py --fp32_mlir f32.mlir --quant_mlir quant.mlir --input top_input_f32.npz --port 9999
Supported functions:
visual 功能
Function
Description
f32_mlir
fp32 mlir file
quant_mlir
quantized mlir file
input
test input data for networks, can be in jpeg or npz format.
port
TCP port used for UI, default port is 10000,the port should be mapped when starting docker
manual_run
if net will be automaticall inferenced when UI is opened, default is false for auto inference