@ccjjww1222

• rknn-toolkit是用于转换RK1代NPU模型的库。

• rk3588s是二代NPU，PC上转换模型的库需要安装的是rknn-toolkit2，板子上安装的叫做rknn_toolkit_lite2

@ccjjww1222 已经发送，有问题请在这个帖子持续回复

@ccjjww1222 NHWC这个配置不需要更改，输入图片不做nhwc转nchw的preprocess就行了。重要的是你需要更改目标板子的型号，你导出的模型是RK356X的，所以在RK3588上跑不通。请查询一下rknn-toolkit的文档，没记错的话应该是config函数存在这个配置选项。

@ccjjww1222

• NPU需要系统sudo权限，请使用sudo -E python3 xxx.py来运行代码。

• 使用yolov5请注意版本，目前测试yolov5 6.0的代码存在bug。可以考虑使用RK官方改进的yolov5做代替。

@ccjjww1222

在板子上安装rknn_toolkit_lite2我们提供了两种方案

• 第一种方案请参考我写的帖子

• 第二种方案是手动安装，您可以留个邮箱地址，我给您发PDF文档。

@ccjjww1222

有问题无非两种情况。

• 第一种情况是模型本身就有问题。
• 第二种情况是模型没问题，rknn推理有问题。

排查的方法有两种。

• 第一种:针对RKNN推理精度问题，rknn-toolkit提供了连板调试的方法，但是我们还未开放这个方法。
• 第二种：请先用onnxruntime跑一遍onnx模型的推理结果，如果onnx推理出来的结果出错，说明是模型本身的问题。如果onnx推理没错，rknn错了，你再留言。

针对这个仓库的话，RKNN精度应该是没问题的，这个RK那边已经测试过了。只可能是模型不匹配，毕竟不可能100%都检测出来。

@1103854485 详细安装文档参考https://github.com/PaddlePaddle/FastDeploy/blob/develop/docs/cn/faq/rknpu2/install_rknn_toolkit2.md

Brief Introduction

FastDeploy is an easy-to-use and high performance AI model deployment toolkit for Cloud, Mobile and Edge with out-of-the-box and unified experience, end-to-end optimization for over 150+ Text, Vision, Speech and Cross-modal AI models. Including image classification, object detection, image segmentation, face detection, face recognition, keypoint detection, matting, OCR, NLP, TTS and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform.

We spent a lot of time on the adaptation of FastDeploy for Coolpi. The model we have adapted can be used in Detection, Face Detection, Face Recognition, Face Alignment, Segmentation, OCR and other fields. These models will greatly improve Coolpi's ability in edge computing.

Model List

In order to facilitate you to choose a reasonable model to apply to your project, we have provided the running speed of the model.

Demo

• Face Detection with SCRFD
• Face Recognition with ArcFace
• Try to deploy YOLOv5 on coolpi

Contribution

Thanks to Baidu FastDeploy team for making such a good tool so that we can easily deploy AI model.

Brief Introduction

With the improvement of AI computing power, Face Detection algorithms have gradually shifted from machine learning to deep learning. Through this tutorial, you will learn how to use FastDeploy to quickly implement face detection on Coolpi.

What is SCRFD

SCRFD is an efficient high accuracy face detection approach which initially described in Arxiv, and accepted by ICLR-2022. In order to facilitate you to quickly understand the parameters of the model, we only give the parameters of common models. If you need to view the detailed data of the model, please go to InsightFace's official Github.

Workflow

# Download CoolPI-AI
git clone https://github.com/yanyitech/coolpi-ai.git

# Go to face detection demo
cd example/face_detection

# Build and make
mkdir build
cd build
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/../../../third_party/fastdeploy-develop
make -j8

# Download model and picture
wget https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640_rknpu2.zip
unzip scrfd_500m_bnkps_shape640x640_rknpu2.zip
wget https://raw.githubusercontent.com/DefTruth/lite.ai.toolkit/main/examples/lite/resources/test_lite_face_detector_3.jpg

# Run
./infer_with_scrfd scrfd_500m_bnkps_shape640x640_rknpu2/scrfd_500m_bnkps_shape640x640_rk3588_quantized.rknn \
              test_lite_face_detector_3.jpg \
              1

How to transform the model

If you need to convert your ONNX model to RKNN model, FastDeploy also provides the corresponding method. You only need to modify the configuration file of the model in the tools/rknpu2/config to achieve rapid model conversion

git clone https://github.com/PaddlePaddle/FastDeploy.git
wget  https://bj.bcebos.com/paddlehub/fastdeploy/rknpu2/scrfd_500m_bnkps_shape640x640.zip
unzip scrfd_500m_bnkps_shape640x640.zip
python  /Path/To/FastDeploy/tools/rknpu2/export.py \
        --config_path tools/rknpu2/config/scrfd_quantized.yaml \
        --target_platform rk3588

Result

Input Data

Output Data

Navigation

• Introduction to FastDeploy
• Introduction to SCRFD

Introduction

YOLOv5 is the world's most loved vision AI, representing Ultralytics open-source research into future vision AI methods, incorporating lessons learned and best practices evolved over thousands of hours of research and development.

FastDeploy is an open source and very easy to use warehouse, which can quickly deploy YOLOv5 on coolpi-4B.

Model Speed Table

In addition to YOLOv5, we also support the deployment of YOLOX, YOLOv7, and PPYOLOE. The following speeds are the end-to-end speeds of the model:

C++ Demo

It is difficult to develop AI applications using C++, but it can be matched with other useful libraries on CoolPI-4B. For your convenience, we provide a demo of C++.

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"

void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {
  auto option = fastdeploy::RuntimeOption();
  option.UseRKNPU2();

  auto format = fastdeploy::ModelFormat::RKNN;

  auto model = fastdeploy::vision::detection::RKYOLOV5(
      model_file, option,format);

  auto im = cv::imread(image_file);

  fastdeploy::vision::DetectionResult res;
  fastdeploy::TimeCounter tc;
  tc.Start();
  if (!model.Predict(im, &res)) {
    std::cerr << "Failed to predict." << std::endl;
    return;
  }
  auto vis_im = fastdeploy::vision::VisDetection(im, res,0.5);
  tc.End();
  tc.PrintInfo("RKYOLOV5 in RKNN");
  std::cout << res.Str() << std::endl;
  cv::imwrite("vis_result.jpg", vis_im);
  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

int main(int argc, char* argv[]) {
  if (argc < 3) {
    std::cout
        << "Usage: infer_demo path/to/model_dir path/to/image run_option, "
           "e.g ./infer_model ./picodet_model_dir ./test.jpeg"
        << std::endl;
    return -1;
  }

  RKNPU2Infer(argv[1], argv[2]);

  return 0;
}

Python Demo

In addition to using C++, you can also use FastDeploy's Python API for rapid development.

# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fastdeploy as fd
import cv2
import os


def parse_arguments():
    import argparse
    import ast
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_file", required=True, help="Path of rknn model.")
    parser.add_argument(
        "--image", type=str, required=True, help="Path of test image file.")
    return parser.parse_args()


if __name__ == "__main__":
    args = parse_arguments()

    model_file = args.model_file
    params_file = ""

    # 配置runtime，加载模型
    runtime_option = fd.RuntimeOption()
    runtime_option.use_rknpu2()

    model = fd.vision.detection.RKYOLOV5(
        model_file,
        runtime_option=runtime_option,
        model_format=fd.ModelFormat.RKNN)

    # 预测图片分割结果
    im = cv2.imread(args.image)
    result = model.predict(im)
    print(result)

    # 可视化结果
    vis_im = fd.vision.vis_detection(im, result, score_threshold=0.5)
    cv2.imwrite("visualized_result.jpg", vis_im)
    print("Visualized result save in ./visualized_result.jpg")

Result

input image

输入图片

output image

I'm looking forward to it.

@jugg 解码和播放运行第一条命令应该就有了？

@jugg 硬解码CPU占用能到50?

@jugg https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu-ports/ 换成这个源试试

@MSUMMERS999 Maybe you need to try burning the image again?

@qwe 具体报什么错误，截图看看

Introduction

YOLOv5 is the world's most loved vision AI, representing Ultralytics open-source research into future vision AI methods, incorporating lessons learned and best practices evolved over thousands of hours of research and development.

FastDeploy is an open source and very easy to use warehouse, which can quickly deploy YOLOv5 on coolpi-4B.

Model Speed Table

In addition to YOLOv5, we also support the deployment of YOLOX, YOLOv7, and PPYOLOE. The following speeds are the end-to-end speeds of the model:

C++ Demo

It is difficult to develop AI applications using C++, but it can be matched with other useful libraries on CoolPI-4B. For your convenience, we provide a demo of C++.

// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "fastdeploy/vision.h"

void RKNPU2Infer(const std::string& model_file, const std::string& image_file) {
  auto option = fastdeploy::RuntimeOption();
  option.UseRKNPU2();

  auto format = fastdeploy::ModelFormat::RKNN;

  auto model = fastdeploy::vision::detection::RKYOLOV5(
      model_file, option,format);

  auto im = cv::imread(image_file);

  fastdeploy::vision::DetectionResult res;
  fastdeploy::TimeCounter tc;
  tc.Start();
  if (!model.Predict(im, &res)) {
    std::cerr << "Failed to predict." << std::endl;
    return;
  }
  auto vis_im = fastdeploy::vision::VisDetection(im, res,0.5);
  tc.End();
  tc.PrintInfo("RKYOLOV5 in RKNN");
  std::cout << res.Str() << std::endl;
  cv::imwrite("vis_result.jpg", vis_im);
  std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

int main(int argc, char* argv[]) {
  if (argc < 3) {
    std::cout
        << "Usage: infer_demo path/to/model_dir path/to/image run_option, "
           "e.g ./infer_model ./picodet_model_dir ./test.jpeg"
        << std::endl;
    return -1;
  }

  RKNPU2Infer(argv[1], argv[2]);

  return 0;
}

Python Demo

In addition to using C++, you can also use FastDeploy's Python API for rapid development.

# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import fastdeploy as fd
import cv2
import os


def parse_arguments():
    import argparse
    import ast
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "--model_file", required=True, help="Path of rknn model.")
    parser.add_argument(
        "--image", type=str, required=True, help="Path of test image file.")
    return parser.parse_args()


if __name__ == "__main__":
    args = parse_arguments()

    model_file = args.model_file
    params_file = ""

    # 配置runtime，加载模型
    runtime_option = fd.RuntimeOption()
    runtime_option.use_rknpu2()

    model = fd.vision.detection.RKYOLOV5(
        model_file,
        runtime_option=runtime_option,
        model_format=fd.ModelFormat.RKNN)

    # 预测图片分割结果
    im = cv2.imread(args.image)
    result = model.predict(im)
    print(result)

    # 可视化结果
    vis_im = fd.vision.vis_detection(im, result, score_threshold=0.5)
    cv2.imwrite("visualized_result.jpg", vis_im)
    print("Visualized result save in ./visualized_result.jpg")

Result

input image

输入图片

output image

@retroman https://github.com/yanyitech/coolpi_4B_docs

@retroman CoolPI Forum is an open forum where developers from all countries communicate. We have created Github Docs Repo. If you have other documents that we need to sort out, please refer to issues.

@ccjjww1222 您误会我的意思了，我的意思是你可以用电脑利用转接器连接Cool-pi开发板的串口。使用串口输入账号密码后就是命令行了。