PDCamera iOS Demo with SSD Model
- Pre-trained Models
- Demo Screenshot
- Fast Installation through QR Code
- Build from Source Code
- Prepare Models
- Prepare PaddlePaddle Inference Library
- Directory Tree
- Integrate Paddle C Library to iOS Project
This iOS demo shows PaddlePaddle running SSD(Single Shot MultiBox Detector）Object detection on iOS devices locally and offline. It loads a pretrained model with PaddlePaddle and uses camera to capture images and call PaddlePaddle's inference ability to show detected objects to users.
vgg_ssd_net models can classify 20 objects.
face_mobilenet_160_91 can only classify human's face.
|pascal_mobilenet_300_66.paddle||300 x 300||66%||23.2MB|
|vgg_ssd_net.paddle||300 x 300||71%||104.3MB|
|face_mobilenet_160_91.paddle||160 x 160||91%||18.4MB|
Simply tap on the screen to toggle settings.
- Models: Select Pascal MobileNet 300 or Face MobileNet 160, App will exit, need to launch to restart.
- Camera: Toggle Front/Back Camera. App will exit, need to launch to restart.
- Accuracy Threshold: Adjust threshold to filter more/less objects based on probability.
- Time Refresh Rate: Adjust the time to refresh bounding box more/less frequently.
Detected object will be highlighted as a bounding box with a classified object label and probability.
Fast Installation through QR Code
To simply run the demo with iPhone/iPad, scan the QR code below, click "Install PDCamera" in the link and the app will be downloaded in the background. After installed, go to Settings -> General -> Device Management -> Baidu USA llc -> Trust "Baidu USA llc"
Build from Source Code
Use latest XCode for development. This demo requires a camera for object detection, therefore you must use a device (iPhone or iPad) for development and testing. Simulators will not work as they cannot access camera.
For developers, feel free to use this as a reference to start a new project. This demo fully demonstrates how to integrate Paddle C Library to iOS and called from Swift.
Swift cannot directly call C API, in order to have client in Swift work, create Objective-C briding header and a Objective-C++ wrapper (.mm files) to access paddle APIs.
(Optional) VGG model is relatively large and takes much higher memory(~800Mb), power, and much slower (~1.5secs) on each inference but it has slightly accuracy gain (See below section) Note: Only runs on iPhone6s or above (iPhone 6 or below will crash due to memory limit) If you want to try it out, download vgg_ssd_net.paddle, then go to XCode target -> Bulid Phases -> Copy Bundle Resources, click '+' to add vgg_ssd_net.paddle
Prepare PaddlePaddle Inference Library
Follow this guide Build PaddlePaddle for iOS to generate paddle libs(include, lib, third_party). Create a folder paddle-ios and add to project root. Put the 3 paddle libs folder under paddle-ios.
$ git clone https://github.com/PaddlePaddle/Mobile.git $ cd Mobile/Demo/iOS/AICamera $ tree . ├── AICamera # sources codes ├── PDCamera.xcodeproj ├── README.md ├── README_cn.md ├── assets ├── models # models │ ├── face_mobilenet_160_91.paddle │ ├── pascal_mobilenet_300_66.paddle │ └── vgg_ssd_net.paddle └── paddle-ios # PaddlePaddle inference library ├── include ├── lib │ ├── libpaddle_capi_engine.a │ ├── libpaddle_capi_layers.a │ └── libpaddle_capi_whole.a └── third_party
Integrate Paddle C Library to iOS Project
- Add the
includedirectory to Header Search Paths
- Add the
veclib.frameworkto your project, if your PaddlePaddle is built with
- Add the libraries of paddle,
libpaddle_capi_engine.a, and all the third party libraries to your project