You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Aug 28, 2024. It is now read-only.
Copy file name to clipboardExpand all lines: ImageSegmentation/README.md
+38-8Lines changed: 38 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,13 +2,13 @@
2
2
3
3
## Introduction
4
4
5
-
This repo offers a Python script that converts the [PyTorch DeepLabV3 model](https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101) to the Mobile Interpreter version and an iOS app that uses the model to segment images.
5
+
This repo offers a Python script that converts the [PyTorch DeepLabV3 model](https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101) to both the Full JIT and the Lite/Mobile Interpreter versions and an iOS app that uses the Full JIT model to segment images. Steps of how to prepare the Lite model and make the code changes in the Xcode project to use the Lite model are also provided.
6
6
7
7
## Prerequisites
8
8
9
9
* PyTorch 1.9 and torchvision 0.10 (Optional)
10
10
* Python 3.8 or above (Optional)
11
-
* iOS Cocoapods LibTorch-Lite 1.9.0 and LibTorchvision 0.10.0
11
+
* iOS Cocoapods LibTorch 1.9.0 or LibTorch-Lite 1.9.0
12
12
* Xcode 12.4 or later
13
13
14
14
## Quick Start
@@ -17,25 +17,25 @@ To Test Run the Image Segmentation iOS App, follow the steps below:
17
17
18
18
### 1. Prepare the Model
19
19
20
-
If you don't have the PyTorch environment set up to run the script below to generate the model file, you can download it to the `ios-demo-app/ImageSegmentation` folder using the link [here](https://pytorch-mobile-demo-apps.s3.us-east-2.amazonaws.com/deeplabv3_scripted.ptl).
20
+
If you don't have the PyTorch environment set up to run the script below to generate the full JIT model file, you can download it to the `ios-demo-app/ImageSegmentation` folder using the link [here](https://pytorch-mobile-demo-apps.s3.us-east-2.amazonaws.com/deeplabv3_scripted.pt).
21
21
22
-
Open a Mac Terminal, first install PyTorch 1.9 and torchvision 0.10 using command like `pip install torch torchvision`, then run the following commands:
22
+
Open a Terminal, first install PyTorch 1.9 and torchvision 0.10 using command like `pip install torch torchvision`, then run the following commands:
23
23
24
24
```
25
25
git clone https://github.com/pytorch/ios-demo-app
26
26
cd ios-demo-app/ImageSegmentation
27
27
python deeplabv3.py
28
28
```
29
29
30
-
The Python script `deeplabv3.py` is used to generate the Lite Interpreter model file`deeplabv3_scripted.ptl` to be used in iOS.
30
+
The Python script `deeplabv3.py` is used to generate both the full JIT and the Lite Interpreter model files `deeplabv3_scripted.pt` and`deeplabv3_scripted.ptl` to be used in iOS.
31
31
32
-
### 2. Use LibTorch-Lite
32
+
### 2. Use LibTorch
33
33
34
-
Run the commands below (note the `Podfile` uses `pod 'LibTorch-Lite', '~>1.9.0'`):
34
+
Run the commands below (note the `Podfile` uses `pod 'LibTorch', '~>1.9.0'`):
35
35
36
36
```
37
37
pod install
38
-
open ImageSegmentation.xcworkspace/
38
+
open ImageSegmentation.xcworkspace
39
39
```
40
40
41
41
### 3. Run the app
@@ -46,6 +46,36 @@ Select an iOS simulator or device on Xcode to run the app. The example image and
46
46
47
47
Note that the `resized` method in `UIImage+Helper.swift` is used to speed up the model inference, but a smaller size may cause the result to be less accurate.
48
48
49
+
## Using the Lite/Mobile Interpreter Model
50
+
51
+
All the other iOS demo apps have been converted to use the new Mobile Interpreter model, except this Image Segmentation demo app, which is used to illustrate how to convert a demo using a full JIT model to one using the mobile interpreter model, by following 3 simple steps.
52
+
53
+
### 1. Prepare the Lite model
54
+
55
+
If you don't have the PyTorch environment set up to run the script `deeplabv3.py` to generate the mobile interpreter model file, you can download it to the `ios-demo-app/ImageSegmentation` folder using the link [here](https://pytorch-mobile-demo-apps.s3.us-east-2.amazonaws.com/deeplabv3_scripted.ptl). Otherwise, or if you prefer to run the script to generate the model yourself, just run `python deeplabv3.py`.
56
+
57
+
Note that to save a model in the mobile interpreter format, simply call `_save_for_lite_interpreter`, as shown at the end of the `deeplabv3.py`:
If you already went through the previous section and have the demo using the full JIT model up and running, close Xcode, go to the `ios-demo-app/ImageSegmentation` directory and run `pod deintegrate` first.
66
+
67
+
In `Podfile`, change `pod 'LibTorch', '~>1.9.0'` to `pod 'LibTorch-Lite', '~>1.9.0'`
68
+
69
+
Then run `pod install` and `open ImageSegmentation.xcworkspace`. Don't forget to drag and drop the `deeplabv3_scripted.ptl` file from step 1 to the project.
70
+
71
+
### 3. Change the iOS code
72
+
73
+
In `InferenceModule.mm`, first change `#import <LibTorch/LibTorch.h>` to `#import <Libtorch-Lite/Libtorch-Lite.h>`, then change `@protected torch::jit::script::Module _impl;` to `@protected torch::jit::mobile::Module _impl;` and `_impl = torch::jit::load(filePath.UTF8String);` to `_impl = torch::jit::_load_for_mobile(filePath.UTF8String);`.
74
+
75
+
Finally, modify `pt` in ViewController.swift's `Bundle.main.path(forResource: "deeplabv3_scripted", ofType: "pt")` to `ptl`.
76
+
77
+
Now you can build and run the app using the Lite/Mobile interpreter model.
78
+
49
79
## Tutorial
50
80
51
81
Read the tutorial [here](https://pytorch.org/tutorials/beginner/deeplabv3_on_ios.html) for detailed step-by-step instructions of how to prepare and run the [PyTorch DeepLabV3 model](https://pytorch.org/hub/pytorch_vision_deeplabv3_resnet101) on iOS, as well as practical tips on how to successfully use a pre-trained PyTorch model on iOS and avoid common pitfalls.
0 commit comments