Skip to content
This repository was archived by the owner on Jan 10, 2025. It is now read-only.

Commit 194f096

Browse files
PierrciPierrci
committed
update docs for gpt2
1 parent 64a9f1a commit 194f096

File tree

6 files changed

+158
-85
lines changed

6 files changed

+158
-85
lines changed

‎README.md‎

Lines changed: 7 additions & 78 deletions
Original file line numberDiff line numberDiff line change
@@ -1,103 +1,32 @@
1-
# TensorFlow Lite Transformers w/ Android demo
1+
# TensorFlow Lite Transformers w/ Android demos
22

33
Convert Transformers models
44
imported from the [🤗 Transformers](https://github.com/huggingface/transformers) library
55
and use them on Android. You can also check out our
66
[swift-coreml-transformers](https://github.com/huggingface/swift-coreml-transformers) repo
77
if you're looking for Transformers on iOS.
88

9-
## DistilBERT for Question Answering
9+
# [Question Answering with DistilBERT](bert)
1010

11-
The app contains a demo of the [DistilBERT](https://arxiv.org/abs/1910.01108) model
11+
Demo of the [DistilBERT](https://arxiv.org/abs/1910.01108) model
1212
(97% of BERT’s performance on GLUE) fine-tuned for Question answering on the SQuAD dataset.
1313
It provides 48 passages from the dataset for users to choose from.
1414

1515
![demo gif](media/distilbert_qa.gif "Demo running offline on a Samsung Galaxy S8")
1616

17-
> Available models:
18-
> * "original" converted DistilBERT (254MB)
19-
> * FP16 post-training-quantized DistilBERT (131MB)
20-
> * "hybrid" (8-bits precision weights) post-training-quantized DistilBERT (64MB)
17+
# [Text Generation with GPT-2/DistilGPT2](gpt2)
2118

22-
### Coming soon: GPT-2, quantization... and much more!
19+
On-device text generation using [GPT-2](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) or [DistilGPT2](https://medium.com/huggingface/distilbert-8cf3380435b5) (same distillation process than DistilBERT, 2x faster and 33% smaller than GPT-2)
2320

24-
---
25-
26-
## Build the demo app using Android Studio
27-
28-
### Prerequisites
29-
30-
* If you don't have already, install
31-
[Android Studio](https://developer.android.com/studio/index.html), following
32-
the instructions on the website.
33-
* Android Studio 3.2 or later.
34-
* You need an Android device or Android emulator and Android development
35-
environment with minimum API 15.
36-
* The `app/libs` directory contains a custom build of
37-
[TensorFlow Lite with TensorFlow ops built-in](https://www.tensorflow.org/lite/guide/ops_select),
38-
which is used by the app. It results in a bigger binary than the "normal" build but allows
39-
compatibility with models such as DistilBERT.
40-
41-
### Building
42-
43-
* Open Android Studio, and from the Welcome screen, select `Open an existing
44-
Android Studio project`.
45-
* From the Open File or Project window that appears, select the directory where you cloned this repo.
46-
* You may also need to install various platforms and tools according to error
47-
messages.
48-
* If it asks you to use Instant Run, click Proceed Without Instant Run.
49-
50-
### Running
51-
52-
* You need to have an Android device plugged in with developer options enabled
53-
at this point. See [here](https://developer.android.com/studio/run/device)
54-
for more details on setting up developer devices.
55-
* If you already have Android emulator installed in Android Studio, select a
56-
virtual device with minimum API 15.
57-
* Click `Run` to run the demo app on your Android device.
58-
59-
## Build the demo using gradle (command line)
60-
61-
### Building and Installing
62-
63-
* Use the following command to build a demo apk:
64-
65-
```
66-
./gradlew build
67-
```
68-
69-
* Use the following command to install the apk onto your connected device:
70-
71-
```
72-
adb install app/build/outputs/apk/debug/app-debug.apk
73-
```
21+
![demo gif](media/gpt2_generation.gif "Demo running offline on a Samsung Galaxy S8, accelerated")
7422

7523
---
7624

77-
## Change the model
78-
79-
To choose which model to use in the app:
80-
* Remove/rename the current `model.tflite` file in `app/src/main/assets`
81-
* Comment/uncomment the model to download in the `download.gradle` config file:
82-
```java
83-
"https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384.tflite": "model.tflite", // <- "original" converted DistilBERT (default)
84-
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384-fp16.tflite": "model.tflite", // <- fp16 quantized version of DistilBERT
85-
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384-8bits.tflite": "model.tflite", // <- hybrid quantized version of DistilBERT
86-
```
87-
8825
## Models generation
8926

90-
Example scripts used to convert models are available in the `models_generation` directory.
27+
Example scripts used to convert the models are available in the `models_generation` directory.
9128
Please note that they require the nightly version of TensorFlow and might thus be unstable.
9229

93-
---
94-
95-
## Credits
96-
97-
The Android app is forked from the `bertqa` example in the
98-
[tensorflow/examples](https://github.com/tensorflow/examples) repository and uses the same
99-
tokenizer with DistilBERT.
100-
10130
## License
10231

10332
[Apache License 2.0](LICENSE)

‎bert/README.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# Question Answering with DistilBERT
2+
3+
The app contains a demo of the [DistilBERT](https://arxiv.org/abs/1910.01108) model
4+
(97% of BERT’s performance on GLUE) fine-tuned for Question answering on the SQuAD dataset.
5+
It provides 48 passages from the dataset for users to choose from.
6+
7+
![demo gif](../media/distilbert_qa.gif "Demo running offline on a Samsung Galaxy S8")
8+
9+
> Available models:
10+
> * "original" converted DistilBERT (254MB)
11+
> * FP16 post-training-quantized DistilBERT (131MB)
12+
> * "hybrid" (8-bits precision weights) post-training-quantized DistilBERT (64MB)
13+
14+
## Build the demo app using Android Studio
15+
16+
### Prerequisites
17+
18+
* If you don't have already, install
19+
[Android Studio](https://developer.android.com/studio/index.html), following
20+
the instructions on the website.
21+
* Android Studio 3.2 or later.
22+
* You need an Android device or Android emulator and Android development
23+
environment with minimum API 26.
24+
* The `libs` directory contains a custom build of
25+
[TensorFlow Lite with TensorFlow ops built-in](https://www.tensorflow.org/lite/guide/ops_select),
26+
which is used by the app. It results in a bigger binary than the "normal" build but allows
27+
compatibility with DistilBERT.
28+
29+
### Building
30+
31+
* Open Android Studio, and from the Welcome screen, select `Open an existing
32+
Android Studio project`.
33+
* From the Open File or Project window that appears, select the directory where you cloned this repo.
34+
* You may also need to install various platforms and tools according to error
35+
messages.
36+
* If it asks you to use Instant Run, click Proceed Without Instant Run.
37+
38+
### Running
39+
40+
* You need to have an Android device plugged in with developer options enabled
41+
at this point. See [here](https://developer.android.com/studio/run/device)
42+
for more details on setting up developer devices.
43+
* If you already have Android emulator installed in Android Studio, select a
44+
virtual device with minimum API 26.
45+
* Be sure the `bert` configuration is selected
46+
* Click `Run` to run the demo app on your Android device.
47+
48+
## Build the demo using gradle (command line)
49+
50+
From the repository root location:
51+
52+
* Use the following command to build a demo apk:
53+
54+
```
55+
./gradlew :bert:build
56+
```
57+
58+
* Use the following command to install the apk onto your connected device:
59+
60+
```
61+
adb install bert/build/outputs/apk/debug/bert-debug.apk
62+
```
63+
64+
## Change the model
65+
66+
To choose which model to use in the app:
67+
* Remove/rename the current `model.tflite` file in `src/main/assets`
68+
* Comment/uncomment the model to download in the `download.gradle` config file:
69+
```java
70+
"https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384.tflite": "model.tflite", // <- "original" converted DistilBERT (default)
71+
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384-fp16.tflite": "model.tflite", // <- fp16 quantized version of DistilBERT
72+
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384-8bits.tflite": "model.tflite", // <- hybrid quantized version of DistilBERT
73+
```
74+
75+
## Credits
76+
77+
The Bert QA app is forked from the `bertqa` example in the
78+
[tensorflow/examples](https://github.com/tensorflow/examples) repository and uses the same
79+
tokenizer with DistilBERT.

‎gpt2/README.md‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Text Generation with GPT2/DistilGPT2
2+
3+
On-device text generation app using [GPT-2](https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf) or [DistilGPT2](https://medium.com/huggingface/distilbert-8cf3380435b5) (same distillation process than DistilBERT, 2x faster and 33% smaller than GPT-2)
4+
5+
![demo gif](../media/gpt2_generation.gif "Demo running offline on a Samsung Galaxy S8, accelerated")
6+
7+
> Available models: GPT-2 (small), GPT-2-FP16 (small), DistilGPT2
8+
9+
## Build the demo app using Android Studio
10+
11+
### Prerequisites
12+
13+
* If you don't have already, install
14+
[Android Studio](https://developer.android.com/studio/index.html), following
15+
the instructions on the website.
16+
* Android Studio 3.2 or later.
17+
* You need an Android device or Android emulator and Android development
18+
environment with minimum API 26.
19+
20+
### Building
21+
22+
* Open Android Studio, and from the Welcome screen, select `Open an existing
23+
Android Studio project`.
24+
* From the Open File or Project window that appears, select the directory where you cloned this repo.
25+
* You may also need to install various platforms and tools according to error
26+
messages.
27+
* If it asks you to use Instant Run, click Proceed Without Instant Run.
28+
29+
### Running
30+
31+
* You need to have an Android device plugged in with developer options enabled
32+
at this point. See [here](https://developer.android.com/studio/run/device)
33+
for more details on setting up developer devices.
34+
* If you already have Android emulator installed in Android Studio, select a
35+
virtual device with minimum API 26.
36+
* Be sure the `gpt2` configuration is selected
37+
* Click `Run` to run the demo app on your Android device.
38+
39+
## Build the demo using gradle (command line)
40+
41+
From the repository root location:
42+
43+
* Use the following command to build a demo apk:
44+
45+
```
46+
./gradlew :gpt2:build
47+
```
48+
49+
* Use the following command to install the apk onto your connected device:
50+
51+
```
52+
adb install gpt2/build/outputs/apk/debug/gpt2-debug.apk
53+
```
54+
55+
## Change the model
56+
57+
To choose which model to use in the app:
58+
* Remove/rename the current `model.tflite` file in `src/main/assets`
59+
* Comment/uncomment the model to download in the `download.gradle` config file:
60+
```java
61+
"https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-fp16-64.tflite": "model.tflite", // <- fp16 quantized version of gpt-2 (small) (default)
62+
// "https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-64.tflite": "model.tflite", // <- "original" gpt-2 (small)
63+
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilgpt2-64.tflite": "model.tflite", // <- distilled version of gpt-2 (small)
64+
```

‎gpt2/build.gradle‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,6 @@ dependencies {
5757

5858
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.2'
5959
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.2'
60-
implementation 'androidx.lifecycle:lifecycle-livedata-ktx:2.2.0-rc02'
61-
implementation 'androidx.lifecycle:lifecycle-viewmodel-ktx:2.2.0-rc02'
62-
implementation 'androidx.activity:activity-ktx:1.1.0-rc02'
60+
implementation 'androidx.lifecycle:lifecycle-viewmodel-ktx:2.2.0-rc03'
61+
implementation 'androidx.activity:activity-ktx:1.1.0-rc03'
6362
}

‎gpt2/download.gradle‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,16 @@ apply plugin: 'de.undercouch.download'
22

33
task downloadLiteModel {
44
def downloadFiles = [
5-
'https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json': 'gpt2-vocab.json',
6-
'https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt': 'gpt2-merges.txt',
7-
// 'https://s3.amazonaws.com/models.huggingface.co/bert/distilbert-base-uncased-distilled-squad-384.tflite': 'model.tflite',
5+
"https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json": "gpt2-vocab.json",
6+
"https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt": "gpt2-merges.txt",
7+
"https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-fp16-64.tflite": "model.tflite",
8+
// "https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-64.tflite": "model.tflite",
9+
// "https://s3.amazonaws.com/models.huggingface.co/bert/distilgpt2-64.tflite": "model.tflite",
810
]
911
downloadFiles.each { key, value ->
1012
download {
1113
src key
12-
dest "$projectDir/src/main/assets/" + value
14+
dest "$projectDir/src/main/assets/$value"
1315
overwrite false
1416
}
1517
}

‎media/gpt2_generation.gif‎

711 KB
Loading

0 commit comments

Comments
 (0)