Skip to content

Commit e38fc2b

Browse files
rossmartinrossmartin
authored
Remove node throttle (#6)
* Remove node throttle * Update readme * Update type for fulfilled promise * Bump package
1 parent e946abe commit e38fc2b

File tree

8 files changed

+45
-142
lines changed

8 files changed

+45
-142
lines changed

‎README.md‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ yarn add react-native-use-file-upload
1414

1515
## Example App
1616

17-
There is an example app in this repo as shown in the above gif. It is located within `example` and there is a small node server script within `example/server` [here](example/server/server.ts). You can start the node server within `example` using `yarn server`. The upload route in the node server intentionally throttles requests to help simulate a real world scenario.
17+
There is an example app in this repo as shown in the above gif. It is located within `example` and there is a small node server script within `example/server` [here](example/server/server.ts). You can start the node server within `example` using `yarn server`.
1818

1919
## Usage
2020

@@ -208,23 +208,23 @@ Requests continue when the app is backgrounded on android but they do not on iOS
208208

209209
The React Native team did a heavy lift to polyfill and bridge `XMLHttpRequest` to the native side for us. [There is an open PR in React Native to allow network requests to run in the background for iOS](https://github.com/facebook/react-native/pull/31838). `react-native-background-upload` is great but if backgrounding can be supported without any external native dependencies it is a win for everyone.
210210

211-
### Why send 1 file at a time instead of multiple in a single request?
211+
### How can I throttle the file uploads so that I can simulate a real world scenario where upload progress takes time?
212212

213-
It is possible to to send multiple files in 1 request. There are downsides to this approach though and the main one is that it is slower. A client has the ability to handle multiple server connections simultaneously, allowing the files to stream in parallel. This folds the upload time over on itself.
213+
You can throttle the file uploads by using [ngrok](https://ngrok.com/) and [Network Link Conditioner](https://developer.apple.com/download/more/?q=Additional%20Tools). Once you have ngrok installed you can start a HTTP tunnel forwarding to the local node server on port 8080 via:
214214

215-
Another downside is fault tolerance. By splitting the files into separate requests, this strategy allows for a file upload to fail in isolation. If the connection fails for the request, or the file is invalidated by the server, or any other reason, that file upload will fail by itself and won't affect any of the other uploads.
216-
217-
### How does the local node server throttle the upload requests?
215+
```sh
216+
ngrok http 8080
217+
```
218218

219-
The local node server throttles the upload requests to simulate a real world scenario on a cellular connection or slower network. This helps test out the progress and timeout handling on the client. It does this by using the [node-throttle](https://github.com/TooTallNate/node-throttle) library. See the `/upload` route in [here](example/server/server.ts) for the details.
219+
ngrok will generate a forwarding URL to the local node server and you should set this as the `url` for `useFileUpload`. This will make your device/simulator make the requests against the ngrok forwarding URL.
220220

221-
### How do I bypass the throttling on the local node server?
221+
You can throttle your connection using Network Link Conditioner if needed. The existing Wifi profile with a 33 Mbps upload works well and you can add a custom profile also. If your upload speed is faster than 100 Mbps you'll see a difference by throttling with Network Link Conditioner. You might not need to throttle with Network Link Conditioner depending on your connection upload speed.
222222

223-
Set the `url` in `useFileUpload` to `http://localhost:8080/_upload`.
223+
### Why send 1 file at a time instead of multiple in a single request?
224224

225-
### The `onDone` and promise from `startUpload` take awhile to resolve in the example app.
225+
It is possible to to send multiple files in 1 request. There are downsides to this approach though and the main one is that it is slower. A client has the ability to handle multiple server connections simultaneously, allowing the files to stream in parallel. This folds the upload time over on itself.
226226

227-
This is because of the throttling and can be bypassed.
227+
Another downside is fault tolerance. By splitting the files into separate requests, this strategy allows for a file upload to fail in isolation. If the connection fails for the request, or the file is invalidated by the server, or any other reason, that file upload will fail by itself and won't affect any of the other uploads.
228228

229229
### Why is `type` and `name` required in the `UploadItem` type?
230230

‎example/package.json‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,10 @@
2424
"@types/multer": "1.4.7",
2525
"@types/node": "18.11.3",
2626
"@types/react-native-sortable-grid": "2.0.4",
27-
"@types/throttle": "1.0.1",
2827
"babel-plugin-module-resolver": "^4.1.0",
2928
"express": "4.18.2",
3029
"metro-react-native-babel-preset": "0.72.3",
3130
"multer": "1.4.5-lts.1",
32-
"throttle": "1.0.3",
3331
"ts-node": "10.9.1"
3432
}
3533
}

‎example/server/server.ts‎

Lines changed: 14 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,5 @@
11
import express from 'express';
22
import multer from 'multer';
3-
import Throttle from 'throttle';
4-
import http from 'http';
53
import os from 'os';
64

75
const app = express();
@@ -23,41 +21,20 @@ const upload = multer({
2321
);
2422
});
2523

26-
app.post('/upload', (req, res) => {
27-
console.log('/upload');
28-
console.log(`Received headers: ${JSON.stringify(req.headers)}`);
29-
30-
// Using the throttle lib here to simulate a real world
31-
// scenario on a cellular connection or slower network.
32-
// This helps test out the progress and timeout handling.
33-
34-
// The below pipes the request stream to the throttle
35-
// transform stream. Then it pipes the throttled stream data
36-
// to the "/_upload" route on this same server via http.request
37-
// Finally we pipe the response stream received from the http.request
38-
// to the original response stream on this route.
39-
const throttle = new Throttle(100 * 1024); // 100 kilobytes per second
40-
req.pipe(throttle).pipe(
41-
http.request(
42-
{
43-
host: 'localhost',
44-
path: '/_upload',
45-
port,
46-
method: 'POST',
47-
headers: req.headers,
48-
},
49-
(requestResp) => {
50-
requestResp.pipe(res);
51-
}
52-
)
53-
);
54-
});
55-
56-
app.post('/_upload', upload.single('file'), (req, res) => {
57-
console.log('req.file: ', req.file);
58-
console.log(`Wrote to: ${req.file?.path}`);
59-
res.status(200).send({ path: req.file?.path });
60-
});
24+
app.post(
25+
'/upload',
26+
(req, _res, next) => {
27+
console.log('/upload');
28+
console.log(`Received headers: ${JSON.stringify(req.headers)}`);
29+
return next();
30+
},
31+
upload.single('file'),
32+
(req, res) => {
33+
console.log('req.file: ', req.file);
34+
console.log(`Wrote to: ${req.file?.path}`);
35+
res.status(200).send({ path: req.file?.path });
36+
}
37+
);
6138

6239
return app.listen(port, () =>
6340
console.log(`Server listening on port ${port}!`)

‎example/src/App.tsx‎

Lines changed: 11 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ import FastImage from 'react-native-fast-image';
1818

1919
import ProgressBar from './components/ProgressBar';
2020
import useFileUpload, { UploadItem, OnProgressData } from '../../src/index';
21-
import { allSettled, sleep } from './util/general';
21+
import { allSettled } from './util/general';
2222
import placeholderImage from './img/placeholder.png';
2323

2424
const hapticFeedbackOptions: HapticOptions = {
@@ -44,10 +44,9 @@ export default function App() {
4444
method: 'POST',
4545
timeout: 60000, // you can set this lower to cause timeouts to happen
4646
onProgress,
47-
onDone: (_data) => {
48-
//console.log('onDone, data: ', data);
47+
onDone: ({ item }) => {
4948
updateItem({
50-
item: _data.item,
49+
item,
5150
keysAndValues: [
5251
{
5352
key: 'completedAt',
@@ -56,20 +55,18 @@ export default function App() {
5655
],
5756
});
5857
},
59-
onError: (_data) => {
60-
//console.log('onError, data: ', data);
58+
onError: ({ item }) => {
6159
updateItem({
62-
item: _data.item,
60+
item,
6361
keysAndValues: [
6462
{ key: 'progress', value: undefined },
6563
{ key: 'failed', value: true },
6664
],
6765
});
6866
},
69-
onTimeout: (_data) => {
70-
//console.log('onTimeout, data: ', data);
67+
onTimeout: ({ item }) => {
7168
updateItem({
72-
item: _data.item,
69+
item,
7370
keysAndValues: [
7471
{ key: 'progress', value: undefined },
7572
{ key: 'failed', value: true },
@@ -114,30 +111,10 @@ export default function App() {
114111
? Math.round((event.loaded / event.total) * 100)
115112
: 0;
116113

117-
// This logic before the else below is a hack to
118-
// simulate progress for any that upload immediately.
119-
// This is needed after moving to FastImage?!?!
120-
const now = new Date().getTime();
121-
const elapsed = now - item.startedAt!;
122-
if (progress === 100 && elapsed <= 200) {
123-
for (let i = 0; i <= 100; i += 25) {
124-
setData((prevState) => {
125-
const newState = [...prevState];
126-
const itemToUpdate = newState.find((s) => s.uri === item.uri);
127-
if (itemToUpdate) {
128-
// item can fail before this hack is done because of the sleep
129-
itemToUpdate.progress = itemToUpdate.failed ? undefined : i;
130-
}
131-
return newState;
132-
});
133-
await sleep(800);
134-
}
135-
} else {
136-
updateItem({
137-
item,
138-
keysAndValues: [{ key: 'progress', value: progress }],
139-
});
140-
}
114+
updateItem({
115+
item,
116+
keysAndValues: [{ key: 'progress', value: progress }],
117+
});
141118
}
142119

143120
const onPressSelectMedia = async () => {

‎example/src/util/general.ts‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,3 @@
1-
export const sleep = (time: number) =>
2-
new Promise((resolve) => setTimeout(resolve, time));
3-
41
export const allSettled = (promises: Promise<any>[]) => {
52
return Promise.all(
63
promises.map((promise) =>

‎example/yarn.lock‎

Lines changed: 2 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1170,13 +1170,6 @@
11701170
"@types/mime" "*"
11711171
"@types/node" "*"
11721172

1173-
"@types/throttle@1.0.1":
1174-
version "1.0.1"
1175-
resolved "https://registry.yarnpkg.com/@types/throttle/-/throttle-1.0.1.tgz#cbf88ec5c63b11c6466f1b73e3760ae41dc16e05"
1176-
integrity sha512-tb2KFn61P0HBt+X5uMGzqlfoSpctymCPp5pQOUDanj7GThQimvrnerQviYhIxz/+tDMEQgWXQiZlznrGIFBsbw==
1177-
dependencies:
1178-
"@types/node" "*"
1179-
11801173
"@types/yargs-parser@*":
11811174
version "21.0.0"
11821175
resolved "https://registry.yarnpkg.com/@types/yargs-parser/-/yargs-parser-21.0.0.tgz#0c60e537fa790f5f9472ed2776c2b71ec117351b"
@@ -1545,14 +1538,6 @@ buffer@^5.5.0:
15451538
base64-js "^1.3.1"
15461539
ieee754 "^1.1.13"
15471540

1548-
buffer@^6.0.3:
1549-
version "6.0.3"
1550-
resolved "https://registry.yarnpkg.com/buffer/-/buffer-6.0.3.tgz#2ace578459cc8fbe2a70aaa8f52ee63b6a74c6c6"
1551-
integrity sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==
1552-
dependencies:
1553-
base64-js "^1.3.1"
1554-
ieee754 "^1.2.1"
1555-
15561541
busboy@^1.0.0:
15571542
version "1.6.0"
15581543
resolved "https://registry.yarnpkg.com/busboy/-/busboy-1.6.0.tgz#966ea36a9502e43cdb9146962523b92f531f6893"
@@ -1886,7 +1871,7 @@ dayjs@^1.8.15:
18861871
resolved "https://registry.yarnpkg.com/dayjs/-/dayjs-1.11.6.tgz#2e79a226314ec3ec904e3ee1dd5a4f5e5b1c7afb"
18871872
integrity sha512-zZbY5giJAinCG+7AGaw0wIhNZ6J8AhWuSXKvuc1KAyMiRsvGQWqh4L+MomvhdAYjN+lqvVCMq1I41e3YHvXkyQ==
18881873

1889-
debug@2, debug@2.6.9, debug@^2.2.0, debug@^2.3.3:
1874+
debug@2.6.9, debug@^2.2.0, debug@^2.3.3:
18901875
version "2.6.9"
18911876
resolved "https://registry.yarnpkg.com/debug/-/debug-2.6.9.tgz#5d128515df134ff327e90a4c93f4e077a536341f"
18921877
integrity sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA==
@@ -2048,11 +2033,6 @@ event-target-shim@^5.0.0, event-target-shim@^5.0.1:
20482033
resolved "https://registry.yarnpkg.com/event-target-shim/-/event-target-shim-5.0.1.tgz#5d4d3ebdf9583d63a5333ce2deb7480ab2b05789"
20492034
integrity sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==
20502035

2051-
events@^3.3.0:
2052-
version "3.3.0"
2053-
resolved "https://registry.yarnpkg.com/events/-/events-3.3.0.tgz#31a95ad0a924e2d2c419a813aeb2c4e878ea7400"
2054-
integrity sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==
2055-
20562036
execa@^1.0.0:
20572037
version "1.0.0"
20582038
resolved "https://registry.yarnpkg.com/execa/-/execa-1.0.0.tgz#c6236a5bb4df6d6f15e88e7f017798216749ddd8"
@@ -2443,7 +2423,7 @@ iconv-lite@0.4.24:
24432423
dependencies:
24442424
safer-buffer ">= 2.1.2 < 3"
24452425

2446-
ieee754@^1.1.13, ieee754@^1.2.1:
2426+
ieee754@^1.1.13:
24472427
version "1.2.1"
24482428
resolved "https://registry.yarnpkg.com/ieee754/-/ieee754-1.2.1.tgz#8eb7a10a63fff25d15a57b001586d177d1b0d352"
24492429
integrity sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA==
@@ -3657,11 +3637,6 @@ process-nextick-args@~2.0.0:
36573637
resolved "https://registry.yarnpkg.com/process-nextick-args/-/process-nextick-args-2.0.1.tgz#7820d9b16120cc55ca9ae7792680ae7dba6d7fe2"
36583638
integrity sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==
36593639

3660-
process@^0.11.10:
3661-
version "0.11.10"
3662-
resolved "https://registry.yarnpkg.com/process/-/process-0.11.10.tgz#7332300e840161bda3e69a1d1d91a7d4bc16f182"
3663-
integrity sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==
3664-
36653640
promise@^8.0.3:
36663641
version "8.3.0"
36673642
resolved "https://registry.yarnpkg.com/promise/-/promise-8.3.0.tgz#8cb333d1edeb61ef23869fbb8a4ea0279ab60e0a"
@@ -3825,16 +3800,6 @@ react@18.1.0:
38253800
dependencies:
38263801
loose-envify "^1.1.0"
38273802

3828-
"readable-stream@>= 0.3.0":
3829-
version "4.2.0"
3830-
resolved "https://registry.yarnpkg.com/readable-stream/-/readable-stream-4.2.0.tgz#a7ef523d3b39e4962b0db1a1af22777b10eeca46"
3831-
integrity sha512-gJrBHsaI3lgBoGMW/jHZsQ/o/TIWiu5ENCJG1BB7fuCKzpFM8GaS2UoBVt9NO+oI+3FcrBNbUkl3ilDe09aY4A==
3832-
dependencies:
3833-
abort-controller "^3.0.0"
3834-
buffer "^6.0.3"
3835-
events "^3.3.0"
3836-
process "^0.11.10"
3837-
38383803
readable-stream@^2.2.2, readable-stream@~2.3.6:
38393804
version "2.3.7"
38403805
resolved "https://registry.yarnpkg.com/readable-stream/-/readable-stream-2.3.7.tgz#1eca1cf711aef814c04f62252a36a62f6cb23b57"
@@ -4258,13 +4223,6 @@ statuses@~1.5.0:
42584223
resolved "https://registry.yarnpkg.com/statuses/-/statuses-1.5.0.tgz#161c7dac177659fd9811f43771fa99381478628c"
42594224
integrity sha512-OpZ3zP+jT1PI7I8nemJX4AKmAX070ZkYPVWV/AaKTJl+tXCTGyVdC1a4SL8RUQYEwk/f34ZX8UTykN68FwrqAA==
42604225

4261-
"stream-parser@>= 0.0.2":
4262-
version "0.3.1"
4263-
resolved "https://registry.yarnpkg.com/stream-parser/-/stream-parser-0.3.1.tgz#1618548694420021a1182ff0af1911c129761773"
4264-
integrity sha512-bJ/HgKq41nlKvlhccD5kaCr/P+Hu0wPNKPJOH7en+YrJu/9EgqUF+88w5Jb6KNcjOFMhfX4B2asfeAtIGuHObQ==
4265-
dependencies:
4266-
debug "2"
4267-
42684226
streamsearch@^1.1.0:
42694227
version "1.1.0"
42704228
resolved "https://registry.yarnpkg.com/streamsearch/-/streamsearch-1.1.0.tgz#404dd1e2247ca94af554e841a8ef0eaa238da764"
@@ -4363,14 +4321,6 @@ throat@^5.0.0:
43634321
resolved "https://registry.yarnpkg.com/throat/-/throat-5.0.0.tgz#c5199235803aad18754a667d659b5e72ce16764b"
43644322
integrity sha512-fcwX4mndzpLQKBS1DVYhGAcYaYt7vsHNIvQV+WXMvnow5cgjPphq5CaayLaGsjRdSCKZFNGt7/GYAuXaNOiYCA==
43654323

4366-
throttle@1.0.3:
4367-
version "1.0.3"
4368-
resolved "https://registry.yarnpkg.com/throttle/-/throttle-1.0.3.tgz#8a32e4a15f1763d997948317c5ebe3ad8a41e4b7"
4369-
integrity sha512-VYINSQFQeFdmhCds0tTqvQmLmdAjzGX1D6GnRQa4zlq8OpTtWSMddNyRq8Z4Snw/d6QZrWt9cM/cH8xTiGUkYA==
4370-
dependencies:
4371-
readable-stream ">= 0.3.0"
4372-
stream-parser ">= 0.0.2"
4373-
43744324
through2@^2.0.1:
43754325
version "2.0.5"
43764326
resolved "https://registry.yarnpkg.com/through2/-/through2-2.0.5.tgz#01c1e39eb31d07cb7d03a96a70823260b23132cd"

‎package.json‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "react-native-use-file-upload",
3-
"version": "0.1.4",
3+
"version": "0.1.5",
44
"description": "A hook for uploading files using multipart form data with React Native. Provides a simple way to track upload progress, abort an upload, and handle timeouts. Written in TypeScript and no dependencies required.",
55
"main": "lib/commonjs/index",
66
"module": "lib/module/index",
@@ -38,7 +38,11 @@
3838
"keywords": [
3939
"react-native",
4040
"ios",
41-
"android"
41+
"android",
42+
"file",
43+
"upload",
44+
"uploader",
45+
"photo"
4246
],
4347
"repository": "https://github.com/rossmartin/react-native-use-file-upload",
4448
"author": "Ross Martin <2498502+rossmartin@users.noreply.github.com> (https://github.com/rossmartin)",

‎src/hooks/useFileUpload.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ export default function useFileUpload<T extends UploadItem = UploadItem>({
2121
[key: string]: XMLHttpRequest;
2222
}>({});
2323

24-
const startUpload = (item: T): Promise<OnDoneData<T> | OnErrorData<T>> => {
24+
const startUpload = (item: T): Promise<OnDoneData<T>> => {
2525
return new Promise((resolve, reject) => {
2626
const formData = new FormData();
2727
formData.append(field, item);

0 commit comments

Comments
 (0)