Skip to content

Commit c8dee19

Browse files
author
Raphaël Droz
authored
Asynchronous file bootstrap hooks, aka async initFileFn (#329)
## Ability to use asynchronous initFileFn. This is a follow-up (and proper implementation) of #296 (reverted in e867d54) This could be use to initialize a stream, a reader, fetch a remote resource, ... during the FlowFile initialization. `asyncAddFile` and `asyncAddFiles` are introduced which support this mechanism. These function return promises of one (or multiple) FlowFile(s). To implement this: - An AsyncFlowFile class extending a FlowFile is created, overriding the bootstrap part - In order to keep code-duplication low, a filterFileList generator yield files in way common to async and non-async addFiles() functions. The possibly async' nature of initializing file affects events firing. (#319) in general and `fileAdded` in particular (#271) and the current confusion between hooks altering bootstraping and simple events. - fileAdded is now assumed an event. If we detect a return value, a warning is sent. - preFilterFile is introduced and allow filtering the raw file before bootstrap() - postFilterFile is introduced and allow filtering the FlowFile after a (possibly async) bootstrap() - filesAdded is preserved About parameters: - `initFileFn` is only used during bootstrap - If such a helper exists, it's tightly related to addFile* functions. As such, it was deemed adequate to provide it as a parameter to `*addFile*()` (but `opts.initFileFn` is still supported) An `Eventizer` class is added where all the necessary code exist. - Hooks alter code processing and could be `synchronous` or not. - Events are CustomEvent dispatched asynchronously for informative purpose. A warning is triggered if an obsolete event name is used ## Other changes * uploadNextChunk: use transpilation instead of relying upon the each() helper * FlowFile: don't bootstrap within the constructor. This must be done manually now * README/CHANGELOG * Also trigger async hooks in sync-mode (but don't wait for them to return) # tests: - switch uploadSpec to createFakeServer - move some useful helpers related to final file consistency - bugfix. xhr_server.restore() is not enough. Recreating the object fix some failures under particular tests sequences - added new tests for the async initFileFn and related asynAddFile(s) functions - modified tests to take into account changes related to hooks & events * builds
1 parent ad9967f commit c8dee19

20 files changed

+4904
-2988
lines changed

CHANGELOG.md

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,18 @@
1+
# 3.0.0
2+
3+
## Breaking Changes
4+
5+
### Events
6+
- Events are distincts from hooks.
7+
- Recognized events and hooks are now lower-snake-case (case-sensitive).
8+
- Hooks are "file-added", "files-added", "files-submitted". "filter-file" is a filtering hook.
9+
- Events are passed a native CustomEvent. IE:
10+
v2: `flow.on('fileRemoved', (file) => { ... });`
11+
v3: `flow.on('file-removed', ({detail: [file]}) => { ... });`
12+
13+
### Other
14+
- FlowFile does not run bootstrap() upon instanciation. This must be done manually (or rely on *addFile* functions).
15+
116
# 2.0.0
217

318
## Features
@@ -34,4 +49,4 @@
3449
that calling **Resumable.upload()** method in these events will not start uploading current
3550
files. To start upload use **filesSubmitted** event instead.
3651
- **throttleProgressCallbacks** parameter was replaced with **progressCallbacksInterval** and it
37-
is now measured in milliseconds.
52+
is now measured in milliseconds.

README.md

Lines changed: 48 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -173,7 +173,17 @@ parameter must be adjusted together with `progressCallbacksInterval` parameter.
173173
* `.cancel()` Cancel upload of all `FlowFile` objects and remove them from the list.
174174
* `.progress()` Returns a float between 0 and 1 indicating the current upload progress of all files.
175175
* `.isUploading()` Returns a boolean indicating whether or not the instance is currently uploading anything.
176-
* `.addFile(file)` Add a HTML5 File object to the list of files.
176+
* `.addFile(file, event = null, initFileFn = undefined)` Add a HTML5 File object to the list of files.
177+
* Accept the same `event` and `initFileFn` parameters thant `addFiles` which is used under the hood.
178+
* `.addFiles([files], event = null, initFileFn = undefined)` Add multiple File objects to the list of files.
179+
* `event` The optional event that trigger the addition (for internal purposes)
180+
* `initFileFn` An override of Flow.initFileFn
181+
* `.asyncAddFile(file, event = null, initFileFn = undefined)` Add a HTML5 File object to the list of files.
182+
* `.asyncAddFiles([files], event = null, initFileFn = undefined)` Add multiple File objects to the list of files.
183+
* `asyncAddFile` and `asyncAddFiles` rely on the same parameters than they non-async counterparts with one
184+
difference: They accept an asynchronous `initFileFn` file function and return, in a promise, the corresponding FlowFiles.
185+
* Note: Calling `asyncAddFile` or `asyncAddFiles` with no `initFileFn` being defined is aimed identical to there non-async
186+
counterpart but this may change in the future [TBD].
177187
* `.removeFile(file)` Cancel upload of a specific `FlowFile` object on the list from the list.
178188
* `.getFromUniqueIdentifier(uniqueIdentifier)` Look up a `FlowFile` object by its unique identifier.
179189
* `.getSize()` Returns the total size of the upload in bytes.
@@ -182,29 +192,46 @@ parameter must be adjusted together with `progressCallbacksInterval` parameter.
182192

183193
#### Events
184194

185-
* `.fileSuccess(file, message, chunk)` A specific file was completed. First argument `file` is instance of `FlowFile`, second argument `message` contains server response. Response is always a string.
186-
Third argument `chunk` is instance of `FlowChunk`. You can get response status by accessing xhr
187-
object `chunk.xhr.status`.
188-
* `.fileProgress(file, chunk)` Uploading progressed for a specific file.
189-
* `.fileAdded(file, event)` This event is used for file validation. To reject this file return false.
190-
This event is also called before file is added to upload queue,
191-
this means that calling `flow.upload()` function will not start current file upload.
192-
Optionally, you can use the browser `event` object from when the file was
193-
added.
194-
* `.filesAdded(array, event)` Same as fileAdded, but used for multiple file validation.
195-
* `.filesSubmitted(array, event)` Same as filesAdded, but happens after the file is added to upload queue. Can be used to start upload of currently added files.
196-
* `.fileRemoved(file)` The specific file was removed from the upload queue. Combined with filesSubmitted, can be used to notify UI to update its state to match the upload queue.
197-
* `.fileRetry(file, chunk)` Something went wrong during upload of a specific file, uploading is being
198-
retried.
199-
* `.fileError(file, message, chunk)` An error occurred during upload of a specific file.
200-
* `.uploadStart()` Upload has been started on the Flow object.
201-
* `.complete()` Uploading completed.
202-
* `.progress()` Uploading progress.
203-
* `.error(message, file, chunk)` An error, including fileError, occurred.
204-
* `.catchAll(event, ...)` Listen to all the events listed above with the same callback function.
195+
Events are native, synchronous and provide information about the internal state of the application without being given a chance to alter it.
196+
197+
* `file-success(<FlowFile> file, <string> message, <FlowChunk> chunk)` A specific file was completed (`message` comes from `xhr.responseText`, `chunk` is instance of `FlowChunk` representing the last chunk for this file. You can get response status by accessing xhr object `chunk.xhr.status`).
198+
* `file-progress(<FlowFile> file, <FlowChunk> chunk)` Upload progressed for a specific file.
199+
* `file-removed(<FlowFile> file)` The specific file was removed from the upload queue. Combined with the `files-submitted` hook, it can be used to notify UI to update its state to match the upload queue.
200+
* `file-retry(<FlowFile> file, <FlowChunk> chunk)` Something went wrong during upload of a specific file, uploading is being retried.
201+
* `file-error(<FlowFile> file, <string> message, <FlowChunk> chunk)` An error occurred during upload of a specific file. `message` comes from `xhr.responseText`.
202+
* `upload-start()` Upload started on the Flow object.
203+
* `complete()` Upload completed.
204+
* `progress()` Upload progress.
205+
* `error(<string> message, <FlowFile> file, <FlowChunk> chunk)` An error, including fileError, occurred.
206+
* `catch-all(<string> event-name, ...)` Receive all above events listed above and their corresponding parameters.
207+
208+
#### Processing hooks
209+
210+
Hooks allows for either synchronous or asynchronous operations and allow altering the regular processing of the file(s) from addition to upload completion.
211+
It's user responsability to use or not the `async` version of `(async)?addFile` and `(async)?addFiles` according to the behavior of its processing hooks.
212+
(Defining `async` callbacks for the `asyncAddFile(s)`)
213+
214+
* `file-added(<FlowFile> file, event) : null` This event is also called before file is added to upload queue and after it's been fully initialized. `event` is the browser `event` object from when the file was added.
215+
* `files-added([<FlowFile> files], event) : null` Same as `file-added`, but used for multiple file validation.
216+
* `files-submitted([<FlowFile> files], event) : null` Same as `files-added`, but happens after the file is added to upload queue. Can be used to start upload of currently added files.
217+
* `filter-file(<FlowFile> file, event) : boolean` The boolean return value decide whether this particular file must be processed or ignored.
218+
219+
### Hooks and events format
220+
- Events and hooks name are case-sensitive, snake-cased and return CustomEvent passed straight to `Flow.on()` callback.
221+
- Sample use `flow.on('file-removed', ({detail: [file]}) => { ... });`
222+
- In an attempt of backward compatibility, some support of camelCase events exist:
223+
```
224+
flow.on('filesAdded', async (files, event) => { // v2 events prototype
225+
if (files instanceof CustomEvent) { // Handle v3+ events
226+
var [files, event] = files.detail;
227+
}
228+
// Do something with files
229+
});
230+
```
205231

206232
### FlowFile
207233
FlowFile constructor can be accessed in `Flow.FlowFile`.
234+
208235
#### Properties
209236

210237
* `.flowObj` A back-reference to the parent `Flow` object.

0 commit comments

Comments
 (0)