You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Ability to use asynchronous initFileFn.
This is a follow-up (and proper implementation) of #296 (reverted in e867d54)
This could be use to initialize a stream, a reader, fetch a remote resource, ... during the FlowFile initialization.
`asyncAddFile` and `asyncAddFiles` are introduced which support this mechanism.
These function return promises of one (or multiple) FlowFile(s).
To implement this:
- An AsyncFlowFile class extending a FlowFile is created, overriding the bootstrap part
- In order to keep code-duplication low, a filterFileList generator yield files
in way common to async and non-async addFiles() functions.
The possibly async' nature of initializing file affects events firing.
(#319) in general and `fileAdded`
in particular (#271) and the current
confusion between hooks altering bootstraping and simple events.
- fileAdded is now assumed an event. If we detect a return value, a warning is sent.
- preFilterFile is introduced and allow filtering the raw file before bootstrap()
- postFilterFile is introduced and allow filtering the FlowFile after a (possibly async) bootstrap()
- filesAdded is preserved
About parameters:
- `initFileFn` is only used during bootstrap
- If such a helper exists, it's tightly related to addFile* functions.
As such, it was deemed adequate to provide it as a parameter to `*addFile*()`
(but `opts.initFileFn` is still supported)
An `Eventizer` class is added where all the necessary code exist.
- Hooks alter code processing and could be `synchronous` or not.
- Events are CustomEvent dispatched asynchronously for informative purpose.
A warning is triggered if an obsolete event name is used
## Other changes
* uploadNextChunk: use transpilation instead of relying upon the each() helper
* FlowFile: don't bootstrap within the constructor. This must be done manually now
* README/CHANGELOG
* Also trigger async hooks in sync-mode (but don't wait for them to return)
# tests:
- switch uploadSpec to createFakeServer
- move some useful helpers related to final file consistency
- bugfix. xhr_server.restore() is not enough. Recreating the object fix some failures under particular tests sequences
- added new tests for the async initFileFn and related asynAddFile(s) functions
- modified tests to take into account changes related to hooks & events
* builds
Copy file name to clipboardExpand all lines: README.md
+48-21Lines changed: 48 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -173,7 +173,17 @@ parameter must be adjusted together with `progressCallbacksInterval` parameter.
173
173
*`.cancel()` Cancel upload of all `FlowFile` objects and remove them from the list.
174
174
*`.progress()` Returns a float between 0 and 1 indicating the current upload progress of all files.
175
175
*`.isUploading()` Returns a boolean indicating whether or not the instance is currently uploading anything.
176
-
*`.addFile(file)` Add a HTML5 File object to the list of files.
176
+
*`.addFile(file, event = null, initFileFn = undefined)` Add a HTML5 File object to the list of files.
177
+
* Accept the same `event` and `initFileFn` parameters thant `addFiles` which is used under the hood.
178
+
*`.addFiles([files], event = null, initFileFn = undefined)` Add multiple File objects to the list of files.
179
+
*`event` The optional event that trigger the addition (for internal purposes)
180
+
*`initFileFn` An override of Flow.initFileFn
181
+
*`.asyncAddFile(file, event = null, initFileFn = undefined)` Add a HTML5 File object to the list of files.
182
+
*`.asyncAddFiles([files], event = null, initFileFn = undefined)` Add multiple File objects to the list of files.
183
+
*`asyncAddFile` and `asyncAddFiles` rely on the same parameters than they non-async counterparts with one
184
+
difference: They accept an asynchronous `initFileFn` file function and return, in a promise, the corresponding FlowFiles.
185
+
* Note: Calling `asyncAddFile` or `asyncAddFiles` with no `initFileFn` being defined is aimed identical to there non-async
186
+
counterpart but this may change in the future [TBD].
177
187
*`.removeFile(file)` Cancel upload of a specific `FlowFile` object on the list from the list.
178
188
*`.getFromUniqueIdentifier(uniqueIdentifier)` Look up a `FlowFile` object by its unique identifier.
179
189
*`.getSize()` Returns the total size of the upload in bytes.
@@ -182,29 +192,46 @@ parameter must be adjusted together with `progressCallbacksInterval` parameter.
182
192
183
193
#### Events
184
194
185
-
*`.fileSuccess(file, message, chunk)` A specific file was completed. First argument `file` is instance of `FlowFile`, second argument `message` contains server response. Response is always a string.
186
-
Third argument `chunk` is instance of `FlowChunk`. You can get response status by accessing xhr
187
-
object `chunk.xhr.status`.
188
-
*`.fileProgress(file, chunk)` Uploading progressed for a specific file.
189
-
*`.fileAdded(file, event)` This event is used for file validation. To reject this file return false.
190
-
This event is also called before file is added to upload queue,
191
-
this means that calling `flow.upload()` function will not start current file upload.
192
-
Optionally, you can use the browser `event` object from when the file was
193
-
added.
194
-
*`.filesAdded(array, event)` Same as fileAdded, but used for multiple file validation.
195
-
*`.filesSubmitted(array, event)` Same as filesAdded, but happens after the file is added to upload queue. Can be used to start upload of currently added files.
196
-
*`.fileRemoved(file)` The specific file was removed from the upload queue. Combined with filesSubmitted, can be used to notify UI to update its state to match the upload queue.
197
-
*`.fileRetry(file, chunk)` Something went wrong during upload of a specific file, uploading is being
198
-
retried.
199
-
*`.fileError(file, message, chunk)` An error occurred during upload of a specific file.
200
-
*`.uploadStart()` Upload has been started on the Flow object.
201
-
*`.complete()` Uploading completed.
202
-
*`.progress()` Uploading progress.
203
-
*`.error(message, file, chunk)` An error, including fileError, occurred.
204
-
*`.catchAll(event, ...)` Listen to all the events listed above with the same callback function.
195
+
Events are native, synchronous and provide information about the internal state of the application without being given a chance to alter it.
196
+
197
+
*`file-success(<FlowFile> file, <string> message, <FlowChunk> chunk)` A specific file was completed (`message` comes from `xhr.responseText`, `chunk` is instance of `FlowChunk` representing the last chunk for this file. You can get response status by accessing xhr object `chunk.xhr.status`).
198
+
*`file-progress(<FlowFile> file, <FlowChunk> chunk)` Upload progressed for a specific file.
199
+
*`file-removed(<FlowFile> file)` The specific file was removed from the upload queue. Combined with the `files-submitted` hook, it can be used to notify UI to update its state to match the upload queue.
200
+
*`file-retry(<FlowFile> file, <FlowChunk> chunk)` Something went wrong during upload of a specific file, uploading is being retried.
201
+
*`file-error(<FlowFile> file, <string> message, <FlowChunk> chunk)` An error occurred during upload of a specific file. `message` comes from `xhr.responseText`.
202
+
*`upload-start()` Upload started on the Flow object.
203
+
*`complete()` Upload completed.
204
+
*`progress()` Upload progress.
205
+
*`error(<string> message, <FlowFile> file, <FlowChunk> chunk)` An error, including fileError, occurred.
206
+
*`catch-all(<string> event-name, ...)` Receive all above events listed above and their corresponding parameters.
207
+
208
+
#### Processing hooks
209
+
210
+
Hooks allows for either synchronous or asynchronous operations and allow altering the regular processing of the file(s) from addition to upload completion.
211
+
It's user responsability to use or not the `async` version of `(async)?addFile` and `(async)?addFiles` according to the behavior of its processing hooks.
212
+
(Defining `async` callbacks for the `asyncAddFile(s)`)
213
+
214
+
*`file-added(<FlowFile> file, event) : null` This event is also called before file is added to upload queue and after it's been fully initialized. `event` is the browser `event` object from when the file was added.
215
+
*`files-added([<FlowFile> files], event) : null` Same as `file-added`, but used for multiple file validation.
216
+
*`files-submitted([<FlowFile> files], event) : null` Same as `files-added`, but happens after the file is added to upload queue. Can be used to start upload of currently added files.
217
+
*`filter-file(<FlowFile> file, event) : boolean` The boolean return value decide whether this particular file must be processed or ignored.
218
+
219
+
### Hooks and events format
220
+
- Events and hooks name are case-sensitive, snake-cased and return CustomEvent passed straight to `Flow.on()` callback.
221
+
- Sample use `flow.on('file-removed', ({detail: [file]}) => { ... });`
222
+
- In an attempt of backward compatibility, some support of camelCase events exist:
0 commit comments