121 lines
		
	
	
		
			4.8 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
		
		
			
		
	
	
			121 lines
		
	
	
		
			4.8 KiB
		
	
	
	
		
			Markdown
		
	
	
	
	
	
| 
								 | 
							
								# Promise-parallel-throttle
							 | 
						||
| 
								 | 
							
								[](https://travis-ci.org/DJWassink/Promise-parallel-throttle)
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								Run a array of Promises in parallel. Kinda like Promise.all(), but throttled!
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								## Install 
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								### NPM
							 | 
						||
| 
								 | 
							
								```bash
							 | 
						||
| 
								 | 
							
								npm i promise-parallel-throttle -S
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								### Yarn
							 | 
						||
| 
								 | 
							
								```bash
							 | 
						||
| 
								 | 
							
								yarn add promise-parallel-throttle
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								## Usage
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								```js
							 | 
						||
| 
								 | 
							
								import Throttle from 'promise-parallel-throttle';
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								//Function which should return a Promise
							 | 
						||
| 
								 | 
							
								const doReq = async (firstName, lastName) => {
							 | 
						||
| 
								 | 
							
								    //Do something async.
							 | 
						||
| 
								 | 
							
								    return firstName + " " + lastName;
							 | 
						||
| 
								 | 
							
								}
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								const users = [
							 | 
						||
| 
								 | 
							
								    {firstName: "Irene", lastName: "Pullman"},
							 | 
						||
| 
								 | 
							
								    {firstName: "Sean",  lastName: "Parr"}
							 | 
						||
| 
								 | 
							
								];
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								//Queue with functions to be run
							 | 
						||
| 
								 | 
							
								const queue = users.map(user => () => doReq(user.firstName, user.lastName));
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								//Default Throttle runs with 5 promises parallel.
							 | 
						||
| 
								 | 
							
								const formattedNames = await Throttle.all(queue);
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								console.log(formattedNames); //['Irene Pullman', 'Sean Parr']
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								## API
							 | 
						||
| 
								 | 
							
								### Throttle.all
							 | 
						||
| 
								 | 
							
								`Throttle.all(tasks, options)`
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								Throttle.all is made to behave exactly like Promise.all but instead of all the tasks running in parallel it runs a maxium amount of tasks in parallel.
							 | 
						||
| 
								 | 
							
								Only the tasks parameter is required while the [options](#options-object) parameter is optional.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								### Throttle.sync
							 | 
						||
| 
								 | 
							
								`Throttle.sync(tasks, options)`
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								Throttle.sync runs all the tasks synchronously. 
							 | 
						||
| 
								 | 
							
								Once again the tasks array is required, the [options](#options-object) are optional. 
							 | 
						||
| 
								 | 
							
								Be aware that this method is simply a wrapper to pass `maxInProgress` with 1. So overwriting this option in the options object would run the tasks again in parallel.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								### Throttle.raw
							 | 
						||
| 
								 | 
							
								`Throttle.raw(tasks, options)`
							 | 
						||
| 
								 | 
							
								 
							 | 
						||
| 
								 | 
							
								 The raw method instead of returning the tasks their results, will return a [result](#result-object--progress-callback) object. 
							 | 
						||
| 
								 | 
							
								 Useful if you wan't more statistics about the execution of your tasks. Once again the tasks are required while the [options](#options-object) are optional.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								#### Option's Object
							 | 
						||
| 
								 | 
							
								|Parameter|Type|Default|Definition|
							 | 
						||
| 
								 | 
							
								|:---|:---|:---|:---|
							 | 
						||
| 
								 | 
							
								|maxInProgress |Integer|5| max amount of parallel threads|
							 | 
						||
| 
								 | 
							
								|failFast |Boolean|true (false for the [raw](#throttleraw) method)| reject after a single error, or keep running|
							 | 
						||
| 
								 | 
							
								|progressCallback |Function|Optional| callback with progress reports|
							 | 
						||
| 
								 | 
							
								|nextCheck |Function|Optional| function which should return a promise, if the promise resolved true the next task is spawn, errors will propagate and should be handled in the calling code|
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								#### Result object / Progress callback
							 | 
						||
| 
								 | 
							
								The `progressCallback` and the `Raw` will return a `Result` object with the following properties:
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								|Property|Type|Start value|Definition|
							 | 
						||
| 
								 | 
							
								|:---|:---|:---|:---|
							 | 
						||
| 
								 | 
							
								|amountDone|Integer|0|amount of tasks which are finished|
							 | 
						||
| 
								 | 
							
								|amountStarted|Integer|0|amount of tasks which started|
							 | 
						||
| 
								 | 
							
								|amountResolved|Integer|0|amount of tasks which successfully resolved|
							 | 
						||
| 
								 | 
							
								|amountRejected|Integer|0|amount of tasks which returned in an error and are aborted|
							 | 
						||
| 
								 | 
							
								|amountNextCheckFalsey|Integer|0|amount of tasks which got a falsey value in the [nextCheck](#nextcheck)|
							 | 
						||
| 
								 | 
							
								|rejectedIndexes|Array|[]|all the indexes in the tasks array where the promise rejected|
							 | 
						||
| 
								 | 
							
								|resolvedIndexes|Array|[]|all the indexes in the tasks array where the promise resolved|
							 | 
						||
| 
								 | 
							
								|nextCheckFalseyIndexes|Array|[]|all the indexes in the tasks array where the [nextCheck](#nextcheck) returned a falsey value|
							 | 
						||
| 
								 | 
							
								|taskResults|Array|[]|array containing the result of every task|
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								#### nextCheck
							 | 
						||
| 
								 | 
							
								All the `Throttle` methods have a `nextCheck` method which will be used to verify if a next task is allowed to start. 
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								The default `nextCheck` is defined like this;
							 | 
						||
| 
								 | 
							
								```js
							 | 
						||
| 
								 | 
							
								const defaultNextTaskCheck = (status, tasks) => {
							 | 
						||
| 
								 | 
							
								    return new Promise((resolve, reject) => {
							 | 
						||
| 
								 | 
							
								        resolve(status.amountStarted < tasks.length);
							 | 
						||
| 
								 | 
							
								    });
							 | 
						||
| 
								 | 
							
								}
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								This function will get a status object as parameter which adheres to the [Result object](#result-object--progress-callback) and it also receives the list of tasks.
							 | 
						||
| 
								 | 
							
								In the default `nextCheck` we simply check if the amount of started exceeds the amount to be done, if not we are free to start an other task.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								This function can be useful to write your own scheduler based on, for example ram/cpu usage.
							 | 
						||
| 
								 | 
							
								Lets say that your tasks use a lot of ram and you don't want to exceed a certain amount.
							 | 
						||
| 
								 | 
							
								You could then write logic inside a `nextCheck` function which resolves after there is enough ram available to start the next task.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								If a custom implementation decides to reject, the error is propagated and should be handled in the user it's code. If a custom implementation returns a falsey value the task will simply not execute and the next task will be scheduled.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								## Example
							 | 
						||
| 
								 | 
							
								Check out the example's directory, it's heavily documented so it should be easy to follow.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								To run the example, at least Node 8.x.x is required, since it supports native async/await.
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								Simply run the example with npm:
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								npm run-script names
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								
							 | 
						||
| 
								 | 
							
								Or with Yarn:
							 | 
						||
| 
								 | 
							
								```
							 | 
						||
| 
								 | 
							
								yarn names
							 | 
						||
| 
								 | 
							
								```
							 |