11# Queuing
22
3+ InEngine.NET's queue functionality allows for commands to be run in the background with a simple publish/consume model.
4+
35## Prerequisites
46
5- Redis is required to use the InEngine.NET Queue feature.
7+ Redis is required to use the InEngine.NET queue feature.
68It can be installed on Ubuntu with this command:
79
810``` bash
@@ -19,22 +21,27 @@ sudo service redis start
1921It is highly recommended to <a href =" https://redis.io/topics/security#authentication-feature " >set a password</a > for Redis.
2022</div >
2123
22- ## Working with Queues
23-
24- ### Publishing Commands
24+ ## Publishing Commands
2525
26- #### Programmatically
26+ ### From Code
2727
2828[ Commands] ( commands ) can be published programmatically with the ** InEngine.Core.Queuing.Broker** class:
2929
3030``` c#
3131Broker .Make ().Publish (new MyCommand ());
3232```
3333
34- #### From the Command Line
34+ Or publish to the secondary queue:
35+
36+ ``` c#
37+ Broker .Make (true ).Publish (new MyCommand ());
38+ ```
39+
40+ ### From the Command Line
41+
3542Commands can be published from the command line as well.
3643Note that all queue commands reside in the ** InEngine.Core** plugin.
37- This is an example of how to publish a command from the CLI by specifying the commands assembly , class name, and arguments:
44+ This is an example of how to publish a command from the CLI by specifying the command's plugin , class name, and arguments:
3845
3946``` bash
4047inengine.exe -pInEngine.Core queue:publish --command-plugin=MyCommandPlugin.dll --command-class=MyCommand --args " text=bar"
@@ -52,9 +59,9 @@ The command verb can also be specified instead of the full class name:
5259inengine.exe -pInEngine.Core queue:publish --command-plugin=InEngine.Core.dll --command-verb=echo--args " text=foo"
5360```
5461
55- ### Consuming Commands
62+ ## Consuming Commands
5663
57- #### Programmatically
64+ ### From Code
5865Consuming a command is also accomplished with the Broker class:
5966
6067``` c#
@@ -70,7 +77,7 @@ Broker.Make(true).Consume();
7077
7178Commands can be consumed from the command line as well with this simple command:
7279
73- #### From the Command Line
80+ ### From the Command Line
7481
7582``` bash
7683inengine.exe -pInEngine.Core queue:consume
@@ -82,22 +89,86 @@ Use the **--secondary** argument to consume the secondary queue instead of the p
8289inengine.exe -pInEngine.Core queue:consume --secondary
8390```
8491
85- #### With the Scheduler
92+ ### With the Scheduler
8693
8794The InEngine scheduler is needed to consume queued messages in the background.
8895There are a variety of [ ways to run the scheduler] ( scheduling/#running-the-scheduler ) .
8996
97+ ## Examining the Queue
98+
99+ ### Viewing Queue Lengths
100+
101+ The ** queue: length ** command shows a quick summary of pending, in-progress, and failed commands in the primary and secondary queues:
102+
103+ ``` bash
104+ inengine.exe -pInEngine.Core queue:length
105+ ```
106+
107+ ### Peek at Commands
108+
109+ The ** queue: peek ** command allows for queued commands to be inspected:
110+
111+ ``` bash
112+ inengine.exe -pInEngine.Core queue:peek --pending --in-progress --failed
113+ ```
114+
115+ It is of course possible to peek in the secondary queues:
116+
117+ ``` bash
118+ inengine.exe -pInEngine.Core queue:peek --pending --secondary
119+ ```
120+
121+ Queued commands can be viewed in JSON which maybe useful for debugging:
122+
123+ ``` bash
124+ inengine.exe -pInEngine.Core queue:peek --pending --json
125+ ```
126+
127+ By default, up to 10 messages will be retrieved, but the number is configurable:
128+
129+ ``` bash
130+ inengine.exe -pInEngine.Core queue:peek --pending --limit=100
131+ ```
132+
133+ The a paginated slice of the queue can be retrieved using the offset argument.
134+ For example, this queue: peek call retrieves the 100-200 queued commands:
135+
136+ ``` bash
137+ inengine.exe -pInEngine.Core queue:peek --pending --limit=100 --offset=100
138+ ```
139+
140+ ## Handling Failed Commands
141+
142+ Commands that throw an exception are put in a special "failed" queue.
143+ They can be republished with the ** queue: republish ** command:
144+
145+ ``` bash
146+ inengine.exe -pInEngine.Core queue:republish
147+ ```
148+
149+ Failed secondary queue commands can be republished as well:
150+
151+ ``` bash
152+ inengine.exe -pInEngine.Core queue:republish --secondary
153+ ```
154+
155+ By default, only 100 failed commands are republished at a time.
156+ The is configurable:
157+
158+ ``` bash
159+ inengine.exe -pInEngine.Core queue:republish --limit=1000
160+ ```
90161
91162## Primary and Secondary Queue
92163
93164Other than the fact that the primary queue is used by default, there is no difference between the primary and secondary queues.
94- However, it is often desirable to have more than 1 queue .
165+ However, it is often desirable to use two queues .
95166For example, long running jobs might be sent to the secondary queue,
96167while jobs that are expected to finish after only a few moments are sent to the primary queue.
97168
98169What about 3, 4, or 900 queues? Managing numerous queues gets to be a pain and, practically speaking, is probably unnecessary.
99170If it is desirable, different [ configuration files] ( configuration ) can be used to run multiple instances of InEngine.NET.
100- Simply create a new [ config file] ( configuration ) with a new QueueName setting and point inengine.exe at it with the -c argument:
171+ Simply create a new config file with a new QueueName setting and point inengine.exe at it with the -c argument:
101172
102173``` bash
103174inengine.exe -cMyCustomSettingsFile.json -pInEngine.Core queue:consume
0 commit comments