Skip to content

Commit fde4655

Browse files
guimachiavelliirevoire
guimachiavelli
and
irevoire
authored
v1.12: New /batches route (#3072)
--------- Co-authored-by: Tamo <[email protected]>
1 parent 191aaa6 commit fde4655

File tree

7 files changed

+287
-3
lines changed

7 files changed

+287
-3
lines changed

‎.code-samples.meilisearch.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1370,6 +1370,12 @@ update_localized_attribute_settings_1: |-
13701370
reset_localized_attribute_settings_1: |-
13711371
curl \
13721372
-X DELETE 'http://localhost:7700/indexes/INDEX_NAME/settings/localized-attributes'
1373+
get_all_batches_1: |-
1374+
curl \
1375+
-X GET 'http://localhost:7700/batches'
1376+
get_batch_1: |-
1377+
curl \
1378+
-X GET 'http://localhost:7700/batches/BATCH_UID'
13731379
13741380
### Code samples for experimental features
13751381
get_embedders_1: |-

‎config/sidebar-reference.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,11 @@
4343
"label": "Tasks",
4444
"slug": "tasks"
4545
},
46+
{
47+
"source": "reference/api/batches.mdx",
48+
"label": "Batches",
49+
"slug": "batches"
50+
},
4651
{
4752
"source": "reference/api/keys.mdx",
4853
"label": "Keys",

‎learn/async/asynchronous_operations.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,10 @@ When you make a [request for an asynchronous operation](#which-operations-are-as
137137
**Terminating a Meilisearch instance in the middle of an asynchronous operation is completely safe** and will never adversely affect the database.
138138
</Capsule>
139139

140+
### Task batches
141+
142+
Meilisearch processes tasks in batches, grouping tasks for the best possible performance. In most cases, batching should be transparent and have no impact on the overall task workflow. Use [the `/batches` route](/reference/api/batches) to obtain more information on batches and how they are processing your tasks.
143+
140144
### Canceling tasks
141145

142146
You can cancel a task while it is `enqueued` or `processing` by using [the cancel tasks endpoint](/reference/api/tasks#cancel-tasks). Doing so changes a task's `status` to `canceled`.

‎learn/async/filtering_tasks.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@ Querying the [get tasks endpoint](/reference/api/tasks#get-tasks) returns all ta
99

1010
This guide shows you how to use query parameters to filter tasks and obtain a more readable list of asynchronous operations.
1111

12+
<Capsule intent="tip" title="Filtering batches">
13+
Filtering batches with [the `/batches` route](/reference/api/batches) follows the same rules as filtering tasks. Keep in mind that many `/batches` parameters such as `uids` target the tasks included in batches, instead of the batches themselves.
14+
</Capsule>
15+
1216
## Requirements
1317

1418
- a command-line terminal

‎learn/async/paginating_tasks.mdx‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ description: Meilisearch uses a task queue to handle asynchronous operations. Th
77

88
By default, Meilisearch returns a list of 20 tasks for each request when you query the [get tasks endpoint](/reference/api/tasks#get-tasks). This guide shows you how to navigate the task list using query parameters.
99

10+
<Capsule intent="tip" title="Paginating batches">
11+
Paginating batches with [the `/batches` route](/reference/api/batches) follows the same rules as paginating tasks.
12+
</Capsule>
13+
1014
## Configuring the number of returned tasks
1115

1216
Use the `limit` parameter to change the number of returned tasks:

‎reference/api/batches.mdx‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
---
2+
title: Batches — Meilisearch API reference
3+
description: The /batches route allows you to monitor how Meilisearch is grouping and processing asynchronous operations.
4+
---
5+
6+
# Batches
7+
8+
The `/batches` route gives information about the progress of batches of [asynchronous operations](/learn/async/asynchronous_operations).
9+
10+
## Batch object
11+
12+
```json
13+
{
14+
"uid": 0,
15+
"details": {
16+
"receivedDocuments": 6,
17+
"indexedDocuments": 6
18+
},
19+
"stats": {
20+
"totalNbTasks": 1,
21+
"status": {
22+
"succeeded": 1
23+
},
24+
"types": {
25+
"documentAdditionOrUpdate": 1
26+
},
27+
"indexUids": {
28+
"INDEX_NAME": 1
29+
}
30+
},
31+
"duration": "PT0.250518S",
32+
"startedAt": "2024-12-10T15:20:30.18182Z",
33+
"finishedAt": "2024-12-10T15:20:30.432338Z",
34+
"progress": {
35+
"steps": [
36+
{
37+
"currentStep": "extracting words",
38+
"finished": 2,
39+
"total": 9,
40+
},
41+
{
42+
"currentStep": "document",
43+
"finished": 30546,
44+
"total": 31944,
45+
}
46+
],
47+
"percentage": 32.8471
48+
}
49+
}
50+
```
51+
52+
### `uid`
53+
54+
**Type**: Integer<br />
55+
**Description**: Unique sequential identifier of the batch. Starts at `0` and increases by one for every new patch.
56+
57+
### `details`
58+
59+
**Type**: Object<br />
60+
**Description**: Basic information on the types tasks in a batch. Consult the [task object reference](/reference/api/tasks#details) for an exhaustive list of possible values.
61+
62+
### `progress`
63+
64+
**Type**: Object<br />
65+
**Description**: Object containing two fields: `steps` and `percentage`. Once Meilisearch has fully processed a batch, its `progress` is set to `null`.
66+
67+
#### `steps`
68+
69+
Information about the current operations Meilisearch is performing in this batch. A step may consist of multiple substeps.
70+
71+
| Name | Description |
72+
| :-----------------| :------------------------------------------------- |
73+
| **`currentStep`** | A string describing the operation |
74+
| **`total`** | The total number of operations in the step |
75+
| **`finished`** | The number of operations Meilisearch has completed |
76+
77+
<Capsule intent="tip" title="">
78+
If Meilisearch is taking longer than expected to process a batch, monitor the `steps` array. If the `finished` field of the last item in the `steps` array does not update, Meilisearch may be stuck.
79+
</Capsule>
80+
81+
#### `percentage`
82+
83+
The percentage of completed operations, calculated from all current steps and substeps. This value is a rough estimate and may not always reflect the current state of the batch due to how different steps are processed more quickly than others.
84+
85+
### `stats`
86+
87+
**Type**: Object<br />
88+
**Description**: Detailed information on the payload of all tasks in a batch.
89+
90+
#### `totalNbTasks`
91+
92+
Number of tasks in the batch.
93+
94+
#### `status`
95+
96+
Object listing the [status of each task](/reference/api/tasks#status) in the batch. Contains five keys whose values correspond to the number of tasks with that status.
97+
98+
#### `types`
99+
100+
List with the `types` of tasks contained in the batch.
101+
102+
#### `indexUids`
103+
104+
List of the number of tasks in the batch separated by the indexes they affect.
105+
106+
### `duration`
107+
108+
**Type**: String<br />
109+
**Description**: The total elapsed time the batch spent in the `processing` state, in [ISO 8601](https://www.ionos.com/digitalguide/websites/web-development/iso-8601/) format. Set to `null` while the batch is processing tasks
110+
111+
### `startedAt`
112+
113+
**Type**: String<br />
114+
**Description**: The date and time when the batch began `processing`, in [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format
115+
116+
### `finishedAt`
117+
118+
**Type**: String<br />
119+
**Description**: The date and time when the tasks finished `processing`, whether `failed`, `succeeded`, or `canceled`, in [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format
120+
121+
## Get batches
122+
123+
<RouteHighlighter method="GET" route="/batches"/>
124+
125+
List all batches, regardless of index. The batch objects are contained in the `results` array.
126+
127+
Batches are always returned in descending order of `uid`. This means that by default, **the most recently created batch objects appear first**.
128+
129+
Batch results are [paginated](/learn/async/paginating_tasks) and can be [filtered](/learn/async/filtering_tasks) with query parameters.
130+
131+
<Capsule intent="danger" title="`/batches` parameters and tasks">
132+
Some query parameters for `/batches`, such as `uids` and `statuses`, target tasks instead of batches.
133+
134+
For example, `?uids=0` returns a batch containing the task with a `taskUid` equal to `0`, instead of a batch with a `batchUid` equal to `0`.
135+
</Capsule>
136+
137+
### Query parameters
138+
139+
| Query Parameter | Default Value | Description |
140+
| ---------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
141+
| **`uids`** | `*` (all task uids) | Select batches containing the tasks with the specified `uid`s. Separate multiple task `uids` with a comma (`,`) |
142+
| **`batchUids`** | `*` (all batch uids) | Filter batches by their `uid`. Separate multiple batch `uids` with a comma (`,`) |
143+
| **`indexUids`** | `*` (all indexes) | Select batches containing tasks affecting the specified indexes. Separate multiple `indexUids` with a comma (`,`) |
144+
| **`statuses`** | `*` (all statuses) | Select batches containing tasks with the specified `status`. Separate multiple task `statuses` with a comma (`,`) |
145+
| **`types`** | `*` (all types) | Select batches containing tasks with the specified `type`. Separate multiple task `types` with a comma (`,`) |
146+
| **`limit`** | `20` | Number of batches to return |
147+
| **`from`** | `uid` of the last created batch | `uid` of the first batch returned |
148+
| **`reverse`** | `false` | If `true`, returns results in the reverse order, from oldest to most recent |
149+
| **`beforeEnqueuedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `enqueuedAt` field |
150+
| **`beforeStartedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `startedAt` field |
151+
| **`beforeFinishedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `finishedAt` field |
152+
| **`afterEnqueuedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `enqueuedAt` field |
153+
| **`afterStartedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `startedAt` field |
154+
| **`afterFinishedAt`** | `*` (all tasks) | Select batches containing tasks with the specified `finishedAt` field |
155+
156+
### Response
157+
158+
| Name | Type | Description |
159+
| :------------ | :------ | :----------------------------------------------------------------------------------------------------------------------------- |
160+
| **`results`** | Array | An array of [batch objects](#batch-object) |
161+
| **`total`** | Integer | Total number of batches matching the filter or query |
162+
| **`limit`** | Integer | Number of batches returned |
163+
| **`from`** | Integer | `uid` of the first batch returned |
164+
| **`next`** | Integer | Value passed to `from` to view the next "page" of results. When the value of `next` is `null`, there are no more tasks to view |
165+
166+
### Example
167+
168+
<CodeSamples id="get_all_batches_1" />
169+
170+
#### Response: `200 Ok`
171+
172+
```json
173+
{
174+
"results": [
175+
{
176+
"uid": 2,
177+
"details": {
178+
"stopWords": [
179+
"of",
180+
"the"
181+
]
182+
},
183+
"progress": null,
184+
"stats": {
185+
"totalNbTasks": 1,
186+
"status": {
187+
"succeeded": 1
188+
},
189+
"types": {
190+
"settingsUpdate": 1
191+
},
192+
"indexUids": {
193+
"INDEX_NAME": 1
194+
}
195+
},
196+
"duration": "PT0.110083S",
197+
"startedAt": "2024-12-10T15:49:04.995321Z",
198+
"finishedAt": "2024-12-10T15:49:05.105404Z"
199+
}
200+
],
201+
"total": 3,
202+
"limit": 1,
203+
"from": 2,
204+
"next": 1
205+
}
206+
```
207+
208+
## Get one batch
209+
210+
<RouteHighlighter method="GET" route="/batches/{batch_uid}"/>
211+
212+
Get a single batch.
213+
214+
### Path parameters
215+
216+
| Name | Type | Description |
217+
| :---------------- | :----- | :----------------------------------- |
218+
| **`batch_uid`** * | String | [`uid`](#uid) of the requested batch |
219+
220+
### Example
221+
222+
<CodeSamples id="get_batch_1" />
223+
224+
#### Response: `200 Ok`
225+
226+
```json
227+
{
228+
"uid": 1,
229+
"details": {
230+
"receivedDocuments": 1,
231+
"indexedDocuments": 1
232+
},
233+
"progress": null,
234+
"stats": {
235+
"totalNbTasks": 1,
236+
"status": {
237+
"succeeded": 1
238+
},
239+
"types": {
240+
"documentAdditionOrUpdate": 1
241+
},
242+
"indexUids": {
243+
"INDEX_NAME": 1
244+
}
245+
},
246+
"duration": "PT0.364788S",
247+
"startedAt": "2024-12-10T15:48:49.672141Z",
248+
"finishedAt": "2024-12-10T15:48:50.036929Z"
249+
}
250+
```

‎reference/api/tasks.mdx‎

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@ The `/tasks` route gives information about the progress of [asynchronous operati
1212
```json
1313
{
1414
"uid": 4,
15-
"indexUid" :"movie",
15+
"batchUids": 0,
16+
"indexUid":"movie",
1617
"status": "failed",
1718
"type": "indexDeletion",
1819
"canceledBy": null,
@@ -35,10 +36,19 @@ The `/tasks` route gives information about the progress of [asynchronous operati
3536
### `uid`
3637

3738
**Type**: Integer<br />
38-
**Description**: Unique sequential identifier of the task
39+
**Description**: Unique sequential identifier of the task.
3940

4041
<Capsule intent="note">
41-
The task `uid` is incremented **globally.**
42+
The task `uid` is incremented across all indexes in an instance.
43+
</Capsule>
44+
45+
### `batchUid`
46+
47+
**Type**: Integer<br />
48+
**Description**: Unique sequential identifier of the batch this task belongs to.
49+
50+
<Capsule intent="note">
51+
The batch `uid` is incremented across all indexes in an instance.
4252
</Capsule>
4353

4454
### `indexUid`
@@ -220,6 +230,7 @@ Task results are [paginated](/learn/async/paginating_tasks) and can be [filtered
220230
| Query Parameter | Default Value | Description |
221231
| :--------------------- | :----------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
222232
| **`uids`** | `*` (all uids) | [Filter tasks](/learn/async/filtering_tasks) by their `uid`. Separate multiple task `uids` with a comma (`,`) |
233+
| **`batchUids`** | `*` (all batch uids) | [Filter tasks](/learn/async/filtering_tasks) by their `batchUid`. Separate multiple `batchUids` with a comma (`,`) |
223234
| **`statuses`** | `*` (all statuses) | [Filter tasks](/learn/async/filtering_tasks) by their `status`. Separate multiple task `statuses` with a comma (`,`) |
224235
| **`types`** | `*` (all types) | [Filter tasks](/learn/async/filtering_tasks) by their `type`. Separate multiple task `types` with a comma (`,`) |
225236
| **`indexUids`** | `*` (all indexes) | [Filter tasks](/learn/async/filtering_tasks) by their `indexUid`. Separate multiple task `indexUids` with a comma (`,`). Case-sensitive |

0 commit comments

Comments
 (0)