diff --git a/troubleshoot/elasticsearch/increase-capacity-data-node.md b/troubleshoot/elasticsearch/increase-capacity-data-node.md index 787f3b77f0..c148fc59e8 100644 --- a/troubleshoot/elasticsearch/increase-capacity-data-node.md +++ b/troubleshoot/elasticsearch/increase-capacity-data-node.md @@ -4,32 +4,27 @@ mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/increase-capacity-data-node.html applies_to: stack: - deployment: - eck: - ess: - ece: - self: products: - id: elasticsearch --- # Increase the disk capacity of data nodes [increase-capacity-data-node] -:::::::{tab-set} +:::::::{applies-switch} -::::::{tab-item} {{ech}} -In order to increase the disk capacity of the data nodes in your cluster: +::::::{applies-item} { ess: } +To increase the disk capacity of the data nodes in your cluster: 1. Log in to the [{{ecloud}} console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. On the **Hosted deployments** panel, click the gear under the `Manage deployment` column that corresponds to the name of your deployment. -3. If autoscaling is available but not enabled, enable it. You can do this by clicking the button `Enable autoscaling` on a banner like the one below: +2. On the **Hosted deployments** panel, select the gear under the **Manage deployment** column that corresponds to the name of your deployment. +3. If autoscaling is available but not enabled, enable it by clicking the **Enable autoscaling** button in a banner like the one below: :::{image} /troubleshoot/images/elasticsearch-reference-autoscaling_banner.png :alt: Autoscaling banner :screenshot: ::: - Or you can go to `Actions > Edit deployment`, check the checkbox `Autoscale` and click `save` at the bottom of the page. + Or you can go to **Actions > Edit deployment**, check the checkbox **Autoscale** and select **Save** from the bottom of the page. :::{image} /troubleshoot/images/elasticsearch-reference-enable_autoscaling.png :alt: Enabling autoscaling @@ -43,18 +38,16 @@ In order to increase the disk capacity of the data nodes in your cluster: :screenshot: ::: - or you can go to `Actions > Edit deployment` and look for the label `LIMIT REACHED` as shown below: + Alternatively, you can go to **Actions > Edit deployment** and look for the label `LIMIT REACHED` as shown below: + + ![Autoscaling limits](/troubleshoot/images/elasticsearch-reference-reached_autoscaling_limits.png "Autoscaling limits reached") - :::{image} /troubleshoot/images/elasticsearch-reference-reached_autoscaling_limits.png - :alt: Autoscaling limits reached - :screenshot: - ::: - If you are seeing the banner click `Update autoscaling settings` to go to the `Edit` page. Otherwise, you are already in the `Edit` page, click `Edit settings` to increase the autoscaling limits. After you perform the change click `save` at the bottom of the page. + If you are seeing the banner, click **Update autoscaling settings** to go to the **Edit** page. Otherwise, if you are already on the **Edit** page, click **Edit settings** to increase the autoscaling limits. After you perform the change select **Save** from the bottom of the page. :::::: -::::::{tab-item} Self-managed -In order to increase the data node capacity in your cluster, you will need to calculate the amount of extra disk space needed. +::::::{applies-item} { self: } +To increase the data node capacity in your cluster, you need to calculate the amount of extra disk space needed. 1. First, retrieve the relevant disk thresholds that will indicate how much space should be available. The relevant thresholds are the [high watermark](elasticsearch://reference/elasticsearch/configuration-reference/cluster-level-shard-allocation-routing-settings.md#cluster-routing-watermark-high) for all the tiers apart from the frozen one and the [frozen flood stage watermark](elasticsearch://reference/elasticsearch/configuration-reference/cluster-level-shard-allocation-routing-settings.md#cluster-routing-flood-stage-frozen) for the frozen tier. The following example demonstrates disk shortage in the hot tier, so we will only retrieve the high watermark: diff --git a/troubleshoot/elasticsearch/increase-cluster-shard-limit.md b/troubleshoot/elasticsearch/increase-cluster-shard-limit.md index 2bad9edb46..d7fc726dc3 100644 --- a/troubleshoot/elasticsearch/increase-cluster-shard-limit.md +++ b/troubleshoot/elasticsearch/increase-cluster-shard-limit.md @@ -4,11 +4,6 @@ mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/increase-cluster-shard-limit.html applies_to: stack: - deployment: - eck: - ess: - ece: - self: products: - id: elasticsearch --- @@ -23,28 +18,14 @@ You might want to influence this data distribution by configuring the [`cluster. To fix this issue, complete the following steps: -:::::::{tab-set} +:::::::{applies-switch} -::::::{tab-item} {{ech}} -In order to get the shards assigned we’ll need to increase the number of shards that can be collocated on a node in the cluster. We’ll achieve this by inspecting the system-wide `cluster.routing.allocation.total_shards_per_node` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings) and increasing the configured value. +::::::{applies-item} { ess: } +To get the shards assigned, you need to increase the number of shards that can be collocated on a node in the cluster. You achieve this by inspecting the system-wide `cluster.routing.allocation.total_shards_per_node` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings) and increasing the configured value. -**Use {{kib}}** +You can run the following steps using either [API console](/explore-analyze/query-filter/tools/console.md) or direct [Elasticsearch API](elasticsearch://reference/elasticsearch/rest-apis/index.md) calls. -1. Log in to the [{{ecloud}} console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. On the **Hosted deployments** panel, click the name of your deployment. - - ::::{note} - If the name of your deployment is disabled your {{kib}} instances might be unhealthy, in which case contact [Elastic Support](https://support.elastic.co). If your deployment doesn’t include {{kib}}, all you need to do is [enable it first](../../deploy-manage/deploy/elastic-cloud/access-kibana.md). - :::: - -3. Open your deployment’s side navigation menu (placed under the Elastic logo in the upper left corner) and go to **Dev Tools > Console**. - - :::{image} /troubleshoot/images/elasticsearch-reference-kibana-console.png - :alt: {{kib}} Console - :screenshot: - ::: - -4. Inspect the `cluster.routing.allocation.total_shards_per_node` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings): +1. Inspect the `cluster.routing.allocation.total_shards_per_node` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings): ```console GET /_cluster/settings?flat_settings @@ -63,7 +44,7 @@ In order to get the shards assigned we’ll need to increase the number of shard 1. Represents the current configured value for the total number of shards that can reside on one node in the system. -5. [Increase](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) the value for the total number of shards that can be assigned on one node to a higher value: +1. [Increase](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-put-settings) the value for the total number of shards that can be assigned on one node to a higher value: ```console PUT _cluster/settings @@ -77,8 +58,8 @@ In order to get the shards assigned we’ll need to increase the number of shard 1. The new value for the system-wide `total_shards_per_node` configuration is increased from the previous value of `300` to `400`. The `total_shards_per_node` configuration can also be set to `null`, which represents no upper bound with regards to how many shards can be collocated on one node in the system. :::::: -::::::{tab-item} Self-managed -In order to get the shards assigned you can add more nodes to your {{es}} cluster and assign the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. +::::::{applies-item} { self: } +To get the shards assigned, you can add more nodes to your {{es}} cluster and assign the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. To inspect which tier is an index targeting for assignment, use the [get index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) API to retrieve the configured value for the `index.routing.allocation.include._tier_preference` setting: @@ -109,7 +90,7 @@ Alternatively, if adding more nodes to the {{es}} cluster is not desired, inspec GET /_cluster/settings?flat_settings ``` - The response will look like this: + The response looks like this: ```console-result { diff --git a/troubleshoot/elasticsearch/increase-shard-limit.md b/troubleshoot/elasticsearch/increase-shard-limit.md index 99f9c27b23..0c15a40879 100644 --- a/troubleshoot/elasticsearch/increase-shard-limit.md +++ b/troubleshoot/elasticsearch/increase-shard-limit.md @@ -4,11 +4,6 @@ mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/increase-shard-limit.html applies_to: stack: - deployment: - eck: - ess: - ece: - self: products: - id: elasticsearch --- @@ -23,28 +18,14 @@ You might want to influence this data distribution by configuring the [index.rou To fix this issue, complete the following steps: -:::::::{tab-set} +:::::::{applies-switch} -::::::{tab-item} {{ech}} -In order to get the shards assigned we’ll need to increase the number of shards that can be collocated on a node. We’ll achieve this by inspecting the configuration for the `index.routing.allocation.total_shards_per_node` [index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) and increasing the configured value for the indices that have shards unassigned. +::::::{applies-item} { ess: } +To get the shards assigned, you need to increase the number of shards that can be collocated on a node. You achieve this by inspecting the configuration for the `index.routing.allocation.total_shards_per_node` [index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) and increasing the configured value for the indices that have shards unassigned. -**Use {{kib}}** +You can run the following steps using either [API console](/explore-analyze/query-filter/tools/console.md) or direct [Elasticsearch API](elasticsearch://reference/elasticsearch/rest-apis/index.md) calls. -1. Log in to the [{{ecloud}} console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. On the **Hosted deployments** panel, click the name of your deployment. - - ::::{note} - If the name of your deployment is disabled your {{kib}} instances might be unhealthy, in which case contact [Elastic Support](https://support.elastic.co). If your deployment doesn’t include {{kib}}, all you need to do is [enable it first](../../deploy-manage/deploy/elastic-cloud/access-kibana.md). - :::: - -3. Open your deployment’s side navigation menu (placed under the Elastic logo in the upper left corner) and go to **Dev Tools > Console**. - - :::{image} /troubleshoot/images/elasticsearch-reference-kibana-console.png - :alt: {{kib}} Console - :screenshot: - ::: - -4. Inspect the `index.routing.allocation.total_shards_per_node` [index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) for the index with unassigned shards: +1. Inspect the `index.routing.allocation.total_shards_per_node` [index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) for the index with unassigned shards: ```console GET /my-index-000001/_settings/index.routing.allocation.total_shards_per_node?flat_settings @@ -64,7 +45,7 @@ In order to get the shards assigned we’ll need to increase the number of shard 1. Represents the current configured value for the total number of shards that can reside on one node for the `my-index-000001` index. -5. [Increase](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) the value for the total number of shards that can be assigned on one node to a higher value: +1. [Increase](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) the value for the total number of shards that can be assigned on one node to a higher value: ```console PUT /my-index-000001/_settings @@ -78,8 +59,8 @@ In order to get the shards assigned we’ll need to increase the number of shard 1. The new value for the `total_shards_per_node` configuration for the `my-index-000001` index is increased from the previous value of `1` to `2`. The `total_shards_per_node` configuration can also be set to `-1`, which represents no upper bound with regards to how many shards of the same index can reside on one node. :::::: -::::::{tab-item} Self-managed -In order to get the shards assigned you can add more nodes to your {{es}} cluster and assing the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. +::::::{applies-item} { self: } +To get the shards assigned, you can add more nodes to your {{es}} cluster and assing the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. To inspect which tier is an index targeting for assignment, use the [get index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) API to retrieve the configured value for the `index.routing.allocation.include._tier_preference` setting: diff --git a/troubleshoot/elasticsearch/increase-tier-capacity.md b/troubleshoot/elasticsearch/increase-tier-capacity.md index a3492763e3..92ed2e2634 100644 --- a/troubleshoot/elasticsearch/increase-tier-capacity.md +++ b/troubleshoot/elasticsearch/increase-tier-capacity.md @@ -4,11 +4,6 @@ mapped_pages: - https://www.elastic.co/guide/en/elasticsearch/reference/current/increase-tier-capacity.html applies_to: stack: - deployment: - eck: - ess: - ece: - self: products: - id: elasticsearch --- @@ -21,27 +16,14 @@ If a warning is encountered with not enough nodes to allocate all shard replicas To accomplish this, complete the following steps: -:::::::{tab-set} +:::::::{applies-switch} -::::::{tab-item} {{ech}} -One way to get the replica shards assigned is to add an availability zone. This will increase the number of data nodes in the {{es}} cluster so that the replica shards can be assigned. This can be done by editing your deployment. But first you need to discover which tier an index is targeting for assignment. Do this using {{kib}}. +::::::{applies-item} { ess: } +You can get the replica shards assigned by adding an availability zone. This action increases the number of data nodes in the {{es}} cluster so that the replica shards can be assigned. You achieve this by editing your deployment. -**Use {{kib}}** - -1. Log in to the [{{ecloud}} console](https://cloud.elastic.co?page=docs&placement=docs-body). -2. On the **Hosted deployments** panel, click the name of your deployment. - - ::::{note} - If the name of your deployment is disabled your {{kib}} instances might be unhealthy, in which case contact [Elastic Support](https://support.elastic.co). If your deployment doesn’t include {{kib}}, all you need to do is [enable it first](../../deploy-manage/deploy/elastic-cloud/access-kibana.md). - :::: - -3. Open your deployment’s side navigation menu (placed under the Elastic logo in the upper left corner) and go to **Dev Tools > Console**. - - :::{image} /troubleshoot/images/elasticsearch-reference-kibana-console.png - :alt: {{kib}} Console - :screenshot: - ::: +This procedure includes steps you complete in {{kib}}, as well as steps using either [API console](/explore-analyze/query-filter/tools/console.md) or direct [Elasticsearch API](elasticsearch://reference/elasticsearch/rest-apis/index.md) calls. +First, you need to discover which tier an index is targeting for assignment. To inspect which tier an index is targeting for assignment, use the [get index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) API to retrieve the configured value for the `index.routing.allocation.include._tier_preference` setting: @@ -49,7 +31,7 @@ To inspect which tier an index is targeting for assignment, use the [get index s GET /my-index-000001/_settings/index.routing.allocation.include._tier_preference?flat_settings ``` -The response will look like this: +The response looks like this: ```console-result { @@ -64,7 +46,9 @@ The response will look like this: 1. Represents a comma separated list of data tier node roles this index is allowed to be allocated on, the first one in the list being the one with the higher priority i.e. the tier the index is targeting. e.g. in this example the tier preference is `data_warm,data_hot` so the index is targeting the `warm` tier and more nodes with the `data_warm` role are needed in the {{es}} cluster. -Now that you know the tier, you want to increase the number of nodes in that tier so that the replicas can be allocated. To do this you can either increase the size per zone to increase the number of nodes in the availability zone(s) you were already using, or increase the number of availability zones. Go back to the deployment’s landing page by clicking on the three horizontal bars on the top left of the screen and choosing **Manage this deployment**. On that page click the **Manage** button, and choose **Edit deployment**. Note that you must be logged in to [https://cloud.elastic.co/](https://cloud.elastic.co/) in order to do this. In the {{es}} section, find the tier where the replica shards could not be assigned. +Now that you know the tier, you want to increase the number of nodes in that tier so that the replicas can be allocated. To do this you can either increase the size per zone to increase the number of nodes in the availability zone(s) you were already using, or increase the number of availability zones. + +In {{kib}}, go to the deployment’s landing page by clicking on the three horizontal bars on the top left of the screen and choosing **Manage this deployment**. On that page click the **Manage** button, and choose **Edit deployment**. Note that you must be logged in to [https://cloud.elastic.co/](https://cloud.elastic.co/) in order to do this. In the {{es}} section, find the tier where the replica shards could not be assigned. :::{image} /troubleshoot/images/elasticsearch-reference-ess-advanced-config-data-tiers.png :alt: {{kib}} Console @@ -80,16 +64,15 @@ Now that you know the tier, you want to increase the number of nodes in that tie * Find the **Availability zones** selection. If it is less than 3, you can select a higher number of availability zones for that tier. -If it is not possible to increase the size per zone or the number of availability zones, you can reduce the number of replicas of your index data. We’ll achieve this by inspecting the [`index.number_of_replicas`](elasticsearch://reference/elasticsearch/index-settings/index-modules.md#dynamic-index-number-of-replicas) index setting index setting and decreasing the configured value. +If it is not possible to increase the size per zone or the number of availability zones, you can reduce the number of replicas of your index data. You achieve this by inspecting the [`index.number_of_replicas`](elasticsearch://reference/elasticsearch/index-settings/index-modules.md#dynamic-index-number-of-replicas) index setting index setting and decreasing the configured value. -1. Access {{kib}} as described above. -2. Inspect the [`index.number_of_replicas`](elasticsearch://reference/elasticsearch/index-settings/index-modules.md#dynamic-index-number-of-replicas) index setting. +1. Inspect the [`index.number_of_replicas`](elasticsearch://reference/elasticsearch/index-settings/index-modules.md#dynamic-index-number-of-replicas) index setting. ```console GET /my-index-000001/_settings/index.number_of_replicas ``` - The response will look like this: + The response looks like this: ```console-result { @@ -105,13 +88,13 @@ If it is not possible to increase the size per zone or the number of availabilit 1. Represents the currently configured value for the number of replica shards required for the index -3. Use the `_cat/nodes` API to find the number of nodes in the target tier: +1. Use the `_cat/nodes` API to find the number of nodes in the target tier: ```console GET /_cat/nodes?h=node.role ``` - The response will look like this, containing one row per node: + The response looks like this, containing one row per node: ```console-result himrst @@ -121,7 +104,7 @@ If it is not possible to increase the size per zone or the number of availabilit You can count the rows containing the letter representing the target tier to know how many nodes you have. See [Query parameters](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cat-nodes) for details. The example above has two rows containing `h`, so there are two nodes in the hot tier. -4. [Decrease](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) the value for the total number of replica shards required for this index. As replica shards cannot reside on the same node as primary shards for [high availability](../../deploy-manage/production-guidance/availability-and-resilience.md), the new value needs to be less than or equal to the number of nodes found above minus one. Since the example above found 2 nodes in the hot tier, the maximum value for `index.number_of_replicas` is 1. +1. [Decrease](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-put-settings) the value for the total number of replica shards required for this index. As replica shards cannot reside on the same node as primary shards for [high availability](../../deploy-manage/production-guidance/availability-and-resilience.md), the new value needs to be less than or equal to the number of nodes found above minus one. Since the example above found 2 nodes in the hot tier, the maximum value for `index.number_of_replicas` is 1. ```console PUT /my-index-000001/_settings @@ -135,8 +118,8 @@ If it is not possible to increase the size per zone or the number of availabilit 1. The new value for the `index.number_of_replicas` index configuration is decreased from the previous value of `2` to `1`. It can be set as low as 0 but configuring it to 0 for indices other than [searchable snapshot indices](../../deploy-manage/tools/snapshot-and-restore/searchable-snapshots.md) may lead to temporary availability loss during node restarts or permanent data loss in case of data corruption. :::::: -::::::{tab-item} Self-managed -In order to get the replica shards assigned you can add more nodes to your {{es}} cluster and assign the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. +::::::{applies-item} { self: } +To get the replica shards assigned, you can add more nodes to your {{es}} cluster and assign the index’s target tier [node role](../../manage-data/lifecycle/index-lifecycle-management/migrate-index-allocation-filters-to-node-roles.md#assign-data-tier) to the new nodes. To inspect which tier an index is targeting for assignment, use the [get index setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-get-settings) API to retrieve the configured value for the `index.routing.allocation.include._tier_preference` setting: @@ -144,7 +127,7 @@ To inspect which tier an index is targeting for assignment, use the [get index s GET /my-index-000001/_settings/index.routing.allocation.include._tier_preference?flat_settings ``` -The response will look like this: +The response looks like this: ```console-result { @@ -167,7 +150,7 @@ Alternatively, if adding more nodes to the {{es}} cluster is not desired, inspec GET /my-index-000001/_settings/index.number_of_replicas ``` - The response will look like this: + The response looks like this: ```console-result { @@ -189,7 +172,7 @@ Alternatively, if adding more nodes to the {{es}} cluster is not desired, inspec GET /_cat/nodes?h=node.role ``` - The response will look like this, containing one row per node: + The response looks like this, containing one row per node: ```console-result himrst