From 90f89a763f0475dbc24368cf30f8749e359961d3 Mon Sep 17 00:00:00 2001 From: Hemant Kumar Date: Wed, 5 Aug 2020 14:33:47 -0400 Subject: [PATCH] Add docs for manual recovery from resize failure --- content/en/docs/concepts/storage/persistent-volumes.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 132e397e75..4933513929 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -254,6 +254,16 @@ FlexVolume resize is possible only when the underlying driver supports resize. Expanding EBS volumes is a time-consuming operation. Also, there is a per-volume quota of one modification every 6 hours. {{< /note >}} +#### Recovering from Failure when Expanding Volumes + +If expanding underlying storage fails, the cluster administrator can manually recover the Persistent Volume Claim (PVC) state and cancel the resize requests. Otherwise, the resize requests are continuously retried by the controller without administrator intervention. + +1. Mark the PersistentVolume(PV) that is bound to the PersistentVolumeClaim(PVC) with `Retain` reclaim policy. +2. Delete the PVC. Since PV has `Retain` reclaim policy - we will not loose any data when we recreate the PVC. +3. Delete the `claimRef` entry from PV specs, so as new PVC can bind to it. This should make the PV `Available`. +4. Re-create the PVC with smaller size than PV and set `volumeName` field of the PVC to the name of the PV. This should bind new PVC to existing PV. +5. Don't forget to restore the reclaim policy of the PV. + ## Types of Persistent Volumes