add processing of kubectl cmd links (#16529)

- add processing of kubectl links in update-imported-docs.
- update component reference generation guide.
- For more information, see issue #16454.

content/en/docs/contribute/generate-ref-docs/kubernetes-components.md

@@ -7,8 +7,7 @@ content_template: templates/task
 
 This page shows how to use the `update-imported-docs` tool to generate
 reference documentation for tools and components in the
-[Kubernetes](https://github.com/kubernetes/kubernetes) and
+[Kubernetes](https://github.com/kubernetes/kubernetes) repository.
-[Federation](https://github.com/kubernetes/federation) repositories.
 
 {{% /capture %}}
 
@@ -66,7 +65,7 @@ The remaining steps refer to your base directory as `<k8s-base>`.
 
 The reference documentation for the Kubernetes components and tools is automatically
 generated from the Kubernetes source code. If you want to change the reference documentation,
-please follow [this guide](/docs/contribute/gen-ref-docs/contribute-upstream).
+please follow [this guide](/docs/contribute/generate-ref-docs/contribute-upstream).
 
 {{< note >}}
 If you only need to generate, but not change, the reference docs, you don't need to
@@ -80,12 +79,13 @@ The `update-imported-docs` tool is located in the `kubernetes/website/update-imp
 directory. The tool performs the following steps:
 
 1. Clones the related repositories specified in a configuration file. For the
-   purpose of generating reference docs, the repositories that are cloned by
+   purpose of generating reference docs, the repository that is cloned by
-   default are `kubernetes-incubator/reference-docs` and `kubernetes/federation`.
+   default is `kubernetes-incubator/reference-docs`.
 1. Runs commands under the cloned repositories to prepare the docs generator and
    then generates the Markdown files.
 1. Copies the generated Markdown files to a local clone of the `kubernetes/website`
    repository under locations specified in the configuration file.
+1. Updates `kubectl` command links from `kubectl`.md to the `kubectl` command reference.
 
 When the Markdown files are in your local clone of the `kubernetes/website`
 repository, you can submit them in a [pull request](/docs/contribute/start/)
@@ -171,12 +171,12 @@ might look like this:
 ...
 
     modified:   content/en/docs/reference/command-line-tools-reference/cloud-controller-manager.md
-    modified:   content/en/docs/reference/command-line-tools-reference/federation-apiserver.md
-    modified:   content/en/docs/reference/command-line-tools-reference/federation-controller-manager.md
     modified:   content/en/docs/reference/command-line-tools-reference/kube-apiserver.md
     modified:   content/en/docs/reference/command-line-tools-reference/kube-controller-manager.md
     modified:   content/en/docs/reference/command-line-tools-reference/kube-proxy.md
     modified:   content/en/docs/reference/command-line-tools-reference/kube-scheduler.md
+    modified:   content/en/docs/reference/setup-tools/kubeadm/generated/kubeadm.md
+    modified:   content/en/docs/reference/kubectl/kubectl.md
 ...
 ```
 
@@ -196,8 +196,7 @@ topics will be visible in the
 
 {{% capture whatsnext %}}
 
-* [Generating Reference Documentation for kubectl Commands](/docs/home/contribute/generated-reference/kubectl/)
+* [Generating Reference Documentation for kubectl Commands](/docs/contribute/generate-ref-docs/kubectl/)
-* [Generating Reference Documentation for the Kubernetes API](/docs/home/contribute/generated-reference/kubernetes-api/)
+* [Generating Reference Documentation for the Kubernetes API](/docs/contribute/generate-ref-docs/kubernetes-api/)
-* [Generating Reference Documentation for the Kubernetes Federation API](/docs/home/contribute/generated-reference/federation-api/)
+* [Contributing to the Upstream Kubernetes Project for Documentation](/docs/contribute/generate-ref-docs/contribute-upstream/)
-* [Contributing to the Upstream Kubernetes Project for Documentation](/docs/contribute/gen-ref-docs/contribute-upstream)
 {{% /capture %}}

update-imported-docs/reference.yml

@@ -8,7 +8,7 @@ repos:
     cd $GOPATH
     git clone https://github.com/kubernetes/kubernetes.git src/k8s.io/kubernetes
     cd src/k8s.io/kubernetes
-    git checkout release-1.14
+    git checkout release-1.16
     make generated_files
     cp -L -R vendor $GOPATH/src
     rm -r vendor
@@ -36,25 +36,25 @@ repos:
   - src: gen-compdocs/build/kubeadm*.md
     dst: content/en/docs/reference/setup-tools/kubeadm/generated/
 
-- name: federation
+#- name: federation
-  remote: https://github.com/kubernetes/federation.git
+#  remote: https://github.com/kubernetes/federation.git
   # Change this to a release branch when federation has release branches.
-  branch: master
+#  branch: master
-  generate-command: hack/generate-docs.sh
+#  generate-command: hack/generate-docs.sh
-  files:
+#  files:
-  - src: docs/admin/federation-apiserver.md
+#  - src: docs/admin/federation-apiserver.md
-    dst: content/en/docs/reference/command-line-tools-reference/federation-apiserver.md
+#    dst: content/en/docs/reference/command-line-tools-reference/federation-apiserver.md
-  - src: docs/admin/federation-controller-manager.md
+#  - src: docs/admin/federation-controller-manager.md
-    dst: content/en/docs/reference/command-line-tools-reference/federation-controller-manager.md
+#    dst: content/en/docs/reference/command-line-tools-reference/federation-controller-manager.md
-  - src: docs/admin/kubefed_init.md
+#  - src: docs/admin/kubefed_init.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_init.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_init.md
-  - src: docs/admin/kubefed_join.md
+#  - src: docs/admin/kubefed_join.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_join.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_join.md
-  - src: docs/admin/kubefed.md
+#  - src: docs/admin/kubefed.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed.md
-  - src: docs/admin/kubefed_options.md
+#  - src: docs/admin/kubefed_options.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_options.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_options.md
-  - src: docs/admin/kubefed_unjoin.md
+#  - src: docs/admin/kubefed_unjoin.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_unjoin.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_unjoin.md
-  - src: docs/admin/kubefed_version.md
+#  - src: docs/admin/kubefed_version.md
-    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_version.md
+#    dst: content/en/docs/reference/setup-tools/kubefed/kubefed_version.md

update-imported-docs/update-imported-docs

@@ -25,9 +25,11 @@ def processLinks(content, remotePrefix, subPath):
                 target.startswith("mailto:") or
                 target.startswith("#")):
             if target.startswith("/"):
-                target = "/".join(remotePrefix, target[1:])
+                targetList = remotePrefix, target[1:]
+                target = "/".join(targetList)
             else:
-                target = "/".join(remotePrefix, subPath, target)
+                targetList = remotePrefix, subPath, target
+                target = "/".join(targetList)
 
         return "[%s](%s)" % (ankor, target)
 
@@ -41,6 +43,25 @@ def processLinks(content, remotePrefix, subPath):
     return content
 
 
+def processKubectlLinks(content):
+    """Update markdown links found in the SeeAlso section of kubectl page.
+       Example:[kubectl annotate](/docs/reference/generated/kubectl/kubectl-commands#annotate)
+    """
+    def analyze(matchObj):
+        ankor = matchObj.group('ankor')
+        target = matchObj.group('target')
+        if (target.endswith(".md") and target.startswith("kubectl")):
+            ankorList = ankor.split("kubectl ")
+            target = "/docs/reference/generated/kubectl/kubectl-commands" + "#" + ankorList[1]
+        return "[%s](%s)" % (ankor, target)
+
+    # Links are in the form '[text](url)'
+    linkRegex = re.compile(r"\[(?P<ankor>.*)\]\((?P<target>.*?)\)")
+    content = re.sub(linkRegex, analyze, content)
+
+    return content
+
+
 def processFile(src, dst, repoPath, repoDir, rootDir, genAbsoluteLinks):
     """Process a file element.
 
@@ -78,6 +99,9 @@ def processFile(src, dst, repoPath, repoDir, rootDir, genAbsoluteLinks):
                     srcDir = os.path.dirname(src)
                     remotePrefix = repoPath + "/tree/master"
                     content = processLinks(content, remotePrefix, srcDir)
+                if dst.endswith("kubectl.md"):
+                    print("Processing kubectl links")
+                    content = processKubectlLinks(content)
                 dstFile.write(content)
         except Exception as ex:
             print("[Error] failed in writing target file '%s': %s"