[LTP] [PATCH 1/1] ci: Add hook to mirror docparse to homepage
Aleks L
aleksandrosansan@gmail.com
Fri Dec 9 12:05:02 CET 2022
Hi Peter, two comments:
+ metadata-mirror:
+ permissions:
+ contents: write # for git push
Since you are using GH_PERSONAL_ACCESS_TOKEN and not the default
GITHUB_TOKEN for push, I think "contents: read" should be sufficient. Could
you please try it?
+ - name: Checkout LTP homepage
+ uses: actions/checkout@v3
+ with:
+ repository: "linux-test-project/linux-test-project.github.com"
+ path: linux-test-project.github.com
+ persist-credentials: false
+ env:
+ GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
I think the GH_PERSONAL_ACCESS_TOKEN env variable here is not used and can
be deleted. The checkout uses the standard GITHUB_TOKEN with read access to
public repos.
Otherwise LGTM
On Fri, Dec 9, 2022 at 11:04 AM Petr Vorel <pvorel@suse.cz> wrote:
> GitHub Actions git push hook generates metadata HTML and push it wo LTP
> homepage.
>
> Hook pushes only if there are actual changes in generated doc.
>
> NOTE: This requires to add Personal access tokens (classic)
> named as GH_PERSONAL_ACCESS_TOKEN with permissions to public_repo.
>
> Signed-off-by: Petr Vorel <pvorel@suse.cz>
> ---
> Hi,
>
> automatic updates of our docparse documentation to our homepage [1]
> (linked in "nightly test documentation" [2]).
>
> This does push to our homepage git repository [3].
>
> This patchset required to add:
>
> 1) Personal Access Token (PAT) to any developer which has write access
> to homepage git repository [3]. In Developer settings -> Personal access
> tokens -> Tokens (classic) [4]), where set:
> Note: GH_PERSONAL_ACCESS_TOKEN
> Select scopes: public_repo (minimal permission)
> Expiration: either never or regularly renew.
>
> 2) Allow PAT in LTP organisation (I dared to already set it)
> Iin linux-test-project group -> Settings -> Third-party Access -> Personal
> access tokens -> Settings [5]
> select:
> Allow access via personal access tokens (classic)
> API and Git access will be allowed using an organization member's personal
> access token (classic)
>
> 3) Add repository action secret to ltp repository
> IN Settings -> Actions -> New repository secret [6]:
> name: GH_PERSONAL_ACCESS_TOKEN
> value: the value of previously created token.
>
> I haven't tested fine-grained personal access tokens, maybe they are
> safer. Using proper GitHub App would be safer (would not include PAT),
> but I have spent already quite a lot of time with this, thus unless
> anybody objects I'd merge this way.
>
> I did example of this run on organisation which I created just for
> testing purposes:
>
> * working example
> action run [7], resulted in commit [8]. Example of unrelated commit [9]
> which does not trigger doc generation [10].
>
> * example of error, when token not set
> https://github.com/foo-pevik/ltp_foo/actions/runs/3650567244
> GH_PERSONAL_ACCESS_TOKEN environment variable is not set
>
> * example of invalid GH_PERSONAL_ACCESS_TOKEN
>
> https://github.com/foo-pevik/ltp_foo/actions/runs/3650579488/jobs/6166734733
> Notice: GH_PERSONAL_ACCESS_TOKEN: ***
> fatal: could not read Password for 'https://***@github.com': No such
> device or address
> Error: Process completed with exit code 128.
>
> Credits:
> * Inspiration from SwiftDocOrg [11], pointed out by Cyril
> * Thanks a lot to Aleks for hint with "persist-credentials: false".
>
> Kind regards,
> Petr
>
> [1] http://linux-test-project.github.io/
> [2] http://linux-test-project.github.io/metadata/metadata.nightly.html
> [3] https://github.com/linux-test-project/linux-test-project.github.com
> [4] https://github.com/settings/tokens/new
> [5]
> https://github.com/organizations/linux-test-project/settings/personal-access-tokens
> [6] https://github.com/foo-pevik/ltp_foo/settings/secrets/actions/new
> [7] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475124
> [8]
> https://github.com/foo-pevik/linux-test-project.github.com_foo/commit/c2b94862da2c5741f08282988dcc0302e0847a69
> [9]
> https://github.com/foo-pevik/ltp_foo/commit/36a66a2f304b449e1c17d72bad1b1db46a30f36f
> [10] https://github.com/foo-pevik/ltp_foo/actions/runs/3651475552
> [11] https://github.com/SwiftDocOrg/github-wiki-publish-action
>
> .github/workflows/metadata-mirror.yml | 141 ++++++++++++++++++++++++++
> 1 file changed, 141 insertions(+)
> create mode 100644 .github/workflows/metadata-mirror.yml
>
> diff --git a/.github/workflows/metadata-mirror.yml
> b/.github/workflows/metadata-mirror.yml
> new file mode 100644
> index 000000000..9182c554d
> --- /dev/null
> +++ b/.github/workflows/metadata-mirror.yml
> @@ -0,0 +1,141 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# Copyright (c) 2022 Petr Vorel <pvorel@suse.cz>
> +
> +name: "Mirror metadata doc to homepage"
> +
> +on:
> + push:
> + branches:
> + - master
> +
> +permissions: {}
> +jobs:
> + metadata-mirror:
> + permissions:
> + contents: write # for git push
> +
> + runs-on: ubuntu-latest
> + if: ${{ github.repository == 'linux-test-project/ltp' }}
> + steps:
> + - name: Check secret
> + env:
> + GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> + run: |
> + if [ -z "$GH_PERSONAL_ACCESS_TOKEN" ]; then
> + echo "::error::GH_PERSONAL_ACCESS_TOKEN environment variable
> is not set"
> + exit 1
> + fi
> +
> + - name: Checkout LTP
> + uses: actions/checkout@v3
> + with:
> + path: ltp
> + # we need to fetch whole history to get 'git describe' working
> for correct version in docs
> + fetch-depth: 0
> +
> + - name: Checkout LTP homepage
> + uses: actions/checkout@v3
> + with:
> + repository: "linux-test-project/linux-test-project.github.com"
> + path: linux-test-project.github.com
> + persist-credentials: false
> + env:
> + GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> +
> + - name: Check metadata need to be updated
> + run: |
> + cd "$GITHUB_WORKSPACE/ltp/testcases/"
> +
> + # check for changes
> + cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> + git grep '<p><strong>Version</strong>:'
> metadata/metadata.nightly.html
> + sed -ne 's/.*<p><strong>Version<\/strong>: \(.*\)<\/p>/\1/p'
> metadata/metadata.nightly.html
> + old_commit=$(sed -ne 's/.*<p><strong>Version<\/strong>:
> \(.*\)<\/p>/\1/p' metadata/metadata.nightly.html)
> + old_commit=$(echo "$old_commit" | sed 's/.*-g\(.*\)/\1/')
> + echo "::notice::old commit in linux-test-project.github.com:
> '$old_commit'"
> +
> + echo "git log $old_commit..HEAD testcases/ | wc -l"
> + git log $old_commit..HEAD testcases/ | wc -l
> +
> + echo "git log $old_commit..HEAD testcases/ 2>&1 | wc -l"
> + git log $old_commit..HEAD testcases/ 2>&1 | wc -l
> +
> + if [ "$(git log $old_commit..HEAD testcases/ 2>&1 | wc -l)" = 0
> ]; then
> + echo "::notice::no file changes in testcases/ directory since
> '$old_commit' => exit without commit and push"
> + exit 0
> + fi
> +
> + - name: Install dependencies
> + run: |
> + apt="apt install -y --no-install-recommends"
> + sudo $apt asciidoctor autoconf automake libjson-perl
> libwww-perl make
> +
> + echo "which asciidoctor"
> + which asciidoctor || { echo "::error::missing asciidoctor";
> exit 1; }
> +
> + - name: Configure LTP
> + run: |
> + cd "$GITHUB_WORKSPACE/ltp/"
> + make autotools && ./configure
> --with-metadata-generator=asciidoctor && make Version || { echo
> "::error::LTP configure failed"; exit 1; }
> +
> + - name: Generate html metadata doc
> + run: |
> + cd "$GITHUB_WORKSPACE/ltp/metadata/"
> + echo "going to generate metadata"
> + make -j$(nproc)
> +
> + - name: Check possible LTP homepage update
> + run: |
> + cd "$GITHUB_WORKSPACE/ltp/"
> + ltp_commit=$(git log --pretty=format:"%h" -1 .)
> + echo "::notice::Search for $ltp_commit in
> metadata/metadata.nightly.html"
> +
> + cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> + grep "<p><strong>Version</strong>:"
> metadata/metadata.nightly.html
> + git pull
> + grep "<p><strong>Version</strong>:"
> metadata/metadata.nightly.html
> + if grep "<p><strong>Version</strong>: .*${ltp_commit}.*</p>"
> metadata/metadata.nightly.html; then
> + echo "::notice::$ltp_commit already in
> metadata/metadata.nightly.html => exit without commit and push"
> + exit 0
> + fi
> +
> + - name: Push generated html metadata to LTP homepage
> + env:
> + GH_PERSONAL_ACCESS_TOKEN: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN
> }}
> + run: |
> + git config --global user.email "actions@github.com"
> + git config --global user.name "Metadata doc mirror"
> +
> + cd "$GITHUB_WORKSPACE/ltp/"
> + commit=$(git log --pretty=format:"%h" -1 .)
> + commit_desc=$(git log --pretty=format:"%h (\"%s\")" -1 .)
> + echo "commit_desc: '$commit_desc'"
> +
> + cd "$GITHUB_WORKSPACE/linux-test-project.github.com"
> +
> + # pull in case previous hook pushed later than checkout here
> had started
> + git pull
> +
> + cp -v "$GITHUB_WORKSPACE/ltp/docparse/metadata.html"
> metadata/metadata.nightly.html
> +
> + # skip if there is only different version and time
> + echo "Check for changes"
> + if ! git diff --unified=0 --no-color | grep -Po
> '(?<=^\+)(?!\+\+).*' | grep -v -e '^<p><strong>Version</strong>:' -e '^Last
> updated'; then
> + echo "::notice::No changes in metadata/metadata.nightly.html
> => exit without commit and push"
> + exit 0
> + fi
> +
> + git add .
> +
> + # only commit if there are changes
> + if git diff-index --quiet HEAD --; then
> + echo "::notice::No files changed in $GITHUB_WORKSPACE/
> linux-test-project.github.com => exit without commit and push"
> + git status
> + exit 0
> + fi
> +
> + printf "metadata.nightly.html: Update to $commit\n\nUpdate
> metadata.nightly.html to $commit_desc\n" > /tmp/msg
> + git commit -F /tmp/msg .
> +
> + echo "::notice::GH_PERSONAL_ACCESS_TOKEN:
> $GH_PERSONAL_ACCESS_TOKEN"
> + git push https://${
> GH_PERSONAL_ACCESS_TOKEN}@github.com/linux-test-project/linux-test-project.github.com.git
> <http://GH_PERSONAL_ACCESS_TOKEN%7D@github.com/linux-test-project/linux-test-project.github.com.git>
> --
> 2.38.1
>
>
More information about the ltp
mailing list