diff options
Diffstat (limited to 'vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html')
| -rw-r--r-- | vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html | 127 |
1 files changed, 70 insertions, 57 deletions
diff --git a/vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html b/vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html index bf281982..3e41bbf7 100644 --- a/vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html +++ b/vhosts/blog/public/posts/2024-04-21/pipefail-option-in-gitlab-ci-cd/index.html @@ -14,8 +14,7 @@ <meta property="og:locale" content="ja_JP"> <link rel="icon" type="image/svg+xml" href="/favicon.svg"> <title>【GitLab】 GitLab CI/CD 上での bash/sh は pipefail が有効になっている｜REPL: Rest-Eat-Program Loop</title> - <link rel="stylesheet" href="/style.css?h=79020a898c7052f79b32e90376a4497d"> - <link rel="stylesheet" href="/hl.css?h=340e65ffd5c17713efc9107c06304f7b"> + <link rel="stylesheet" href="/style.css?h=60eb349e583f5bd51518a7eb98598043"> </head> <body class="single"> <header class="header"> @@ -89,14 +88,16 @@ 例: </p> - <pre class="highlight" filename=".gitlab-ci.yml" language="yaml"><code class="highlight"><span class="hljs-attr">hello-world:</span> - <span class="hljs-attr">stage:</span> <span class="hljs-string">test</span> - <span class="hljs-attr">image:</span> <span class="hljs-string">alpine:latest</span> - <span class="hljs-attr">script:</span> - <span class="hljs-bullet">-</span> <span class="hljs-string">'echo "Hello, World!"'</span> - <span class="hljs-attr">rules:</span> - <span class="hljs-bullet">-</span> <span class="hljs-attr">if:</span> <span class="hljs-string">'$CI_MERGE_REQUEST_IID'</span> - <span class="hljs-attr">when:</span> <span class="hljs-string">always</span></code></pre> + <div class="codeblock" filename=".gitlab-ci.yml" language="yaml"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#22863A">hello-world</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#22863A"> stage</span><span style="color:#24292E">: </span><span style="color:#032F62">test</span></span> +<span class="line"><span style="color:#22863A"> image</span><span style="color:#24292E">: </span><span style="color:#032F62">alpine:latest</span></span> +<span class="line"><span style="color:#22863A"> script</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#032F62">'echo "Hello, World!"'</span></span> +<span class="line"><span style="color:#22863A"> rules</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#22863A">if</span><span style="color:#24292E">: </span><span style="color:#032F62">'$CI_MERGE_REQUEST_IID'</span></span> +<span class="line"><span style="color:#22863A"> when</span><span style="color:#24292E">: </span><span style="color:#032F62">always</span></span></code></pre> + </div> <p> ここで、<code>script</code> に指定したコマンドが失敗する (exit status が 0 以外になる) と、即座に実行が停止され、ジョブは失敗する。 @@ -106,14 +107,16 @@ では、次のようなケースだとどうなるか。 </p> - <pre class="highlight" filename=".gitlab-ci.yml" language="yaml"><code class="highlight"><span class="hljs-attr">hello-world:</span> - <span class="hljs-attr">stage:</span> <span class="hljs-string">test</span> - <span class="hljs-attr">image:</span> <span class="hljs-string">alpine:latest</span> - <span class="hljs-attr">script:</span> - <span class="hljs-bullet">-</span> <span class="hljs-string">'exit 1 | exit 0'</span> - <span class="hljs-attr">rules:</span> - <span class="hljs-bullet">-</span> <span class="hljs-attr">if:</span> <span class="hljs-string">'$CI_MERGE_REQUEST_IID'</span> - <span class="hljs-attr">when:</span> <span class="hljs-string">always</span></code></pre> + <div class="codeblock" filename=".gitlab-ci.yml" language="yaml"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#22863A">hello-world</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#22863A"> stage</span><span style="color:#24292E">: </span><span style="color:#032F62">test</span></span> +<span class="line"><span style="color:#22863A"> image</span><span style="color:#24292E">: </span><span style="color:#032F62">alpine:latest</span></span> +<span class="line"><span style="color:#22863A"> script</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#032F62">'exit 1 | exit 0'</span></span> +<span class="line"><span style="color:#22863A"> rules</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#22863A">if</span><span style="color:#24292E">: </span><span style="color:#032F62">'$CI_MERGE_REQUEST_IID'</span></span> +<span class="line"><span style="color:#22863A"> when</span><span style="color:#24292E">: </span><span style="color:#032F62">always</span></span></code></pre> + </div> <p> 失敗するコマンドをパイプに接続した。通常 Bash では、パイプの最後のコマンドの exit code が全体の exit code になる。 @@ -126,10 +129,12 @@ 前述したようなケースにおいて、途中で失敗したときに全体を失敗させるには、<code>pipefail</code> オプションを有効にする。 </p> - <pre class="highlight" language="bash"><code class="highlight"><span class="hljs-comment"># On にする</span> -<span class="hljs-built_in">set</span> -o pipefail -<span class="hljs-comment"># Off にする</span> -<span class="hljs-built_in">set</span> +o pipefail</code></pre> + <div class="codeblock" language="bash"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#6A737D"># On にする</span></span> +<span class="line"><span style="color:#005CC5">set</span><span style="color:#005CC5"> -o</span><span style="color:#032F62"> pipefail</span></span> +<span class="line"><span style="color:#6A737D"># Off にする</span></span> +<span class="line"><span style="color:#005CC5">set</span><span style="color:#032F62"> +o</span><span style="color:#032F62"> pipefail</span></span></code></pre> + </div> <p> こうすると、パイプ全体が失敗するようになる。この設定は、デフォルトだと off になっている。 @@ -143,14 +148,16 @@ 次のような GitLab CI/CD ジョブが失敗してしまった。 </p> - <pre class="highlight" filename=".gitlab-ci.yml" language="yaml"><code class="highlight"><span class="hljs-attr">hoge:</span> - <span class="hljs-attr">stage:</span> <span class="hljs-string">test</span> - <span class="hljs-attr">image:</span> <span class="hljs-string">alpine:latest</span> - <span class="hljs-attr">script:</span> - <span class="hljs-bullet">-</span> <span class="hljs-string">'cat hoge.txt | grep piyo | sed -e "s/foo/bar/g"'</span> - <span class="hljs-attr">rules:</span> - <span class="hljs-bullet">-</span> <span class="hljs-attr">if:</span> <span class="hljs-string">'$CI_MERGE_REQUEST_IID'</span> - <span class="hljs-attr">when:</span> <span class="hljs-string">always</span></code></pre> + <div class="codeblock" filename=".gitlab-ci.yml" language="yaml"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#22863A">hoge</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#22863A"> stage</span><span style="color:#24292E">: </span><span style="color:#032F62">test</span></span> +<span class="line"><span style="color:#22863A"> image</span><span style="color:#24292E">: </span><span style="color:#032F62">alpine:latest</span></span> +<span class="line"><span style="color:#22863A"> script</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#032F62">'cat hoge.txt | grep piyo | sed -e "s/foo/bar/g"'</span></span> +<span class="line"><span style="color:#22863A"> rules</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#22863A">if</span><span style="color:#24292E">: </span><span style="color:#032F62">'$CI_MERGE_REQUEST_IID'</span></span> +<span class="line"><span style="color:#22863A"> when</span><span style="color:#24292E">: </span><span style="color:#032F62">always</span></span></code></pre> + </div> <p> <code>grep</code> コマンドは、パターンにマッチする行が一行もなかったとき、exit code 1 を返す。よって、<code>pipefail</code> が on になっていると、このジョブは失敗する。現在の <code>pipefail</code> がどうなっているか確かめるため <code>set +o</code> で全オプションを出力させたところ、<code>pipefail</code> が on になっていた。 @@ -160,20 +167,22 @@ しかし、先述したように Bash における <code>pipefail</code> のデフォルト値は off のはずだ。実際に、ローカルで <code>alpine:latest</code> を動かしてみたところ、 </p> - <pre class="highlight"><code>$ docker run --rm alpine:latest sh -c "set +o" -set +o errexit -set +o noglob -set +o ignoreeof -set +o monitor -set +o noexec -set +o xtrace -set +o verbose -set +o noclobber -set +o allexport -set +o notify -set +o nounset -set +o vi -set +o pipefail</code></pre> + <div class="codeblock"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span>$ docker run --rm alpine:latest sh -c "set +o"</span></span> +<span class="line"><span>set +o errexit</span></span> +<span class="line"><span>set +o noglob</span></span> +<span class="line"><span>set +o ignoreeof</span></span> +<span class="line"><span>set +o monitor</span></span> +<span class="line"><span>set +o noexec</span></span> +<span class="line"><span>set +o xtrace</span></span> +<span class="line"><span>set +o verbose</span></span> +<span class="line"><span>set +o noclobber</span></span> +<span class="line"><span>set +o allexport</span></span> +<span class="line"><span>set +o notify</span></span> +<span class="line"><span>set +o nounset</span></span> +<span class="line"><span>set +o vi</span></span> +<span class="line"><span>set +o pipefail</span></span></code></pre> + </div> <p> 確かに <code>pipefail</code> は無効になっている。 @@ -190,9 +199,11 @@ set +o pipefail</code></pre> <code>.gitlab-ci.yml</code> で明示的には書いていないので、GitLab Runner (GitLab CI/CD のスクリプトを実行するプログラム) が勝手に追加しているに違いない。そう仮説を立てて <a href="https://gitlab.com/gitlab-org/gitlab-runner" rel="noreferrer" target="_blank">GitLab Runner のリポジトリ</a> を調査したところ、<a href="https://gitlab.com/gitlab-org/gitlab-runner/-/blob/c75da0796a0e3048991dccfdf2784e3d931beda4/shells/bash.go#L276" rel="noreferrer" target="_blank">ソースコード中の以下の箇所</a> で <code>set -o pipefail</code> していることが判明した (コメントは筆者による)。 </p> - <pre class="highlight" language="go"><code class="highlight"><span class="hljs-comment">// pipefail オプションが存在しない環境にも対応するため、</span> -<span class="hljs-comment">// 先に set -o でオプション一覧を表示させたあと、set -o pipefail している</span> -buf.WriteString(<span class="hljs-string">"if set -o | grep pipefail > /dev/null; then set -o pipefail; fi; set -o errexit\n"</span>)</code></pre> + <div class="codeblock" language="go"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#6A737D">// pipefail オプションが存在しない環境にも対応するため、</span></span> +<span class="line"><span style="color:#6A737D">// 先に set -o でオプション一覧を表示させたあと、set -o pipefail している</span></span> +<span class="line"><span style="color:#24292E">buf.</span><span style="color:#6F42C1">WriteString</span><span style="color:#24292E">(</span><span style="color:#032F62">"if set -o | grep pipefail > /dev/null; then set -o pipefail; fi; set -o errexit</span><span style="color:#005CC5">\n</span><span style="color:#032F62">"</span><span style="color:#24292E">)</span></span></code></pre> + </div> </section> <section id="section--how-to-solve"> @@ -201,16 +212,18 @@ buf.WriteString(<span class="hljs-string">"if set -o | grep pipefail > / 通常の Bash スクリプトを書く場合と同様に、<code>pipefail</code> が on になっていては困る場所だけ off にしてやればよい。 </p> - <pre class="highlight" diff="true" filename=".gitlab-ci.yml" language="yaml"><code class="highlight"> <span class="hljs-attr">hoge:</span> - <span class="hljs-attr">stage:</span> <span class="hljs-string">test</span> - <span class="hljs-attr">image:</span> <span class="hljs-string">alpine:latest</span> - <span class="hljs-attr">script:</span> -<span class="hljs-string">+</span> <span class="hljs-bullet">-</span> <span class="hljs-string">'set +o pipefail'</span> - <span class="hljs-bullet">-</span> <span class="hljs-string">'cat hoge.txt | grep piyo | sed -e "s/foo/bar/g"'</span> -<span class="hljs-string">+</span> <span class="hljs-bullet">-</span> <span class="hljs-string">'set -o pipefail'</span> <span class="hljs-comment"># この例の場合、ここで終わりなので戻さなくてもよい</span> - <span class="hljs-attr">rules:</span> - <span class="hljs-bullet">-</span> <span class="hljs-attr">if:</span> <span class="hljs-string">'$CI_MERGE_REQUEST_IID'</span> - <span class="hljs-attr">when:</span> <span class="hljs-string">always</span></code></pre> + <div class="codeblock" diff="true" filename=".gitlab-ci.yml" language="yaml"> + <pre class="shiki github-light" style="background-color:#f5f5f5;color:#24292e" tabindex="0"><code><span class="line"><span style="color:#22863A"> hoge</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#22863A"> stage</span><span style="color:#24292E">: </span><span style="color:#032F62">test</span></span> +<span class="line"><span style="color:#22863A"> image</span><span style="color:#24292E">: </span><span style="color:#032F62">alpine:latest</span></span> +<span class="line"><span style="color:#22863A"> script</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#032F62">+ - 'set +o pipefail'</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#032F62">'cat hoge.txt | grep piyo | sed -e "s/foo/bar/g"'</span></span> +<span class="line"><span style="color:#032F62">+ - 'set -o pipefail'</span><span style="color:#6A737D"> # この例の場合、ここで終わりなので戻さなくてもよい</span></span> +<span class="line"><span style="color:#22863A"> rules</span><span style="color:#24292E">:</span></span> +<span class="line"><span style="color:#24292E"> - </span><span style="color:#22863A">if</span><span style="color:#24292E">: </span><span style="color:#032F62">'$CI_MERGE_REQUEST_IID'</span></span> +<span class="line"><span style="color:#22863A"> when</span><span style="color:#24292E">: </span><span style="color:#032F62">always</span></span></code></pre> + </div> </section> <section id="section--remarks"> |