「问题修复」「cargo」warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)

1 问题描述

在使用cargo进行复杂软件安装时， 依赖比较多的库， 需要快速下载， 出现了该问题“Couldn't resolve host name (Could not resolve host: crates)”， 错误日志如下：

$ cargo install mdbook
    Updating `git://mirrors.ustc.edu.cn/crates.io-index` index
  Installing mdbook v0.4.4
  Downloaded byteorder v1.3.4 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded inotify-sys v0.1.4 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
warning: spurious network error (2 tries remaining): [6] Couldn't resolve host name (Could not resolve host: crates)
  Downloaded input_buffer v0.3.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
。。。// 跳过部分日志
error: failed to compile `mdbook v0.4.4`, intermediate artifacts can be found at `/tmp/cargo-installzrH2PU`

Caused by:
  failed to download from `https://crates-io.proxy.ustclug.org/api/v1/crates/tokio-tungstenite/0.11.0/download`

Caused by:
  [6] Couldn't resolve host name (Could not resolve host: crates)

看到该问题， 还特意检查了网络， 设置了dns，

在文件 /etc/resolv.conf 中增加了阿里的dns域名服务器， 问题没有解决;

nameserver 223.5.5.5
nameserver 223.6.6.6

查找资料找到了问题根因和解决办法;

2 解决办法

临时规避，禁止并行化下载安装， 命令行输入： CARGO_HTTP_MULTIPLEXING=false cargo install mdbook

$ CARGO_HTTP_MULTIPLEXING=false cargo install mdbook
pdating `git://mirrors.ustc.edu.cn/crates.io-index` index
  Installing mdbook v0.4.4
  Downloaded cfg-if v0.1.10 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded bytes v0.5.6 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded futures-io v0.3.8 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded http-body v0.3.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded markup5ever_rcdom v0.1.0 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded num-integer v0.1.44 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded phf_shared v0.8.0 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded rand_pcg v0.2.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
  Downloaded serde_urlencoded v0.6.1 (registry `git://mirrors.ustc.edu.cn/crates.io-index`)
。。。
Compiling markup5ever_rcdom v0.1.0
   Compiling ammonia v3.1.0
   Compiling hyper v0.13.9
   Compiling warp v0.2.5
   Compiling mdbook v0.4.4
    Finished release [optimized] target(s) in 3m 01s
  Installing /home/neo/.cargo/bin/mdbook
   Installed package `mdbook v0.4.4` (executable `mdbook`)

编译时间略长， 也还可以接受。。。

3 引申阅读

问题根因分析：

详见：https://github.com/ustclug/discussions/issues/294

主要原因如下：

 knight42 commented on Apr 30

我找到这个问题的原因了，是因为我们的 nginx 会根据 client ip 对请求数作限制，默认是 4r/s 即每秒 4 个请求，如果超过的话就会返回 503。而我们的 nginx 配置里有这么一段

error_page 500 502 503 504 =302 $scheme://$proxy_host$uri;

即当上游返回 503 的时候，nginx 会让 client 302 到 $scheme://$proxy_host$uri，而这个时候 nginx 还没开始做 proxy，$proxy_host 应该还是空的，
所以 client 会看到 nginx 返回的 302 的响应里的 location 是 https:///crates/futures-io/futures-io-0.3.4.crate.

目前我先对 crates-io.proxy.ustclug.org 稍微放开了一下限制

limit_req_zone $binary_remote_addr zone=rust-crates:10m rate=20r/s;
limit_req zone=rust-crates burst=150;

同时调整了一下配置，减少了一次 302 跳转，@s977120 你现在可以试试看有没有问题。

不过感觉这个只是临时的 workaround，要彻底解决应该要让 cargo 限制一下并发请求数。

最终解决方案， 要控制cargo下载时的并发数， 估计要看一下源码，确认如何实现的。当前尝试使用cargo install -j 4 时，问题依旧;

使用说明如下：

$ cargo install --help
cargo-install 
Install a Rust binary. Default location is $HOME/.cargo/bin

USAGE:
    cargo install [OPTIONS] [--] [crate]...

OPTIONS:
    -q, --quiet                     No output printed to stdout
        --version <VERSION>         Specify a version to install
        --git <URL>                 Git URL to install the specified crate from
        --branch <BRANCH>           Branch to use when installing from git
        --tag <TAG>                 Tag to use when installing from git
        --rev <SHA>                 Specific commit to use when installing from git
        --path <PATH>               Filesystem path to local crate to install
        --list                      list all installed packages and their versions
    -j, --jobs <N>                  Number of parallel jobs, defaults to # of CPUs
    -f, --force                     Force overwriting existing crates or binaries
        --no-track                  Do not save tracking information
        --features <FEATURES>...    Space or comma separated list of features to activate
        --all-features              Activate all available features
        --no-default-features       Do not activate the `default` feature
        --profile <PROFILE-NAME>    Install artifacts with the specified profile
        --debug                     Build in debug mode instead of release mode
        --bin <NAME>...             Install only the specified binary
        --bins                      Install all binaries
        --example <NAME>...         Install only the specified example
        --examples                  Install all examples
        --target <TRIPLE>...        Build for the target triple
        --target-dir <DIRECTORY>    Directory for all generated artifacts
        --root <DIR>                Directory to install packages into
        --index <INDEX>             Registry index to install from
        --registry <REGISTRY>       Registry to use
    -v, --verbose                   Use verbose output (-vv very verbose/build.rs output)
        --color <WHEN>              Coloring: auto, always, never
        --frozen                    Require Cargo.lock and cache are up to date
        --locked                    Require Cargo.lock is up to date
        --offline                   Run without accessing the network
    -Z <FLAG>...                    Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details
    -h, --help                      Prints help information

ARGS:
    <crate>...    

Run `cargo help install` for more detailed information.