Script for vendoring dependencies of rust std libs

This commit creates a script `vendor.sh` that can be used to vendor our
dependencies from crates.io. In addition to vendoring the direct
dependencies of our rust crates, it also vendors the dependencies of the
rust standard libraries.

This script requires a minor update to the Dockerfile because we need to
install the sources of the rust standard libraries and copy a file when
we are the "root" user.

Author: NickeZ

.ci/run-container-ci

@@ -25,7 +25,7 @@
 set -e
 set -x
 
-CONTAINER=shiftcrypto/firmware_v2:40
+CONTAINER=shiftcrypto/firmware_v2:41
 
 if [ "$1" == "pull" ] ; then
 	docker pull "$CONTAINER"

Dockerfile

@@ -129,9 +129,18 @@ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | CARGO_HOME=/opt/
 RUN rustup target add thumbv7em-none-eabi
 RUN rustup component add rustfmt
 RUN rustup component add clippy
+RUN rustup component add rust-src
 RUN CARGO_HOME=/opt/cargo cargo install cbindgen --version 0.26.0 --locked
 RUN CARGO_HOME=/opt/cargo cargo install bindgen-cli --version 0.69.4 --locked
 
+# Until cargo vendor supports vendoring dependencies of the rust std libs we
+# need a copy of this file next to the toml file. It also has to be world
+# writable so that invocations of `cargo vendor` can update it. Below is the
+# tracking issue for `cargo vendor` to support rust std libs.
+# https://github.com/rust-lang/wg-cargo-std-aware/issues/23
+RUN cp "$(rustc --print=sysroot)/lib/rustlib/src/rust/Cargo.lock" "$(rustc --print=sysroot)/lib/rustlib/src/rust/library/test/"
+RUN chmod 777 $(rustc --print=sysroot)/lib/rustlib/src/rust/library/test/Cargo.lock
+
 COPY tools/prost-build-proto prost-build-proto
 RUN CARGO_HOME=/opt/cargo cargo install --path prost-build-proto --locked
 

src/rust/README.md

@@ -35,3 +35,8 @@ The bottom-most layer are bindings generated from C header files:
 
 We generate one header file `rust.h` and ever product specific function is `#ifdeffed` with
 `RUST_PRODUCT_*` macro.
+
+# Vendoring
+
+Run the vendoring script `vendor.sh` to vendor dependencies from crates.io. The
+script will ensure that also rust std libs dependencies are vendored.

src/rust/vendor.sh

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+#
+# Script for vendoring our dependencies, including the deps of core/alloc.
+#
+# This script must be called from the <git-project-root>/src/rust directory. It will place the
+# dependencies in a directory called "vendor" in the current working directory.
+#
+# For some reason Cargo needs to find the dependencies of all rust std libs. Since "test" depends
+# on all the other ones, we take the toml-file from it. This means that we vendor libs that we
+# don't use in the end (like hashbrown and getopts).
+#
+# The invocation below depends on the fact that rust std libs "Cargo.lock" has been manually copied
+# to be next to the Cargo.toml file in the test directory.
+#
+# Copying the Cargo.lock file in the rust sysroot image requires root permissions. Therefore it is
+# done in the Dockerfile in our setup.
+
+RUST_SYSROOT="$(rustc --print=sysroot)"
+
+RUSTC_BOOTSTRAP=1 cargo vendor \
+    --manifest-path Cargo.toml \
+    --sync "$RUST_SYSROOT/lib/rustlib/src/rust/library/test/Cargo.toml" \
+    vendor