[CI] Upload average CPU utilization of CI jobs to DataDog
This PR adds a new CI step that uploads the average CPU utilization of the current GH job to Datadog. I want to add more metrics in follow-up PRs.
r? `@jdno`
try-job: dist-i686-msvc
try-job: aarch64-apple
try-job: x86_64-gnu-llvm-18
Before the workspace split, the library was covered by the weekly `cargo
update` cron job. Now that the library has its own workspace, it doesn't
get these updates.
Add `library/Cargo.toml` to the job so updates happen again.
use key-value format in stage0 file
Currently, we are working on the python removal task on bootstrap. Which means we have to extract some data from the stage0 file using shell scripts. However, parsing values from the stage0.json file is painful because shell scripts don't have a built-in way to parse json files.
This change simplifies the stage0 file format to key-value pairs, which makes it easily readable from any environment.
See the zulip thread for more details: https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Using.20different.20format.20in.20the.20stage0.20file
CI: add a script for dynamically computing CI job matrix
It would be great if was easier to run specific CI workflows locally, and also to allow us to spawn a specific CI workflow by bors, to enable running arbitrary try builds. See discussion [here](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20workflows.20refactoring).
This PR is a first step in that direction.
- Moves the definition of CI runners and (for now) PR jobs into a separate `jobs.yml` file.
- Adds a simple Python script that reads the file, decides which jobs should be active for the current CI workflow, and prints them as JSON to their output.
- The PR job then reads this output and generates its job matrix based on it.
By moving the job definitions from `ci.yml` into a separate file, we can handle it programmatically, which should make it easier to both do local execution of CI jobs and also to do arbitrary try builds.
Cancel `cargo update` job if there's no updates
Previously there were always updates so we didn't hit this. Since #122489, this job runs on a more frequent schedule and causes errors if there have been no changes in that timespan.
This led to a weird error on https://github.com/rust-lang/rust/pull/122646#issuecomment-2004339093 - because of this I've replaced the `exit 1`s here with `gh run cancel` so we don't have false 'failed' jobs in the logs.
