| commit | 2854335fe548d552b90e221410cdf48b1743c447 | [log] [tgz] |
|---|---|---|
| author | Kameron Lutes <[email protected]> | Thu Dec 11 06:45:01 2025 |
| committer | Kameron Lutes <[email protected]> | Thu Dec 11 06:51:46 2025 |
| tree | 7f309c88cded308898583f2913bfea48ef2e420f | |
| parent | 21fb973c4059908357178107034bccccf80b69d2 [diff] |
Roll third_party/crossbench/ aac006fd9..4a2c78af2 (17 commits) https://chromium.googlesource.com/crossbench/+log/aac006fd9958..4a2c78af2dde $ git log aac006fd9..4a2c78af2 --date=short --no-merges --format='%ad %ae %s' 2025-12-10 chromium-autoroll Roll Buildtools from d7c2ffbac256 to 797cad2655a9 (1 revision) 2025-12-10 kalutes Add 'check_rect' option to 'wait_for_element' 2025-12-10 cbruni Add jetstream3 and jetstream2 benchmark alias 2025-12-09 chromium-autoroll Roll Buildtools from fc67fc1cee41 to d7c2ffbac256 (1 revision) 2025-12-08 chromium-autoroll Roll Buildtools from a497a3a27e9e to fc67fc1cee41 (3 revisions) 2025-12-07 chromium-autoroll Roll Buildtools from eca5f0685c48 to a497a3a27e9e (1 revision) 2025-12-05 maximsheshukov Sort recent builds by build number 2025-12-05 maximsheshukov Set default commit to the recent build 2025-12-05 maximsheshukov Set the default for the `--repeat` flag to 30 instead of 100 2025-12-04 maximsheshukov Collect Pinpoint CLI user metrics 2025-12-04 etiennep [tracign] Add v8.memory tracing category 2025-12-03 chromium-autoroll Roll WebPageReplay from 052833a44dd2 to 2bfcbbe006bc (1 revision) 2025-12-03 khokhlov Release LoadLine v2 benchmark 2025-12-03 maximsheshukov Rename requests.py to http_requests.py 2025-12-01 kalutes Don't log path parsing warning for xpath strings 2025-12-01 chromium-autoroll Roll Buildtools from aec4f3c79e5a to eca5f0685c48 (1 revision) 2025-12-01 maximsheshukov Display complete pinpoint server error message Created with: roll-dep third_party/crossbench Change-Id: Ia3f69d61ef216e743ea1b03142050115693c756e Reviewed-on: https://chromium-review.googlesource.com/c/chromium/web-tests/+/7249430 Reviewed-by: Charles Dick <[email protected]>
web-testsweb-tests contains:
Do not git clone web-tests. Use the fetch command included with depot_tools.
mkdir src cd src gcloud auth login fetch web-tests cd web-tests
Don't forget to run gclient sync every time you pull new changes from origin.
web-tests uses poetry to manage python dependencies.
sudo apt install python3.11 python3-poetry
Alternatively, install poetry in a python venv:
python3 -m venv web-tests-venv source web-tests-venv/bin/activate pip install poetry
Check that you have poetry on your path and make sure you have the right $PATH settings.
poetry --help || echo "Please update your \$PATH to include poetry bin location"; # Depending on your setup, add one of the following to your $PATH: echo "`python3 -m site --user-base`/bin"; python3 -c "import sysconfig; print(sysconfig.get_path('scripts'))";
Install the necessary dependencies from the lock file using poetry:
cd cuj/crossbench/runner export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring poetry env use 3.11 poetry install
Setting PYTHON_KEYRING_BACKEND to keyring.backends.null.Keyring disables keyring and prevents poetry install from getting stuck waiting for user input in the GUI.
Before running a test against an android target, make sure your device is connected through adb:
adb devices > List of devices attached > 192.168.20.194:5555 device
Replace <DEVICE ID> below with the actual device ID from adb devices:
cd cuj/crossbench/runner poetry run python run.py --platform adb --device <DEVICE ID>
Before running a test against a ChromeOS target, make sure passwordless SSH is available to the device. Either an IP address or a SSH host is supported as the device id.
Replace <DEVICE> below with the IP address or hostname of your device:
cd cuj/crossbench/runner poetry run python run.py --platform cros --device <DEVICE>
Running against a local browser on linux is minimally supported, but may require manual changes to test configuration.
cd cuj/crossbench/runner poetry run python run.py --platform local
The minimal invocation of the runner will attempt to run all benchmarks, CUJs, and corresponding variants in series.
To run a subset of tests, use the --tests flag. --tests supports Python regex format for matching the test names.
poetry run python run.py --platform adb --device <DEVICE ID> --tests speedometer.*
To specify only certain variants of a test, you can use the --variants flag. --variants also supports Python regex format for matching variants.
poetry run python run.py --platform adb --device <DEVICE ID> --tests local-conference --variants 16p
By default the runner will use ‘Chrome’ if the --browser flag is not specified. The format accepted by the --browser flag depends on the platform. For Android, use the package name of the installed browser. For ChromeOS or local, use the path to the browser executable.
Some CUJs require secrets to perform privileged actions (such as a test account username/password, or auth tokens for the Google Meet Bond API). Place your secrets in secrets.hjson and pass the file to the runner:
poetry run python run.py --platform adb --device <DEVICE ID> --secrets /home/me/secrets.hjson --tests docs
Tests can be repeated for a number of iterations or for a specified amount of time using the --playback flag. This flag is supported by crossbench and will iterate the post-setup sections of a CUJ and collect metrics for the entire invocation (instead of splitting metrics by iteration).
poetry run python run.py --platform adb --device <DEVICE ID> --tests tab-stress --variants blank-tab --playback 50x
poetry run python run.py --platform adb --device <DEVICE ID> --tests tab-stress --variants blank-tab --playback 2h
To get started contributing to web-tests, refer to the docs.