 Magnus Müller
|
3ecee462a2
|
Update GitHub Actions workflow permissions to allow writing comments on pull requests and issues, enhancing interaction capabilities for automated testing processes.
|
2025-06-07 13:22:43 +02:00 |
|
|
Magnus Müller
|
bdf29c34fb
|
Add PR comment functionality to GitHub Actions workflow for agent task evaluation results. This includes a summary of passed tasks, percentage score, and status emoji based on task outcomes, enhancing visibility of evaluation results directly in pull requests.
|
2025-06-07 13:16:30 +02:00 |
|
|
Magnus Müller
|
8d9b24b03a
|
Add summary output for agent tasks evaluation in CI workflow
|
2025-06-07 11:27:06 +02:00 |
|
|
Magnus Müller
|
52c60f8367
|
Remove dependency on tests job in evaluate-tasks step of CI workflow
|
2025-06-07 11:11:23 +02:00 |
|
|
Magnus Müller
|
576519ee40
|
Enhance CI workflow by adding agent tasks evaluation step and updating evaluate_tasks.py to output evaluation results
|
2025-06-07 10:59:24 +02:00 |
|
|
Magnus Müller
|
3666d2b077
|
Add agent tasks evaluation script and update CI workflow to include it
|
2025-06-07 10:49:03 +02:00 |
|
|
Magnus Müller
|
074169f80f
|
Merge branch 'main' into tests/eval
|
2025-06-07 09:15:41 +02:00 |
|
|
Nick Sweeting
|
4358924964
|
only install chromium in tests
|
2025-06-06 19:32:20 -07:00 |
|
|
Magnus Müller
|
42dac3dce1
|
Add API key environment variables to GitHub Actions workflow for enhanced test capabilities
|
2025-06-07 01:11:48 +02:00 |
|
|
Nick Sweeting
|
8504bc4c7b
|
parallelize playwright tests using loop scope=session and pytest-xdist
|
2025-06-06 02:03:25 -07:00 |
|
|
Alezander9
|
61f9c31a3d
|
feat: support changing eval task set
|
2025-06-03 10:27:34 -07:00 |
|
|
BroskyBrowser
|
15cb992618
|
fix: expose ANCHOR_BROWSER_API_KEY variable in the evals workflow .yaml
|
2025-06-01 12:58:14 +02:00 |
|
|
Nick Sweeting
|
bfb6b26274
|
Merge branch 'main' into new-eval
|
2025-05-25 18:58:08 -07:00 |
|
|
Nick Sweeting
|
1d2cb46d73
|
Update claude.yml
|
2025-05-25 06:07:37 -04:00 |
|
|
Nick Sweeting
|
6d0758764a
|
Add Claude PR Assistant workflow
|
2025-05-25 03:04:06 -07:00 |
|
|
Alezander9
|
ad71ba8d29
|
add branch name selection into workflow
|
2025-05-24 19:12:01 -07:00 |
|
|
Alezander9
|
1f113fa640
|
Merge remote-tracking branch 'upstream/main' into new-eval
|
2025-05-24 10:25:43 -07:00 |
|
|
Nick Sweeting
|
04a3c881df
|
add docker setting for ci
|
2025-05-23 22:22:10 -07:00 |
|
|
Nick Sweeting
|
cacc7c2020
|
allow running publish manually
|
2025-05-23 22:18:09 -07:00 |
|
|
Nick Sweeting
|
8e8f9a2381
|
allow running publish manually
|
2025-05-23 22:17:09 -07:00 |
|
|
Nick Sweeting
|
4196e79faa
|
fix publish action
|
2025-05-23 22:15:34 -07:00 |
|
|
Nick Sweeting
|
3d10260543
|
fix missing link between find_tests and test job in CI
|
2025-05-23 19:22:55 -07:00 |
|
|
Nick Sweeting
|
37a36dbd28
|
catch failure case up-front
|
2025-05-23 19:12:46 -07:00 |
|
|
Nick Sweeting
|
6a1ed628e3
|
properly split filenamees out of ls results in test discovery
|
2025-05-23 19:10:25 -07:00 |
|
|
Nick Sweeting
|
063f103efd
|
more warning on filure to list tests
|
2025-05-23 19:05:52 -07:00 |
|
|
Nick Sweeting
|
06ee004a88
|
add assertion to tests discovery
|
2025-05-23 18:33:18 -07:00 |
|
|
Nick Sweeting
|
9fcd5cd7b2
|
debugging tests discovery
|
2025-05-23 18:32:28 -07:00 |
|
|
Nick Sweeting
|
e19e1c5dfc
|
fix ci tests
|
2025-05-23 18:29:59 -07:00 |
|
|
Nick Sweeting
|
815ae48938
|
fix pypi publish action
|
2025-05-23 18:21:18 -07:00 |
|
|
Alezander9
|
45dd0a26c2
|
update eval workflow with new arguments
|
2025-05-23 14:57:18 -07:00 |
|
|
Alezander9
|
a3dd8b004b
|
update eval workflow with new arguments
|
2025-05-23 14:46:14 -07:00 |
|
|
Nick Sweeting
|
27ca169393
|
reduce mumber of redundant CI builds
|
2025-05-23 04:46:21 -04:00 |
|
|
Nick Sweeting
|
3940462d8d
|
Potential fix for code scanning alert no. 28: Workflow does not contain permissions
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
|
2025-05-23 03:50:15 -04:00 |
|
|
Nick Sweeting
|
6b8360c475
|
better logging
|
2025-05-22 23:17:21 -07:00 |
|
|
Alezander9
|
aab470243f
|
update user message default argument
|
2025-05-22 15:53:49 -07:00 |
|
|
Alezander9
|
529e43fdd1
|
update args in workflow script to match new format
|
2025-05-22 11:36:47 -07:00 |
|
|
Alezander9
|
4a7e9113ca
|
add claude 4 support and cleanup eval script arguments
|
2025-05-22 10:54:00 -07:00 |
|
|
Nick Sweeting
|
18554e2834
|
autodetect tests for ci by looking in folder
|
2025-05-22 05:34:04 -07:00 |
|
|
Alezander9
|
0fbfc82da0
|
switch eval workflow to use new consolidated branch
|
2025-05-21 14:39:09 -07:00 |
|
|
Alezander9
|
a8d661b2d0
|
consolidated changes: adapt refactored eval service to work with new browser and on github actions
|
2025-05-21 14:36:57 -07:00 |
|
|
Nick Sweeting
|
bdb9bc81a3
|
install both chrome and chromium channels
|
2025-05-20 03:55:57 -07:00 |
|
|
Nick Sweeting
|
312a738ce9
|
better browser info logging at startup and tests
|
2025-05-20 03:50:04 -07:00 |
|
|
Nick Sweeting
|
836e1ddbf0
|
rename test
|
2025-05-20 02:33:33 -07:00 |
|
|
Nick Sweeting
|
eb8e7d52e0
|
only run docker build on main branch pushes
|
2025-05-16 05:42:58 -04:00 |
|
|
Nick Sweeting
|
71329da1d5
|
use repo name for ghcr
|
2025-05-13 22:09:46 -07:00 |
|
|
Nick Sweeting
|
56a9ed7374
|
more dockerfile fixes
|
2025-05-13 21:24:32 -07:00 |
|
|
Nick Sweeting
|
a771816566
|
also push to ghcr
|
2025-05-13 21:16:42 -07:00 |
|
|
Nick Sweeting
|
150dc9efde
|
also push to ghcr
|
2025-05-13 20:34:36 -07:00 |
|
|
Nick Sweeting
|
ac08799202
|
improve docker autobuild tagging
|
2025-05-13 20:11:45 -07:00 |
|
|
Nick Sweeting
|
5dc7a3cee6
|
feat: add dockerhub workflow (#1608)
|
2025-05-13 20:10:36 -07:00 |
|