Commit Graph

161 Commits

Author SHA1 Message Date
Aditya Mandaleeka 5d75f75b2a Bump node version to 22. (#46) 2026-02-19 18:36:47 -08:00
Aditya Mandaleeka ae6f4b00a9 Use gh pr comment instead of third party dependency. (#45) 2026-02-19 18:07:54 -08:00
Aditya Mandaleeka e1f7fe3165 Replace eval pipeline with skill-validator. (#44) 2026-02-19 17:27:21 -08:00
Viktor Hofer fd2d4409db Replace peter-evans/create-or-update-comment with gh CLI
The third-party action peter-evans/create-or-update-comment@v4 is not
allowed by repository policy. Replace with inline gh pr comment command
which is pre-installed on GitHub-hosted runners.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-19 23:59:36 +01:00
Viktor Hofer 5e5b930282 Temporarily dissable skill warnAsError 2026-02-19 23:55:17 +01:00
Viktor Hofer 7d69f47377 Disable reference-exists false-positive test 2026-02-19 23:55:17 +01:00
Viktor Hofer 3362be3cd5 Eval workflow structure (#41)
* Restructure repo to use src/<component> layout

Adopt the same repository structure as dotnet/skills:
- Move skills/ and tests/ under src/dotnet/
- Add plugin.json for the csharp-scripts component
- Add AGENTS.md with plugin versioning instructions
- Update marketplace.json files to point to new paths
- Update CI workflows for new directory structure
- Move contributor guidance from README into CONTRIBUTING.md
- Streamline README to match dotnet/skills format

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add evaluation workflow and dashboard infrastructure

Add GitHub Actions workflow for skill evaluation with parallel test
execution, response grading, and summary generation. Include:

- CI workflow (.github/workflows/evaluation.yml) with matrix-based
  parallel test runs and configurable parameters
- Evaluation scripts (eng/evaluation/) for invoking Copilot, parsing
  stats, evaluating responses, and generating summaries
- Dashboard (eng/dashboard/) with HTML/JS visualization and benchmark
  data generation script
- Updated CONTRIBUTING.md with evaluation documentation reference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-19 23:22:33 +01:00
Viktor Hofer 9c60c3fd44 Merge pull request #40 from ViktorHofer/RepoLayoutRefactoring
Restructure repo to use src/<component> layout
2026-02-19 23:15:30 +01:00
Tim Heuer 6fd0acd2e6 ci: added validator for custom-agents [skip ci] 2026-02-04 17:47:05 -08:00
Tim Heuer 56df391833 ci: fixing workflow for validation [skip ci] 2026-02-04 13:36:14 -08:00
Tim Heuer 41da31e928 feat: add initial marketplace manifest, contributing, readme, and validation workflow 📚 2026-02-04 07:44:22 -08:00