We could add a manual trigger but I'm not sure we'd be able to see the results without being embedded in a PR.

Can we add it to the commit message?

So rather than triggering all the time code is touched only trigger when code is touched and a commit contains [benchmark]? This could be a pain as people will need to know to trigger the benchmark but could be a good option if working on R side code that impacts the stan model?

Or alternatively, change it to look for PR comments but that might be complicated.

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 63d8631 is merged into main:

•   :ballot_box_with_check:default: 59.5s -> 57.5s [-25.54%, +18.95%]
•   :ballot_box_with_check:no_delays: 48.4s -> 50.6s [-7.14%, +16.49%]
•   :ballot_box_with_check:random_walk: 14.9s -> 15.7s [-5.98%, +17.37%]
•   :ballot_box_with_check:stationary: 35.5s -> 36.1s [-4.93%, +8.22%]
•   :ballot_box_with_check:uncertain: 57.9s -> 55.9s [-15.61%, +8.77%]
  Further explanation regarding interpretation and methodology can be found in the documentation.

These are both great suggestions but I think would either require a separate workflow file or some fairly involved logic inside the workflow body to check whether the benchmark was triggered by a change in stan code or a trigger commit message.

As a simpler solution how about we add a path trigger for a file called e.g. .benchmark which could be temporarily added to a PR to trigger touchstone? We’ll just have to remember to always remove it before merging.

This is how benchmark results would change (along with a 95% confidence interval in relative change) if a4aad24 is merged into main:

•   :ballot_box_with_check:default: 45.6s -> 48.6s [-11.94%, +24.95%]
• ❗🐌no_delays: 38.9s -> 42s [+1.07%, +14.91%]
•   :ballot_box_with_check:random_walk: 12.6s -> 12.4s [-10.7%, +7.15%]
•   :ballot_box_with_check:stationary: 30.5s -> 32.2s [-4.75%, +16.17%]
•   :ballot_box_with_check:uncertain: 46.8s -> 47.2s [-12.32%, +14.01%]
  Further explanation regarding interpretation and methodology can be found in the documentation.