- Flakiness is defined on a rolling 14-day window.
times_flakedis the sum of all flakes detected for a specific test over the last 14 days, qualified by the job it’s been run in, so if Test A is run in Job 1 and Job 2, those are kept as separate counts
- Is the `times_flaked` counter collected over a specific timespan? e.g: past day, week? Or is it an all time aggregate? For the count of all the tests that have flaked, is this an all time aggregate?To answer these questions: these are both collected over the most recent 14 day timespan; we delete any records that have a workflow
created_attimestamp of older than 14 days
- So long as new workflows are created that flake on the same test, that workflow
created_attimestamp will be updated
If the customer fixes the flaky test, and it doesn’t flake for 14 days, we will no longer flag it as flaky.
If the customer stops running the test, after 14 days, we will no longer flag it as flaky.
Article is closed for comments.