Feature #44345
closed
Use SimpleCov's built-in HTML formatter instead of Redmine's own formatter
Description
Redmine uses SimpleCov to measure test coverage. Running bin/rails test:coverage runs the test suite with coverage enabled and generates an HTML report in the coverage directory.
The report is currently generated by Redmine's own formatter, consisting of source:trunk/test/coverage/html_formatter.rb@24890 and two ERB templates (source:trunk/test/coverage/views@24890). I propose replacing it with SimpleCov's built-in HTML formatter.
Before (Redmine's custom formatter):
After (SimpleCov's built-in formatter):
- We no longer have to maintain the custom formatter.
- The report can be filtered by file name and coverage rate, and sorted by any column. The current report is a single flat table.
- The SimpleCov CLI becomes usable.
bundle exec simplecov open,report, anduncoveredreadcoverage/coverage.json, which the custom formatter does not generate. - Files are grouped into tabs (Controllers, Models, Helpers, Libraries, ...) instead of being listed in a single table.
- Branch and method coverage can be displayed if enabled later. The current templates support only line coverage.
The Redmine-specific links in the report header are dropped: the revision used to generate the report and the Continuous_integration wiki page. These were intended for reports published at https://www.redmine.org/builds/, which is no longer in operation.
Files
Updated by Marius BĂLTEANU about 1 month ago
- Status changed from New to Resolved
- Assignee set to Marius BĂLTEANU
- Target version changed from Candidate for next major release to 7.1.0
- Resolution set to Fixed
Nice!
I've committed the changes, thanks!
Updated by Marius BĂLTEANU about 1 month ago
- File 0001-Add-SimpleCov-coverage-to-Github-Actions-test-workfl.patch added
- Status changed from Resolved to Closed
Go MAEDA, what do you think if we run one test suite from the matrix with coverage enabled? Another option is to have a nightly run.
Updated by Marius BĂLTEANU about 1 month ago
- File deleted (
0001-Add-SimpleCov-coverage-to-Github-Actions-test-workfl.patch)
Updated by Marius BĂLTEANU about 1 month ago
- Status changed from Closed to Resolved
Updated by Go MAEDA about 1 month ago
Marius BĂLTEANU wrote in #note-2:
Go MAEDA, what do you think if we run one test suite from the matrix with coverage enabled? Another option is to have a nightly run.
Just to make sure I understand correctly: to view the coverage report, we would need to open the Actions tab, download the artifact as a ZIP file, and extract it locally. Is that right?
If so, I probably wouldn't look at it very often, simply because it takes several steps to get there. That said, I don't object to adding it at all.
Updated by Marius BĂLTEANU about 1 month ago
Because I do not run the whole test suite on my local environment very often, my ultimate goal is to set up a coverage badge displaying the percentage value, as well as a visual breakdown of coverage trends over time.
Based on a quick research, there are two main paths for trend tracking:- SaaS Integration (e.g., Codecov / Coveralls): Hooks into GitHub Actions to store test history automatically and generate both the badge and trend graphs out of the box.
- Self-hosted: Store historical
.last_run.jsonexecution data somewhere and then, render a custom dynamic badge via Shields.io. I'm not sure how to deal with trend history.
Updated by Go MAEDA 19 days ago
- File clipboard-202609101423-xkfqs.png clipboard-202609101423-xkfqs.png added
- File clipboard-202609101446-ykpds.png clipboard-202609101446-ykpds.png added
- File enable-simplecov-cover_views-and-track_tests.patch enable-simplecov-cover_views-and-track_tests.patch added
Now that SimpleCov 1.2 is in trunk (r25065), I propose enabling its cover_views and track_tests options for SimpleCov's HTML formatter.
cover_views:
ERB templates under app/views are now measured and included in the "Views" tab.
track_tests:
The report shows which tests executed each covered line. In the HTML report, every covered line shows a link such as "2 tests"; clicking the link lists the tests that exercised the line.
Updated by Go MAEDA 14 days ago
Marius BĂLTEANU wrote in #note-7:
Because I do not run the whole test suite on my local environment very often, my ultimate goal is to set up a coverage badge displaying the percentage value, as well as a visual breakdown of coverage trends over time.
The scope of this issue was to replace Redmine's custom HTML formatter with SimpleCov's built-in formatter, and that change has already been committed.
Running coverage in CI, tracking coverage over time, and displaying a badge are worth discussing, but I think these topics are outside the scope of this issue and involve several additional decisions. How about closing this issue and opening a separate one to continue that discussion?
Updated by Marius BĂLTEANU 14 days ago
- Status changed from Resolved to Closed
Sure, no problem, I will open a new issue once I have some working code.
Updated by Marius BĂLTEANU 14 days ago
- Assignee changed from Marius BĂLTEANU to Go MAEDA