option to disable line warnings with `enable_coverage_for_eval` and rails views

[dgarwood]

Couldn't find an issue that was directly related to this, please forgive me if I've overlooked something.

Love enable_coverage_for_eval since that allows getting coverage metrics for rails views, which was something that until Ruby 3.2, you had to do without. However, turning it on results in a lot of warnings, relative to this method in lib/simplecov/source_file.rb:

# lib/simplecov/source_file.rb:250-253
    # Warning to identify condition from Issue #56
    def coverage_exceeding_source_warn
      warn "Warning: coverage data provided by Coverage [#{coverage_data['lines'].size}] exceeds number of lines in #{filename} [#{src.size}]"
    end

Reviewing Issue#56, it wasn't clear why this warning occurs, other than we know that we're getting more lines reported for a source file than the code in the file actually has. I have not been able to determine why this is, but suspect it might be due to erb inserting lines in the file.

Below is a copy of the patch I created to allow turning these warnings off with an ENV var. Granted this is a crude fix, but it helps to clean up our test output.

My request is getting the below patch or something like it included in simplecov. Ultimately, these warnings are not helpful in their current form for view specs, since it's not clear why this is an issue or what can be done to resolve it.

$ ruby -e "puts RUBY_DESCRIPTION"
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]

Rails 7.0.4.3
RSpec 3.12.0
SimpleCov 0.22.0

tests are run with bin/rspec spec manually or via guard.

# spec/spec_helper.rb
# frozen_string_literal: true

unless RSpec.configuration.files_to_run.one?
  require 'simplecov'
  require_relative 'support/simplecov_warnings_patch'
  SimpleCov.start 'rails' do
    enable_coverage :branch
    enable_coverage_for_eval
    # minimum_coverage line: 85, branch: 70
    # maximum_coverage_drop 5
  end
end

...
<more spec_helper>

# spec/support/simplecov_warnings_patch.rb
# frozen_string_literal: true

module SimpleCov
  class SourceFile
    # this is needed for SimpleCov Issue#56
    # it also shows up for views when we `enable_coverage_for_eval`
    # patching here so that we can turn this off via ENV VAR
    def coverage_exceeding_source_warn
      return unless ENV['SHOW_EXCESS_LINE_WARNING']

      message = "Warning: coverage data provided by Coverage [#{coverage_data['lines'].size}] "
      message += "exceeds number of lines in #{filename} [#{src.size}]"
      warn message
    end
  end
end

This is an example view spec for devise (v4.8.1) that reproduces the issue (remove the unless wrapper on SimpleCov config to get results from specs for a single file):

# spec/views/devise/registrations/new.html.erb_spec.rb
# frozen_string_literal: true

require 'rails_helper'

RSpec.describe 'devise/registrations/new.html.erb' do
  let(:resource) { build_stubbed :user }
  let(:devise_mapping) { Devise.mappings[:user] }

  before do
    render template: 'devise/registrations/new', \
           locals: { resource:, resource_name: :user, devise_mapping: }
  end

  it 'accepts username' do
    expect(rendered).to have_css '#user_username'
  end

  it 'accepts email' do
    expect(rendered).to have_css '#user_email'
  end
end

Again, main focus here is to have an env var or config option added to simplecov that allows silencing the warnings from the coverage_exceeding_source_warn method.

[dnalbach]

I am experiencing this too and am in favor of the ENV var fix.

[joshuapinter]

I'm also seeing this since enabling enable_coverage_for_eval.

Silencing them could work in the meantime but hopefully a better solution that correctly identifies the # of lines for erb files is possible as well.

[oniram88]

I think that the problem is with only one line,
every erb to ruby compilation add a first line: #coding:UTF-8

require "erb"
example = "hello <%= 'world' %>"
compiler = ERB.new(example)
puts compiler.src

get =>

#coding:UTF-8
_erbout = +''; _erbout.<< "hello ".freeze; _erbout.<<(( 'world' ).to_s); _erbout

this will then be executed from eval, src is 2 lines, coverage 1

I suppose that if you have nested renders in the same view you will have one line per nested template added to the total count of lines that will be evalueted, but i'm not sure.

[awilfox]

I came here looking for a similar option. It's hard to even find the test output in my terminal through all these warnings, which is disappointing because I was super excited to finally have coverage for Rails views (and other forms of eval). Having this coverage data actually led me to fixing two minor bugs in our codebase, so it's definitely valuable to have.

Perhaps there could be an option to disable the warning only in files ending in erb?

[jana4u]

I came here looking for a similar option. It's hard to even find the test output in my terminal through all these warnings, which is disappointing because I was super excited to finally have coverage for Rails views (and other forms of eval). Having this coverage data actually led me to fixing two minor bugs in our codebase, so it's definitely valuable to have.

Perhaps there could be an option to disable the warning only in files ending in erb?

I did it like this:

module SimpleCov
  class SourceFile
    private

    alias_method :coverage_exceeding_source_warn_original, :coverage_exceeding_source_warn

    def coverage_exceeding_source_warn
      unless filename.end_with?(".erb")
        coverage_exceeding_source_warn_original
      end
    end
  end
end

It solves the issue with warnings for .erb files, but I would really prefer something like this to be part of the gem itself. At least as an option.

Without something like this turning on enable_coverage_for_eval in Rails app is really not useful because output is flooded by warnings for .erb files.

[joshuapinter]

🙏 @sferik Thanks for merging in a fix. Any idea when a new release will be made?

[sferik]

@joshuapinter I’m still waiting for @PragTob to re-review #1165, which he won’t be able to do until next week at the earliest. I also need to figure out what I want to do about #1068 for 1.0, which is another large-ish task. I’m hoping we can ship 1.0 sometime in May. If you’re really hankering to use the next version sooner, you can point your Gemfile at this git repo, like so:

gem "simplecov", git: "https://github.com/simplecov-ruby/simplecov.git"

If you decide to do that, it would be very helpful if you reported any bugs that you found.

[joshuapinter]

@sferik Thanks for the quick reply!

Yup, I was hankering so I pointed to the specific ref where it got merged in (least surprises that way):

  gem "simplecov",
    github:  "simplecov-ruby/simplecov",                 # Using a commit that fixes warning messages when using `enable_coverage_for_eval` to monitor coverage of ERB files: "Warning: coverage data provided by Coverage [39] exceeds number of lines"
    ref:     "2729bdfe8d0e89279f3719af5b60cdf1e06f5cf3", # TODO: Replace when new release is made: https://github.com/simplecov-ruby/simplecov/pull/1172
    require: false

We just merged that into main today and so far everything is running well with no warning messages. 👌