Student gets no feedback on number of specs passed but others do

[mtc2013]

[zhangaaron]

Can you clarify what the issue here is? The number of specs here passed is 13/13, number of cucumber scenarios passed is 3/3.

[tansaku]

@zhangaaron: @rosewcs345 is saying some other students (in her SUNY Binghamton class) don't get this information. (@mtc2013 filed this on her behalf)

Ideally we need a screen shot of the other kind of output they are seeing in order to understand this better ...

[mtc2013]

@tansaku could you post the picture of the results returned from the grader when submitting this student's hw through @rosewcs345 spoc? I don't have access.

[tansaku]

I think this is it:

[rosewcs345]

rspec passes locally, but not accepted by autograder.
Students with this issue used FactoryGirl

[tansaku]

so I am essentially replicating this in #93 - I get exactly the same message in the local acceptance test as I do in the grader:

Submission message after grade: Your submission was recorded at 2010-06-19 18:52:28 +0100 : submission is on time.
Score: 40.0/100.0
----------------------------------------
Running student tests found in features/ spec/:
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -S bundle exec cucumber  --profile default
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -I/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/lib:/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-support-3.3.0/lib /Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb failed

----------------------------------------

----------------------------------------
Checking coverage for:
  controllers >= 90.00%
  models >= 90.00%
----------------------------------------
  all files: 84.21%% coverage
  controllers: 80.43%% coverage
  models: 100.00%% coverage
  mailers: 100.00%% coverage
  helpers: 100.00%% coverage
  libraries: 100.00%% coverage

Failed coverage test (controllers coverage too low).
  Score: 0/20.0
----------------------------------------

----------------------------------------
Running reference Cucumber scenarios:
  3 out of 3 scenarios passed
  Score: 40/40
----------------------------------------
Total score: 40.0 / 100.0
Completed in 19.305187 seconds.

[tansaku]

and if I open the zipped file from the student all the specs pass and the coverage is good:

[tansaku@Samuels-MBP:~/Documents/GitHub/saasbook/rag/features/support/hw4 (92_factorygirl_in_hw4)]$ 
→ be rspec
...................

Finished in 0.20686 seconds (files took 2.21 seconds to load)
19 examples, 0 failures

Coverage report generated for RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/features/support/hw4/coverage. 0.0 / 0.0 LOC (100.0%) covered.

[tansaku]

however if I go to the submission folder where the grader has unpackaged the student code (and done various things with it) then we get the following:

[tansaku@Samuels-MBP:~/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42 (92_factorygirl_in_hw4)]$ 
→ be rspec
...................

Finished in 0.20125 seconds (files took 1.51 seconds to load)
19 examples, 0 failures

Coverage report generated for Cucumber Features, RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42/coverage. 57 / 57 LOC (100.0%) covered.

which is very similar ...

[tansaku]

now the grader doesn't run bundle exec rspec, it runs a special rake task like so ....

[tansaku@Samuels-MBP:~/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42 (92_factorygirl_in_hw4)]$ 
→ bundle exec rake saas:run_student_tests
----BEGIN CUCUMBER----
--------------------------------------------------------------------------------
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -S bundle exec cucumber  --profile default
Using the default profile...
Feature: search for movies by director

  As a movie buff
  So that I can find movies with my favorite director
  I want to include and serach on director information in movies I enter

  Background: movies in database      # features/add_director.feature:7
    Given the following movies exist: # features/step_definitions/director_steps.rb:1
      | title        | rating | director     | release_date |
      | Star Wars    | PG     | George Lucas | 1977-05-25   |
      | Blade Runner | PG     | Ridley Scott | 1982-06-25   |
      | Alien        | R      |              | 1979-05-25   |
      | THX-1138     | R      | George Lucas | 1971-03-11   |

  Scenario: add director to existing movie                # features/add_director.feature:16
    When I go to the edit page for "Alien"                # features/step_definitions/web_steps.rb:48
    And I fill in "Director" with "Ridley Scott"          # features/step_definitions/web_steps.rb:60
    And I press "Update Movie Info"                       # features/step_definitions/web_steps.rb:52
DEPRECATION: Using `should` from rspec-expectations' old `:should` syntax without explicitly enabling the syntax is deprecated. Use the new `:expect` syntax or explicitly enable `:should` with `config.expect_with(:rspec) { |c| c.syntax = :should }` instead. Called from /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42/features/step_definitions/web_steps.rb:107:in `block in <top (required)>'.
    Then the director of "Alien" should be "Ridley Scott" # features/step_definitions/director_steps.rb:7

  Scenario: find movie with same director                       # features/add_director.feature:22
    Given I am on the details page for "Star Wars"              # features/step_definitions/web_steps.rb:44
    When I follow "Find Movies With Same Director"              # features/step_definitions/web_steps.rb:56
    Then I should be on the Similar Movies page for "Star Wars" # features/step_definitions/web_steps.rb:230
    And I should see "THX-1138"                                 # features/step_definitions/web_steps.rb:105
    But I should not see "Blade Runner"                         # features/step_definitions/web_steps.rb:123

  Scenario: can't find similar movies if we don't know director (sad path) # features/add_director.feature:29
    Given I am on the details page for "Alien"                             # features/step_definitions/web_steps.rb:44
    Then I should not see "Ridley Scott"                                   # features/step_definitions/web_steps.rb:123
    When I follow "Find Movies With Same Director"                         # features/step_definitions/web_steps.rb:56
    Then I should be on the home page                                      # features/step_definitions/web_steps.rb:230
    And I should see "'Alien' has no director info"                        # features/step_definitions/web_steps.rb:105

3 scenarios (3 passed)
17 steps (17 passed)
0m0.414s
Coverage report generated for Cucumber Features, RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42/coverage. 57 / 57 LOC (100.0%) covered.
--------------------------------------------------------------------------------
----END CUCUMBER----
----BEGIN RSPEC----
--------------------------------------------------------------------------------
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -I/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/lib:/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-support-3.3.0/lib /Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
...................

Finished in 0.19556 seconds (files took 1.46 seconds to load)
19 examples, 0 failures

Coverage report generated for Cucumber Features, RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-10-42/coverage. 57 / 57 LOC (100.0%) covered.
--------------------------------------------------------------------------------
----END RSPEC----

but running that in the submission directory above also generates a 100% test coverage ...

[tansaku]

so stepping through the HW4 grader in the debugger is not helping me very much, but if I look in the coverage directory in the submission directory I can see that a <100% coverage is being reported there:

[tansaku]

showing gaps in the movies controller like so:

[tansaku]

and if I run bundle exec rake saas:run_student_tests again in that submission folder then the coverage updates to 100%, indicating that somehow on the first run of that task, that perhaps not everything is in place ...?

[tansaku]

so I hooked the grader with the debugger to stop just before it starts parsing the test coverage files. The coverage files then show another different story

Is there some race condition here where the parsing of the coverage files starts before they have finished being generated? That doesn't make sense ...

[tansaku]

so with the grader hooked on the debugger I ran the rake task and got the failure in the terminal:

Coverage report generated for Cucumber Features, RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-39-08/coverage. 48 / 57 LOC (84.21%) covered.
--------------------------------------------------------------------------------
----END CUCUMBER----
----BEGIN RSPEC----
--------------------------------------------------------------------------------
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -I/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/lib:/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-support-3.3.0/lib /Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
....FF.............

Failures:

  1) MoviesController GET #index should sort movies by title
     Failure/Error: expect(assigns(:movies)).to match_array(Movie.order(:title))
       expected collection contained:  [#<Movie id: 11, title: "Star Wars", rating: "PG", description: "Star Wars is a movie about...", release_date: "1977-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 12, title: "Blade Runner", rating: "PG", description: "Blade Runner is a movie about...", release_date: "1982-06-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 13, title: "Alien", rating: "R", description: ".........", release_date: "1979-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 14, title: "THX-1138", rating: "R", description: "THX is a movie about", release_date: "1971-03-11 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 15, title: "Superman", rating: "PG", description: "Superman is a movie about", release_date: "1973-03-21 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "">]
       actual collection contained:    [#<Movie id: 1, title: "Aladdin", rating: "G", description: nil, release_date: "1992-11-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 2, title: "The Terminator", rating: "R", description: nil, release_date: "1984-10-26 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 3, title: "When Harry Met Sally", rating: "R", description: nil, release_date: "1989-07-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 4, title: "The Help", rating: "PG-13", description: nil, release_date: "2011-08-10 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 5, title: "Chocolat", rating: "PG-13", description: nil, release_date: "2001-01-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 6, title: "Amelie", rating: "R", description: nil, release_date: "2001-04-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 7, title: "2001: A Space Odyssey", rating: "G", description: nil, release_date: "1968-04-06 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 8, title: "The Incredibles", rating: "PG", description: nil, release_date: "2004-11-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 9, title: "Raiders of the Lost Ark", rating: "PG", description: nil, release_date: "1981-06-12 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 10, title: "Chicken Run", rating: "G", description: nil, release_date: "2000-06-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 11, title: "Star Wars", rating: "PG", description: "Star Wars is a movie about...", release_date: "1977-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 12, title: "Blade Runner", rating: "PG", description: "Blade Runner is a movie about...", release_date: "1982-06-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 13, title: "Alien", rating: "R", description: ".........", release_date: "1979-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 14, title: "THX-1138", rating: "R", description: "THX is a movie about", release_date: "1971-03-11 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 15, title: "Superman", rating: "PG", description: "Superman is a movie about", release_date: "1973-03-21 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "">]
       the extra elements were:        [#<Movie id: 1, title: "Aladdin", rating: "G", description: nil, release_date: "1992-11-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 2, title: "The Terminator", rating: "R", description: nil, release_date: "1984-10-26 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 3, title: "When Harry Met Sally", rating: "R", description: nil, release_date: "1989-07-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 4, title: "The Help", rating: "PG-13", description: nil, release_date: "2011-08-10 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 5, title: "Chocolat", rating: "PG-13", description: nil, release_date: "2001-01-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 6, title: "Amelie", rating: "R", description: nil, release_date: "2001-04-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 7, title: "2001: A Space Odyssey", rating: "G", description: nil, release_date: "1968-04-06 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 8, title: "The Incredibles", rating: "PG", description: nil, release_date: "2004-11-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 9, title: "Raiders of the Lost Ark", rating: "PG", description: nil, release_date: "1981-06-12 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 10, title: "Chicken Run", rating: "G", description: nil, release_date: "2000-06-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>]
     # ./spec/controllers/movies_controller_spec.rb:108:in `block (3 levels) in <top (required)>'

  2) MoviesController GET #index should sort movies by rating
     Failure/Error: expect(assigns(:movies)).to match_array(Movie.order(:release_date))
       expected collection contained:  [#<Movie id: 11, title: "Star Wars", rating: "PG", description: "Star Wars is a movie about...", release_date: "1977-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 12, title: "Blade Runner", rating: "PG", description: "Blade Runner is a movie about...", release_date: "1982-06-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 13, title: "Alien", rating: "R", description: ".........", release_date: "1979-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 14, title: "THX-1138", rating: "R", description: "THX is a movie about", release_date: "1971-03-11 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 15, title: "Superman", rating: "PG", description: "Superman is a movie about", release_date: "1973-03-21 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "">]
       actual collection contained:    [#<Movie id: 1, title: "Aladdin", rating: "G", description: nil, release_date: "1992-11-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 2, title: "The Terminator", rating: "R", description: nil, release_date: "1984-10-26 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 3, title: "When Harry Met Sally", rating: "R", description: nil, release_date: "1989-07-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 4, title: "The Help", rating: "PG-13", description: nil, release_date: "2011-08-10 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 5, title: "Chocolat", rating: "PG-13", description: nil, release_date: "2001-01-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 6, title: "Amelie", rating: "R", description: nil, release_date: "2001-04-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 7, title: "2001: A Space Odyssey", rating: "G", description: nil, release_date: "1968-04-06 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 8, title: "The Incredibles", rating: "PG", description: nil, release_date: "2004-11-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 9, title: "Raiders of the Lost Ark", rating: "PG", description: nil, release_date: "1981-06-12 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 10, title: "Chicken Run", rating: "G", description: nil, release_date: "2000-06-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 11, title: "Star Wars", rating: "PG", description: "Star Wars is a movie about...", release_date: "1977-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 12, title: "Blade Runner", rating: "PG", description: "Blade Runner is a movie about...", release_date: "1982-06-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 13, title: "Alien", rating: "R", description: ".........", release_date: "1979-05-25 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "Ridley Scott">, #<Movie id: 14, title: "THX-1138", rating: "R", description: "THX is a movie about", release_date: "1971-03-11 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "George Lucas">, #<Movie id: 15, title: "Superman", rating: "PG", description: "Superman is a movie about", release_date: "1973-03-21 00:00:00", created_at: "2016-03-24 12:46:32", updated_at: "2016-03-24 12:46:32", director: "">]
       the extra elements were:        [#<Movie id: 1, title: "Aladdin", rating: "G", description: nil, release_date: "1992-11-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 2, title: "The Terminator", rating: "R", description: nil, release_date: "1984-10-26 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 3, title: "When Harry Met Sally", rating: "R", description: nil, release_date: "1989-07-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 4, title: "The Help", rating: "PG-13", description: nil, release_date: "2011-08-10 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 5, title: "Chocolat", rating: "PG-13", description: nil, release_date: "2001-01-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 6, title: "Amelie", rating: "R", description: nil, release_date: "2001-04-25 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 7, title: "2001: A Space Odyssey", rating: "G", description: nil, release_date: "1968-04-06 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 8, title: "The Incredibles", rating: "PG", description: nil, release_date: "2004-11-05 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 9, title: "Raiders of the Lost Ark", rating: "PG", description: nil, release_date: "1981-06-12 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>, #<Movie id: 10, title: "Chicken Run", rating: "G", description: nil, release_date: "2000-06-21 00:00:00", created_at: "2016-03-24 12:39:11", updated_at: "2016-03-24 12:39:11", director: nil>]
     # ./spec/controllers/movies_controller_spec.rb:115:in `block (3 levels) in <top (required)>'

Finished in 0.21762 seconds (files took 1.59 seconds to load)
19 examples, 2 failures

Failed examples:

rspec ./spec/controllers/movies_controller_spec.rb:104 # MoviesController GET #index should sort movies by title
rspec ./spec/controllers/movies_controller_spec.rb:112 # MoviesController GET #index should sort movies by rating

Coverage report generated for Cucumber Features, RSpec to /Users/tansaku/Documents/GitHub/saasbook/rag/submissions/506af89a6181960fc69f47f1fbc8d708/assignment4/2016-03-24-12-39-08/coverage. 57 / 57 LOC (100.0%) covered.
/Users/tansaku/.rvm/rubies/ruby-2.2.2/bin/ruby -I/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/lib:/Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-support-3.3.0/lib /Users/tansaku/.rvm/gems/ruby-2.2.2/gems/rspec-core-3.3.2/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb failed

[tansaku]

what I would like to do now is leave the grader to finish running, and then compare what, if any, files are changed over the completion of the grader activity - I would do that with git, but the submissions directory is nested inside the rag git space ...