New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test TruffleRuby in CI #2797
Test TruffleRuby in CI #2797
Conversation
Is there an easy way to disable AppVeyor from from building this branch? |
@eregon Not sure but maybe adding |
573d4c1
to
531d882
Compare
I rebased this and it should now use the latest 19.1.0 TruffleRuby release, but TravisCI is down, oh well. |
@eregon it looks like your change to TruffleRuby last month addresses a significant number of the errors in the last Travis CI run. It also appears that 19.1.1 was released after that commit. So — assuming your changes are in TruffleRuby 19.1.1 — I think if you rebase this, it may have fewer failures. |
@duckinator 19.1.1 only contains security fixes and since there was none for TruffleRuby, it's exactly the same as the 19.1.0 TruffleRuby standalone. So, we have to wait the 19.2.0 release to get that fix. I don't think that fix alone will be enough. Please let me work alone on this for now, since I know most about TruffleRuby and I think right now there is little the RubyGems contributors can do to help. |
What we really need to make better progress on this is nightly builds (oracle/truffleruby#1483). I'm trying to get those, but it takes a while to put all the pieces together. It's rather frustrating that RubyGems tests mostly just work when we run a good part of them in TruffleRuby's CI, but for some reason it's a lot harder to make them run in TravisCI. |
@eregon understood. 👍 |
Making some progress, the tests were somehow expecting |
2864: Use the standard RUBY_ENGINE_VERSION instead of JRUBY_VERSION r=bronzdoc a=eregon * RUBY_ENGINE and RUBY_ENGINE_VERSION are defined on every modern Ruby. * There is no such constant as TRUFFLERUBY_VERSION or RBX_VERSION. This blocks #2797. Also see rubygems/bundler#7275 I will abide by the [code of conduct](https://github.com/rubygems/rubygems/blob/master/CODE_OF_CONDUCT.md). Co-authored-by: Benoit Daloze <eregontp@gmail.com>
7275: Use the standard RUBY_ENGINE_VERSION instead of JRUBY_VERSION r=deivid-rodriguez a=eregon * RUBY_ENGINE and RUBY_ENGINE_VERSION are defined on every modern Ruby. * There is no such constant as TRUFFLERUBY_VERSION or RBX_VERSION. This blocks rubygems/rubygems#2797. Also see rubygems/rubygems#2864 Co-authored-by: Benoit Daloze <eregontp@gmail.com>
2864: Use the standard RUBY_ENGINE_VERSION instead of JRUBY_VERSION r=bronzdoc a=eregon * RUBY_ENGINE and RUBY_ENGINE_VERSION are defined on every modern Ruby. * There is no such constant as TRUFFLERUBY_VERSION or RBX_VERSION. This blocks #2797. Also see rubygems/bundler#7275 I will abide by the [code of conduct](https://github.com/rubygems/rubygems/blob/master/CODE_OF_CONDUCT.md). Co-authored-by: Benoit Daloze <eregontp@gmail.com>
2864: Use the standard RUBY_ENGINE_VERSION instead of JRUBY_VERSION r=bronzdoc a=eregon * RUBY_ENGINE and RUBY_ENGINE_VERSION are defined on every modern Ruby. * There is no such constant as TRUFFLERUBY_VERSION or RBX_VERSION. This blocks #2797. Also see rubygems/bundler#7275 I will abide by the [code of conduct](https://github.com/rubygems/rubygems/blob/master/CODE_OF_CONDUCT.md). Co-authored-by: Benoit Daloze <eregontp@gmail.com>
9e1cf86
to
f6e5e6b
Compare
@eregon Can you update with the current master branch? We moved to GitHub Actions instead of Travis CI. See https://github.com/rubygems/rubygems/blob/master/.github/workflows/ubuntu-rvm.yml#L16 |
dbc5bae
to
a2bce0a
Compare
Making progress here, by running it locally and fixing the missing
|
f7bf725
to
89a0b9b
Compare
7275: Use the standard RUBY_ENGINE_VERSION instead of JRUBY_VERSION r=deivid-rodriguez a=eregon * RUBY_ENGINE and RUBY_ENGINE_VERSION are defined on every modern Ruby. * There is no such constant as TRUFFLERUBY_VERSION or RBX_VERSION. This blocks #2797. Also see #2864 Co-authored-by: Benoit Daloze <eregontp@gmail.com>
52f44e9
to
ce3c13b
Compare
Latest run in CI:
Just 2 failures left. |
Thanks so much for your work on this @eregon 👍. |
cb9e93d
to
90fb194
Compare
Thanks, it's in large part due to @bjfish too :) It's now down to 2 failures, |
… tests * Clearly require the benchmark stdlib instead of far away in test_case.rb
* When testing in the rubygems/rubygems repository, the previous code would move the lib/ dir at the end of $LOAD_PATH, which would cause to load a mix of lib/ RubyGems and in-stdlib-dir RubyGems, which blows up.
* But not enforce that error looks like /full/path/to/ruby/bin/ruby: No such file or directory but instead allow truffleruby: No such file or directory A typical output for gem_make.out looks like: current directory: /.../rubygems/tmp/test_rubygems_112388/gemhome/gems/a-2 /.../ruby-2.6.6/bin/ruby -I /.../rubygems/lib -r ./siteconf20200422-112388-nyrcy0.rb extconf.rb '' /.../ruby-2.6.6/bin/ruby: No such file or directory -- extconf.rb (LoadError)
* Test that the correct version is loaded and that the default gem is not loaded at all.
* The --testing-rubygems flag is needed as otherwise TruffleRuby shows extra output when detecting it's using gem homes of other Rubies.
This is now passing, TruffleRuby now passes all RubyGems tests 🎉 🎉 🎉 This is thanks to @bjfish and my work which fixed many things in TruffleRuby and also spent a fair amount of time understanding what some of the tests try to do and why they failed, which took a long time (>1 year since #2700, working on this a little bit every while) to get everything passing. https://github.com/rubygems/rubygems/runs/658849554?check_suite_focus=true
And that's with zero test excluded! There are a few improvements to existing tests to be more reliable in this PR, as well as an added test for upgraded default gems as the existing test was not testing much. Unfortunately these tests take a long time on TruffleRuby, I would think mostly because they use a lot of subprocesses and load quite a bit of code (+ of course tests are typically worst-case for a JIT, running a lot of different code just once). We're generally working on improving loading code speed in TruffleRuby so this should improve over time. @deivid-rodriguez @hsbt Could you review and merge? |
* JRuby doesn't support multi-line -e.
* This is useful by showing the time for each command.
@@ -12,16 +12,18 @@ jobs: | |||
runs-on: ubuntu-latest | |||
strategy: | |||
matrix: | |||
ruby: [ 2.3.8, 2.4.10, 2.5.8, 2.6.6, 2.7.1, jruby-9.2.11.1 ] | |||
ruby: [ 2.3.8, 2.4.10, 2.5.8, 2.6.6, 2.7.1, jruby-9.2.11.1, truffleruby-head ] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we test with the stable version of truffleruby?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The latest release doesn't pass the tests, so it's not an option.
I'll make sure to run RubyGems tests in TruffleRuby's CI too (already the case, but some tests are excluded and no longer need to be).
TruffleRuby's master
(and therefore truffleruby-head
) is only updated if the entire TruffleRuby CI passes, so it should not cause failures.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. We can add the stable version of TruffleRuby after releasing from the current master version.
Description:
We'd like to test TruffleRuby in RubyGems' CI, but it's not working yet.
This is related to #2700, but with additional fixes to run tests.
cc @hsbt
I will abide by the code of conduct.