Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ruby 3.0 Support #6878

Closed
headius opened this issue Oct 5, 2021 · 11 comments · Fixed by #6927
Closed

Ruby 3.0 Support #6878

headius opened this issue Oct 5, 2021 · 11 comments · Fixed by #6927
Milestone

Comments

@headius
Copy link
Member

headius commented Oct 5, 2021

Ruby 2.7 Note

This list, as mentioned below, is changed or added features since the Ruby 2.7.0 release. However, since JRuby skipped Ruby 2.7 (JRuby 9.3 was 2.6 compatible and 9.4 is 3.0 compatible), we will also need to audit that list of features and ensure that any changes exclusive to 2.7 are also implemented.

See #6464 for the checklist of Ruby 2.7 features.

Ruby specs for 2.7 and 3.0

Many specs have already been added for Ruby 2.7 and Ruby 3.0 behaviors, but there are always gaps! Please help write a complete set of executable Ruby specifications for these releases. Your efforts will help Rubyists worldwide have a better understanding of Ruby and better tools to use it.

Ruby 2.7: ruby/spec#745

Ruby 3.0: ruby/spec#823

Full list of changes for Ruby 3.0.0

NOTE: https://rubyreferences.github.io/rubychanges/3.0.html gives more details for many features and changes. From https://github.com/ruby/ruby/blob/master/doc/NEWS-3.0.0.md and ruby/spec#823:

This document is a list of user visible feature changes since the 2.7.0 release, except for bug fixes.

Note that each entry is kept to a minimum, see links for details.

Language changes

  • Keyword arguments are now separated from positional arguments.
    Code that resulted in deprecation warnings in Ruby 2.7 will now
    result in ArgumentError or different behavior. [Feature #14183]
  • Procs accepting a single rest argument and keywords are no longer
    subject to autosplatting. This now matches the behavior of Procs
    accepting a single rest argument and no keywords.
    [Feature #16166]
    pr = proc{|*a, **kw| [a, kw]}
    
    pr.call([1])
    # 2.7 => [[1], {}]
    # 3.0 => [[[1]], {}]
    
    pr.call([1, {a: 1}])
    # 2.7 => [[1], {:a=>1}] # and deprecation warning
    # 3.0 => [[[1, {:a=>1}]], {}]
  • Arguments forwarding (...) now supports leading arguments.
    [Feature #16378]
    def method_missing(meth, ...)
      send(:"do_#{meth}", ...)
    end
  • Pattern matching (case/in) is no longer experimental. [Feature #17260]
  • One-line pattern matching is redesigned. [EXPERIMENTAL]
    • => is added. It can be used like a rightward assignment.
      [Feature #17260]
      0 => a
      p a #=> 0
      
      {b: 0, c: 1} => {b:}
      p b #=> 0
    • in is changed to return true or false. [Feature #17371]
      # version 3.0
      0 in 1 #=> false
      
      # version 2.7
      0 in 1 #=> raise NoMatchingPatternError
  • Find-pattern is added. [EXPERIMENTAL]
    [Feature #16828]
    case ["a", 1, "b", "c", 2, "d", "e", "f", 3]
    in [*pre, String => x, String => y, *post]
      p pre  #=> ["a", 1]
      p x    #=> "b"
      p y    #=> "c"
      p post #=> [2, "d", "e", "f", 3]
    end
  • Endless method definition is added. [EXPERIMENTAL]
    [Feature #16746]
    def square(x) = x * x
  • Interpolated String literals are no longer frozen when
    # frozen-string-literal: true is used. [Feature #17104]
  • Magic comment shareable_constant_value added to freeze constants #7457 Magic comment shareable_constant_value added to freeze constants.See {Magic Comments}[rdoc-ref:doc/syntax/comments.rdoc@Magic+Comments] for more details.
    [Feature #17273] (Parses in JRuby but currently ignored due to not supporting Ractor)
  • Deprecation warnings are no longer shown by default (since Ruby 2.7.2).
    Turn them on with -W:deprecated (or with -w to show other warnings too).
    [Feature #16345] (Add support for -W:category flag #7436)
  • $SAFE and $KCODE are now normal global variables with no special behavior.
    C-API methods related to $SAFE have been removed.
    [Feature #16131] [Feature #17136], $KCODE and $-k vars are normal globals now #7369
  • yield in singleton class definitions in methods is now a SyntaxError
    instead of a warning. yield in a class definition outside of a method
    is now a SyntaxError instead of a LocalJumpError. [Feature #15575]
  • Missing error when a class variable is overtaken by an ancestor #7437 When a class variable is overtaken by the same definition in an ancestor class/module, a RuntimeError is now raised (previously, it only issued a warning in verbose mode). Additionally, accessing a class variable from the toplevel scope is now a RuntimeError.
    [Bug #14541]
  • Assigning to a numbered parameter is now a SyntaxError instead of
    a warning.

Command line options

  • --help option
    When the environment variable RUBY_PAGER or PAGER is present and has a non-empty value, and the standard input and output are tty, the --help option shows the help message via the pager designated by the value. [Feature #16754] (Pager support for --help #7458)

  • --backtrace-limit option
    The --backtrace-limit option limits the maximum length of a backtrace. [Feature #8661]

Core classes updates

Outstanding ones only.

Array

  • The following methods now return Array instances instead of
    subclass instances when called on subclass instances:
    [Bug #6087]
    • Array#drop
    • Array#drop_while
    • Array#flatten
    • Array#slice!
    • Array#slice / Array#[]
    • Array#take
    • Array#take_while
    • Array#uniq
    • Array#*
  • Can be sliced with Enumerator::ArithmeticSequence
  dirty_data = ['--', 'data1', '--', 'data2', '--', 'data3']
  dirty_data[(1..).step(2)] # take each second element
["data1", "data2", "data3"]

Binding

  • Binding#eval when called with one argument will use "(eval)"
    for __FILE__ and 1 for __LINE__ in the evaluated code.
    [Bug #4352] [Bug #17419]

ConditionVariable

  • ConditionVariable#wait may now invoke the block/unblock scheduler
    hooks in a non-blocking context. [Feature #16786]

Dir

ENV

  • ENV.except has been added, which returns a hash excluding the
    given keys and their values. [Feature #15822]
  • Windows: Read ENV names and values as UTF-8 encoded Strings
    [Feature #12650]

Encoding

Fiber

See #7438.

  • Fiber.new(blocking: true/false) allows you to create non-blocking
    execution contexts. [Feature #16786]
  • Fiber#blocking? tells whether the fiber is non-blocking. [Feature #16786]
  • Fiber#backtrace and Fiber#backtrace_locations provide per-fiber backtrace.
    [Feature #16815]
  • The limitation of Fiber#transfer is relaxed. [Bug #17221]

GC

  • GC.auto_compact= and GC.auto_compact have been added to control
    when compaction runs. Setting auto_compact= to true will cause
    compaction to occur during major collections. At the moment,
    compaction adds significant overhead to major collections, so please
    test first! [Feature #17176]

Hash

  • Hash#transform_keys and Hash#transform_keys! now accept a hash that maps
    keys to new keys. [Feature #16274]
  • Hash#except has been added, which returns a hash excluding the
    given keys and their values. [Feature #15822]

IO

  • IO#nonblock? now defaults to true. [Feature #16786]
  • IO#wait_readable, IO#wait_writable, IO#read, IO#write and other
    related methods (e.g. IO#puts, IO#gets) may invoke the scheduler hook
    #io_wait(io, events, timeout) in a non-blocking execution context.
    [Feature #16786]

Kernel

  • Kernel#clone when called with the freeze: false keyword will call
    #initialize_clone with the freeze: false keyword.
    [Bug #14266]
  • Kernel#clone when called with the freeze: true keyword will call
    #initialize_clone with the freeze: true keyword, and will
    return a frozen copy even if the receiver is unfrozen.
    [Feature #16175]
  • Kernel#eval when called with two arguments will use "(eval)"
    for __FILE__ and 1 for __LINE__ in the evaluated code.
    [Bug #4352]
  • Kernel#lambda now warns if called without a literal block.
    [Feature #15973]
  • Kernel.sleep invokes the scheduler hook #kernel_sleep(...) in a
    non-blocking execution context. [Feature #16786]

Module

  • Module#include and Module#prepend now affect classes and modules
    that have already included or prepended the receiver, mirroring the
    behavior if the arguments were included in the receiver before
    the other modules and classes included or prepended the receiver.
    [Feature #9573], Also include/prepend into existing include hiers #7377
    class C; end
    module M1; end
    module M2; end
    C.include M1
    M1.include M2
    p C.ancestors #=> [C, M1, M2, Object, Kernel, BasicObject]
  • Module#public, Module#protected, Module#private, Module#public_class_method,
    Module#private_class_method, toplevel "private" and "public" methods
    now accept single array argument with a list of method names. [Feature #17314] (Expand this array logic #6933)
  • Module#attr_accessor, Module#attr_reader, Module#attr_writer and Module#attr
    methods now return an array of defined method names as symbols.
    [Feature #17314]
  • Module#alias_method now returns the defined alias as a symbol.
    [Feature #17314]

Mutex

  • Mutex is now acquired per-Fiber instead of per-Thread. This change
    should be compatible for essentially all usages and avoids blocking when
    using a scheduler. [Feature #16792]

Proc

  • Proc#== and Proc#eql? are now defined and will return true for
    separate Proc instances if the procs were created from the same block.
    [Feature #14267]

Queue / SizedQueue

  • Queue#pop, SizedQueue#push and related methods may now invoke the
    block/unblock scheduler hooks in a non-blocking context.
    [Feature #16786]

Ractor

  • New class added to enable parallel execution. See rdoc-ref:ractor.md for
    more details.

Random

  • Random::DEFAULT now refers to the Random class instead of being a Random instance,
    so it can work with Ractor.
    [Feature #17322]
  • Random::DEFAULT is deprecated since its value is now confusing and it is no longer global,
    use Kernel.rand/Random.rand directly, or create a Random instance with Random.new instead.
    [Feature #17351]

String

  • The following methods now return or yield String instances
    instead of subclass instances when called on subclass instances:
    [Bug #10845]
    • String#*
    • String#capitalize
    • String#center
    • String#chomp
    • String#chop
    • String#delete
    • String#delete_prefix
    • String#delete_suffix
    • String#downcase
    • String#dump
    • String#each_char
    • String#each_grapheme_cluster
    • String#each_line
    • String#gsub
    • String#ljust
    • String#lstrip
    • String#partition
    • String#reverse
    • String#rjust
    • String#rpartition
    • String#rstrip
    • String#scrub
    • String#slice!
    • String#slice / String#[]
    • String#split
    • String#squeeze
    • String#strip
    • String#sub
    • String#succ / String#next
    • String#swapcase
    • String#tr
    • String#tr_s
    • String#upcase

Symbol

  • Symbol#to_proc now returns a lambda Proc. [Feature #16260]
  • Symbol#name has been added, which returns the name of the symbol
    if it is named. The returned string is frozen. [Feature #16150]

Fiber

See #7438.

  • Introduce Fiber.set_scheduler for intercepting blocking operations and
    Fiber.scheduler for accessing the current scheduler. See
    rdoc-ref:fiber.md for more details about what operations are supported and
    how to implement the scheduler hooks. [Feature #16786]
  • Fiber.blocking? tells whether the current execution context is
    blocking. [Feature #16786]
  • Thread#join invokes the scheduler hooks block/unblock in a
    non-blocking execution context. [Feature #16786]

Thread

Warning

  • Warning#warn now supports a category keyword argument.
    [Feature #17122]

Stdlib updates

Set

Socket

Compatibility issues

Excluding feature bug fixes.

  • Regexp literals and all Range objects are frozen. [Feature #8948] [Feature #16377] [Feature #15504]
    /foo/.frozen? #=> true
    (42...).frozen? # => true
  • EXPERIMENTAL: Hash#each consistently yields a 2-element array. [Bug #12706], Always yield two-elt ary from Hash#each to lambda #7373
    • Now { a: 1 }.each(&->(k, v) { }) raises an ArgumentError
      due to lambda's arity check.
  • When writing to STDOUT redirected to a closed pipe, no broken pipe
    error message will be shown now. [Feature #14413]
  • TRUE/FALSE/NIL constants are no longer defined.
  • Integer#zero? overrides Numeric#zero? for optimization. [Misc #16961]
  • Enumerable#grep and Enumerable#grep_v when passed a Regexp and no block no longer modify
    Regexp.last_match. [Bug #17030]
  • Requiring 'open-uri' no longer redefines Kernel#open.
    Call URI.open directly or use URI#open instead. [Misc #15893]
  • SortedSet has been removed for dependency and performance reasons. (no performance impact in JRuby and the gem depends on abandoned rbtree C extension but is a no-op on JRuby)

Stdlib compatibility issues

  • Default gems
    • The following libraries are promoted to default gems from stdlib.
      • English
      • abbrev
      • base64
      • drb
      • debug
      • erb
      • find
      • net-ftp
      • net-http
      • net-imap
      • net-protocol
      • open-uri
      • optparse
      • pp (Only do RubyVM patches if class exists ruby/pp#4)
      • prettyprint
      • resolv-replace
      • resolv
      • rinda
      • set
      • securerandom
      • shellwords
      • tempfile
      • tmpdir
      • time
      • tsort
      • un
      • weakref
    • The following extensions are promoted to default gems from stdlib.
      • digest
      • io-nonblock
      • io-wait
      • nkf
      • pathname
      • syslog
      • win32ole
  • Bundled gems
    • net-telnet has been removed from the Ruby standard library.
    • xmlrpc has been removed from the Ruby standard library.
    • SDBM has been removed from the Ruby standard library. [Bug #8446]
    • WEBrick has been removed from the Ruby standard library.

Implementation improvements

  • New method cache mechanism for Ractor. [Feature #16614]
    • Inline method caches pointed from ISeq can be accessed by multiple Ractors
      in parallel and synchronization is needed even for method caches. However,
      such synchronization can be overhead so introducing new inline method cache
      mechanisms, (1) Disposable inline method cache (2) per-Class method cache
      and (3) new invalidation mechanism. (1) can avoid per-method call
      synchronization because it only uses atomic operations.
      See the ticket for more details.
  • The number of hashes allocated when using a keyword splat in
    a method call has been reduced to a maximum of 1, and passing
    a keyword splat to a method that accepts specific keywords
    does not allocate a hash.
  • super is optimized when the same type of method is called in the previous call
    if it's not refinements or an attr reader or writer.

Miscellaneous changes

  • Methods using ruby2_keywords will no longer keep empty keyword
    splats, those are now removed just as they are for methods not
    using ruby2_keywords.
  • When an exception is caught in the default handler, the error
    message and backtrace are printed in order from the innermost.
    [Feature #8661]
  • Accessing an uninitialized instance variable no longer emits a
    warning in verbose mode. [Feature #17055]inxlusw
@headius headius added this to the JRuby 9.4.0.0 milestone Oct 5, 2021
@headius headius mentioned this issue Oct 11, 2021
28 tasks
@edipofederle
Copy link
Contributor

@headius could you please tick the "--backtrace-limit option" here ? thanks

@headius
Copy link
Member Author

headius commented Oct 29, 2021

@edipofederle Done, and fixed formatting and added a checkbox for --help!

@headius headius linked a pull request Nov 10, 2021 that will close this issue
@headius
Copy link
Member Author

headius commented Nov 12, 2021

GC.auto_compact is added but does nothing (we do not have that level of control over JVM GCs).

@headius
Copy link
Member Author

headius commented Nov 12, 2021

The Kernel#clone changes are partially implemented in this gist, but the Ruby changes here seem to trigger a bug in IR:

https://gist.github.com/headius/32ecf9c7b669e48e34e99958fa8a8d0d

@headius
Copy link
Member Author

headius commented Nov 18, 2021

The replacement of our own patched DRb with the gem version has led to a hang in the test:mri:stdlib suite. This hang seems to implicate our weak map logic as being insufficient for the most recent DRb code, but I have not yet reduced it (DRb is very dense code and RPC is its own kind of fun).

ruby/drb#7

@headius
Copy link
Member Author

headius commented Nov 18, 2021

To clarify... the gem version of DRb attempts to use _id2ref, which is not enabled for this test suite on our CI. I attempted to use our patched logic (weak map) but that version hangs halfway through the DRb suite for reasons unknown. The same logic passed the same DRb test files in earlier versions.

@edipofederle
Copy link
Contributor

@headius mind to tick the " Hash#transform_keys and Hash#transform_keys! now accept a hash that maps
keys to new keys. [Feature #16274]" since is done here ?

Thanks

@headius
Copy link
Member Author

headius commented Dec 20, 2021

@edipofederle Ok!

@headius
Copy link
Member Author

headius commented Jan 27, 2022

The english gem appears to install correctly now.

@headius
Copy link
Member Author

headius commented Sep 22, 2022

Updated based on current master.

@headius
Copy link
Member Author

headius commented Nov 16, 2022

Remaining items are in #7459.

@headius headius closed this as completed Nov 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants