dry-rb 1.0: upgrading validations, types and schemas
I’m enthusiastic about dry-rb gems. Actually, I’ve never worked on Ruby projects without a dry-rb gem. However, some people are sceptical, as a lot of core dry-rb gems are still in their 0.x
phase, which leads to a lot of breaking changes and hours of refactoring.
I’m happy to see dry-rb mature: dry-monads entered 1.0 phase in Summer 2018, and now two more libraries hit v1.0 milestones: dry-types and dry-struct; and dry-validation is in its 1.0 RC phase.
I haven’t updated my dry-rb gems for a couple of months, so I’ve missed a lot of breaking changes. Finally, I decided to upgrade the gems and write about the process. I’ll take a swing at automating my upgrade process as much as I can.
Prerequisites
Here’s what my dry-rb gems look like:
$ bundle list | grep dry
* dry-auto_inject (0.4.6)
* dry-configurable (0.7.0)
* dry-container (0.6.0)
* dry-core (0.4.7)
* dry-equalizer (0.2.1)
* dry-events (0.1.0)
* dry-inflector (0.1.2)
* dry-initializer (2.5.0)
* dry-logic (0.4.2)
* dry-matcher (0.7.0)
* dry-monads (1.2.0)
* dry-struct (0.6.0)
* dry-transaction (0.13.0)
* dry-types (0.13.2)
* dry-validation (0.12.1)
I’ve got 15 gems, but I only care about four of them: monads, types, struct and validation. Since monads are up-to-date, I’m only going to talk about types, struct and validation.
In this post, I’ll try to give a step-by-step guide that will simplify the upgrading process. It won’t give a 100% working solutions, but it will probably save you a couple of hours.
Note. I use macOS with GNU sed (gsed
) instead of built-in sed
command. So if you want to follow my instructions, install it via brew install gnu-sed
. Since I’m using fish instead of bash
/ zsh
, some commands might need slight modifications to work.
Note. I wrote this article while upgrading the dry-rb gems on my project. I decided to do it gradually — so you might encounter some redundant steps. If you do, please contact me via email and I’ll upgrade it.
dry-validation to dry-schema
The gem we knew as dry-validation
has evolved from a complex schema validation & coercion into a high-level contract DSL with domain logic.
Meanwhile, it has become so complex they decided to break it down into two gems: dry-validation and dry-schema. The latter provides the old functionality of dry-validation
— the schema validations, coercions, and they fixed all known issues. dry-validation
adds domain rules and validations on top of that.
I don’t want to go around and update everything manually, so I’m going to replace dry-validation
with dry-schema
as much as I can, and manually refactor the rest.
Step 1. Upgrade dry-validation to 0.13
. It’s the last version before the switch, so if your builds pass — you’re good to go. You’ll have to update dry-types to 0.14
too.
Step 2. Replace dry-validation with equivalent dry-schema version (0.1.0) and replace all Dry::Validation
occurrences with Dry::Schema
. Also replace all Dry::Validation.Schema
with Dry::Validation.define
.
$ bundle remove dry-validation && bundle add dry-schema --version 0.1.0`
$ grep -rl 'Dry::Validation' ./**/*.rb | xargs gsed -i 's/Dry::Validation/Dry::Schema/g'
$ grep -rl 'Dry::Schema.Schema' ./**/*.rb | xargs gsed -i 's/Dry::Schema.Schema/Dry::Schema.define/g'
If you’ve used struct extension, don’t forget to search for Dry::Schema.load_extensions
and remove :struct
from the list.
Step 3. Replace .each(&:type?)
predicates with .each(:type?)
. The same goes for maybe
, filled
and value
. You might get ArgumentError: no receiver given
if you don’t.
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\.\(filled\|value\|each\|maybe\)(&/.\1(/g'
Step 4. Refactor schemas that use arrays as input.
The feature has been removed and it’s not coming back until dry-schema 1.0. Here’s an issue with the feature.
The refactoring will look like this:
# Before
ItemSchema = Dry::Schema.Params do
each do
schema do
required(item_id).filled(:int?)
required(option_ids).each(:int?)
end
end
end
ItemSchema.call(input)
# After
ItemSchema = Dry::Schema.Params do
required(:input).each do
schema do
required(:item_id).filled(:int?)
required(:option_ids).each(:int?)
end
end
end
ItemSchema.call(input: input)
Step 5. Check you’ve ever inherited from Dry::Validation
schemas. If you did, do the following transformations:
- Rename classes
Dry::Validation::Schema::Params
→Dry::Schema::Params
Dry::Validation::Schema::JSON
→Dry::Schema::JSON
Dry::Validation::Schema
→Dry::Schema
- Replace
define!
block withdefine
- Move
configure
block underdefine
# Before:
class ApplicationSchema < Dry::Validation::Schema::Params
configure do
config.messages = :i18n
end
end
# After:
class ApplicationSchema < Dry::Schema::Params
define do
config.messages = :i18n
end
end
And update its subclasses:
# Before
class MySchema < ApplicationSchema
configure do
config.messages = :yaml
end
define! do
...
# your params go here
end
end
# After
class MySchema < ApplicationSchema
define do
config.messages = :yaml
...
# your params go here
end
end
Step 6. Update DSL inheritance.
Replace Dry::Schema.Params(BaseClass)
with Dry::Schema.Params(parent: BaseClass)
.
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/Dry::Schema\(\(::\)\|\.\)\(Params\|JSON\|Schema\)(\([[:alnum:]]*\))/Dry::Schema\1\3(parent: \4)/g'
Before you proceed Skip steps 7 and 8 if you’ve never used type specs API.
Step 7. Remove config.type_specs
from your schemas
$ grep -rl 'config.type_specs' ./**/*.rb | xargs gsed -i '/config\.type_specs/d'
Step 8. Remove type spec usages from required
and optional
.
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(required\|optional\)(\(:[[:alnum:]_]*\), [[:print:]]*)\(\.\|$\)/\1(\2)\3/g'
Updating dry-schema to 0.3
Step 9. Update your gemfile to specify gem 'dry-schema', '~> 0.3.0'
and run bundle install
Step 10. If you’re using I18n, move errors
under dry_schema
namespace. This way,
en:
errors:
array?: must be an array
will turn into
en:
dry_schema:
errors:
array?: must be an array
Step 11. Find any schema
macro usages and replace them with hash
, as schema
no longer prepends value(:hash?)
check.
# Before
required(:foo).schema do
end
# After
required(:foo).hash do
end
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/schema \(do\|{\)/hash \1/g'
Step 12. Find any each
macro usages and replace them with array
to add type check. Since Ruby has a Enumerable#each
function, we can’t automate it, but we can still find possible occurrences:
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'each \(do\|{\)'
Feel free to skip if you feel like you don’t need type checks.
Step 13. Load hints extension if you use monads or .messages
.
Dry::Schema.load_extensions(:hints)
The leap towards 1.0.0
Step 14. Update dry-struct, dry-types and dry-schema and run bundle install
.
gem 'dry-schema', '~> 1.1.0'
gem 'dry-struct', '~> 1.0.0'
gem 'dry-types', '~> 1.0.0'
Step 15. Replace Dry::Types.module
with Dry.Types(default: :nominal)
If you’ve never used nominal types (i.e. Types::Hash
, Types::Integer
), feel free to use Dry.Types
instead.
$ gsed -i 's/Dry::Types\.module/Dry.Types(default: :nominal)/g' ./**/*.rb
Step 16. Replace legacy hash schemas with new ones. See https://dry-rb.org/gems/dry-types/0.15/hash-schemas/
Step 17. Update error message config
- Replace
config.messages
withconfig.messages.backend
- Replace
config.messages_file = '/path/to/my/errors.yml'
withconfig.messages.load_paths << '/path/to/my/errors.yml'
- Replace
config.namespace = :user
withconfig.messages.namespace = :user
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.messages =/config.messages.backend =/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.messages_file =/config.messages.load_paths <</g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/config\.namespace =/config.messages.namespace =/g'
Step 18. Symbolize all string keys
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(required\|optional\)(\(\'\|"\)\([[:alnum:]_]*\)\(\'\|"\)/\1(:\3/g'
Step 19. Replace Types.Definition
with Types.Nominal
$ gsed -i 's/Types\.Definition/Types.Nominal/' ./**/*.rb
Step 20. If you rely on Types::Params
and Types::JSON
not to raise an exception on invalid input, decorate the definitions with .lax
$ gsed -i 's/Types::JSON::\([[:alnum:]]*\)/Types::JSON::\1.lax/g' ./**/*.rb
$ gsed -i 's/Types::Params::\([[:alnum:]]*\)/Types::Params::\1.lax/g' ./**/*.rb
Step 21. Replace :type?
predicates with type checks wherever you need this
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:str?/\1(:string/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:int?/\1(:integer/g'
$ grep -rl 'Schema' ./**/*.rb | xargs gsed -i 's/\(filled\|maybe\|value\)(:date?/\1(:date/g'
Step 22. Result#{messages, errors, hints}
now return MessageSet
, which can be converted to Hash
. So we need to go and update the usages everywhere. Also Result#to_monad
now wraps entire Result
object, so we have to update our code.
# Before
render errors: Schema.call(params).errors
render errors: Schema.call(params).to_monad.failure
# After
render errors: Schema.call(params).errors.to_h
render errors: Schema.call(params).to_monad.failure.errors.to_h
I’ve used the scripts to help me look and trace those values:
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.messages'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.errors'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.to_monad'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.failure'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.value_or'
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n '\.value!'
Refactoring to dry-validation
The steps above should be good enough to update most of the features, but if you ‘ve ever used high-level rules, validation blocks, you have two options: either remove those features from your schemas, or use dry-validation 1.0. I decided to refactor most of my schemas, that’s what came out of it.
There are things to keep in mind during the update:
dry-validation
is a library to validate domain logic and rules. The core concept is aContract
.- All contracts must be instantiated — no more
Schema.call
. We need to useContract.new.call
now - The idiomatic way to define a contract is to use standard Ruby syntax:
class Contract < Dry::Validation::Contract
as opposed to dry-schema’sDry::Schema.Params { }
Step 23. Update dependency injection. The new version uses dry-initializer under the hood, so it works like this:
- use
option
for keyword arguments - use
param
for positional arguments
You’ll have to pass the arguments when you instantiate the method
# Before
Schema = Dry::Validation.Schema do
configure do
option :repo
end
...
end
Schema.with(repo: my_repo).call(params)
# After
class Contract < Dry::Validation::Contract
option :repo
end
contract = Contract.new(repo: repo)
contract.call(params)
I used the script to find the files I need to refactor:
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'option :[[:alnum:]_]*$'
Step 24. Rewrite rules and validations. I can’t provide a comprehensive migration guide because I’ve just refactored everything and tried to make my specs pass without giving it much thought.
# Before
class CreditCardSchema < Dry::Validation::Schema::Params
configure do
config.type_specs = true
end
define! do
required(:number, :string).filled(format?: /\A\d{13,19}\z/)
required(:month, :string).filled(format?: /\A(0?[1-9]|1[012])\z/)
validate(expired: %i[year month]) do |year, month|
Date.new("20#{year}".to_i, month.to_i).end_of_month >= Date.current
end
end
end
# After
class CreditCardSchema < Dry::Validation::Contract
params do
required(:month).filled(:string, format?: /\A(0?[1-9]|1[012])\z/)
required(:year).filled(:string, format?: /\A\d{2}\z/)
end
rule(:year, :month) do
year = values[:year]
month = values[:month]
if Date.new("20#{year}".to_i, month.to_i).end_of_month < Date.now
key(:expired).failure(:expired)
# ^ a little duplication here to produce the expected error message
# without refactoring anything else
end
end
end
I used this script to search for all schemas that need rewriting:
$ grep -rl 'Schema' ./**/*.rb | xargs grep -n 'rule\|validate('
Step 25 (optional). If you’re using Reform, you’re in for a disappointment, especially if you’ve been using its dry-validation
DSL.
We have Reform 2.2.4 with ActiveModel validations, so we forked it and removed all the dry-validation stuff. Feel free to fork and use!
Step 26. Fix the rest of failing specs. All done!
Recap
The upgrade process took me about 3 work days of refactoring, and I was glad I learned basic sed
to help me — it’s annoying to do so much manual work.
However, I think the improvements are worth it. The ones I like the most:
dry-types
is stricter and less verbose now — if you’re not including nominal types, thenTypes::String
is the same asTypes::Strict::String
- The known dry-validation bugs were fixed
- Decreased complexity of schema validations
- New library to design domain validations and contracts
I urge you to try the new dry-rb gems — and write about your experience. If you’ve upgraded your gems and wrote a post about your journey and update process — please send me an email and I’ll add a link to your page. And of course, it would be great to see new contributions to official docs.
References
- Introducing dry-schema @ solnic.codes
- How it all started
- dry-schema
- dry-validation
- dry-types
Update (01.06.2019). flash-gordon pointed out that you don’t need to wrap config into the configure
block. So I’ve replaced
define do
configure do
config.xxx = yyy
end
end
with a less nested version:
define do
config.xxx = yyy
end
Update (07.06.2019). solnic pointed out that I made a typo in Step 10: it used to say dry_struct
instead of dry_schema
. I’ve updated the step accordingly.