Add delegated type to Active Record #39341

dhh · 2020-05-18T20:48:21Z

Class hierarchies can map to relational database tables in many ways. Active Record, for example, offers purely abstract classes, where the superclass doesn't persist any attributes, and single-table inheritance, where all attributes from all levels of the hierarchy are represented in a single table. Both have their places, but neither are without their drawbacks.

The problem with purely abstract classes is that all concrete subclasses must persist all the shared attributes themselves in their own tables (also known as class-table inheritance). This makes it hard to do queries across the hierarchy. For example, imagine you have the following hierarchy:

Entry < ApplicationRecord
Message < Entry
Comment < Entry

How do you show a feed that has both Message and Comment records, which can be easily paginated? Well, you can't! Messages are backed by a messages table and comments by a comments table. You can't pull from both tables at once and use a consistent OFFSET/LIMIT scheme.

You can get around the pagination problem by using single-table inheritance, but now you're forced into a single mega table with all the attributes from all subclasses. No matter how divergent. If a Message has a subject, but the comment does not, well, now the comment does anyway! So STI works best when there's little divergence between the subclasses and their attributes.

But there's a third way: Delegated types. With this approach, the "superclass" is a concrete class that is represented by its own table, where all the superclass attributes that are shared amongst all the "subclasses" are stored. And then each of the subclasses have their own individual tables for additional attributes that are particular to their implementation. This is similar to what's called multi-table inheritance in Django, but instead of actual inheritance, this approach uses delegation to form the hierarchy and share responsibilities.

Let's look at that entry/message/comment example using delegated types:

# Schema: entries[ id, account_id, creator_id, created_at, updated_at, entryable_type, entryable_id ]
class Entry < ApplicationRecord
  belongs_to :account
  belongs_to :creator
  delegated_type :entryable, types: %w[ Message Comment ]
end

module Entryable
  extend ActiveSupport::Concern

  included do
    has_one :entry, as: :entryable, touch: true
  end
end

# Schema: messages[ id, subject ]
class Message < ApplicationRecord
  include Entryable
  has_rich_text :content
end

# Schema: comments[ id, content ]
class Comment < ApplicationRecord
  include Entryable
end

As you can see, neither Message nor Comment are meant to stand alone. Crucial metadata for both classes resides in the Entry "superclass". But the Entry absolutely can stand alone in terms of querying capacity in particular. You can now easily do things like: Account.entries.order(created_at: :desc).limit(50).

Which is exactly what you want when displaying both comments and messages together. The entry itself can be rendered as its delegated type easily, like so:

# entries/_entry.html.erb
<%= render "entries/entryables/#{entry.entryable_name}", entry: entry %>

# entries/entryables/_message.html.erb
<div class="message">
  Posted on <%= entry.created_at %> by <%= entry.creator.name %>: <%= entry.message.content %>
</div>

# entries/entryables/_comment.html.erb
<div class="comment">
  <%= entry.creator.name %> said: <%= entry.comment.content %>
</div>

Sharing behavior with concerns and controllers

The entry "superclass" also serves as a perfect place to put all that shared logic that applies to both messages and comments, and which acts primarily on the shared attributes. Imagine:

class Entry < ApplicationRecord
  include Eventable, Forwardable, Redeliverable
end

Which allows you to have controllers for things like ForwardsController and RedeliverableController that both act on entries, and thus provide the shared functionality to both messages and comments.

Creating new records

You create a new record that uses delegated typing by creating the delegator and delegatee at the same time, like so:

Entry.create! message: Comment.new(content: "Hello!"), creator: Current.user

If you need more complicated composition, or you need to perform dependent validation, you should build a factory method or class to take care of the complicated needs. This could be as simple as:

class Entry < ApplicationRecord
  def self.create_with_comment(content, creator: Current.user)
    create! message: Comment.new(content: content), creator: creator
  end
end

Adding further delegation

The delegated type shouldn't just answer the question of what the underlying class is called. In fact, that's an anti-pattern most of the time. The reason you're building this hierarchy is to take advantage of polymorphism. So here's a simple example of that:

class Entry < ApplicationRecord
  delegated_type :entryable, types: %w[ Message Comment ]
  delegates :title, to: :entryable
end

class Message < ApplicationRecord
  def title
    subject
  end
end

class Comment < ApplicationRecord
  def title
    content.truncate(20)
  end
end

Now you can list a bunch of entries, call Entry#title, and polymorphism will provide you with the answer.

…3 + HEY

activerecord/lib/active_record/delegated_type.rb

Co-authored-by: Aaron Patterson <aaron.patterson@gmail.com>

terracatta

Two small nits I saw reading through the docs. Super cool!

activerecord/lib/active_record/delegated_type.rb

Co-authored-by: Jason Meller <jason@kolide.co>

victorperez

Really nice! I just leaved some suggestions for your consideration.

activerecord/lib/active_record/delegated_type.rb

tomdalling · 2020-05-19T12:49:53Z

Is the name "delegated type" up for review? I don't see any delegation happening in the code. It looks more like a "subtype", or "secondary type", or something like that.

nwjlyons · 2020-05-19T12:58:54Z

It seems like a subclass/subtype. Django calls it Multi-table inheritance

pixeltrix · 2020-05-19T13:46:30Z

What's the story for creating records? Are you intending to use nested attributes? Also do you start with the Entry or do you create records using the delegated types Comment or Message ?

dhh · 2020-05-19T17:09:11Z

@nwjlyons Yeah, that's similar, at least in terms of the table backing. It's a bit different in the sense that this is forming a hierarchy with composition and delegation rather than inheritance. But would be good to reference it.

@pixeltrix Good point. In both BC3 and HEY, we have factories creating the entry/entryable (or recording/recordable) combos, which need to do a lot more work than just creating the link. But the basic pattern is just Entry.create! entryable: Message.new(subject: "hello!"), creator: Current.user.

dhh · 2020-05-19T17:10:25Z

@tomdalling The type is delegated through the named role. So Entry delegates its type to entryable, which is then answered by Message or Comment. I mean, it's good to recognize that we're doing these gymnastics because of the impedance mismatch. If this was pure OO, we wouldn't bother.

Qqwy · 2020-05-19T17:19:25Z

This is very interesting! From the text as it is currently written, though, it is not entirely clear what the advantage would be of this new technique vs. using plain composition.
Maybe this could be added?

JanDintel · 2020-05-19T17:27:17Z

I like the documentation, but I don't completely understand what the query methods return.

Does Entry.all return the Entry objects from the entries table or Message and Comment objects from the messages and comments tables? Same applies for Account.entries.

Based on reading the code it's the first, but it is a bit hard to understand from the documentation.

activerecord/lib/active_record/delegated_type.rb

mstroming

It's unclear to me what the db tables look like in this case. Can you provide an example of the schema for the Entry, Message and Comment tables?

It seems that polymorphic associations are still utilized? I like that this approach here addresses the drawbacks of STI and Abstract Models, but the lack of referential integrity in polymorphic associations is still concerning.

victorperez · 2020-05-20T18:07:33Z

It's unclear to me what the db tables look like in this case. Can you provide an example of the schema for the Entry, Message and Comment tables?

It seems that polymorphic associations are still utilized? I like that this approach here addresses the drawbacks of STI and Abstract Models, but the lack of referential integrity in polymorphic associations is still concerning.

There is already an Schema note in the documentation:

rails/activerecord/lib/active_record/delegated_type.rb

Line 39 in c810411

    
           #   # Schema: entries[ id, account_id, creator_id, created_at, updated_at, entryable_type, entryable_id ]

rails/activerecord/lib/active_record/delegated_type.rb

Line 54 in c810411

# # Schema: messages[ id, subject ]

rails/activerecord/lib/active_record/delegated_type.rb

Line 60 in c810411

# # Schema: comments[ id, content ]

mstroming · 2020-05-20T18:56:17Z

@victorperez Thanks. I missed that.

@dhh For me, maintaining data integrity in the db is usually more important than utilizing polymorphism here. Is there a way to accomplish the delegated type while allowing the db to enforce referential integrity?

dhh · 2020-05-20T19:08:20Z

@mstroming There is not. This relies on polymorphic associations, and you're not going to get vanilla foreign key constraints on those. But you could probably setup a stored procedure somehow to enforce it, if you really wanted to.

jonathanhefner · 2020-05-20T19:48:13Z

Is the name "delegated type" up for review? I don't see any delegation happening in the code. It looks more like a "subtype", or "secondary type", or something like that.

That was my initial reaction too. I think because we are used to talking about delegating behavior, whereas this is delegating subtyping. Or in other words, delegating the ability to be extended with specialized behavior.

I feel like the following API could help make the gymnastics more clear:

class Entry < ApplicationRecord
  delegates_specialization_to :entryable
end

class Message < ApplicationRecord
  specialization_of :entryable
end

class Comment < ApplicationRecord
  specialization_of :entryable
end

In the above code, specialization_of would include Entryable, and it could handle invoking the latter half of define_delegated_type_methods. I also think it would be pretty slick if delegates_specialization_to dynamically created (or re-opened) the Entryable module.

lazaronixon · 2020-05-21T03:14:43Z

I don't know if I got it right. But how does it differ from the polymorphic association? I always thought that bc3 recordable pattern used it.

tomdalling · 2020-05-21T06:09:50Z

I like "specialization", as in "an Entry has a specialization" and "a Comment is a specialization of an Entry".

DelegatedType doesn't fit any definition of delegation that I'm familiar with. There are no attributes or methods being forwarded, and Entrys don't look/behave like Comments using Ruby magic or anything like that. I think calling this "delegation" will confuse a lot of people.

terracatta · 2020-05-21T17:39:00Z

My suggestion for naming

class Entry < ApplicationRecord
  describes_an :entryable, types: %w[ Message Comment ] 
  # `describes_a` would also be a valid keyword if you wanted to say
  # `describes_a :readable` or something else that doesn't start with a vowel.
end

jonathanhefner · 2020-05-22T00:16:57Z

I think calling this "delegation" will confuse a lot of people.

Reading my comment again, I can see how delegates_specialization_to might also be read like "delegates Entry#specialization to Entryable#specialization", instead of "delegates the ability to specialize to Entryable".

It might be more clear as delegates_specializing_to, but another option could be specializes_via.

dhh · 2020-05-22T00:26:06Z

I think I might inadvertently have shared plans for a bike shed and opened the floor to which color it should be painted. My sincerest apologies 😄

"Entry delegates it's type to the entryable role, of which we currently have Message and Comment". The delegation methods, like entry.message?, is a type predicate that's answered by the entryable (even if we shortcut it for performance by going straight to the polymorphic class column).

I shall expand the example to show how half the point here is to erect a delegation scheme. For example, in HEY, we'd delegate #content to the entryable, so Entry#content ends up delegating to Entryable#content, which is answered by Message using the rich text, and by Comment using plain text (you need to make sure they duck adequately, of course).

Anyway, I appreciate the feedback, but I didn't find any of the suggestions offered more compelling than what we have here. I'll get this finished up this weekend! ✌️

tomdalling · 2020-05-22T06:16:48Z

I'll leave my suggested naming (using "specialization") here: https://gist.github.com/tomdalling/bc585d824466d102e635d5d03502e095

I appreciate that your time is precious @dhh so no need to reply, and that the decision is ultimately yours, of course. I just thought that if there was any time to improve the naming it would be now, before rolling it out to thousands of devs/projects. I don't think of that as bikeshedding, personally.

Co-authored-by: Víctor Pérez <v.rodriguez@volcanic.co.uk>

kaspth · 2020-05-23T22:24:25Z

Obviously the bikeshed should be Ruby red 😄🙌

Woot, nice! 🤘🥳

aamdani · 2020-05-23T23:30:56Z

In the final example around adding further delegation, with title for entryables, Message and Comment, could we add the include Entryable line somewhere in the class definitions for clarity's sake? Unless I've misunderstood the implementation details.

This is in order for the class that's signing up to include this "virtual interface" to be responsible for implementing the delegated :title message, right?

matteeyah · 2020-05-24T01:08:39Z

Since this is currently in master, this'll be included in Rails 6.1, right?

dhh · 2020-05-24T03:30:43Z

I think so. But the value here is really the pattern, less the code. You could copy/paste this code into your ApplicationRecord and use it right now 👍 On Sun, May 24, 2020 at 1:08 AM "github.com" <notifications@github.com> wrote: “ Since this is currently in master, this'll be picked up for Rails 6.1, right?” — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or unsubscribe.

adammiribyan · 2020-05-24T09:20:11Z

activerecord/lib/active_record/delegated_type.rb

+  # You create a new record that uses delegated typing by creating the delegator and delegatee at the same time,
+  # like so:
+  #
+  #   Entry.create! message: Comment.new(content: "Hello!"), creator: Current.user


Entry.create! message: Comment.new… is kinda confusing, did you mean Entry.create! comment: Comment.new…?

Also, does it dynamically imply and set the entryable_id and entryable_type: “Comment”?

Further up in the thread, @dhh mentions the format as being: Entry.create! entryable: Message.new(subject: "hello!"), creator: Current.user which seems to make more sense.

matteeyah · 2020-05-26T01:44:27Z

@dhh Definitely! I'm really glad that you released this design pattern. I was able to extract redundant columns out of a database table. Refactoring this made me sleep easier at night, so thanks for that!

I did a spike to come up with a PoC for introducing this into the codebase of a product that I'm working on (https://github.com/Respondo/respondo/pull/225) by monkey-patching ActiveRecord with delegated types. It's amazing how can a small code change in ActiveRecord facilitate a big change in the domain model.

robertomiranda · 2020-10-17T16:38:15Z

I've published a Backport gem for Rails >= 5

mirror source code: https://github.com/robertomiranda/delegated_type
published here: https://rubygems.org/gems/delegated_type

jhirn · 2020-11-06T13:16:39Z

I've been achieving this in psql using HSTORE, then JSONB, columns and STI for heterogeneous attributes for a very long time.

I'm happy to have official support for this in Rails as I’d always prefer and off-the-shelf over a homegrown solution, but would love to to hear thoughts on multi-table vs JSON to avoid sparsely populated tables. Keep I mind this gem was only built to support my original needs and is nowhere near as polished as the solution above.

The only knee jerk curiosity I have is the efficiency of unioning tables vs querying a single table when retrieving a heterogeneous collection of subtypes but without experimenting a little. The API for this PR is definitely nicer in many regards, but at the physical layer I'm not so sure.

Thanks for this and STI to begin with. Due to JSON columns, I overuse and defend that feature to a point I'm not even sure I should talk about publicly 😇

https://github.com/madeintandem/jsonb_accessor

https://madeintandem.com/blog/2013-3-single-table-inheritance-hstore-lovely-combination/

twada · 2020-11-09T07:22:17Z

@dhh From the perspective of mapping objects and tables, "class-table inheritance" mentioned here is actually a Concrete Table Inheritance, Delegated types newly introduced here looks like a Class Table Inheritance (CTI).

I'm very excited to use this great feature!

adammiribyan · 2020-11-14T19:08:22Z

In case anybody finds this helpful, a few extra pieces that fit nicely with the delegated types concept:

N+1 prevention DSL sugar

module Entryable
  TYPES = %w[ List Spot ]
end

class Entry < ApplicationRecord
  delegated_type :entryable, types: Entryable::TYPES

  scope :with_entryables, -> { includes(:entryable) }
end

Automatically matching entry's url helper based on the entryable_type

Rails.application.routes.draw do
  direct :entry do |model|
    route_for model.entryable_name, model
  end
end

class SpotsController < ApplicationController
  def create
    @entry = Entry.create! entryable: Spot.new(params.require(:spot).permit(:address))
    redirect_to @entry # Redirects to e.g. /spots/47, with 47 being the newly created Entry id.
  end
end

slimdave · 2020-12-06T11:27:24Z

@mstroming There is not. This relies on polymorphic associations, and you're not going to get vanilla foreign key constraints on those. But you could probably setup a stored procedure somehow to enforce it, if you really wanted to.

Coming late to this I'm afraid, but referential integrity cannot be enforced with stored procedures/triggers etc. Those approaches are always subject to race conditions because user sessions can only see other sessions' committed changes.

kurt-mueller-osumc · 2020-12-16T16:53:57Z

In the schema outlined in the original post, I could have an Entry that has an entryable_id that doesn't exist in the table specified by the entryable_type. I could also create Messages and Comments with no backing Entry record.

In order to attempt to make illegal states unrepresentable, having an entry_idcolumn on messages and comments with a foreign key constraint ensures that a Message or Comment cannot exist without a backing Entry. I don't need any stored procedures to enforce that, only database constraints.

mstroming · 2020-12-16T16:57:05Z

In the schema outlined in the original post, I could have an Entry that has an entryable_id that doesn't exist in the table specified by the entryable_type. I could also create Messages and Comments with no backing Entry record.

In order to attempt to make illegal states unrepresentable, having an entry_idcolumn on messages and comments with a foreign key constraint ensures that a Message or Comment cannot exist without a backing Entry. I don't need any stored procedures to enforce that, only database constraints.

The existence of polymorphic associations does not allow the database to enforce referential integrity, however, because no foreign keys can be defined.

kurt-mueller-osumc · 2020-12-16T16:59:48Z

In the schema outlined in the original post, I could have an Entry that has an entryable_id that doesn't exist in the table specified by the entryable_type. I could also create Messages and Comments with no backing Entry record.
In order to attempt to make illegal states unrepresentable, having an entry_idcolumn on messages and comments with a foreign key constraint ensures that a Message or Comment cannot exist without a backing Entry. I don't need any stored procedures to enforce that, only database constraints.

The existence of polymorphic associations does not allow the database to enforce referential integrity, however, because no foreign keys can be defined.

If entry_id existed on messages and comments and pointed to the entries table, I definitely could have a foreign key contraint because it's not a polymorphic association. And a message or comment could not exist without a corresponding entry record.

slimdave · 2020-12-16T17:02:36Z

In the schema outlined in the original post, I could have an Entry that has an entryable_id that doesn't exist in the table specified by the entryable_type. I could also create Messages and Comments with no backing Entry record.
In order to attempt to make illegal states unrepresentable, having an entry_idcolumn on messages and comments with a foreign key constraint ensures that a Message or Comment cannot exist without a backing Entry. I don't need any stored procedures to enforce that, only database constraints.

The existence of polymorphic associations does not allow the database to enforce referential integrity, however, because no foreign keys can be defined.

If entry_id existed on messages and comments and pointed to the entries table, I definitely could have a foreign key contraint because it's not a polymorphic association. And a message or comment could not exist without a corresponding entry record.

True, but it's a fundamentally different relationship in terms of the possible cardinalities of the join in that case.

kurt-mueller-osumc · 2020-12-16T17:10:33Z

True, but it's a fundamentally different relationship in terms of the possible cardinalities of the join in that case.

Good point. In my example, cardinalities would be fundamentally different: an Entry could have_many :messages and have_many :comments. In the original example, a Message could have_many :entries, etc. In either case, there's no way to enforce the cardinalities at the database level (not that I'm aware of).

The dash of sugar used to support the key architectural pattern in BC…

b716da0

…3 + HEY

rails-bot bot added the activerecord label May 18, 2020

tenderlove reviewed May 18, 2020

View reviewed changes

activerecord/lib/active_record/delegated_type.rb Outdated Show resolved Hide resolved

dhh and others added 2 commits May 18, 2020 13:58

Update activerecord/lib/active_record/delegated_type.rb

5b2b12d

Co-authored-by: Aaron Patterson <aaron.patterson@gmail.com>

Update delegated_type.rb

9b58d43

terracatta reviewed May 19, 2020

View reviewed changes

activerecord/lib/active_record/delegated_type.rb Outdated Show resolved Hide resolved

activerecord/lib/active_record/delegated_type.rb Show resolved Hide resolved

Update activerecord/lib/active_record/delegated_type.rb

c810411

Co-authored-by: Jason Meller <jason@kolide.co>

victorperez reviewed May 19, 2020

View reviewed changes

activerecord/lib/active_record/delegated_type.rb Outdated Show resolved Hide resolved

activerecord/lib/active_record/delegated_type.rb Outdated Show resolved Hide resolved

D-D-M reviewed May 20, 2020

View reviewed changes

activerecord/lib/active_record/delegated_type.rb Outdated Show resolved Hide resolved

mstroming reviewed May 20, 2020

View reviewed changes

dhh and others added 3 commits May 23, 2020 09:29

Merge branch 'master' into delegated-type

1dbead5

Update activerecord/lib/active_record/delegated_type.rb

af0ef8e

Co-authored-by: Víctor Pérez <v.rodriguez@volcanic.co.uk>

Update activerecord/lib/active_record/delegated_type.rb

e2811dc

Co-authored-by: Víctor Pérez <v.rodriguez@volcanic.co.uk>

Add changelog entry

aacc858

dhh changed the title ~~The dash of sugar used to support the key architectural pattern in BC3 + HEY~~ Add delegated type to Active Record May 23, 2020

dhh added 2 commits May 23, 2020 15:12

Spelling

fb502cd

Merge branch 'master' into delegated-type

62288e5

dhh merged commit fd8fd4a into master May 23, 2020

dhh deleted the delegated-type branch May 23, 2020 22:14

adammiribyan reviewed May 24, 2020

View reviewed changes

sambostock mentioned this pull request Jun 10, 2020

Consolidate delegated type interpolated strings #39584

Closed

bharget mentioned this pull request Aug 20, 2020

Adds role types accessor for delegated types #40073

Closed

p8 mentioned this pull request Oct 29, 2020

Add post about 6.1.beta1 release rails/weblog#222

Merged

robacarp mentioned this pull request Dec 19, 2020

Delegated models luckyframework/avram#569

Open

blackrat mentioned this pull request Jul 1, 2021

CPDTP-224 Prepare for Profile Declarations switchover DFE-Digital/early-careers-framework#611

Closed

3 tasks

pedrocarmona mentioned this pull request Sep 16, 2021

[idea] Serialize Delegated Type jsonapi-serializer/jsonapi-serializer#203

Open

soartec-lab mentioned this pull request Oct 4, 2022

Add guide for ActiveRecord::DelegatedType [skip ci] #46190

Merged

9 tasks

rioug mentioned this pull request Sep 29, 2023

Fix Rails/HasManyOrHasOneDependent openfoodfoundation/openfoodnetwork#11441

Merged

Add delegated type to Active Record #39341

Add delegated type to Active Record #39341

Conversation

dhh commented May 18, 2020 • edited

Sharing behavior with concerns and controllers

Creating new records

Adding further delegation

terracatta left a comment

Choose a reason for hiding this comment

victorperez left a comment • edited

Choose a reason for hiding this comment

tomdalling commented May 19, 2020

nwjlyons commented May 19, 2020

pixeltrix commented May 19, 2020

dhh commented May 19, 2020

dhh commented May 19, 2020

Qqwy commented May 19, 2020

JanDintel commented May 19, 2020 • edited

mstroming left a comment • edited

Choose a reason for hiding this comment

victorperez commented May 20, 2020

mstroming commented May 20, 2020 • edited

dhh commented May 20, 2020

jonathanhefner commented May 20, 2020

lazaronixon commented May 21, 2020 • edited

tomdalling commented May 21, 2020

terracatta commented May 21, 2020 • edited

jonathanhefner commented May 22, 2020

dhh commented May 22, 2020

tomdalling commented May 22, 2020

kaspth commented May 23, 2020

aamdani commented May 23, 2020

matteeyah commented May 24, 2020 • edited

dhh commented May 24, 2020 via email

adammiribyan May 24, 2020

Choose a reason for hiding this comment

stevenbuccini Jun 24, 2020

Choose a reason for hiding this comment

matteeyah commented May 26, 2020 • edited

robertomiranda commented Oct 17, 2020

jhirn commented Nov 6, 2020 • edited

twada commented Nov 9, 2020

adammiribyan commented Nov 14, 2020 • edited

slimdave commented Dec 6, 2020

kurt-mueller-osumc commented Dec 16, 2020 • edited

mstroming commented Dec 16, 2020

kurt-mueller-osumc commented Dec 16, 2020 • edited

slimdave commented Dec 16, 2020

kurt-mueller-osumc commented Dec 16, 2020

dhh commented May 18, 2020 •

edited

victorperez left a comment •

edited

JanDintel commented May 19, 2020 •

edited

mstroming left a comment •

edited

mstroming commented May 20, 2020 •

edited

lazaronixon commented May 21, 2020 •

edited

terracatta commented May 21, 2020 •

edited

matteeyah commented May 24, 2020 •

edited

matteeyah commented May 26, 2020 •

edited

jhirn commented Nov 6, 2020 •

edited

adammiribyan commented Nov 14, 2020 •

edited

kurt-mueller-osumc commented Dec 16, 2020 •

edited

kurt-mueller-osumc commented Dec 16, 2020 •

edited