Excluding columns from inferred schema

Issues
1

Hi all,

I’m working on some migrations to remove columns from a table - and I’d like to first exclude the columns from the returned datasets (like the ignored_columns method in ActiveRecord, as recommended by strong_migrations). This is so I can deploy a change to have the column ignored, then later actually remove the column from the database (via migrations) without impacting the running code.

I see that a schema object has an exclude method - but I’m not quite sure of the best place to use it. I was hoping it’d be available in the schema block in a relation, along the lines of:

schema(infer: true) do
  associations do
    # ...
  end

  exclude :useless_column
end

But that’s not the case. Is something like this possible in some other manner? (I really don’t want to remove the infer option :sweat:).

1 Like
2

You can define a custom dataset and exclude it there, it’s gonna be a bit raw but should work. See here ROM - Relations

3

So something like this should do the trick? (I agree it’s not ideal, but better than specifying all attributes within the schema)

dataset do
  select(:list, :all, :other, :columns)
end
4

@pat you can do exclude there too

5

Unfortunately exclude within the dataset block is for a negated where.

As an alternative, I tried the following:

dataset do
  select(*(columns - %I[ignored_column_a ignored_column_b]))
end

… but as best as I can tell, this doesn’t get used when the schema infers attributes - so the result is the attributes still exist, but then aren’t populated when querying the database, which understandably raises an exception.

I’ve also looked into creating a schema plugin for this purpose, but plugins are invoked before the inferrer, so there’s no attributes to remove.

My next thought was to change the attributes inferrer on the schema, which… well, this works, but it’s far from elegant, and it hooks into the filter_columns method which is tagged as private:

class ExcludingInferrer < ROM::SQL::Schema::AttributesInferrer
  option :excluded

  def filter_columns(schema)
    super.reject { |name, _definition| excluded.include?(name) }
  end
end

schema_inferrer schema_inferrer.with(
  attributes_inferrer: -> (schema, gateway, options) do
    builder = ROM::SQL::Schema::TypeBuilder[gateway.database_type]
    inferrer = ExcludingInferrer.new(type_builder: builder, excluded: %I[ignored_column_a ignored_column_b], **options)
    inferrer.(schema, gateway)
  end
)

I don’t suppose there’s something I’ve missed about tweaking a schema just before it’s finalised? I’m not against submitting a PR, I’m just not sure what a preferred interface around something like this would be.

6

Okay, have found a neater way with a relation plugin:

class ExcludeAttributes
  def self.apply(relation, names:)
    relation.schema_inferrer Inferrer.new(
      excluded_attributes: names,
      **relation.schema_inferrer.options
    )
  end

  class Inferrer < ROM::SQL::Schema::Inferrer
    option :excluded_attributes, default: -> { [] }

    def call(...)
      super(...).tap do |hash|
        hash[:attributes].reject! do |attribute|
          excluded_attributes.include?(attribute.name)
        end
      end
    end
  end
end

Registered as so:

ROM.plugins do
  adapter :sql do
    register :exclude_attributes, ExcludeAttributes, type: :relation
  end
end

And then in the relation definition:

class Users < ROM::Relation[:sql]
  schema :users, infer: true do
    # ... associations, etc
  end

  use :exclude_attributes, names: %i[first_name last_name]

  # ...
end

I think I’m happy enough with this approach, but if there’s suggestions on improvements, please do let me know :slight_smile:

3 Likes
7

Oh wow this is really cool @pat! Thanks for sharing. We could make it a first-class feature in ROM 6.x.

1 Like
8

We could make it a first-class feature in ROM 6.x.

That’d be great! I can look into writing up a PR if you’d like :slight_smile:

2 Likes
9

This is awesome, thanks for sharing

2 Likes
10

Thanks! We could add it in 6.1. Currently there’s laser-focus on getting 6.0 done for Hanami 2.0 with essential changes/improvements done exclusively. Everything extra will come after 6.0.

11

Hanami 2.0 has been released, but ROM is still on 5.3. Is there an update on what the current plan is?

12

5.3 was released specifically to support Hanami 2.0. My team is using them together without issue.

Last I heard, official persistence support with ROM was slated for 2.1. There are no full-time developers working on any of this, so timelines are loose.

If you have Hanami-specific questions related to ROM it’s probably best to ask over on the Hanami board, I can help with that. It’s possible I have the most complicated ROM configuration on Hanami 2.0 currently.

13

At the moment I am planning to use ROM only, and Roda on top for the API, as that aspect of the code is very thin. It is almost all business logic, and ROM + Ruby should be all we need. My interest in the plan for ROM 6.0 is around whether we should start working with the ROM 6.0 alpha / main branch, or start working with 5.3. I think at this point we should stick to 5.3 as 6.0 still seems a bit fluid.