Tel: (+44) 1482 234021 | info@wearesauce.io

Ecto Preloading

No lazy loading

One feature I really like about Ecto is that it doesn't have lazy loading. I remember when I first came across lazy loading, I thought it was great, but it's so easy to end up with N+1 issues etc that it was refreshing when I realised Ecto didn't support it. It basically meant that from now I had to actually plan my queries out.

So for example if I have a query to pull out my User data and then I want to find out the Group data that is associated to each one, in the days of lazy loading I would just do something like this

defmodule Project.User do

  use Project.Web, :model

  schema "users" do
    field :first_name, :string
    field :last_name, :string


    has_many :groups, Project.Group

  end

end

This is our rough Ecto model

users = Project.Repo.all(User)
Enum.each(users, fn(user) ->
  user_groups = user.groups
end)

Ok, this is a bad example as the code will fail, but it's meant to show how lazy loading would have kicked in, to go and get the Group data for that user. I know, an experienced developer would account for this lazy loading in something like ActiveRecord, but less experience developers may miss this and end up with some fairly messy database executions.

How to Preload

To fix the above code, and to make is work in Ecto, we'd do the following

users =
  User
  |> Repo.all
  |> Repo.preload(:groups)

Enum.each(users, fn(user) ->
  user_groups = user.groups
end
`

Again, not the nicest code in the world, but it's just an example to show how Group data is preloaded.

Nested preloading

Sometimes your Ecto models have more complex relationships. So, in this case Users and Groups should really be a many_to_many, so the models will look like this

defmodule Project.User do

  use Project.Web, :model

  schema "users" do
    field :first_name, :string
    field :last_name, :string

    has_many :users_groups, Project.UserGroup

  end

end

defmodule Project.UserGroup do
  use Project.Web, :model

  schema "users_groups" do
    belongs_to :user, Project.User
    belongs_to :group, Project.Group
  end
end

defmodule Project.Group do

  user Project.Web, :model

  schema "groups" do
    field :name, :string

    has_many :user_groups, Project.UserGroup
  end
end

This could be mapped different in the schema, using many_to_many, but I wanted to show the nesting clearly.

To go back to the original example we want to grab all the users and preload the groups data for them. There are a different syntax options here

users =
  User
  |> Repo.all
  |> Repo.preload([users_groups: :group])

or

users =
  User
  |> Repo.all
  |> Repo.preload({:users_groups, :group})

Personally I prefer the first way of doing it here, as it reflects the structure of the model more.

If you are new to Ecto, you may have considered that you'd might preload it like this

users =
  User
  |> Repo.all
  |> Repo.preload([:users_groups, :group])

This would only work if your User model had this schema

defmodule Project.User do

  use Project.Web, :model

  schema "users" do
    field :first_name, :string
    field :last_name, :string

    has_many :users_groups, Project.UserGroup
    has_many :groups, Project.UserGroup

  end

end

See the example below in Preloading mulitple relationships

Preloading multiple relationships

It is possible to preload multiple different relationships. If we add in a Device model and each user can have multiple devices, the updated User model would look like this

defmodule Project.User do

  use Project.Web, :model

  schema "users" do
    field :first_name, :string
    field :last_name, :string

    has_many :users_groups, Project.UserGroup
    has_many :devices, :Project.Device

  end

end

To preload all the Device and Group data we would do the following

users =
  User
  |> Repo.all
  |> Repo.preload([users_groups: :group, :devices])

or (alternative syntax)

users =
  User
  |> Repo.all
  |> Repo.preload({:users_groups, :group}, :devices])

I suspect most developers would now find the second version here clearer at first glance.