Getting started with Ecto in Phoenix

If you’ve ever created a Phoenix application that communicates with a database, you’ve used Ecto, which is the standard database interface tool for Phoenix applications. Ecto provides rich functionalities and is divided into four parts:

1. Ecto.Repo: Repositories provide an abstraction on the data store. We can create, update, destroy, and query existing entries through the repository
2. Ecto.Schema: Schemas are used to convert external data into structs for Elixir. They are often used for mapping database tables to Elixir data
3. Ecto.Query: Queries enable us to retrieve information from a given repository. Ecto queries are secure and composable
4. Ecto.Changeset: Changesets enable changes to be tracked and verified prior to being applied to the data

In this article, we’ll learn about Ecto and its different parts, their functionality, and how to use them properly in your project.

Jump ahead:

• Getting started with our Phoenix application
• Database migrations
• Using Ecto schemas
• Ecto.Repo
• Ecto.Changeset
• Understading Ecto associations
  • The has many/belongs to Ecto association
  • The has one/belongs to Ecto association

Getting started with our Phoenix application

In this tutorial, we’re building a simple e-commerce application that helps users access products from different stores. Our database will have the following tables:

• Stores: The stores table will have a name field and product field as a foreign key referencing the products table
• Products: The products table will have a name field, description field, price field, and a store field (a foreign key) that references the stores table
• Users: The users table will have a username field and will also have a single store that references the stores table

Our first step is to create a Phoenix application. We can do so by running the following command:

mix phx.new catalog

Make sure to have a database instance running, which you can check out in this tutorial for how to run PostgreSQL on Docker Once the application is created, cd catalog, edit the configuration at config/dev.exs to match your database credentials. Finally, run ecto.create.

Database migrations

Database migrations are a way of telling the database what changes we need to make. These changes are usually made incrementally so they can be reversed at any point in time.

Ecto provides a simple command for creating migrations. The command has the following structure:

mix ecto.gen.migration <migration_name>

To create our store migration, we use the following command:

mix ecto.gen.migration create_store

The newly created migration can be found in the priv/repo/migrations directory. Migration names usually take the format of <timestamp>_<migration_name>.exs.

In the create_store migrate file, we are simply creating a stores table, then we add the name field and timestamps:

defmodule Catalog.Repo.Migrations.CreateStore do
  use Ecto.Migration
  def change do
    create table("stores") do
      add :name, :string

      timestamps()
    end
  end
end

Following the steps above, we create two more migrations: create_user and create_product. Then, we’ll add the following code to the create_user migration file:

defmodule Catalog.Repo.Migrations.User do
  use Ecto.Migration
  def change do
    create table("users") do
      add :name, :string

      timestamps()
    end
  end
end

Next, add the code below to the create_product migration file:

defmodule Catalog.Repo.Migrations.Product do
  use Ecto.Migration
  def change do
    create table("products") do
      add :name, :string
      add :price, :float
      add :description, :string

      timestamps()
    end
  end
end

After the migrations are created, run the mix ecto.migrate command to make any changes to the database.

Using Ecto schemas

Schemas are used to map external data into Elixir structs. This means that these schemas allow us to have Elixir representations of external data (e.g., database tables). To create our schemas, replicate the commands below:

mix phx.gen.schema User users name:string --no-migration

mix phx.gen.schema Store stores name:string --no-migration

mix phx.gen.schema Product products name:string description:string price:float --no-migration

The first argument to the phx.gen.schema is the schema module, followed by its plural name, which should be the same as the table name. The other arguments are the fields we want to create. The --no-migrate flag signals phx.gen.schema not to create a migration file, because we already created them.

The generated schema files can be found in the lib/catalog directory if we look at the schema generated in the store.ex file:

defmodule Catalog.Store do
  use Ecto.Schema
  import Ecto.Changeset
  schema "stores" do
    field :name, :string

    timestamps()
  end

  def changeset(store, attrs) do
    store
    |> cast(attrs, [:name])
    |> validate_required([:name])
  end
end

Notice that the schema has the same fields as our migration file. This is what we mean when we say that schemas are used to map external data into Elixir structs. Schemas lets us know what fields are present in an external data object. We’ll have a look at the changeset function soon, but for now, let’s test the schema so far.

Ecto.Repo

To test our schema, run iex -S mix in your terminal. This command spins up an interactive shell where we can test out our schema and carry out some other operations:

iex(1)> alias Catalog.Repo
Catalog.Repo
iex(2)> alias Catalog.{User, Store, Product}
[Catalog.User, Catalog.Store, Catalog.Product]

The alias keyword allows us to create aliases. Therefore, we can use Repo rather than Catalog.Repo to keep it short and simple. Using Repo, or repository, we can create, update, destroy, and query existing entries:

iex(3)> {:ok, newStore} = Repo.insert(%Store{name: "Logan Spark Store"})
...
{:ok,
 %Catalog.Store{
   __meta__: #Ecto.Schema.Metadata<:loaded, "stores">,
   id: 1,
   name: "Logan Spark Store",
   inserted_at: ~N[2023-04-17 01:47:01],
   updated_at: ~N[2023-04-17 01:47:01]
 }}

 {:ok, newProduct} = Repo.insert(%Product{name: "Spark Armour", price: 200.99, description: "The latest sneakers from the Logan Spark collection"})
...
{:ok,
 %Catalog.Product{
   __meta__: #Ecto.Schema.Metadata<:loaded, "products">,
   id: 1,
   description: "The latest sneakers from the Logan Spark collection",
   name: "Spark Armour",
   price: 200.99,
   inserted_at: ~N[2023-04-17 02:12:32],
   updated_at: ~N[2023-04-17 02:12:32]
 }}

Repo.insert creates data in the given table. In the code snippet above, we created a store with the name “Logan Spark Store.” Using Repo, we can also perform other operations, as shown below:

iex(5)> Repo.all(Store)
...
[
  %Catalog.Store{
    __meta__: #Ecto.Schema.Metadata<:loaded, "stores">,
    id: 1,
    name: "Logan Spark Store",
    inserted_at: ~N[2023-04-17 02:10:34],
    updated_at: ~N[2023-04-17 02:10:34]
  }
]

Repo.get(Product, newProduct.id)
...
%Catalog.Product{
  __meta__: #Ecto.Schema.Metadata<:loaded, "products">,
  id: 1,
  description: "The latest sneakers from the Logan Spark collection",
  name: "Spark Armour",
  price: 200.99,
  inserted_at: ~N[2023-04-17 02:12:32],
  updated_at: ~N[2023-04-17 02:12:32]
}

Ecto.Changeset

Changesets allow filtering, casting, validation, and defining constraints when working with structs. Changesets tell Ecto how to change your data. Here’s a changeset definition we saw previously in the schema section:

 def changeset(product, attrs) do
    product
    |> cast(attrs, [:name, :description, :price])
    |> validate_required([:name, :description, :price])
  end

Let’s go into our interactive shell to see how Ecto changesets work:

iex -S mix

We can create a changeset with our Product schema:

iex(9)> changeset = Product.changeset(%Product{}, %{name: "Mcatty tulip", description: "Tulips from the Mcatty gardens", price: 100.00})
#Ecto.Changeset<
  action: nil,
  changes: %{
    description: "Tulips from the Mcatty gardens",
    name: "Mcatty tulip",
    price: 100.0
  },
  errors: [],
  data: #Catalog.Product<>,
  valid?: true
>

 |> cast(attrs, [:name, :description, :price])

The attributes listed in cast in the changeset function definition above are the only attributes that make it into the changes property in our changeset. If we add another property that isn’t listed in cast, then the changeset ignores such values. We can see this in the example below:

iex(9)> changeset = Product.changeset(%Product{}, %{name: "Mcatty tulip", description: "Tulips from the Mcatty gardens", price: 100.00, inStock: true})
#Ecto.Changeset<
  action: nil,
  changes: %{
    description: "Tulips from the Mcatty gardens",
    name: "Mcatty tulip",
    price: 100.0
  },
  errors: [],
  data: #Catalog.Product<>,
  valid?: true
>

Despite adding the inStock property to the changeset, it doesn’t appear in the changes field.

Using changesets for validation

Changesets are also used for validation purposes:

|> validate_required([:name, :description, :price])

In validate_required, we declared some fields as required. Now, let’s see what happens when we don’t include one or more of those fields in our changeset:

iex(10)> changeset = Product.changeset(%Product{}, %{name: "Mcatty tulip", description: "Tulips from the Mcatty gardens"})
#Ecto.Changeset<
  action: nil,
  changes: %{
    description: "Tulips from the Mcatty gardens",
    name: "Mcatty tulip"
  },
  errors: [price: {"can't be blank", [validation: :required]}],
  data: #Catalog.Product<>,
  valid?: false
>

The changeset returns an error. We can check the error from the changeset:

iex(11)> changeset.errors
[price: {"can't be blank", [validation: :required]}]

We can also check if the changeset is valid:

iex(12)> changeset.valid?
false

There are many more transformations and validations that can be carried out using changesets.

Understanding Ecto associations

Associations help to establish relationships between schemas. With associations, we can embed documents. Ecto associations might take one of the following forms, the latter of which does not have an application in the app we’re building in this tutorial:

• Has many/belongs to
• Has one/belongs to
• Many-to-many

The has many/belongs to Ecto association

In the case of our application, a store has many products and every product must belong to a store. Let’s create an association between the stores schema and the product schema. We’ll also use this as an opportunity to see how to update our database tables using migrations.

First, we have to alter the products table to have an association with the stores table. We can do that with migrations:

mix ecto.gen.migration add_store_product_association

In the add_store_product_association migration file:

defmodule Catalog.Repo.Migrations.AddStoreProductAssociation do
  use Ecto.Migration
  def change do
    alter table("products") do
      add :store_id, references(:stores)

    end
  end
end

Then, run mix ecto.migrate to update the database. We’ll also update the products schema by adding the belongs_to association:

schema "products" do
   ...
    belongs_to :store, Catalog.Store

    timestamps()
end

Update the stores schema by adding the has_many association:

schema "stores" do
    ...
    has_many :products, Catalog.Product

    timestamps()
end

The has one/belongs to Ecto association

The association between users and stores follows this pattern. A user has one store and a store belongs to a user. To work with this association, first generate the migration:

mix ecto.gen.migration add_user_store_association

Then, in the add_user_store_association migration file, we’ll add the following code:

defmodule Catalog.Repo.Migrations.AddUserStoreAssociation do
  use Ecto.Migration
  def change do
    alter table("stores") do
      add :user_id, references(:users)

    end
  end
end

Next, we’ll run mix ecto.migrate to update the database, and update the stores schema by adding the belongs_to association:

schema "stores" do
   ...
    belongs_to :user, Catalog.User

    timestamps()
end

Finally, we’ll update the users schema by adding the has_one association:

schema "users" do
    ...
    has_one :store, Catalog.Store

    timestamps()
 end

Let’s see the relationships between our schemas in our interactive Elixir shell. Run iex -S mix. Next, we can recompile our code and insert a new user into the database:

iex(11)> recompile()
Compiling 2 files (.ex)
:ok
iex(12)> Repo.insert(%User{name: "Alex"})
...
{:ok,
 %Catalog.User{
   __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
   id: 1,
   name: "Alex",
   store: #Ecto.Association.NotLoaded<association :store is not loaded>,
   inserted_at: ~N[2023-04-19 17:55:26],
   updated_at: ~N[2023-04-19 17:55:26]
 }}

We can see that there’s a store field on our User struct. Ecto added that for us, because we specified the store field as a relationship on the User schemas.

Let’s create some items that implement our newly created associations:

iex(13)> {:ok, store} = Repo.insert(%Store{name: "Alex store", user: user})
...
iex(14)> {:ok, product} = Repo.insert(%Product{name: "Alex turtle necks", price: 20.99, description: "Cheap turtle necks to make you comfortable", store: store})
...

Now, #Ecto.Association.NotLoaded<association :store is not loaded simply tells us that the association is not loaded. We can fix that with Repo.preload:

iex(15)> Repo.get(User, 1) |> Repo.preload([:store])
...
%Catalog.User{
  __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
  id: 1,
  name: "Alex",
  store: %Catalog.Store{
    __meta__: #Ecto.Schema.Metadata<:loaded, "stores">,
    id: 2,
    name: "Alex store",
    products: #Ecto.Association.NotLoaded<association :products is not loaded>,
    user_id: 1,
    user: #Ecto.Association.NotLoaded<association :user is not loaded>,
    inserted_at: ~N[2023-04-19 20:16:28],
    updated_at: ~N[2023-04-19 20:16:28]
  },
  inserted_at: ~N[2023-04-19 17:55:26],
  updated_at: ~N[2023-04-19 17:55:26]
}

We can perform nested preload as follows:

iex(16)> Repo.get(User, 1) |> Repo.preload([store: :user])
...
%Catalog.User{
  __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
  id: 1,
  name: "Alex",
  store: %Catalog.Store{
    __meta__: #Ecto.Schema.Metadata<:loaded, "stores">,
    id: 2,
    name: "Alex store",
    products: #Ecto.Association.NotLoaded<association :products is not loaded>,
    user_id: 1,
    user: %Catalog.User{
      __meta__: #Ecto.Schema.Metadata<:loaded, "users">,
      id: 1,
      name: "Alex",
      store: #Ecto.Association.NotLoaded<association :store is not loaded>,
      inserted_at: ~N[2023-04-19 17:55:26],
      updated_at: ~N[2023-04-19 17:55:26]
    },
    inserted_at: ~N[2023-04-19 20:16:28],
    updated_at: ~N[2023-04-19 20:16:28]
  },
  inserted_at: ~N[2023-04-19 17:55:26],
  updated_at: ~N[2023-04-19 17:55:26]
}

Conclusion

In this introduction to Ecto, we learned about the different parts of the database interface tool, including Ecto.Repo, Ecto.Schema, and Ecto.Changeset. Using appropriate code examples and the interactive Elixir shell, we had some hands-on experience with the different parts of Ecto. Don’t forget to check out the GitHub repository for this project.

Although this article was meant to serve as an introduction, I hope you now have the confidence to take on more advanced Ecto concepts and use these concepts to create amazing applications. For more Phoenix projects using Ecto, check out Authentication with Phoenix and Building a REST API with Elixir and Phoenix.