Getting Started with IndexedDB

updated on 16 August 2022

This article will teach you about IndexedDB with a small tutorial, and compare IndexedDB to some of the other options available. IndexedDB is used for storing data in the browser and is particularly important for web application that need to work offline, like most progressive web applications.

First, let’s cover some context of why you might need to store data in a web browser. Data is everywhere within web applications - user interactions create data, look up data, update data and delete data. Without a way to store this data you can’t allow user interactions to persist state across multiple uses of a web application.You’ll often use databases like MySQL, Postgres, MongoDB, Neo4j, ArangoDB and others to handle this storage, but what if you want your application to work offline? 

This is especially important within the growing trend of progressive web applications, applications that replicate the feel of a native app, but are in the browser. These progressive web applications must work offline, and so require a storage option. Luckily, there are several tools on how to store data in browser where it can be accessed online and offline.

Browser Storage Options

Web standards provide you with three primary APIs on how to store data in the browser:

  • Cookies: This data is stored in the browser and cookies have a size limit of 4k. Often when a server responds to a request they may include a SET-COOKIE header which gives the browser a key and value to store. The client should then include this cookie in the headers of future requests, which will allow the server to recognize browser sessions and more. Often these cookies have an HTTP-Only attribute, which means the cookie cannot be accessed through a client-side script. This makes cookies a poor choice for holding offline data.
  • LocalStorage/SessionStorage: Localstorage/Sessionstorage is a key value store built into the browser where each key has a size limit of 5mb. LocalStorage stores data until it is deleted while sessionStorage will clear itself when the browser closes. Otherwise, their APIs are the same.  You can add a key-value pair with window.localStorage.setItem("Key", "Value") and retrieve a value with window.localStorage.getItem("Key"). Note that the LocalStorage API is synchronous and so using it does block other activity in the browser, which can be an issue. You can read more about LocalStorage here.
  • IndexedDB: A full on document database built into the browser with no storage limits, it allows you to access the data asynchronously so great for preventing complex operations from blocking rendering and other activities. This is what we will dive into in-depth below. 

Within the context of these options, localStorage is a good choice for simple operations and storing data in small amounts. For more complex or regular operations IndexedDB may be the better option, particularly if you need to fetch data asynchronously. 

The IndexedDB API is more complicated than the LocalStorage API. So let’s build something with IndexedDB to give you a better feel for how it works!

[Repo with Code from the following exercise for reference]

Tutorial Setup

  • Open your favorite IDE/Text Editor
  • Create a new HTML file, let’s call it index.html

In the index.html put the following starter code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>IndexedDB Todo List</title>
</head>
<body>
    <!-- OUR CONTENT -->
    <main>
        <h1>IndexedDB Todo-List</h1>
        <div id="form">
            <input type="text" placeholder="new todo here">
            <button>Add Todo</button>
        </div>
        <div id="todos">
            <ul></ul>
        </div>
    </main>
    
    
    <!-- OUR JAVASCRIPT -->
    <script>
        // Variables holding our inputs
        const textInput = document.querySelector("[type='text']")
        const button = document.querySelector("button")
        // Array to hold todos
        const todos = []
        // function to render todos
        function renderTodos(){
            const ul = document.querySelector("#todos ul")
            ul.innerHTML = ""
            for (todo of todos){
                ul.innerHTML += `<li>${todo}</li>`
            }
        }
        renderTodos()
    </script>
    <!-- OUR STYLING -->
    <style>
        body {
            text-align: center;
        }
        
        h1 {
            color: brown;
        }
    </style>
</body>
</html>

This sets up a basic shell for a todo list. Now we can start setting up IndexedDB. Open this file in your browser. If you’re using VScode, an extension like liveserver will be useful.

Getting Connected to IndexedDB

IndexedDB support is pretty good but we’d still like to check if the browser supports an implementation of the API so you can add the following function to check.

      // Function to check indexedDB implementation and return it
      function getIndexDB() {
        const indexedDB =
          window.indexedDB ||
          window.mozIndexedDB ||
          window.webkitIndexedDB ||
          window.msIndexedDB ||
          window.shimIndexedDB;
          if (indexedDB){
              return indexedDB
          }
          console.error("indexedDB not supported by this browser")
          return null
      }

This function will either return the browser implementation of IndexedDB or log that it is not supported by the browser. You can log the result of calling getIndexDB  in your browser to confirm that your browser supports IndexedDB.  Below you can see the compatibility list of desktop web browsers from caniuse. You can find the full list including mobile browsers here.

Caniuse's compatibility list of desktop web browsers-yqcb8

Now let’s open a database with indexedDB.open("database name", 1). The first argument to .open is the name of the database and the second argument is the version of the database. You should increment the version number in the .open call if you wish to fire a onupgradeneeded. The .open method will return an object that has several properties, including onerror, onupgradeneeded and onsuccess, each of which accepts a callback which executes when the relevant event occurs.

      const indexedDB = getIndexDB()
      // console.log(indexedDB)
      const request = indexedDB.open("todoDB", 1)
      console.log(request)
      renderTodos();

You should see a console.log that shows an IDBOpenDBRequest object being logged. IndexedDB is event-based, which fits it’s asynchronous model. Next let’s listen to the possible events that will occur as the database starts up. First we’ll listen for the request.onerror event in case there are any errors accessing the database.

      const indexedDB = getIndexDB()
      // console.log(indexedDB)
      const request = indexedDB.open("todoDB", 1)
      //console.log(request)
      //onerror handling
      request.onerror = (event) => console.error("IndexDB Error: ", event)

      renderTodos();

The next event we will hook into is the request.onupgradeneeded event, which runs whenever an attempt is made to open a database with a version number higher than the database’s current version number. This is the function in which to create your stores/tables and their schemas. This function will only execute once per version number. So if you decide to change the onupgradeneeded callback to update your schema or create new stores, then the version number should also be incremented in the next .open call. A store is essentially the equivalent of a table in traditional databases.

      const indexedDB = getIndexDB();
      // console.log(indexedDB)
      const request = indexedDB.open("todoDB", 1);
      //console.log(request)
      //onerror handling
      request.onerror = (event) => console.error("IndexDB Error: ", event);
      //onupgradeneeded
      request.onupgradeneeded = () => {
        // grab the database connection
        const db = request.result;
        // define a new store
        const store = db.createObjectStore("todos", {
          keyPath: "id",
          autoIncrement: true,
        });
        // specify a property as an index
        store.createIndex("todos_text", ["text"], {unique: false})
      };
      
      renderTodos();

In the on upgrade needed call we do the following:

  • grab the database object (you know its available if the onupgradeneeded function is running)
  • create a new store/table/collection called todos with a key of id that is an autoincrementing number (the unique identifier of records)
  • specify todos_text as an index, which allows us to search the database by todos_text later. You don’t have to create an index if you don’t plan on searching by a particular property.

Last you want to handle the request.onsuccess event that runs after the database connection and stores are all setup and configured. You will want to use this opportunity to pull the list of todos and inject them into our array. (yes, there is no todos, yet)

      //onsuccess
      request.onsuccess = () => {
          console.log("Database Connection Established")
          //grab the database connection
          const db = request.result
          // create a transaction object
          const tx = db.transaction("todos", "readwrite")
          //create a transaction with our store
          const todosStore = tx.objectStore("todos")
          //get all todos
          const query = todosStore.getAll()
          //use data from query
          query.onsuccess =  () => {
              console.log("All Todos: ", query.result)
              for (todo of query.result){
                  todos.push(todo.text)
              }
              renderTodos()
          }
      }

In our on success we do the following:

  • get the database connection
  • create a transaction
  • specify which store we are transacting on
  • run a getAll query to get all documents/records in that store
  • In the query specific onsuccess event we loop through the todos, push them into the todos array and call renderTodos() so they are rendered to the dom

You should see a console.log with an empty array in your console.

[Troubleshooting tip: If you are running a hot reloading web server like liveserver you may see an error that there are no stores. This is because the onupgradeneeded function executed prior to you finishing writing the function. Thus, it won’t execute again for that version number. The solution is to increment the table version number, which will create an onupgradeneeded, and the onupgradeneeded callback will execute on the next page refresh.]

Now that we have the database setup with can follow this same pattern for any other events we want to happen. For example, let’s create an event when you click the button that will add a new todo not only to the dom but to the database so it’ll show up on page refreshes.

      // button event
      button.addEventListener("click", (event) => {
          // setup a transaction
          const db = request.result
          const tx = db.transaction("todos", "readwrite")
          const todosStore = tx.objectStore("todos")
          // add a todo
          const text = textInput.value
          todos.push(text) // add to todo array
          todosStore.put({text}) // add to indexedDB
          renderTodos() // update dom
      })

Now you can add todos and since you are using IndexedDB it will work whether you are online or offline as you can see in this gif.

Adding todos using IndexedDB-aq53y

Add some todos and when you refresh the page you shall see the todos persist. They will also show up in the console.log of the query result, with each todo having a unique ID. The full code up to this point should now look like so:

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <title>IndexedDB Todo List</title>
  </head>
  <body>
    <!-- OUR CONTENT -->
    <main>
      <h1>IndexedDB Todo-List</h1>
      <div id="form">
        <input type="text" placeholder="new todo here" />
        <button>Add Todo</button>
      </div>
      <div id="todos">
        <ul></ul>
      </div>
    </main>
    <!-- OUR JAVASCRIPT -->
    <script>
      // Variables holding our inputs
      const textInput = document.querySelector("[type='text']");
      const button = document.querySelector("button");
      // Array to hold todos
      const todos = [];
      // function to render todos
      function renderTodos() {
        const ul = document.querySelector("#todos ul");
        ul.innerHTML = "";
        for (todo of todos) {
          ul.innerHTML += `<li>${todo}</li>`;
        }
      }
      // Function to check indexedDB implementation and return it
      function getIndexDB() {
        const indexedDB =
          window.indexedDB ||
          window.mozIndexedDB ||
          window.webkitIndexedDB ||
          window.msIndexedDB ||
          window.shimIndexedDB;
        if (indexedDB) {
          return indexedDB;
        }
        console.log("indexedDB not supported by this browser");
        return null;
      }
      const indexedDB = getIndexDB();
      // console.log(indexedDB)
      const request = indexedDB.open("todoDB", 2);
      //console.log(request)
      //onerror handling
      request.onerror = (event) => console.error("IndexDB Error: ", event);
      //onupgradeneeded
      request.onupgradeneeded = () => {
        // grab the database connection
        const db = request.result;
        // define a new store
        const store = db.createObjectStore("todos", {
          keyPath: "id",
          autoIncrement: true,
        });
        // specify a property as an index
        store.createIndex("todos_text", ["text"], {unique: false})
      };
      //onsuccess
      request.onsuccess = () => {
          console.log("Database Connection Established")
          //grab the database connection
          const db = request.result
          // create a transaction object
          const tx = db.transaction("todos", "readwrite")
          //create a transaction with our store
          const todosStore = tx.objectStore("todos")
          //get all todos
          const query = todosStore.getAll()
          //use data from query
          query.onsuccess =  () => {
              console.log("All Todos: ", query.result)
              for (todo of query.result){
                  todos.push(todo.text)
              }
              renderTodos()
          }
      }
      // button event
      button.addEventListener("click", (event) => {
          // setup a transaction
          const db = request.result
          const tx = db.transaction("todos", "readwrite")
          const todosStore = tx.objectStore("todos")
          // add a todo
          const text = textInput.value
          todos.push(text) // add to todo array
          todosStore.put({text}) // add to indexedDB
          renderTodos() // update dom
      })

      renderTodos();
    </script>
    <!-- OUR STYLING -->
    <style>
      body {
        text-align: center;
      }
      h1 {
        color: brown;
      }
    </style>
  </body>
</html>

Other methods on todosStore object that can be used for different types of transaction:

  • clear - deletes all documents/records in store
  • add - insert a record with the given id (will error if it already exists)
  • put - insert or update a record with given id (will update if already exists)
  • get - get record with particular id
  • getAll - get all record/documents from the store
  • count - returns the count of records in the store
  • createIndex - create object to query based on a declared index
  • delete - delete document with the given id

Performance and other Considerations

A couple of things you want to consider:

  • Storing files as blobs may not be supported in all browsers, you’ll find better support storing them as ArrayBuffers.
  • Some browsers may not support writing to IndexedDB when in private browsing mode
  • IndexedDB creates structured clones when writing your objects which does block the main thread so if you have large objects filled with even more nested objects this can cause some latency.
  • If the user shuts down the browser there is a chance any incomplete transactions will be aborted. 
  • If another browser tab has the application open with a newer database version number, it will be blocked from upgrading till all older version tabs are closed/reloaded. Luckily, you can use the onblocked event to trigger alerts to notify the user they need to do this.

You can find more limitations of IndexedDB in the MDN Documentation.

Whilst indexedDB is great for making your app work offline, it should not be your main datastore. Once there in an internet connection you would probably want to sync indexedDB with your external database, so that the user’s information is not lost if they clear their browser data.

Conclusion

IndexedDB gives you a powerful asynchronous document database in your browser. The IndexedDB API can be a little cumbersome but there are libraries like Dexie that give you wrappers around IndexedDB which are much easier to use.

Meticulous

Meticulous is a tool for software engineers to easily create end-to-end tests. Use our open source CLI to open an instrumented browser which records your actions and translates them into a test. Meticulous makes it easy to integrate these tests into your CI.

Meticulous has an option to automatically mock out all network calls when simulating a recorded sequence of actions. If you use this option, you do not need a backend environment to use Meticulous and Meticulous tests never cause side effects (like affecting analytics) or hit your backend.

Try out the open-source CLI and create your first test in 60 seconds using our docs or watch the demo.

Authored by Alex Merced

Read more