Quickly backup any collection on your Firestore database

After spending hours trying to figure out how to export Firestore to BigQuery (and then actually being able to use that data outside of Firestore), I ran into a CLI tool that lets you export any part of your database as JSON that you can use everywhere.

Today you’ll learn how to use node-firestore-import-export to backup (or feed data to) your Firestore database.

We’ll need 2 things to get started, first, we need to download the service account file for our Firebase project, you can downloading by going into Project Overview > Project Settings > Service Accounts, and then clicking on the button that says “Generate new Private Key”.

Keep this in a safe place, since it gives access to your firebase project.

Once we have our service account, we’ll want to install an npm package called node-firestore-import-export, for that, we can go ahead and open our terminal and type:

npm install -g node-firestore-import-export

It’s a CLI tool that has 2 main features, download data from anywhere in your database as JSON, or import data to your Firestore database in any path you want.

For example, let’s say we have a database with 2 collections as shown below, one collection for users, and another one for food items.

We’re going to export our users, for that, we want to open the terminal and navigate to where we downloaded the service account file.

Once there, we’ll use the terminal and type:

firestore-export --accountCredentials my-service-account-file.json --backupFile user.backup.json --nodePath users --prettyPrint

Where:

• firestore-export is the command we’re running.
• --accountCredentials tells the command we’re about to pass our service account file and my-service-account-file.json is the actual service account file I downloaded.
• --backupFile tells the command we’re about to pass the name of the file we want to create to save our downloaded data, which in this case is user.bacup.json.json.
• --nodePath users tells the tool to only download the documents from the users collection (You can ignore this one and download the entire database).
• --prettyPrint is a flag that tells the tool it should format the JSON file so that it’s not everything in one single line and it’s easier to read.

In the end, you’ll download the file and see something like this as a result inside of it:

{
  "lZ36pgW4bzTntSPck6hHr92OAk83": {
    "name": "Jorge Vergara",
    "__collections__": {
      "party": {
        "axB7I8Lhmp7YRChipqGb": {
          "date": "2021-10-19T20:54:00-05:00",
          "cost": 50000,
          "ticketPrice": 121221212,
          "revenue": 0,
          "name": "dededed",
          "__collections__": {}
        }
      }
    }
  }
}

In my case I had one user and inside the user another subcollection called party.

You can do the same to import data, the only thing that changes is the command you’re running, for example, let’s say we want to open the user.backup.json file we created, and change or add a property to all of our users, then re-import it.

To import it, you’d open the terminal and type:

firestore-import --accountCredentials my-service-account-file.json --backupFile user.backup.json --nodePath users

Notice how the command changes from firestore-export to firestore-import and that you don’t need the --prettyPrint flag anymore, other than that, we’re writing the same.

Considerations

There are a few considerations to keep in mind.

1. The export command will count against your read quota, so if you have to export a big database on an app that is very active, you might end up paying for that export.
2. The import command will overwrite anything you point it to. If you point it to a collection that exists there, it will overwrite the documents you’re sending.

Always use it taking those 2 considerations in mind.

Need help with anything Firebase related? Do let me know, I’m happy to help, the easiest way to reach out is through twitter as @javebratt