Export json from Firestore

Question

As we can download json file at Firebase RTDB console, are there any way to export json file of Firestore collection/document data?

One of my main objectives is to c

Answer 1

I've written a tool that traverses the collections/documents of the database and exports everything into a single json file. Plus, it will import the same structure as well (helpful for cloning/moving Firestore databases). Since I've had a few colleagues use the code, I figured I would publish it as an NPM package. Feel free to try it and give some feedback.

https://www.npmjs.com/package/node-firestore-import-export

Answer 2

1. Create a blank folder (call it firebaseImportExport ) and run npm init
2. Go to the source Firebase project -> Settings -> Service Accounts
3. Click on the Generate new private key button and rename the file as source.json and put it in the firebaseImportExport folder
4. Do the same (step 2 & 3) for the destination project and rename the file as destination.json
5. Install the npm i firebase-admin npm package.
6. Write the following code in the index.js

const firebase = require('firebase-admin');

var serviceAccountSource = require("./source.json");
var serviceAccountDestination = require("./destination.json");

const sourceAdmin = firebase.initializeApp({
    credential: firebase.credential.cert(serviceAccountSource),
    databaseURL: "https://**********.firebaseio.com" // replace with source
});

const destinationAdmin = firebase.initializeApp({
    credential: firebase.credential.cert(serviceAccountDestination),
    databaseURL: "https://$$$$$.firebaseio.com"
  }, "destination");

const collections = [ "books", "authors",   ...]; // replace with your collections

    var source = sourceAdmin.firestore();
    var destination = destinationAdmin.firestore();
   collections.forEach(colName => {
    source.collection(colName).get().then(function(querySnapshot) {
        querySnapshot.forEach(function(doc) {
            destination.collection(colName).doc(doc.id).set({...doc.data()});
        });
    });
   });

Answer 3

There is not, you'd need to come up with your own process such as querying a collection and looping over everything.

Update

As of August 7th, 2018, we do have a managed export system that allows you to dump your data into a GCS bucket. While this isn't JSON, it is a format that is the same as Cloud Datastore uses, so BigQuery understands it. This means you can then import it into BigQuery.

Answer 4

If someone wants a solution using Python 2 or 3.

Edit: note that this does not backup the rules

Fork it on https://github.com/RobinManoli/python-firebase-admin-firestore-backup

First install and setup Firebase Admin Python SDK: https://firebase.google.com/docs/admin/setup

Then install it in your python environment:

pip install firebase-admin

Install the Firestore module:

pip install google-cloud-core
pip install google-cloud-firestore

(from ImportError: Failed to import the Cloud Firestore library for Python)

Python Code

# -*- coding: UTF-8 -*-

import firebase_admin
from firebase_admin import credentials, firestore
import json

cred = credentials.Certificate('xxxxx-adminsdk-xxxxx-xxxxxxx.json') # from firebase project settings
default_app = firebase_admin.initialize_app(cred, {
    'databaseURL' : 'https://xxxxx.firebaseio.com'
})

db = firebase_admin.firestore.client()

# add your collections manually
collection_names = ['myFirstCollection', 'mySecondCollection']
collections = dict()
dict4json = dict()
n_documents = 0

for collection in collection_names:
    collections[collection] = db.collection(collection).get()
    dict4json[collection] = {}
    for document in collections[collection]:
        docdict = document.to_dict()
        dict4json[collection][document.id] = docdict
        n_documents += 1

jsonfromdict = json.dumps(dict4json)

path_filename = "/mypath/databases/firestore.json"
print "Downloaded %d collections, %d documents and now writing %d json characters to %s" % ( len(collection_names), n_documents, len(jsonfromdict), path_filename )
with open(path_filename, 'w') as the_file:
    the_file.write(jsonfromdict)

Answer 5

Open any of your clientside firebase apps (React, Angular, etc.). Use this code anywhere to log console and copy

const products = await db
  .collection("collectionName")
  .where("time", ">", new Date("2020-09-01"))
  .get()


const json = JSON.stringify(products.docs.map((doc) => ({ ...doc.data() })))
console.log(json)

Answer 6

Firestore is still early in its development so please check the docs on backups for any information pertaining to Firestore.

I found this npm package, node-firestore-backup, to be easy and useful.

Note that the --accountCredentials path/to/credentials/file.json is referring to a service account key json file that you can get by following instructions from https://developers.google.com/identity/protocols/application-default-credentials.

1. Go to the API Console Credentials page.
2. From the project drop-down, select your project.
3. On the Credentials page, select the Create credentials drop-down, then select Service account key.
4. From the Service account drop-down, select an existing service account or create a new one.
5. For Key type, select the JSON key option, then select Create. The file automatically downloads to your computer.
6. Put the *.json file you just downloaded in a directory of your choosing. This directory must be private (you can't let anyone get access to this), but accessible to your web server code.