Firebase Realtime Database / Cloud Firestore Connection Python Django

It is always interesting to combine firebase with another server such as elasticsearch or crawler or mailing server on a separate server.

Then of course a connection is needed, we use python in this example.

sudo pip install firebase-admin

import firebase_admin

from firebase_admin import credentials

cred = credentials.Certificate("path/to/serviceAccountKey.json")

firebase_admin.initialize_app(cred)

https://firebase.google.com/docs/admin/setup#python 

Init and Retrieve Data from 

https://firebase.google.com/docs/database/admin/start 

https://firebase.google.com/docs/database/admin/retrieve-data

def handle(self, *args, **options):

        cred = credentials.Certificate("/")

        firebase_admin.initialize_app(cred, {

                'databaseURL': 'https://database.firebaseio.com'

        })

        jobs = Job.objects.all()[:3]

        ref = db.reference("/jobs")

        import pdb;pdb.set_trace()

        for job in jobs:

            json = self.create_json(job)

            ref.push(json)

    def create_json(self, job):

        json = {

        "id": job.source_unique,

        "company_name": self.clean_json_value(job.company_name),

        "description": self.clean_json_value(job.description),

        "url": self.clean_json_value(job.url),

        "title": self.clean_json_value(job.title),

        "city": self.clean_json_value(job.city),

        "zip_code": self.clean_json_value(job.zip_code)

        }

        return json

snapshot = ref.order_by_child('id').equal_to().get()

If you get an error something like this.


firebase_admin.exceptions.InvalidArgumentError: Index not defined, add ".indexOn": "id", for path "/jobs", to the rules

https://firebase.google.com/docs/database/security/indexing-data 

Add index in your database rules 

for job in jobs:

            json = self.create_json(job)

            snapshot = ref.order_by_child('id').equal_to(job.source_unique).get()

            if not snapshot:

                ref.push(json)

 

Example read data via Javascript

var firebaseConfig = { .... }

 

firebase.initializeApp(firebaseConfig);
 var db = firebase.database();

    var ref = db.ref("jobs");
    ref.orderByChild("id").equalTo(job_id).on("value", function(snapshot) {
        obj = snapshot.val();
        window.location = obj[Object.keys(obj)[0]]["url"];
        //console.log(snapshot.val());
    });

    
    var ref2 = db.ref("jobs/-MWEL0phaE5glY_PjsPa/url");
    ref2.on('value', (snapshot) => {
        data = snapshot.val();
        console.log(data);
    });

Cloud Firestore 

 If database is efficient with object handling and frequent updates then Firestore is effective for complicated queries. Here is very good document about difference.

https://firebase.google.com/docs/database/rtdb-vs-firestore

Connection and authentication is the same as by Realtime Database but there are three layers to our connection.

Database > Collection > Document

from firebase_admin import credentials, firestore

db = firestore.client() # this connects to our Firestore database
collection = db.collection(‘jobs’) # opens 'places' collection
doc = collection.document(‘python-developer’) # specifies the 'rome' document

Each layer has its own set of methods that enable us to perform different operations at the database, collection, or document level.


We use the get method to retrieve data. Let's use this method to get pythondeloper documentation:

doc = collection.document(‘python-developer’)

res = doc.get().to_dict()

print(res)

When collecting, we can also perform a .get() operation to return an array of all documents contained in it. If we have two documents, it will look like this:

To select advanced queries or quick test you can use this tool.

Add documents from the django query set.

jobs = Job.objects.filter(for_index=False)

        for job in jobs:

            json = self.create_json(job)

            if not collection.document(job.source_unique).get().exists:

                res = collection.document(job.source_unique).set(json)

                print(res)

Delete by key. 

for job in offline_jobs:

            print("Delete:" + str(job.source_unique))

            collection.document(job.source_unique).delete()

Perform simple and compound queries in Cloud Firestore

https://firebase.google.com/docs/firestore/query-data/queries

python

        firebase_admin.initialize_app(cred)
        db = firestore.client()
        jobs = db.collection("jobs")
        jobs.where(u"category", u"==", category)
        jobs = self.convert_to_dict(jobs.get())

Very important use unicode in WHERE and stream if it collection filter

 

 

Get started with Cloud Firestore Security Rules

https://firebase.google.com/docs/firestore/security/get-started

Use Shell to test your Rules 

Simple Example Rules

rules_version = '2';

service cloud.firestore {

  match /databases/{database}/documents {

    match /clicks/{document=**} {

      allow write: if true;

      allow read: if false;

    }

    match /jobs/{document=**} {

      allow write: if false;

      allow read: if true;

    }

    match /{document=**} {

      allow read, write: if false;

    }

    

  }

}

Why use Cloudflare with Firebase?

• Security & Protection
• Cost Savings & Prevention
• Speed

Have you heard of the infamous Firebase cost increase by 7000%? If you are using the Blaze plan, you will be charged as needed. If you do not monitor costs carefully, you may receive unwelcome bills at the end of the month.


If you save timestamps you can get all the information from logs. Like IP, User agent etc..