Skip to main content

Firebase Realtime Database / Cloud Firestore Connection Python Django

It is always interesting to combine firebase with another server such as elasticsearch or crawler or mailing server on a separate server.



Then of course a connection is needed, we use python in this example.

sudo pip install firebase-admin



import firebase_admin

from firebase_admin import credentials


cred = credentials.Certificate("path/to/serviceAccountKey.json")

firebase_admin.initialize_app(cred)



https://firebase.google.com/docs/admin/setup#python 



Init and Retrieve Data from 

https://firebase.google.com/docs/database/admin/start 

https://firebase.google.com/docs/database/admin/retrieve-data




def handle(self, *args, **options):

        cred = credentials.Certificate("/")

        firebase_admin.initialize_app(cred, {

                'databaseURL': 'https://database.firebaseio.com'

        })

        jobs = Job.objects.all()[:3]

        ref = db.reference("/jobs")

        import pdb;pdb.set_trace()

        for job in jobs:

            json = self.create_json(job)

            ref.push(json)



    def create_json(self, job):

        json = {

        "id": job.source_unique,

        "company_name": self.clean_json_value(job.company_name),

        "description": self.clean_json_value(job.description),

        "url": self.clean_json_value(job.url),

        "title": self.clean_json_value(job.title),

        "city": self.clean_json_value(job.city),

        "zip_code": self.clean_json_value(job.zip_code)

        }

        return json



snapshot = ref.order_by_child('id').equal_to().get()




If you get an error something like this.


firebase_admin.exceptions.InvalidArgumentError: Index not defined, add ".indexOn": "id", for path "/jobs", to the rules

https://firebase.google.com/docs/database/security/indexing-data 


Add index in your database rules 




for job in jobs:

            json = self.create_json(job)

            snapshot = ref.order_by_child('id').equal_to(job.source_unique).get()

            if not snapshot:

                ref.push(json)

 



Example read data via Javascript


var firebaseConfig = { .... }

 

firebase.initializeApp(firebaseConfig);
 var db = firebase.database();


    var ref = db.ref("jobs");
    ref.orderByChild("id").equalTo(job_id).on("value", function(snapshot) {
        obj = snapshot.val();
        window.location = obj[Object.keys(obj)[0]]["url"];
        //console.log(snapshot.val());
    });


    
    var ref2 = db.ref("jobs/-MWEL0phaE5glY_PjsPa/url");
    ref2.on('value', (snapshot) => {
        data = snapshot.val();
        console.log(data);
    });



Cloud Firestore 


 If database is efficient with object handling and frequent updates then Firestore is effective for complicated queries. Here is very good document about difference.

https://firebase.google.com/docs/database/rtdb-vs-firestore

Connection and authentication is the same as by Realtime Database but there are three layers to our connection.

Database > Collection > Document


from firebase_admin import credentials, firestore


db = firestore.client()  # this connects to our Firestore database

collection = db.collection(‘jobs’)  # opens 'places' collection

doc = collection.document(‘python-developer’)  # specifies the 'rome' document








Each layer has its own set of methods that enable us to perform different operations at the database, collection, or document level.


We use the get method to retrieve data. Let's use this method to get pythondeloper documentation:


doc = collection.document(‘python-developer’)

res = doc.get().to_dict()

print(res)



When collecting, we can also perform a .get() operation to return an array of all documents contained in it. If we have two documents, it will look like this:






To select advanced queries or quick test you can use this tool.






Add documents from the django query set. 


jobs = Job.objects.filter(for_index=False)


        for job in jobs:

            json = self.create_json(job)

            if not collection.document(job.source_unique).get().exists:

                res = collection.document(job.source_unique).set(json)

                print(res)

Delete by key. 


for job in offline_jobs:

            print("Delete:" + str(job.source_unique))

            collection.document(job.source_unique).delete()


Perform simple and compound queries in Cloud Firestore

https://firebase.google.com/docs/firestore/query-data/queries

python 

        firebase_admin.initialize_app(cred)
        db = firestore.client()
        jobs = db.collection("jobs")
        jobs.where(u"category", u"==", category)
        jobs = self.convert_to_dict(jobs.get())


Very important use unicode in WHERE and stream if it collection filter

 

 

Get started with Cloud Firestore Security Rules

https://firebase.google.com/docs/firestore/security/get-started


Use Shell to test your Rules 




Simple Example Rules


rules_version = '2';

service cloud.firestore {

  match /databases/{database}/documents {

    match /clicks/{document=**} {

      allow write: if true;

      allow read: if false;

    }

    match /jobs/{document=**} {

      allow write: if false;

      allow read: if true;

    }

    match /{document=**} {

      allow read, write: if false;

    }

    

  }

}



Why use Cloudflare with Firebase?


  1. Security & Protection

  2. Cost Savings & Prevention

  3. Speed


Have you heard of the infamous Firebase cost increase by 7000%? If you are using the Blaze plan, you will be charged as needed. If you do not monitor costs carefully, you may receive unwelcome bills at the end of the month.


If you save timestamps you can get all the information from logs. Like IP, User agent etc..




Comments

Popular posts from this blog

Pgpool PgBouncer Postgresql streaming replication, load balancing and administration

The term scalability refers to the ability of a software system to grow as the business that uses it grows. PostgreSQL provides some features to help you build scalable solutions, but strictly speaking, PostgreSQL itself is not scalable. It can effectively use the following resources from one computer. Now, we will show you some configurations that may be useful for your use case. However, this can be problematic when distributing the database solution to multiple computers, because the standard PostgreSQL server can only run on a single computer. In this article, we will study different extension schemes and their implementation in PostgreSQL. Replication can be used in many expansion scenarios. Its main purpose is to create and maintain a backup database when the system fails. This is especially true for physical replication. However, replication can also be used to improve the performance of PostgreSQL-based solutions. Sometimes third-party tools can be used to implement complex exp

Tekstverwerking python Text processing python SpaCy, TensorFlow, NLTK, Allen-NLP, Stanford-NLP

 Dit post maakt gebruik van spaCy, een populaire Python-bibliotheek die de taalgegevens en algoritmen bevat die je nodig hebt om teksten in natuurlijke taal te verwerken. Zoals u in dit post zult leren, is spaCy gemakkelijk te gebruiken omdat het containerobjecten biedt die elementen van natuurlijke taalteksten vertegenwoordigen, zoals zinnen en woorden. Deze objecten hebben op hun beurt attributen die taalkenmerken vertegenwoordigen, zoals delen van spraak. Op het moment van schrijven bood spaCy voorgetrainde modellen aan voor Engels, Duits, Grieks, Spaans, Frans, Italiaans, Litouws, Noors BokmÃ¥l, Nederlands, Portugees en meerdere talen gecombineerd. Bovendien biedt spaCy ingebouwde visualizers die u programmatisch kunt aanroepen om een grafische weergave van de syntactische structuur van een zin of benoemde entiteiten in een document te genereren.   De spaCy-bibliotheek ondersteunt ook native geavanceerde NLP-functies die andere populaire NLP-bibliotheken voor Python niet hebben. Spa

Google Closure

   Closure Library De Closure-bibliotheek is een JavaScript-bibliotheek, vergelijkbaar met andere moderne producten zoals jQuery, Angular, Vue.js, Dojo en MooTools. De coderingsstijl en het gebruik van opmerkingen in de Closure-bibliotheek zijn op maat gemaakt voor Closure Compiler. In vergelijking met andere JavaScript-bibliotheken is het de belangrijkste onderscheidende factor van Closure Compiler. Een eenvoudig compressie-experiment ontdekte dat wanneer Closure Compiler wordt gebruikt in plaats van YUI Compressor, de Closure Lib-code met 85% kan worden verminderd, wat een enorme impact kan hebben op de codecompressiecapaciteit van de compiler.    De implementatie van de  closure bibliotheek richt zich ook op leesbaarheid en prestaties. Wees zuinig bij het maken van objecten, maar wees genereus bij het benoemen en opnemen van objecten. Het heeft ook een prachtig gebeurtenissysteem, ondersteuning voor klassen en overerving en verschillende UI-componenten, waaronder bijvoorbeeld een ri