Skip to main content


Showing posts from 2021

Setup Firebase Hosting and custom domain

  First  step: Create a new project in your account.. In command line firebase init Follow the steps in Wizard and choose the project that you want to use.  If you get some errors like  Error: It looks like you haven't used Cloud Firestore in this project before. Go to to create your Cloud Firestore database. % firebase init Or Error: Your project vacaturemailscom must be on the Blaze (pay-as-you-go) plan to complete this command. Required API can't be enabled until the upgrade is complete. To upgrade, visit the following URL You can fix this problem by using the firebase admin panel.  When Firebase initialization completes.  You can use command  firebase deploy So, now after first deploy you can add custom domain Add TXT records in your DNS so hosting can verify your ownership.  If you get this error message.  If you receive this message then you have to wait for DNS renewal for ma

max_result_window haystack or Elasticsearch DSL

Problems with Elasticsearch limit.. Solving problem with labeling.. When you filter more than 10000 objects you can't browse because there is a limit on elasticsearch. paginate-search-results is difficult in haystack.  If this index is not used so intensively then you can increase maximum in elastic.  Elasticsearch show indexes curl -X GET "" green open jobs_vacaturestoday     5 1     3622      0 52.1mb     26mb green open profile_vacaturestoday 5 1   769725      0  3.7gb   1.8gb curl -X PUT "" -H 'Content-Type: application/json' -d' {   "index" : {     "max_result_window" : 2000000   } } '

Config vim ~/.vimrc

If a lot of your work is done in VIM, it would be interesting to set it up a bit easier. Here is an example of a config that I often use. vim ~/.vimrc   set encoding=utf8 set paste set expandtab set textwidth=0 set tabstop=4 set softtabstop=4 set shiftwidth=4 set smartindent set backspace=indent,eol,start set incsearch set ignorecase set ruler set wildmenu set commentstring=\ #\ %s set clipboard+=unnamed syntax on

SyncfirebaseCloudFirestore Sync sources and publish updated category

Synchronize XML from different sources, to filter out duplicates and print static customer vacancies in files system. That is internal working method, we are not currently publishing scripts in open source projects because the project is still too raw.  On server ./crontab/syncjobs.bash It contains python3 syncfirebaseCloudFirestore python3 deleteold yes | python3 rebuild_index --using=default On client for firebase deploy ./publish_categories.bash If not installed. sudo pip3 install firebase-admin sudo pip3 install -U Jinja2 Old version slugify is not working in py3. sudo pip3 uninstall slugify sudo pip3 install python-slugify eslint: command not found /bin/sh: eslint: command not found npm ERR! code ELIFECYCLE npm ERR! syscall spawn npm ERR! file /Users/sergejdergatsjev/.cache/firebase/runtime/shell npm ERR! errno ENOENT npm ERR! functions@ lint: `eslint .` npm ERR! spawn ENOENT npm ERR! npm ERR! Failed at the functions@ lint script. npm

Change UFW config or disable Firewall on your server mount Raid

Sometimes it can happen that you have not configured your firewall correctly and locked yourself out so that the server is no longer accessible. Activate rescue CD and restart server with Linux 64 bit. Lsblk mkdir /media/mydrive mount /dev/md2 /media/mydrive ls /media/mydrive Now you can access config files in your main operating system. UFW configs you can find in Ubuntu vim /etc/ufw/user.rules So you can reach it in this way. vim /media/mydrive/etc/ufw/user.rules Then you can add rules with your own IP and restart the server… Or you can disable UFW with vim /media/mydrive/etc/ufw/ufw.conf

CRM installation on test server

In General, we can say that the bakup and the code are fully functional. I deployed it in one day without roughly straining. Generally 2 - 3 hours if needed. apt-get install postgresql sudo -u postgres createuser -D -A -P username sudo -u postgres createdb -O user db psql -U user -W pwd -h localhost -p 5432 -f crm.sql git clone https:// apt install libcurl4-openssl-dev libssl-dev pip3 install -r  requirements.txt apt install python3-dev apt-get install  python-dev apt-get install build-essential Troubleshooting When i start runserver then.  raise MissingDependency( haystack.exceptions.MissingDependency: The 'elasticsearch5' backend requires the Elastic wget -qO - | sudo apt-key add - apt-get install apt-transport-https Echo "deb stable main" | sudo tee /etc/apt/sources.list.d/elastic-7.x.list apt-get update && sudo apt-get install elasticsearch Not helped  pip3 ins

Static site generation from keyword.txt file and publish it in Firebase

  From keywords.txt file we will generate templates to fill it with content.  vim keywords.txt Add one keyword per line..  For example,  Aanbieding Binnendeuren Aanbiedingen binnendeur python Then you will see generated templates. For example: input/02-aanbieding-binnendeuren.html input/03-aanbiedingen-binnendeur.html Fill generated templates with content.  Then  vim  and update from .. to counts..  This will read file names from the filesystem and generate a sitemap in the current input directory.  input/sitemap02-10.html Then run python include script to make AMP pages from templates.  Python  Then publish all these generated files.  firebase deploy And add it in the Git directory so you can follow changes. 

Firebase install CLI Mac

 You can install the Firebase CLI using a method that matches  your  Mac operating system, experience level, and/or use case. No matter how you install the CLI, you can access the same features and firebase commands.  So this is the command to install it. curl -sL | bash    

Python3 on Mac

First install development tools then you can do the following   sudo python3 -m pip install --upgrade pip  or somthing what you need in python3 sudo python3 -m pip install slugify

Read-only file system hardware fault in RAID

When you are getting errors from a file system like this..   cd /hom-bash: cannot create temp file for here-document: Read-only file system -bash: cannot create temp file for here-document: Read-only file system -bash: cannot create temp file for here-document: Read-only file system You have new mail. Last login: Tue Jun 15 13:16:58 2021 from mktemp: failed to create file via template ‘/tmp/virtualenvwrapper-initialize-hook-XXXXXXXXXX’: Read-only file system touch: cannot touch ‘’: No such file or directory ERROR: virtualenvwrapper could not create a temporary file name. It could be a hardware problem. Due to a hardware fault, you have to replace your server while taking over the hard drives. Furthermore both of my drives are faulty with one of them not even recognised by the system anymore. Mount your hard drive and back up my data and accept the full configuration loss for the server in order for us to replace them. When my server has been booted into our rescue system

Live streaming from video camera to laptop OBS Facebook, Youtube, Twitch, RTMP server

I did a live stream a few days ago. I would save and share configuration because it was not immediately obvious how to do it.. Config camera for streaming it will be in Menu something like REC SET and choose AVCHD 1080 / 50i or XAVC S HD but or XAVC S 4K if your network connection can stream it. In my case AVCHD compression was the best choice. First step is getting video input to computer for that you need HDMI cable and a CAM Link This link is very good because all possible cameras will fit in this setup. HDMI is very useful. Next thing we need OBS it will convert and optimize streams for channels. Install it on your computer and configure it like in the screenshot below. Facebook stream get it from your page or account Pay attention! You can use the browser and then choose a cam link there but you can't configure speed and resolution of the stream and can get problems with bandwidth. Maybe that is a good choice to stream webcam but not successful if you want to

Other storage on your Macintosh HD

  If you look at your Mac's storage capacity, part of it will consist of an "Other" bar. At first glance, you can't tell what exactly these files are, which also makes it very difficult to delete them. The 'Other' storage bar can consist of several things. Documents, plug-ins and the cache are stored here. These files are important to keep your Mac running smoothly, but if you're not careful this part can fill up quickly. Especially if you have a MacBook with limited storage capacity, it may sometimes be necessary to slim down the amount with 'Other' storage. To do this you will first need to find the Other category. Here's how you can see how much space is taken up by other storage: Click the Apple logo at the top left of your Mac screen. Choose About This Mac > Storage. You will now see how your Mac storage is composed, including 'Others'   It may take a while before the information is visible. Now that you know how much space the

How do I add an extra / additional hard drive?

The technical support team can add an extra SSD. For installing the drive, we need to shut down your server (downtime ~ 30 min). If as requested we have added the drive to the server and restarted it. You can format the filesystem and add drive to your OS. In this case Ubuntu server. lsblk NVMe (Non-Volatile Memory Express) is a specification for accessing drives through PCI Express; it is tailored to meet the needs of non-volatile memory. For example, the connection via PCIe Gen3 x4 (32 GBit/s) provides five times faster access speed and lower latency than the traditional SATA 3 (6GBit/s) interface. Due to the high degree of parallelism and low energy consumption, they allow users to meet current and future memory needs. mkfs.ext4 -j -L backup /dev/nvme2n1  apt-get install  nvme-cli Getting information about hard drive  nvme list nvme id-ctrl /dev/nvme2n1|egrep "sn |mn " blkid /dev/nvme2n1 /dev/nvme2n1: LABEL="backup" UUID="378cdb81-98c8-44b7-95a2-22bd60a0707b

Perform commands with Python over ssh

I need to run some commands on a remote computer. Manually, I will log in with ssh and run the command. How would you automate this in Python? You  need for example to log in to the remote computer with a (known) password, so I can't just use cmd = ssh user @ remotehost, and then use the paramiko module for authentication and automation on the server side.  Very simple example: ssh = paramiko.SSHClient() ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect(server, username=username, password=password) ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command(cmd_to_execute) If you are using ssh keys, do: k = paramiko.RSAKey.from_private_key_file(keyfilename) k = paramiko.DSSKey.from_private_key_file(keyfilename) ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) ssh.connect(hostname=host, username=user, pkey=k) Or  commands.getstatusoutput("ssh machine 1 'your script'") Paramiko is a Python (2.7, 3.4+) implementation of the SSHv

Can't Use apt i.e. /boot is 100% full

  Note: Only in cases where cleanup cannot be done with apt due to 100% full/boot Get a list of kernel images and determine which will be removed. This command will display the installed kernels, except for the currently running kernel dpkg --list 'linux-image*'|awk '{ if ($1=="ii") print $2}'|grep -v `uname -r` linux-image-4.15.0-46-generic linux-image-4.15.0-50-generic linux-image-4.15.0-51-generic linux-image-4.15.0-52-generic linux-image-4.15.0-54-generic Linux-image-4.15.0-55-generic First, get the list of currently running Linux kernels and run: v="$(uname -r | awk -F '-virtual' '{ print $1}')" 4.15.0-47-generic Use brace expansion to keep your boot clear, you can write a command to delete all files in /boot that are not related to the kernel. Remember to exclude the current and two latest kernel images. In the example above, it is rm -rf /boot/*-4.15.0-{46,50,51,52,54,55}-* Or move these files in another directory. mv /boot/*-4

Sendgrid simple integration on localhost python TinyDB sendgridclient

  Configure mail to send email from a local host without any platform. Sometimes, you need a quick application to test an idea, but don't want to invest in a separate server with DKIM, SPF and other tools installed, but only need to test a business case for 2-3 months. The perfect tool for sendgrid that you can control via your laptop. There is no need to pay for the server and django, and there is no need to use a completely naive integration without a code solution. This API integration is at least 3 times cheaper than any other solution on the market, and it can be used for professional purposes. Note this integration only works for small lists of about 200k-300k maximum. Otherwise database is too slow and updates are slower than postgres or other databases.   Register your account on the Sendgrid website. Then, at this point in the API configuration wizard are out of date docs. Ignore it!  We use the code found on github. See the example below.   This is an old version in docs