Upload Files on Google Drive using CURL

Initiate an automated backup on Google Drive using shell script.

Steps :

a) Generate client id and secret key.

b) Add Google logged in user as a Test user in consent screen.

c) Enable Google drive API

d) Generate Authorization Code (One Time)

https://accounts.google.com/o/oauth2/auth?client_id=xxxxxxxx&redirect_uri=http://localhost&response_type=code&scope=https://www.googleapis.com/auth/drive&access_type=offline

e) Generate Refresh Token (One Time)

curl --request POST --data "code=xxxxxxxx&client_id=xxxxxxxxxxxx&client_secret=xxxxxxxxxxxx&redirect_uri=http://localhost&grant_type=authorization_code" https://oauth2.googleapis.com/token

f) Generate Access Token (Always)

curl --request POST --data "client_id=xxxxxxxxxxx&client_secret=xxxxxxxxxxxxxxx&refresh_token=xxxxxxxxxxxxx&grant_type=refresh_token" https://oauth2.googleapis.com/token

g) Upload File on Google Drive (Always)

curl -X POST -L \
    -H "Authorization: Bearer xxxxxxxxxxxxxxxxxxxxxxxxx" \
    -F "metadata={name : 'imagename.png'};type=application/json;charset=UTF-8" \
    -F "file=@imagename.png;type=image/png" \
   "https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart"

Note : Refresh Tokens expire in 1 week if your app is not set as production. Change publishing status of your app from testing to production to use your refresh token always.

The Publishing Status option can be found in 'Oauth Consent Screen' which is under API & Services.

See complete instructions in this video. 

 

Alert notification on Ubuntu Linux about certain task after given time duration

This happens always with me. I boil milk regularly and it always got boiled over. The problem is I cannot stand 10 mins while watching it, not even with phone. Generally I leave the place immediately and start finishing my pending work on the laptop. I tell myself, I am going to check after 5 min but I get occupied in the work and many times I remember when I get burning smell. Not only it is wastage of milk but also the expensive LPG.

Now I have found the solution only because I start working on my Ubuntu laptop after burning the stove, if I would be busy with my phone, this solution would not work and I would have to find some alarm or similar feature.

I run an alert command which opens a GUI pop-up box. The challenge here was, when I first implemented the command, it opens the pop-up but in the same terminal window where command is running. Of course I was working in some other window so I would never know about the pop-up. Now this solution opens the pop-up over any window and I am able to know about my task.

I had to install a package named wmctrl

sudo apt install wmctrl

Now I run the command

sleep 480; (sleep 1 && wmctrl -F -a "top" -b add,above) &(zenity --info --title="top" --text="Watch the Milk")

If I want alert pop up after 8 min. 480 is in seconds . If I need alert after 5 min I use command.

sleep 300; (sleep 1 && wmctrl -F -a "top" -b add,above) &(zenity --info --title="top" --text="My Alert Message")

It opens the pop up with message over any window and it reminds me about the task.

Jenkins - No Plugin available for Execute Shell script on remote host

You were using this plugin in some previous versions of Jenkins but you are not getting ssh plugin after upgrade in the newer versions.

This plugin is very useful if you want to run the script on remote host.

This is the solution for you.

Download hpi file

https://plugins.jenkins.io/ssh/#releases

and import it using

Manage Jenkins > Manage Plugins > Advanced > Upload

 

Restart Jenkins, the option will be there in the Build Environment and Build.

mysql not inserting int greater than 2147483647

Solution :

Your column type might be int or similar. Use bigint as column type.

You will not face the issue.

Automated Script to Increase Views for a YouTube Video

YouTube's analytics system has been made with very advanced AI. When users try to gain the views with unauthorized methods, it detects very easily. It finds if the video was opened using browser or app, IP of the client, how many times same video was played, time duration of the video, logged in username, if user was not signed in then some parameters to detect authenticity and many other parameters which find out that the viewer is genuine or not.

Users want to increase the views immediately, they are impatient that's why they play same video multiple times, it might increase the view count in the studio immediately but after sometimes or days, the views will be reduced and that's why genuine views are better for you. They are less but they have high watch time and CTR (click through rate), fake views will not do any good for your channel.

We are going to see two scripts which use Google Chrome Driver and Proxy Details to create genuine environment. This may be a slow process to increase the view count but first it is fully automated, at least you need not open video in multiple tabs manually and second developer has tried his best to make the process authentic.

Here are two automated methods which increase the views of the video but there is no guarantee that YouTube will not detect and reduce the count.

Script : 1

Install 'Tor' before running the script.

git clone https://github.com/but3k4/youtube-views.git

cd youtube-views

pip install -r requirements.txt

python3 bot.py --visits 2 --url https://www.youtube.com/watch?v=ajT1Qbm1wGE --verbose

Script : 2

git clone https://github.com/MShawon/YouTube-Viewer.git

cd YouTube-Viewer

pip install -r requirements.txt

python3 youtube_viewer.py

JMeter - Regular Expression Extractor - CSRF Token Login - Django and Laravel Login with JMeter

CSRF Token is a common practice to prevent the Cross Site Request Forgery Attacks. If you are testing such web applications where a CSRF token is sent with the http request, you might find difficult to achieve the results with JMeter.

Most common hurdle is the Login Page because this is the place where you find CSRF token parameter with username and password. Actually CSRF Token should be used with every request where data is getting inserted in the database but generally developers do not use in the other web forms.

So How to Log into the application which has CSRF token implemented in the login page.

Here is the complete guide to handle the CSRF Tokens in the JMeter.

JMeter - Regular Expression Extractor - CSRF Token Login - Django and Laravel Login with JMeter

CSRF Token is a common practice to prevent the Cross Site Request Forgery Attacks. If you are testing such web applications where a CSRF token is sent with the http request, you might find difficult to achieve the results with JMeter.

Most common hurdle is the Login Page because this is the place where you find CSRF token parameter with username and password. Actually CSRF Token should be used with every request where data is getting inserted in the database but generally developers do not use in the other web forms.

So How to Log into the application which has CSRF token implemented in the login page.

Here is the complete guide to handle the CSRF Tokens in the JMeter.

Docker - Attach port to a running container | Map a port of host with the container

When you run a container, you attach port with the run command as docker does not provide any option to attach a port for a running container.

Generally you use parameter '-p' with run command to map the host port with the container

docker run -it -p 80:80 -p 3306:3306 -v /var/www/html:/var/www/html ubuntu:18.04 /bin/bash

If you want to attach a port or multiple ports to a running container, here is the solution.

1. Stop the container and cd into the container directory

cd /var/lib/docker/containers/d260db74672bf96c07536835229b1b3609c74f24ba54e4c4d0e314b24d01ae19

(long string is container id)

2. Edit files configv2.json and hostconfig.json

3. Add your extra require port settings in these two files
hostconfig.json

"PortBindings":{"21/tcp":[{"HostIp":"","HostPort":"21"}],"27017/tcp":[{"HostIp":"","HostPort":"27017"}],"3306/tcp":[{"HostIp":"","HostPort":"3306"}],"80/tcp":[{"HostIp":"","HostPort":"80"}]},

configv2.json

"ExposedPorts":{"21/tcp":{},"27017/tcp":{},"3306/tcp":{},"80/tcp":{}},

Here we have attached four host ports with the container.

4. Restart Docker, start container and log into it. You should be able to access the service of container in the host using attached port.

Using this method you can connect single port or multiple ports.

Note : You can map one port of host with the other of container. Example you can map port 89 of host with port 80 of docker container. This way you can access web server (apache / nginx ) service of container on port 89 of host.

You can see practical example in this video to understand the steps better.

 

Docker - Attach Volume to a running container | Map a directory of host with the container

When you run a container, you attach volume with the run command as docker does not provide any option to attach a volume for a running container.

Generally you use parameter '-v' with run command to map the host directory with the container

docker run -it -p 80:80 -p 3306:3306 -v /var/www/html:/var/www/html ubuntu:18.04 /bin/bash

You want to attach the volume because you want to access the contents of host directory in the container.

If you want to attach a volume or multiple volumes to a running container, here is the solution.

1. Stop the container and cd into the container directory

cd /var/lib/docker/containers/d260db74672bf96c07536835229b1b3609c74f24ba54e4c4d0e314b24d01ae19

(long string is container id)

2. Edit files configv2.json and hostconfig.json

3. Add your extra require volume settings in these two files
hostconfig.json

{"Binds":["/var/www/html:/var/www/html","/opt:/opt"],

configv2.json

"MountPoints":{"/var/www/html":{"Source":"/var/www/html","Destination":"/var/www/html","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/opt/lampp/htdocs/InvoicePlane","Target":"/var/www/html/InvoicePlane"},"SkipMountpointCreation":false},"/opt":{"Source":"/opt","Destination":"/opt","RW":true,"Name":"","Driver":"","Type":"bind","Propagation":"rprivate","Spec":{"Type":"bind","Source":"/opt","Target":"/opt"},"SkipMountpointCreation":false}},

Here we have mounted two host directories inside the container. /var/www/html and /opt of host is visible in the same directory of the container.

4. Restart Docker, start container and log into it. Host Directory content should be available in the container.

Using this method you can connect single volume or multiple volumes.

Note : You can map one directory of host in the other directory of container. Example you can map /var/www/html of host inside /opt/lampp/htdocs of docker container or vice versa.
 

You can see practical example in this video to understand the steps better.

 

 

Run Skype as a root user in Linux Ubuntu CentOS Fedora

Skype is not working for root in ubuntu

Now Skype does not work for root user in Linux.

In recent versions, they have also removed the option --no-sandbox for root user.

Here is the solution to open Skype as root user in Linux without any security loophole and without --no-sandbox

Once you have installed latest version of Skypeforlinux in your Ubuntu, CentOS, Fedora or any other distribution, now follow the tutorial to open the skype as a root user in Linux.

 

Run Skype version 60.0+ on Centos 7, Centos 8, Ubuntu 20.04 or Ubuntu 18.04 as a root

Run Skype version 70.0+ on Centos 7, Centos 8, Ubuntu 20.04 or Ubuntu 18.04 as a root

Skype crashes when I run it as a root user in Linux | skypeforlinux not launch anymore | Skype for linux as a root user in Ubuntu | Open Skypeforlinux as a root user | skype not running as root user in linux | skype now working with sudo

Django Migration Error - '%s=%s' % (k, v) for k, v in params.items(),

Django Migration Error :

  File "/home/pavi/ldap/lib/python3.8/site-packages/django/contrib/admin/__init__.py", line 4, in <module>
    from django.contrib.admin.filters import (
  File "/home/pavi/ldap/lib/python3.8/site-packages/django/contrib/admin/filters.py", line 10, in <module>
    from django.contrib.admin.options import IncorrectLookupParameters
  File "/home/pavi/ldap/lib/python3.8/site-packages/django/contrib/admin/options.py", line 12, in <module>
    from django.contrib.admin import helpers, widgets
  File "/home/pavi/ldap/lib/python3.8/site-packages/django/contrib/admin/widgets.py", line 151
    '%s=%s' % (k, v) for k, v in params.items(),

Solution :

Update file at line number 151

Add the Parenthesis
  File "/home/pavi/ldap/lib/python3.8/site-packages/django/contrib/admin/widgets.py", line 151

('%s=%s' % (k, v) for k, v in params.items()),

Postman Error : The requested URL /upload/drive/v3/files was not found on this server. That’s all we know.

Error : The requested URL /upload/drive/v3/files was not found on this server. That’s all we know. 

Solution :

The issue is in the Headers section.
Add always default header Host, Do not skip it while sending the request.

Django sqlite Migration error

Error : django.db.utils.OperationalError: no such table: main.auth_user__old

Solution :
Do not use Django 2.0.0 with sqlite database. The issue is fixed in later versions of Django.
Django 2.1.5 does not have this error.

pip install django==2.1.5

Google Drive api uploads file name as "Untitled"

If you are uploading a file on google drive using Google API, you might face incorrect filename issue. If uploaded filename is not the real name of the file, its name is Untitled, here is the solution for you.

You need to send metadata json with the post data. Metadata json contains filename and content type. When google api finds metadata json with the post data, it uploads the file with correct name.

Here is the tutorial to upload the file on the google drive using postman.

https://linuxamination.blogspot.com/2021/08/upload-files-on-google-drive-using.html

If you are using curl or any other scripting method to upload the files on drive, you need to send metadata json with the post data.

It will solve the issue.

Complete Solution :

Upload Files on Google Drive using Postman

SR. No.		Parameter	Value
1		Request Type	Post
2		URL	https://www.googleapis.com/upload/drive/v3/files?uploadType=multipart
3.1	Authorization	Authorization Type	Oauth2.0
3.2		Available Token	<here-is-the-token>
3.3		Token Name	Authorization
3.4		Grant Type	Authorization Code
4.1	Headers	Authorization	Bearer <here-is-the-token>
4.2		Content-Type	multipart/form-data; boundary=<calculated when request is sent>
4.3		Content-Length	<calculated when request is sent>
4.4		Host	<calculated when request is sent>
5.1	Body	Key	Empty
5.2		File	filename.json
5.3		Key	Empty
5.4		File	file.png

Screenshots : (Click on image to see bigger view)

 

a) filename.json

    Content-Type: application/json; charset=utf-8
    {
      "name": "file.png",
      "mimeType": "image/png"
    }

b) file.png

    An image file

Run Google Chrome as root in Linux without no sandbox

If you are facing issue while running Google Chrome or other softwares as a root user on Linux systems like Ubuntu, CentOS, KALI, Fedora etc, here is the guide for you.

# Create a User for Google Chrome or other software

useradd chrome -s /bin/bash -d /home/chrome -m

# Create a shell script to run Google Chrome as root

nano /root/Documents/chrome.sh

# Add following content in the script

xhost +
sudo -H -u chrome DISPLAY=:0 google-chrome

# Execution Permission to script

chmod a+x /root/Documents/chrome.sh

 # Run the script by command line

bash /root/Documents/chrome.sh

or double click on the sh file. 

You should be able to run Chrome or other softwares like skype as a root user.

Django Authentication using LDAP Users

 Django Login with LDAP users

LDAP was designed to share common access details between applications. If a user is created in the LDAP server, the user does not need to register in other applications. You can use LDAP user credentials to login to the other apps.

All major applications provide configuration options with LDAP. Suppose you use Gitlab for your code repository manager. Gitlab provides configuration option with LDAP, it means you can log into the Gitlab with LDAP users. You do not need to register in the Gitlab for your login. Of course you can register there to login but why to do multiple registrations when you can manage multiple applications access using single database.

Similarly if you are developing a web application in Django. You can use Postgres or Sqlite database for your application but if the users who are going to register in the application already registered in LDAP, we can simply configure Django with LDAP and use LDAP credentials to log into Django Application.

Here is the method to configure Django with LDAP.

1. Create a virtual environment with Python. Here I have used Python 3.8.2

virtualenv djangoldap -p /usr/bin/python3

2. Activate the environment.

cd djangoldap && source bin/activate

3. Install Django Auth Ldap Module. I have used version 2.0.0

pip install django-auth-ldap==2.0.0

4. Now install Django 2. You can also install Django 1. It depends on your requirements.

pip install django==2.1.5

5. Now create a Django project using django-admin command. My project name is ldappro.

django-admin startproject ldappro

6. Run the Migrations

cd ldappro && python manage.py migrate

7. Add the LDAP configuration in the settings.py at bottom of the page below STATIC_URL

import ldap
from django_auth_ldap.config import LDAPSearch, LDAPGroupQuery,GroupOfNamesType,PosixGroupType

AUTH_LDAP_SERVER_URI = 'ldap://localhost'
AUTH_LDAP_BIND_DN = 'cn=admin,dc=example,dc=com'
AUTH_LDAP_BIND_PASSWORD = 'YourLDAPPassword'
AUTH_LDAP_USER_SEARCH = LDAPSearch('dc=example,dc=com',ldap.SCOPE_SUBTREE, '(uid=%(user)s)')
AUTH_LDAP_GROUP_SEARCH = LDAPSearch('dc=example,dc=com',ldap.SCOPE_SUBTREE, '(objectClass=top)')
AUTH_LDAP_GROUP_TYPE = PosixGroupType(name_attr="cn")
AUTH_LDAP_MIRROR_GROUPS = True

    # Populate the Django user from the LDAP directory.
AUTH_LDAP_REQUIRE_GROUP = "cn=enabled,ou=groups,dc=example,dc=com"

AUTH_LDAP_USER_ATTR_MAP = {
        "first_name": "givenName",
        "last_name": "sn",
        "email": "mail",
        "username": "uid",
        "password": "userPassword",
}
AUTH_LDAP_PROFILE_ATTR_MAP = {
        "home_directory": "homeDirectory"
}
AUTH_LDAP_USER_FLAGS_BY_GROUP = {
        "is_active": "cn=active,ou=groups,dc=example,dc=com",
        "is_staff": "cn=staff,ou=groups,dc=example,dc=com",
        "is_superuser": "cn=superuser,ou=groups,dc=example,dc=com"
}
    
AUTH_LDAP_ALWAYS_UPDATE_USER = True
AUTH_LDAP_FIND_GROUP_PERMS = True
AUTH_LDAP_CACHE_TIMEOUT = 3600
    
AUTH_LDAP_FIND_GROUP_PERMS = True
    
    # Keep ModelBackend around for per-user permissions and maybe a local
    # superuser.
AUTHENTICATION_BACKENDS = (
        'django_auth_ldap.backend.LDAPBackend',
        'django.contrib.auth.backends.ModelBackend',
)

8. Now run the python server.

python manage.py runserver

9. Open phpldapadmin and Create two 'Generic : Organizational Unit' users and groups .

10. Add a generic : Posix Group 'Active' under the OU groups and then add a generic : user account 'John Doe' under OU users. Select GID 'Active' while creating the user account.

11. After creating the user account, add two more objectClasses for the user i.e. person and organizationalPerson.

12. Add a generic : Posix Group 'Enabled' under groups with enabled checkbox of user jdoe.

13. Add a generic : Posix Group 'Staff' under groups with enabled checkbox of user jdoe.

14. Add a generic : Posix Group 'superuser' under groups with enabled checkbox of user jdoe.

15. Once user is added in the all the Posix Groups, we can log into the django using the LDAP user. 

 16. Now use the credentials of the user. Username of the user can be found on the user account page of the user and password was already set by you while creating the user.

You should be able to loginto the django using LDAP Credentials.

You can find complete solution in the following video.

JMETER LDAP Extended Request Error - javax.naming.NamingException: context is null

If you are creating a test plan to perform load testing on LDAP server using JMETER, you might have added a sampler 'LDAP Extended Request' in the Thread Group of your Test Plan.

While performing operations such as Add Test, Modify Test, Search Test or Delete Test, if you are getting following error

javax.naming.NamingException: context is null

 Here is the solution for you.

You should not use the sampler 'LDAP Extended Request' for your Performance Test Plan. Add Test, Modify Test and other necessary operations can be performed by another sampler 'LDAP Request' successfully. You will not get above error using this sampler.

Here is the complete guide to perform the Load Testing on LDAP Server using JMETER.

 

LDAP Server Performance Testing by JMETER | LDAP Server Load Testing by JMETER

A guide to perform Load Testing on your LDAP Server by JMETER.

Watch the video to learn all the necessary steps to perform Load Testing on LDAP Server. This is an easy and simple method. Any beginner or intermediate QA Engineer can learn to do performance testing on LDAP Server.

It does not require any scripting. A very basic knowledge of JMETER is enough to perform this automated task.

php LDAP admin Import Sample Users | Import Sample Data LDAP Server

LDAP Server : Import Test Users

php LDAP admin is a user friendly client to operate LDAP server. If you want to use LDAP user data for your task, you might need dummy users. There are multiple reasons such as security policy when real users' data cannot be used for any automated task. 

Here we will see how you can import sample or dummy user data which is same as real data.

Steps :

1. Download zip file of LDIF Generator from here.

2. Extract it and open the software using command

java -jar LDIFGen.jar

3. Add Domain Component (dc) name of your LDAP server in the field "Base added to Generate Records". In my case it was

dc=example, dc=com

4. Number of records can be 500 or 5000 depends on your requirement.

5. Field 'Directory where input data is stored' : This is the path of the 'data' directory which you can find in the extracted LDIF generator folder. 

Make sure D is capital in the path name if it is capital in the extracted folder.

6. Output directory path can be anywhere on the system. Make sure you have write permission in the folder.

7. Click on 'Run' button. An output file output.ldif file will be generated.

8. Open php LDAP Admin and click on import option in the left menu. Select the output.ldif file and click on Proceed.

9. All sample users will be imported into the LDAP server.

You can see the complete procedure in the following video.

 

 

php LDAP Admin - Fatal error: Cannot redeclare password_hash

phpldapadmin - Fatal error: Cannot redeclare password_hash() in /var/www/html/phpldapadmin/lib/functions.php on line 2225

Make sure you have php 7.0 for phpldapadmin. The solution is tested on php 7.0

Solution :

If you install the phpldapadmin using command

sudo apt install phpldapadmin

in Ubuntu or any other Debian based system. You will not get this error  

Same is applied for Centos or Fedora.

sudo yum install phpldapadmin

This is an easy solution that's why you should not use the downloaded zip package. Use default package installation on command line. There should be less chances that you get the error.

But if you have downloaded phpldapadmin 1.2.2 or 1.2.3 zip file from sourceforge.net, you might get this error. If you have any other version, still this solution will fix the error.

Make sure you have php 7.0 where you have deployed the downloaded package of phpldapadmin.

Solution :

Change text password_hash to password_hash_custom in the all files of the lib directory.

This sed command will do the trick. Make sure you are in the phpldapadmin directory before executing the command.

sudo sed -i 's/password_hash/password_hash_custom/g' lib/*

After updating the text password_hash, the above error will be fixed but there are chances that you might get following error.

E_WARNING: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead

To solve this error update the whole code of the function function dn_unescape($dn) in /var/www/html/phpldapadmin/lib/functions.php with

function dn_unescape($dn) {
    if (DEBUG_ENABLED && (($fargs=func_get_args())||$fargs='NOARGS'))
        debug_log('Entered (%%)',1,0,__FILE__,__LINE__,__METHOD__,$fargs);

    if (is_array($dn)) {
        $a = array();

        foreach ($dn as $key => $rdn) {
            $a[$key] = preg_replace_callback('/\\\([0-9A-Fa-f]{2})/',
                function ($m) {
                    return ''.chr(hexdec('\\1')).'';
                },
                $rdn
            );
        }

        return $a;

    } else {
         return  preg_replace_callback('/\\\([0-9A-Fa-f]{2})/',
             function ($m) {
                return ''.chr(hexdec('\\1')).'';
            },
            $dn
        );
    }
}

This will solve the front error but if you click on the tree icon to maximize the tree items, you might get following error again.

E_WARNING: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead

To solve the error completely update the whole code of the private function function dn_unescape($dn) in /var/www/html/phpldapadmin/lib/ds_ldap.php with

    private function unescapeDN($dn) {
        if (DEBUG_ENABLED && (($fargs=func_get_args())||$fargs='NOARGS'))
            debug_log('Entered (%%)',17,0,__FILE__,__LINE__,__METHOD__,$fargs);

        if (is_array($dn)) {
            $a = array();
            foreach ($dn as $key => $rdn) {
                $a[$key] = preg_replace_callback('/\\\([0-9A-Fa-f]{2})/',
                    function ($m) {
                        return ''.chr(hexdec('\\1')).'';
                    },
                    $rdn
                    );
            }

            return $a;

        } else
             return preg_replace_callback('/\\\([0-9A-Fa-f]{2})/',
                function ($m) {
                    return ''.chr(hexdec('\\1')).'';
                },
                $dn
            );
    }

It should fix all errors of function.php and you should be able to use phpldapadmin smoothly.

You can see the complete solution here.

 

ejabberd Error - CRASH REPORT Process with 0 neighbours exited with reason

CRASH REPORT Process <0.645.0> with 0 neighbours exited with reason: {process_limit,{max_queue,xxxx}} in p1_fsm:terminate/8 line 755

Solution : 1

Change @all@ to @online@ in shared roster group. 

 

Now it does not disconnect immediately because it loads only online users instead of all registered users. 

 

@all@ was showing error because ejabberd was loading all registered users in the list that's why it was getting crashed but @online@ is not giving any error now because all registered users are not online yet. 

 

If you have more than 1000 registered users. This solution might work for you as all 1000 users might not be online at once and there are high chances that system will not have enough load.

Solution 2 :
Update following attribute in the ejabberd config file conf/ejabberd.yml

max_fsm_queue: 10000

You will find the attribute under "port 5222:" section

listen:
  -
    port: 5222
    module: ejabberd_c2s
    certfile: "/home/ubuntu/ejabberd-16.06/conf/server.pem"
    starttls: true
    resume_timeout: 0
    protocol_options:
      - "no_sslv3"
    max_stanza_size: 65536
    shaper: c2s_shaper
    access: c2s
    max_fsm_queue: 100000

I have tried this and I did not get above error for 2000 users even if @all@ is selected in shared roster group.

I would suggest you to implement both solutions.

ffmpeg - Create a Video from Images

To create a video from images, you must have used software like Kdenlive and Shotcut. Yeah it is quite easy and fast in Shotcut but you cannot use UI softwares everytime for your task.

If you need to automate this task, you must need scripting. ffmpeg is an excellent command line tool to perform audio and video jobs.

Suppose you have six-seven images which you want to convert in a video, you have to consider these two important points.

1) Images should not appear and disappear abruptly. They should have some effect. Fade in and Fade out is the best effect you can add for your video.

2) All images should have same size, you will find this important when you will see a video of images with different size.

Here I have used image names 1.jpg, 2.png, 3.png, 4.png, 5.png, 6.png and 7.png. You can use jpeg/jpg images also. I have used first image as a black image, it provides correct effect of fade in when second image (originally my first) appears. Resolution of my images are 1312 x 706

This is the command to generate video from images.

ffmpeg -loop 1 -t 5 -i 1.jpg -loop 1 -t 5 -i 2.png -loop 1 -t 5 -i 3.png -loop 1 -t 5 -i 4.png -loop 1 -t 5 -i 5.png -loop 1 -t 5 -i 6.png -loop 1 -t 5 -i 7.png -filter_complex "[0:v]fade=t=out:st=4:d=1[v0]; \
 [1:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v1]; \
 [2:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v2]; \
 [3:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v3]; \
 [4:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v4]; \
 [5:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v5]; \
 [6:v]fade=t=in:st=0:d=1,fade=t=out:st=4:d=1[v6]; \
 [v0][v1][v2][v3][v4][v5][v6]concat=n=7:v=1:a=0,format=yuv420p[v]" -map "[v]" ouptput.mp4

First try with seven images for creating a video and make good understanding of command. Then you can reduce or increase number of images, you will find where exactly you have to make changes in the command.

php 5.3 / 5.4 on Ubuntu 20.04 / 18.04

Deploy php 5.3 or php 5.4 web application on latest Ubuntu LTS 20.04 or Ubuntu 18.04

If you want to deploy your older php application on latest Ubuntu or Centos server, 32 bit library error can be an issue. You might have tried to download and install older version of xampp but your newer 64 bit OS does not support 32 bit xampp hence you are not able to install it.

Do not worry about the issue. Here is the solution for you.

You need to install xampp inside a docker container and deploy your application there.

1. Install docker.

There are plenty of guides available which can help you to install docker on your Linux OS. Once docker is installed, follow the guide.

2. Pull an Ubuntu 12.04 image using command.

docker pull ubuntu:12.04

3. Launch a container. Here we need to map host system's port 80 with docker container's port 80 and /opt folder of host system with /opt folder of container. You can choose different port and directory of host system also.

But if you choose port 80 of host system, make sure no service is running on it. Kill all the services of port 80 of host and launch a container with mapped port 80 and mapped volume /opt using following command.

docker run -i -t -p 80:80 -v /opt:/opt ubuntu:12.04 /bin/bash

4. Now you are inside the container. You can confirm by command `uname -m` that it is 32 bit, now you can install xampp of required php version.

To reach inside the directory /opt of container, Run command

cd /opt; ls

You should be able to see all the folders of /opt of your host system in the container.

5. Now open your browser and download the xampp of php 5.3.5 using following url.

https://sourceforge.net/projects/xampp/files/XAMPP%20Linux/1.7.4/xampp-linux-1.7.4.tar.gz

Extract the compressed file and copy folder lampp in /opt.

If you have an existing folder lampp in /opt, make sure you have renamed it before copying php 5.3.5 lampp in the /opt, otherwise it can overwrite your existing lampp setup.

6. Start the lampp in the container terminal using command

/opt/lampp/lampp start

If you have closed the container terminal, login again using following commands.

sudo service docker start

sudo docker start `sudo docker ps -a | grep "12.04" | awk '{print $1}'`

sudo docker exec -t `sudo docker ps -a | grep "12.04" | awk '{print $1}'` /bin/bash

/opt/lampp/lampp start

7. Once lampp is started successfully, open a tab in the browser and open url

http://localhost/

You should be able to open xampp dashboard. Verify php version on phpinfo page.

Deploy the application in the same way you deploy on your host system.

Note : 

1. If you want to shift on existing lampp, stop the container and rename the current lampp as 12.04 and existing lampp as lampp in /opt.

2. Whatever changes you make in lampp folder of 12.04 in /opt host system, it will be updated in the container because /opt of host system is mapped with /opt of container.

 

Docker rename a "none" image

If name of a docker image is imported as <none>, you can rename it quite easily.

Run command in the host.

docker tag imageid imagename

Where imageid is id of the image which is showing <none> and image name is the new name of the image which you want to replace with <none>.

You can see the solution in the following video.

Docker Error - debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module)

Docker Error :

debconf: (Can't locate Term/ReadLine.pm in @INC (you may need to install the Term::ReadLine module)
debconf: delaying package configuration, since apt-utils is not installed.

Solution :

Run commands in the ubuntu container where you are getting the error

apt install apt-utils

echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections

Docker Error - Error initializing network controller: could not delete the default bridge network

Docker service is not starting because of following error.

Starting Docker Application Container Engine...
level=info msg="[graphdriver] using prior storage driver \"aufs\""
level=info msg="Graph migration to content-addressability took 0.00 seconds"
level=info msg="Firewalld running: false"
level=fatal msg="Error starting daemon: Error initializing network controller: could not delete the default bridge network: network bridge has active endpoints"
docker.service: Main process exited, code=exited, status=1/FAILURE
Failed to start Docker Application Container Engine.
docker.service: Unit entered failed state.
docker.service: Failed with result 'exit-code'.

Solution :
Remove /var/lib/docker/network folder.

I moved the folder to home dir and restarted docker as I thought if something is broken, I can replace it again to its original place but nothing was broken, I did not need the folder again and the error was gone. I could restart docker successfully.

git refog shows git password in plain text

Why git reflogs are important, check here. If not opened, paste this url manually.

https://linuxamination.blogspot.com/2021/04/git-reflog-and-its-importance.html

CI / CD was implemented to save the time of everyone in the team who pulls the code on development, staging and production server.

It makes the 'git pull' process automated. Gitlab CI / CD and Jenkins are used by DevOps to achieve this. If you have used Jenkins, you must have created pipelines or jobs to update the code frequently on the server.

Jenkins has its own security policies and if you see carefully, you will find Jenkins asks your git password but it does not show password in plain text, the places where password is visible, either it is in hash or it is with stars like '******@123'.

But if you use third party applications like Jenkins or a Shell Script to pull the code using git on the server,  anyone who has access of your server's project directory can read your git password in plain text.

You should try this yourself. cd into your project directory on the server and run command `git reflog`

If reflogs are enabled, you can see the pulled commits with password in the plain text.

If your all code is tested properly on QA server and you do not need this utility on production server, you can simply turn off saving reflogs by setting the following option in the remote git repository.

[core]
  logAllRefUpdates = 0

You can remove your git reflogs using following command.

This command will remove all the relogs older than 15 days.

git reflog expire --expire=15.days.ago --expire-unreachable=now --all

You can change the day number to 1 and it will delete all the reflogs of yesterday and older.

git reflog and its importance

Command `git reflog` is very helpful when you want to revert changes. It is little different than `git log` command as `git log` lists all the commit ids but using `git reflog` you can see only those commit ids which are pulled on the server.

Suppose a developer works on a task and commits frequently. He/She has committed 11 commits i.e. commit 1, commit 2, .... commit 11 before pushing these changes on the server. DevOps pulls these changes on the server. Now developer commits 9 more changes before pushing them on the server so he/she has pushed two times on the server but he/she has committed code 20 times. If this second push has broken something on the server and It is asked DevOps to revert the push.

DevOps checks the commits using command `git log` but he/she sees 20 commits, DevOps is not sure about the commit number before second pull, he can discuss with developer but developer might not sure too as his/her concentration was on the code & error and not on the commit number. It was DevOps responsibility about every details of the pull / push and here `git reflog` comes to the rescue.

It lists only those commits which were pulled on the server. The number of times developer pushed the code, it lists only those commit number. It is independent of the number of commits done by developer. Yes, if developer pushes code after every commit then it will list all the commit numbers but developers do not push code in the repository after every commit.

 

 

git - check branch of detached head

If your git branch is in detached head state and you want to know the branch name of this detached head state, this is a simple solution for you.

cd into your git project directory and run the command. 

git branch --contains HEAD

This way you can find the correct branch of your git project dir.

In above screenshot, head is detached from branch branch-two.

If user wants to know the branch name from where head was detached, above command is the solution.

You can see in the above screenshot that we can find the detached head branch using above command.

pip install error - sys.stderr.write(f"ERROR: {exc}")

If you are getting the following error while installing a pip package, this solution should work for you.

sys.stderr.write(f"ERROR: {exc}")

Solution :

A) If you are installing a pip package with default python (2.7) and getting above error, run following command

curl https://bootstrap.pypa.io/2.7/get-pip.py --output get-pip.py
python get-pip.py

B) If you are installing a pip package with python3 and you are installing a package using pip3 command, run following command

curl https://bootstrap.pypa.io/3.5/get-pip.py --output get-pip.py
python3 get-pip.py

Change the python version in url, it depends on your system's python version.

C)  If you are installing a pip package in a virtual environment with python3.5,  run following command in your virtual environment

curl https://bootstrap.pypa.io/3.5/get-pip.py --output get-pip.py
python3 get-pip.py

Command python or python3 both should work same in the virtual env.

phpLiteAdmin : No option visible to create / rename / delete database

If you have installed phpLiteAdmin in your Apache but you are unable to see the options to create / rename / delete database. Here is the solution for you.

Solution :

Your php code of  phpLiteAdmin does not have sufficient permission to show these options. You need to change the owner of the folder phpLiteAdmin.

If you have installed phpLiteAdmin in CentOS, your httpd user is apache, change the owner of the folder.

chown -R apache:apache phpLiteAdmin

If you have installed phpLiteAdmin in Ubuntu, your apache user is www-data, change the owner of the folder.

chown -R www-data:www-data phpLiteAdmin

If you have installed phpLiteAdmin in Lampp/Xampp, your apache user is daemon, change the owner of the folder.

chown -R daemon:daemon phpLiteAdmin

Now open again phpLiteAdmin in the browser. You should be able to see all options.

You can also try with 777 permission to the folder phpLiteAdmin but this is not a good practice to give full permissions to a folder of Document Root.

AWS S3 Copy Error - Connection was closed before we received a valid response from endpoint URL

 aws s3 cp error :

Error : upload failed: ./file.txt to s3://bucketname/file.txt Connection was closed before we received a valid response from endpoint URL: "https://bucketname.s3.amazonaws.com/file.txt"

Solution :

You are getting this error because you did not configure your aws bucket region in your ec2 instance. 

I did not get this error every time. When my bucket was in Mumbai region (ap-south-1), I did not need to configure bucket region in my ec2, I got this error when one of my bucket was in North California (us-west-1) region but it is a good practice if you configure your bucket region always.

Configure aws with region.

aws configure

It will ask for access key and secret key. If you have then you can enter, if you do not have like you attached the role with your ec2 instance, then you can skip it by simply pressing return key.

Now it will ask 'Default region name'. This is the attribute for that you are configuring aws. Check your bucket's region and enter here.

Example : If your bucket is in North California then your region will be us-west-1.

Skip the 'Default output format' and try to copy again using `aws s3 cp` command.

You should not get the same error again.