Lab

This python project is about converting a number from one system to another with the appropriate commands for each system (binary, decimal, octal, hexadecimal).

How to run the program

1. Using command line

1. Download the script (nsc.py) to your computer.
2. Open a command line window and type python or python3 (based on your Python version) followed by the script path.

python3 ../path_to_script/nsc.py

2. Using an IDE or a Text Editor

Run the script using an IDE or an advanced text editor. Some IDEs are IDLE, PyCharm, Visual Studio Code, Sublime Text and others.

Number systems used in the program

• Decimal (base 10): uses digits 0–9.
• Binary (base 2): uses digits 0 and 1.
• Octal (base 8): uses digits 0–7.
• Hexadecimal (base 16): uses digits 0–9 and characters A–F.

Functions used in the program

• convert: transforms a number from one system to another.
• remove: removes the extra printed characters "0b", "0o" or "0x".
• invert: inverts 0 to 1 and 1 to 0 in string format.

How it works

The user gives the initial number and selects its system. The number is converted to an integer using one of:

int(num, 2)  # binary
int(num)     # decimal
int(num, 8)  # octal
int(num, 16) # hexadecimal

The result is stored in temp. The user then picks the target system:

temp      # decimal
bin(temp) # binary
oct(temp) # octal
hex(temp) # hexadecimal

Negative numbers

Example: converting -123 to hexadecimal.

# 1. Transform positive 123 to binary:
a = bin(123) # 1111011

# 2. Pad with 0's to 8 digits:
a = '0'*(8-len(a)) + a # 01111011

# 3. Invert every digit:
b = ''
for let in a:
    b += invert(let) # 10000100

# 4. Add 1 and clean the prefix:
c = remove(bin(int(b, 2) + int('1', 2)))

# 5. Convert to hexadecimal:
result = hex(int(c, 2)) # 85

# 6. Pad with F's to mark it negative:
# FFFFFF85

4th option: convert to all systems at once

if my_system == 1:
    if system_choice == 1:
        result = temp
    elif system_choice == 2:
        result = oct(temp)
    elif system_choice == 3:
        result = hex(temp)
    elif system_choice == 4:
        result = "dec : {}, oct : {}, hex : {}".format(temp, oct(temp), hex(temp))

A simple C++ example of a social network.

How to run the program (Ubuntu)

1. Open terminal.
2. Install gcc (g++), the C++ compiler.

   $ sudo apt install g++

   Or install build-essential, which also installs libraries as well as the gcc compiler.

   $ sudo apt-get install build-essential

3. Check the gcc version.

   $ gcc --version

4. Go to the folder of the program.

   $ cd /path_to_folder/folder

5. Compile the C++ program (main.cpp).

   $ sudo g++ main.cpp

6. Run the C++ program.

   $ ./main

Folders and Files

Folders

The only folder that exists is the folder of the header files (header_files). More details about the header files later.

User files

Each user file has as its file name the name of the corresponding user, and contains the other users with whom they are friends. For example, the file named kostas contains all friends of user kostas.

The following files have already been created for easier understanding and use of this social network:

• kostas
• maria
• tom
• george
• anna
• john

Network files

• conversations — Contains conversations between users and their friends.

  Logged_in_user : post_of_logged_in_user
                   reply_from_friend_user (name_of_friend)

• database — Contains pairs of usernames and passwords from all registered users in the Social Network.

  user1
  password_of_user1
  user2
  password_of_user2

• likes — Contains posts of users and likes from friends and their names.

  Logged_in_user : post_of_logged_in_user
                   LIKE! (name_of_friend)

• posts — Contains posts from all registered users in the Social Network.

  user_name_1
  post_of_user_name_1
  user_name_2
  post_of_user_name_2

• Requests — Contains friend requests that have been sent from a user to another user with whom they are not friends.

  user_sent_request -> user_to_accept/reject_request

• TempReq — A temporary file used in the program.
• users — Contains only the names of all registered users in the Social Network.

Basic header files

The main header files used in the program are:

iostream, fstream, sstream, unistd.h

Local header files

The header files contained in the header files folder, used in the program, are:

• mainmenu.h
• registration.h
• login.h
• menu2.h
• viewposts.h
• viewfriendposts.h
• menu3.h
• friendrequest.h
• friends.h
• deletefriends.h
• listusers.h
• goback.h
• exxit.h

mainmenu.h

The very first menu that the user sees. The user can select one of the 3 available choices by pressing the corresponding number:

1. Register
2. Log In
3. Exit the program

If the user presses a number different from the given ones, he goes back to the first menu.

registration.h

The user types their desired username and password. If either already exists, the user is asked to repeat the process. If the process is done correctly, he can choose from 3 options:

1. Log In
2. Return to main menu
3. Exit the program

login.h

Initially a check is made whether the user is already registered or not. Then the user enters his username and password and goes to the second menu.

menu2.h

After the user logs in successfully, he enters the Social Network menu where he can choose one of the following options:

1. My wall
2. Friend’s wall
3. Send friend request
4. Accept/Reject friend request
5. My friends
6. Delete friend
7. Back
8. Log out
9. Exit the program

viewposts.h

By choosing My wall from the above menu, the user can see one of the following:

1. His own posts
2. His friends’ posts
3. Conversations with friends
4. Likes from friends on his own posts

Finally he is asked whether to return to the Social Network menu or not.

viewfriendposts.h

By selecting Friend’s wall from the menu, the user can choose one of the following options:

1. Post a message on his wall
2. Reply to a post
3. Leave a Like
4. Main Menu

• If he chooses the first one (i), he can write the post that he wants, and this will be added to the posts file at the end after the login name is added above.
• If he chooses the second option (ii), he can choose one of his friends, view the friend’s posts, and choose the post he wants to reply to. The user then writes the reply, which is stored in the conversations file as a reply to the friend’s post, saved with his name in brackets at the end. Finally, the logged-in user can reply to another post of the same friend if he wants.
• If he chooses the third option (iii), the process is exactly the same as before, but instead of a reply, the user leaves a LIKE! which is stored in the likes file under the selected friend’s post, before the older likes.
• If he chooses the 4th option (iv), he returns to the Social Network menu.

menu3.h

By selecting Send friend request, the logged-in user can send a friend request to one of the registered users with whom he is not already friends. By pressing the number that corresponds to the desired friend-user, the request is saved in the Requests file as:

user_to_sent_request -> user_to_accept/reject_request

Finally, the user can also choose whether to send a friend request to other non-friend users or not.

friendrequest.h

By selecting Accept/Reject friend request, the logged-in user can check all the requests he has received and press the number corresponding to the name of the user whose request he wants to accept. When a request is accepted, the two users are added to each other’s file as friends, and the request is deleted from the Requests file. If there are additional friend requests but the user exits the Accept/Reject friend request menu, this is equivalent to rejecting the rest of the requests.

friends.h

By selecting My friends, the logged-in user can check the other registered users with whom he is friends, and then return to the Social Network menu if he wants.

deletefriends.h

By selecting Delete friend, the logged-in user can see a list of his friends and, if he wants, delete one of them by typing the number corresponding to that friend. The two involved users are then removed from each other’s file. Finally, the user can delete more friends if he wants.

listusers.h

Counts the number of all registered users by reading the pairs (username, password) from the database file.

goback.h

By selecting Back, the logged-in user can return to the menu or leave the Social Network and the program. If the user selects the Log out option, he returns to the first menu of the program.

exxit.h

By selecting Exit the program, the logged-in user leaves the Social Network. The program terminates and a message appears, like the following:

Program termination
Thanks for your visit logged_in_user_name !

A project made for finding Easter’s date with Gauss’s Algorithm.

The symbolism [x]n = (x mod n) is used to denote the remainder of the division x divided by n. Also, the symbol Y is used to represent a year.

Some definitions and explanations of the following terms have been provided:

• Metonic Cycle
• Golden Number of a year

  golden_number(year) = [Y]19 + 1
  gn(Y) = [Y]19 + 1

• Epact of a year

  E[Y] = [11 · (gn(Y) - 1) + 8]30
  E[Y] = [11 · gn(Y) - 3]30

• Pascha Full Moon date

  2 + [19 · [Y]19 + 16]30 of April

For a complete overview of the project you can read the PDF file “Finding the date of Easter with Gauss’s Algorithm: An implementation in Python”, in which the above definitions are explained in detail. You are free to use this file, just leave a reference to its URL.

In addition, at the end of the work, both Orthodox and Catholic Easter dates for the years 2000 to 2100 are presented. The years where Orthodox and Catholic dates coincide are highlighted. Below you can see a part of the results for the years 2000 to 2055.

All the mathematical calculations and algorithms presented in this work have also been implemented in Python for a clearer picture of the results — Python file: easter_date.py. View file in Github.

Dice Roller — it’s a two player game with dice

Watch the game in action on the Youtube video. Or you can play the game now.

Looking at the picture below (main page of the game) there are some options which the two players should adjust.

1. Read the game instructions, found by pressing the second button (Instructions) on the top right.

   a. Set the player names by clicking on Player 1 and Player 2.
   b. Choose the desired winning score. If you don't, it will be set to 100 by default.
   c. Roll the dices and watch for the rules. Press the Hold button when you have to.
   d. A player loses his entire score when he rolls two 1's or two 6's.
   e. The first player who will reach the winning score is the winner.
   f. Game starts from Player 1 by default when the Roll button is pressed.

   

2. Set their names on Player1 and Player2 respectively by clicking on them.

3. Choose the desired score by pressing the first button (Winning Score) on the top right. If the players don’t set a score, then the score will be set to 100 by default.

   

4. Game starts from Player1 by default when the ROLL DICE button is pressed.

5. The active player has a gray background. This player rolls the dice and continues to roll until the desired winning score is reached. If he rolls two 1’s or two 6’s then he loses his turn and at the same time loses the score he has won so far. To avoid this, he should press the HOLD button and hold the current score.

6. The first player who will reach the winning score is the winner.

Below are 2 images. One is during the game and the other one is when Player1 has already won.

If the screen is smaller or the user wants to make it smaller, the main page takes some of the following forms.

The idea behind this project is to create a scientific calculator which will be easy to use by any user. Anyone can execute from the very basic arithmetic operations to some more complex mathematical functions.

Tkinter Scientific Calculator

Import libraries

In order to avoid the full use of a Python editor or a command line window, we are going to create a basic GUI. The Tkinter package (“Tk interface”) is the standard Python interface to the Tcl/Tk GUI toolkit.

We also use the math and numpy libraries in order to build basic mathematical functions (sin, cosine, log, …)

from tkinter import *
import math
import numpy as np

GUI Creation – Initialization

At first we build the general structure of the GUI which will contain the buttons.

tk_calc = Tk()
tk_calc.configure(bg="#293C4A", bd=10)  # Background color and padding
tk_calc.title("Scientific Calculator")   # GUI title

Functions

We define some necessary mathematical functions which we are going to use inside our functions’ buttons. For example, the percent() function finds the result of the division of a number by 100.

def percent():
    global calc_operator
    temp = str(eval(calc_operator + '/100'))
    calc_operator = temp
    text_input.set(temp)

We also define some functions for the correct operation of our calculator. For example, we define the button_delete() function which we use for the DEL button. This function allows the user to delete, one by one from the end, the digits that have been entered on the calculator’s entry screen.

def button_delete():
    global calc_operator
    text = calc_operator[:-1]
    calc_operator = text
    text_input.set(text)

Button construction

Now it is time to fill our calculator with buttons. We place one button for each function, each operation and for each one of the 10 digits (0 to 9). The basic syntax of a Tkinter button is the following.

button = Button(container, *options)

In order to place each row of buttons we use the grid method. For example, the 7th row of our calculator contains digits 4, 5, 6 and the operators * and /.

button_4 = Button(tk_calc, button_params_main, text='4', command=lambda:button_click('4')).grid(row=7, column=0, sticky="nsew")
button_5 = Button(tk_calc, button_params_main, text='5', command=lambda:button_click('5')).grid(row=7, column=1, sticky="nsew")
button_6 = Button(tk_calc, button_params_main, text='6', command=lambda:button_click('6')).grid(row=7, column=2, sticky="nsew")
mul = Button(tk_calc, button_params_main, text='*', command=lambda:button_click('*')).grid(row=7, column=3, sticky="nsew")
div = Button(tk_calc, button_params_main, text='/', command=lambda:button_click('/')).grid(row=7, column=4, sticky="nsew")

Some explanations for each button and the function it represents are the following:

1st row

• abs — The absolute value of a number (e.g. abs(-5) = 5)
• mod — From modulo. The operation to find the remainder of the division of one number by another. In Python we use the symbol % (e.g. 5 mod 2 = 5 % 2 = 1)
• div — Floor division returns the result of the division rounded down to the nearest integer. In Python we use the symbol // (e.g. 8 div 3 = 8 // 3 = 2)
• x! — The factorial of the number x (e.g. 4! = 24)
• e — Euler’s number. A mathematical constant approximately equal to 2.71828

2nd row

• sin — Sine of an angle θ in degrees (e.g. sin(90) = 1)
• cos — Cosine of an angle θ in degrees (e.g. cos(180) = -1)
• tan — Tangent of an angle θ in degrees (e.g. tan(45) = 1)
• cot — Cotangent of an angle θ in degrees (e.g. cot(45) = 1/tan(45) = 1)
• π — Archimedes’ constant, defined as the ratio of a circle’s circumference to its diameter. Approximately equal to 3.14159.

3rd row

• x² — x raised to the power of 2 (e.g. 4² = 16)
• x³ — x raised to the power of 3 (e.g. 5³ = 125)
• xⁿ — x raised to any power (e.g. 2⁴ = 16)
• x^-1 — x raised to the power of (-1). The inverse of x (e.g. 2^-1 = 0.5)
• 10^x — Powers of 10 (e.g. 10³ = 1000)

4th row

• ²√ — Square root of a number (e.g. ²√144 = 12)
• ∛ — Cube root of a number (e.g. ∛8 = 2)
• ⁿ√ — Any root of a number (e.g. ⁴√16 = 2)
• log10 — The logarithm of a number with base 10 (e.g. log10(1000) = 3)
• ln — The logarithm of a number with base e (e.g. ln(e) = 1)

5th row

• ( — Left parenthesis
• ) — Right parenthesis
• ± — Change the sign of a number
• % — Find the percentage of a number (e.g. 5% = 0.05)
• e^x — Exponential function (e.g. e² ≈ 7.389)

6th, 7th, 8th & 9th rows

• The basic number buttons (0 to 9)
• The basic math operators (+, -, *, /)
• The equal sign (=) and decimal point (.)
• DEL — Deletes one or more digits from the end of the entry
• AC — Clears the whole entry
• EXP — Multiplies any number by a power of 10 (e.g. 2 * 10³ = 2000)

Tkinter mainloop

The mainloop() method plays a vital role in Tkinter, as it is the core application loop that waits for events and helps update the GUI — in simple terms, this is event-driven programming. If mainloop() is not called, nothing will appear on the screen window.

tk_calc.mainloop()

For more details visit Github link.

Real time drowsiness detection (blinks or yawns) with OpenCV using facial landmarks.

For a more detailed explanation of this project (more details and theory) read drowsiness_detection.pdf.

You are free to use this file, just leave a reference to this url-link « https://github.com/kostasthanos/Drowsiness-Detection ».

Drowsiness detection on sleepy driver

Main Idea

The main idea of this project is to detect drowsiness. In other words, we are trying to detect eye blinks or yawns and decide about drowsiness. We will use the dlib library for detecting faces and predicting the facial landmarks of a person.

import dlib

# DLIB - Face Detector
detector = dlib.get_frontal_face_detector()

# DLIB - Predictor
predictor = dlib.shape_predictor('Models/shape_predictor_68_face_landmarks.dat')

Using the above detector and predictor, after converting the frame (video capture) into grayscale, we detect faces on it. After that it’s time to determine the facial landmarks.

while True:
    _, frame = cap.read()
    h, w = frame.shape[:2]   # Height and Width of frame

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)  # Grayscale
    faces = detector(gray, 0)  # Detect faces in the gray frame

    # Loop through each face
    for face in faces:
        facial_landmarks = predictor(gray, face)  # Determine facial landmarks

Below you can see a representation of the numbered facial landmarks.

Eyes and Lips

Now that we have already detected the facial landmarks we can concentrate on the eyes and lips of a face in order to detect blinks or yawns respectively.

Eyes

As shown in the facial landmarks image, we can define two lists with numbers corresponding to the left and right eye.

# Landmark indexes for eyes
left_eye = [36, 37, 38, 39, 40, 41]
right_eye = [42, 43, 44, 45, 46, 47]

Blink Detection

In order to build a function that will help us determine if there is a blink or not, we could use the Eye Aspect Ratio (E.A.R.) value as defined in the paper Real-Time Eye Blink Detection using Facial Landmarks (Soukupová, Čech).

Let’s enumerate the facial landmarks of a single eye with points P1 to P6.

We will calculate the two vertical distances between points P2–P6 (dist1) and P3–P5 (dist2), and the horizontal (largest) distance between points P1–P4 (dist3). So the E.A.R. (from now on ear) is defined as:

ear = (dist1 + dist2) / (2 * dist3)

The above explanation and calculation refers to only one eye. In order to use both eyes’ ear, we can find the average eye aspect ratio from the two eyes (left and right).

Is it a blink or not?

How do we decide if there is a blink or not by the ear value? We can set an ear threshold. This means that if the current value is less than the threshold, then it’s a blink.

How do we do this? We can plot the ear value for each frame. Below is a simple example that I made to define my ear threshold value.

As a result, I will set my ear threshold value to 0.3 at the beginning of our code.

# E.A.R. threshold
ear_thresh = 0.3

Lips

To find out if there is a yawn or not, we will work exactly as before on the eyes section. First let’s define a list with numbers corresponding to the lips part of a face, as shown in the facial landmarks image.

# Landmark indexes for lips
lips = [60, 61, 62, 63, 64, 65, 66, 67]

Yawn Detection

We will define a function similar to the eyes’ function, in order to calculate the Lips Aspect Ratio (L.A.R.). Let’s enumerate the facial landmarks corresponding to the “interior” lips with points L1 to L8.

Now we can calculate the vertical distance between points L3 and L7 (dist1) and the horizontal distance between points L1 and L5 (dist2). So the L.A.R. (from now on lar) is defined as:

lar = dist1 / dist2

Is it a yawn or not?

As in blink detection, we are going to set a lar threshold. Here it means that if the current value is greater than the threshold, then it is a yawn.

This plot is quite different from the one for ear threshold values per frame. Here there are 4 different time periods (frame periods) with different situations. These situations are:

• Lips closed
• Speaking
• Yawning
• Smiling

As a result, I will set my lar threshold value to 0.5 at the beginning of our code.

# L.A.R. threshold
lar_thresh = 0.5

Drowsiness Detection

By combining the eye aspect ratio (ear) and lips aspect ratio (lar) values, we can decide if there is drowsiness or not.

if total_yawns > 2 or total_blinks > 3:
    cv2.putText(frame, "ALERT", (w - 120, 160), font, 1.2, (0, 0, 255), 4)

For a more detailed explanation of this project, read drowsiness_detection.pdf.

This project focuses on the implementation of a program able to display the keys of a keyboard on the screen and allow the user, with the help of a webcam, to select any character when the desired color (or finger) is “virtually” inside the rectangle (ROI) which corresponds to a key.

Feel free to check the following video on YouTube for a better, visual understanding of this project and its concepts.

Color settings and color-tracking

Define the lower and upper color for the color-tracking part.

lower_color = np.array([75, 169, 45], np.uint8)
upper_color = np.array([96, 255, 255], np.uint8)

In order to get this range we can use OpenCV’s trackbars. A code snippet on how to set trackbars can be found on the project Hand Detection and Finger Counting.

On-screen keyboard settings

At first we define the rectangles corresponding to each key by setting the dimensions width and height, as well as the distance between two consecutive keys.

import numpy as np

width, height = 65, 65
dist = 15  # Distance between two consecutive rectangles

# Initial position of text in frame
start_x = int((video.shape[1] - (10 * width + 9 * dist)) / 2)
start_y = 50

Keyboard characters strings

We define a string named alphabet which contains the numbers from 0 to 9, the letters of the English alphabet and some extra characters. These extra characters correspond to actions-commands.

# Set a string with letters, numbers and symbols
numbers = "1234567890"  # String containing 10 numbers (0-9)
alphabet = "1234567890QWERTYUIOP:ASDFGHJKL?ZXCVBNM,.<->[_]"

# Strings used in basic 'command' buttons
symbol_strings = ["BACK", "DELETE", "ENTER", "Aa", "SPACE", "NUM2SYM"]

Last 6 alphabet symbols

The extra symbols (< - > [ _ ]), which are the 6 final alphabet symbols, are used to create some extra actions during the virtual typing. The explanation of these symbols is the following.

• < — Delete one character (BACK)
• – — Delete all (DELETE)
• > — Newline character (ENTER)
• [ — Change between capitals/lowercase (Aa)
• _ — Leave one character empty (SPACE)
• ] — Change between numbers/symbols (NUM2SYM)

Text size and Position

We find the size of each alphabet character so that we can place it in the middle of the corresponding rectangle. We use cv2.putText to put text on screen.

# Find the size of each character based on font, font-scale and font-weight
text_size = cv2.getTextSize(text, font, fontScale, fontThickness)
text_width = text_size[0][0]
text_height = text_size[0][1]

# Text position for each character
text_x = int((width - text_width) / 2) + x
text_y = int((height + text_height) / 2) + y

Draw rectangles & symbols-characters on screen

In order to draw the rectangles and display the symbols defined in the previous strings, we use two basic functions: one for the basic alphabet symbols and numbers (draw_letters(let_index, text)) and one for the last 6 extra characters (draw_extra_chars(let_index, text)). To display each row on a new line, we use a loop with if conditions. For example, to display the second row of the keyboard, the condition (based on the alphabet length) will be:

# Draw the rectangles
width, height = 65, 65  # For the rectangles
for i in range(alphabet_length):
    if i >= 10 and i < 21:
        x, y = start_x - int(start_x / 1.5) + (i - 10) * (width + dist), start_y + (height + dist)
        cv2.rectangle(frame, (x, y), (x + width, y + height), (255), 2)

# Draw the letters
if let_index >= 10 and let_index < 21:
    cv2.putText(frame, text, (text_x - 10, text_y), font, fontScale, text_color, fontThickness)

On the other hand, if we want to draw the rectangle and word corresponding to a special character — for example the word BACK — then we should add the following code.

# Draw the rectangle
if i == 40:
    x, y = start_x, start_y + 5 * (height + int(1.5 * dist))
    width += 70

# Draw the word "BACK"
if let_index == 40:
    cv2.putText(frame, symbol_strings[0], (text_x - 32, text_y - 5), font, fontScale, text_color, fontThickness)

Finally, the keyboard that we see on our screen will have the following structure.

If the user wants to select symbols like ! or @ instead of numbers on the first row, they should move the desired (detection) color inside the rectangle containing the word NUM2SYM. The keyboard layout will change to the following.

Black frame with pencil

An extra black window-frame containing the selected characters (output message).

paper = np.zeros((250, 1000, 3), np.uint8)
cv2.namedWindow("Paper")

# Starting position of text in "Paper" frame
start_write_x = 5
start_write_y = 35

In this window someone can see the following:

• Selected characters — text “typed” so far
• Active writing line
• Upper/Lower case option (green color for the selected one)
• Press p or P to print and save the message to a txt file

An example will be the following image.

Text file name

Every time someone presses p or P, the message is saved to a txt file with a name in the format:

messagenum_DD-MM-YY_HH:MM -      num : Number of message in file
                          - DD-MM-YY : Current date (day-month-year)
                          -    HH:MM : Current time (hour:minutes)

e.g. message1_26-07-20_21:30.txt

Print-write selected character

First, find the max contour of the desired (detection) color in the frame. Find the center of the max contour and check if the center is inside a rectangle of the rows containing letters, numbers and symbols in the frame.

contour = max(contours, key=cv2.contourArea)  # Find the maximum contour each time (on each frame)
M = cv2.moments(contour)

# Draw each contour
cv2.drawContours(frame, [contour], -1, (0, 255, 0), 2)

# Find each contour center
if int(M["m00"]) != 0:
    cx = int(M["m10"] / M["m00"])
    cy = int(M["m01"] / M["m00"])
    cv2.circle(frame, (cx, cy), 2, (0, 0, 255), 2)  # Draw center of each contour

    # Check if center is inside a letter box per row of rectangles in frame
    if cy > start_y and cy < start_y + height:
        for k in range(10):
            if cx > start_x + k * (width + dist) and cx < start_x + k * (width + dist) + width:
                let_index = k

Every 20 frames, let the user add a new letter-character (beep sound).

if frames == 20:
    frames = 0

for i in range(len(alphabet)):
    draw_letters(i, letters[i])  # Use of draw_letters function
    if let_index == i:
        my_text += letters[i]

For a better, visual understanding of this project and its concepts, watch the video on YouTube.

This project focuses on the implementation of a program for detecting a human hand based on color. Using the necessary commands and logic, we can also count the number of detected fingers.

For this implementation we are using the Python language. We also need some packages to import. The basic package that we are using for color detection is OpenCV.

Feel free to check the following video on YouTube for a better, visual understanding of this project and its concepts.

Import libraries

import numpy as np
import math
import cv2

Preparing the Environment

We are going to define a smaller window (test_window) inside the main frame which will be the ROI (Region of Interest). Only inside this window the tests will be visible.

# Define the Region Of Interest (ROI) window
top_left = (245, 50)
bottom_right = (580, 395)
cv2.rectangle(frame, (top_left[0] - 5, top_left[1] - 5), (bottom_right[0] + 5, bottom_right[1] + 5), (0, 255, 255), 3)

Using the shape function we can get the size (width and height) of the frame.

# Frame shape: height and width
h, w = frame.shape[:2]  # h, w = 480, 640

Create a mask for the hand

Focus only on the user’s hand. In this part the hand must be isolated from the background. To achieve this we are going to apply Gaussian Blur on the ROI.

test_window_blurred = cv2.GaussianBlur(test_window, (5, 5), 0)

The window is in BGR format by default. We are going to convert it to HSV format.

# Convert ROI only to HSV format
hsv = cv2.cvtColor(test_window_blurred, cv2.COLOR_BGR2HSV)

Color Trackbars

In order to find the user’s skin color (array values), the user can modify the trackbars until the hand is the only thing that is visible. To enable the trackbars window, someone must define it before starting the program. So after importing the necessary packages, add this part of the code.

def nothing(x):
    pass

cv2.namedWindow("trackbars")
cv2.createTrackbar("Lower-H", "trackbars", 0, 179, nothing)
cv2.createTrackbar("Lower-S", "trackbars", 0, 255, nothing)
cv2.createTrackbar("Lower-V", "trackbars", 0, 255, nothing)
cv2.createTrackbar("Upper-H", "trackbars", 179, 179, nothing)
cv2.createTrackbar("Upper-S", "trackbars", 255, 255, nothing)
cv2.createTrackbar("Upper-V", "trackbars", 255, 255, nothing)

After that, it’s time to define a range for the colors, based on arrays.

# Find finger (skin) color using trackbars
low_h = cv2.getTrackbarPos("Lower-H", "trackbars")
low_s = cv2.getTrackbarPos("Lower-S", "trackbars")
low_v = cv2.getTrackbarPos("Lower-V", "trackbars")
up_h = cv2.getTrackbarPos("Upper-H", "trackbars")
up_s = cv2.getTrackbarPos("Upper-S", "trackbars")
up_v = cv2.getTrackbarPos("Upper-V", "trackbars")

# Create a range for the colors (skin color)
lower_color = np.array([low_h, low_s, low_v])
upper_color = np.array([up_h, up_s, up_v])

Finally, get the mask.

# Create a mask
mask = cv2.inRange(hsv, lower_color, upper_color)
cv2.imshow("Mask", mask)  # Show mask frame

Computing the maximum contour and its convex hull

For each frame on the video capture, find the maximum contour inside the ROI.

if len(contours) > 0:
    # Find the maximum contour each time (on each frame)
    # --Max Contour--
    max_contour = max(contours, key=cv2.contourArea)
    # Draw maximum contour (blue color)
    cv2.drawContours(test_window, max_contour, -1, (255, 0, 0), 3)

# Find the convex hull "around" the max_contour
# --Convex Hull--
convhull = cv2.convexHull(max_contour, returnPoints=True)
# Draw convex hull (red color)
cv2.drawContours(test_window, [convhull], -1, (0, 0, 255), 3, 2)

Finding the point with the minimum y-value

This is the highest point of the convex hull.

min_y = h  # Set the minimum y-value equal to frame's height value
final_point = (w, h)
for i in range(len(convhull)):
    point = (convhull[i][0][0], convhull[i][0][1])
    if point[1] < min_y:
    min_y = point[1]
    final_point = point
# Draw a circle (black color) to the point with the minimum y-value
cv2.circle(test_window, final_point, 5, (0, 0, 0), 2)

Finding the center of the max_contour

The center of the max contour is defined by the point (cx, cy) using cv2.moments().

M = cv2.moments(max_contour)  # Moments

# Find the center of the max contour
if M["m00"] != 0:
    cX = int(M["m10"] / M["m00"])
    cY = int(M["m01"] / M["m00"])
    # Draw circle (red color) in the center of max contour
    cv2.circle(test_window, (cX, cY), 6, (0, 0, 255), 3)

Calculating the Defect points

Find and draw the polygon that is defined by the contour.

# --Contour Polygon--
contour_poly = cv2.approxPolyDP(max_contour, 0.01 * cv2.arcLength(max_contour, True), True)
# Draw contour polygon (white color)
cv2.fillPoly(test_window, [max_contour], text_color)

The result of the command

defects = cv2.convexityDefects(contour_poly, hull)

is an array where each row contains the values:

• start point
• end point
• farthest point
• approximate distance to farthest point

Then find these points plus the mid points on each frame as below.

points = []
for i in range(defects.shape[0]):  # Len of arrays
    start_index, end_index, far_pt_index, fix_dept = defects[i][0]
    start_pts = tuple(contour_poly[start_index][0])
    end_pts = tuple(contour_poly[end_index][0])
    far_pts = tuple(contour_poly[far_pt_index][0])
    mid_pts = (int((start_pts[0] + end_pts[0]) / 2), int((start_pts[1] + end_pts[1]) / 2))
    points.append(mid_pts)

    #--Start Points-- (yellow color)
    cv2.circle(test_window, start_pts, 2, (0, 255, 255), 2)



#--End Points-- (black color)
cv2.circle(test_window, end_pts, 2, (0, 0, 0), 2)



#--Far Points-- (white color)
cv2.circle(test_window, far_pts, 2, text_color, 2)

Finger Counting

In order to do the finger counting we should find a way to check how many fingers are displayed. To do this we are calculating the angle between the start point, defect point and end point, as shown below.

# --Calculate distances--
# If p1 = (x1, y1) and p2 = (x2, y2) are two points, then the distance between them is
# Dist : sqrt[(x2-x1)^2 + (y2-y1)^2]

# Distance between the start and the end defect point
a = math.sqrt((end_pts[0] - start_pts[0]) ** 2 + (end_pts[1] - start_pts[1]) ** 2)
# Distance between the farthest (defect) point and the start point
b = math.sqrt((far_pts[0] - start_pts[0]) ** 2 + (far_pts[1] - start_pts[1]) ** 2)
# Distance between the farthest (defect) point and the end point
c = math.sqrt((end_pts[0] - far_pts[0]) ** 2 + (end_pts[1] - far_pts[1]) ** 2)

angle = math.acos((b ** 2 + c ** 2 - a ** 2) / (2 * b * c))  # Find each angle
# If angle > 90 then the farthest point is "outside the area of fingers"
if angle <= 90:
    count += 1
    frame[0:40, w - 40:w] = (0)

Display Finger Counter

for c in range(5):
    if count == c:
        cv2.putText(frame, str(count + 1), (w - 35, 30), font, 2, text_color, 2)

if len(points) <= 1:
    frame[0:40, w - 40:w] = (0)
        cv2.putText(frame, "1", (w - 35, 30), font, 2, text_color, 2)

Conclusion

This project is aiming at understanding topics such as contours, convex hull and contour polygon.

It also focuses on the defect points which we find on the detected hand.

For a better, visual understanding of this project and its concepts, watch the video on YouTube.