Final Year Project - SIM-UOW BCompSc

Final Year Project - The Hand-Tracking-Nator

For my bachelor's final year project, the topic my group was assigned was to research and develop a program related to gaming with our bare hands, from a machine learning approach. Towards this end, my team decided to pursue the approach of developing a general use, customizable software for translating hand gestures to keyboard input that can be effectively used for a wide selection of different games.

The final product of our efforts is the Hand-Tracking-Nator, a program developed in Python, with the Mediapipe, OpenCV and Tensorflow libraries at its core. The program is capable of smooth and accurate hand gesture recognition using a dataset of roughly 60,000 data points, and triggering keyboard input by performing its corresponding hand gesture, as well as a host of other features for user customization of the software.

I did the integration of OpenCV and Mediapipe into our project for implementing the frame capture from camera input and hand landmark detection respectively. These features are then used for recording data points of the performed hand gestures into a dataset, which is then processed using Tensorflow's random forest algorithm to build the machine learning model used for the gesture prediction, the results of which is displayed by our program's frontend user interface.

Relevant links: