GESTURE VISION: GESTURE CONTROLLED VIRTUAL MOUSE
The proliferation of touchless human–computer interaction methods has motivated the development of vision-based cursor control systems requiring no physical hardware. This paper presents Gesture Vision, a complete Gesture Controlled Virtual Mouse (GVVM) system that enables real-time cursor manipulation and mouse action execution through hand gestures captured via a standard webcam. The system employs Google's MediaPipe Hands framework for 21-landmark hand tracking, a rule-based gesture classification engine mapping ten distinct finger configurations to mouse actions, and a One-Euro filter-based cursor smoothing pipeline that eliminates jitter and enables fluid pointer movement by adapting its cutoff frequency to signal velocity. Implemented in Python using OpenCV and PyAutoGUI, the system achieves sub-30ms end-to-end inference latency at 30 FPS on commodity CPU hardware. The system integrates an advanced eye-tracking module using MediaPipe FaceMesh with single left-iris tracking, an EMA-based gaze range learning algorithm, and adaptive EAR baseline calibration for robust blink-triggered clicking. A voice command controller supports over 50 spoken commands — including cursor movement, window management, clipboard, and system operations — with both online (Google Speech API) and fully offline (PocketSphinx) recognition modes. The complete application is deployed as a lightweight desktop executable compatible with Windows, Linux, and macOS, requiring only a standard webcam and no specialised hardware or trained neural network at inference time.Keywords—Gesture recognition, virtual mouse, MediaPipe Hands, hand landmark detection, human– computer interaction, OpenCV, PyAutoGUI, One-Euro filter, eye tracking, voice control, touchless control, computer vision, real-time systems, FaceMesh, blink detection.
V, A., D, K. & M, L. (2026). Gesture Vision: Gesture Controlled Virtual Mouse. International Journal of Science, Strategic Management and Technology, 02(05). https://doi.org/10.55041/ijsmt.v2i5.012
V, Arjun, et al.. "Gesture Vision: Gesture Controlled Virtual Mouse." International Journal of Science, Strategic Management and Technology, vol. 02, no. 05, 2026, pp. . doi:https://doi.org/10.55041/ijsmt.v2i5.012.
V, Arjun,Kubendran. D, and Logeshwaran. M. "Gesture Vision: Gesture Controlled Virtual Mouse." International Journal of Science, Strategic Management and Technology 02, no. 05 (2026). https://doi.org/https://doi.org/10.55041/ijsmt.v2i5.012.
2.Mitra and T. Acharya, "Gesture recognition: A survey," IEEE Trans. Systems, Man, Cybernetics C, vol. 37, no. 3, pp. 311–324, 2007.
3.Bradski, "The OpenCV library," Dr. Dobb's Journal of Software Tools, 2000.
4.Flusser, T. Suk, and B. Zitova, 2D and 3D Image Analysis by Moments. Wiley, 2016.
5.Lugaresi et al., "MediaPipe: A framework for building perception pipelines," arXiv:1906.08172, 2019.
6.M. Hasan and P. K. Mishra, "Hand gesture modelling and recognition using geometric features: A review," CanadianImage Processing and Computer Vision, 2012.
7.Welch and G. Bishop, "An introduction to the Kalman filter," Univ. North Carolina at Chapel Hill, Tech. Rep. TR 95-041, 1995.
8.S. Rautaray and A. Agrawal, "Vision based hand gesture recognition for human computer interaction: A survey," Artificial Intelligence Review, vol. 43, no. 1, pp. 1–54, 2015.
9.Casiez, N. Roussel, and D. Vogel, "1€ Filter: A simple speed-based low-pass filter for noisy input in interactive systems," in Proc. ACM CHI, 2012, pp. 2527–2530.