Browsing by Author "Berkol, Ali"

Now showing 1 - 3 of 3

Lip Reading Using Various Deep Learning Models with Visual Turkish Data
(GAZI UNIVERSITY JOURNAL OF SCIENCE, 2024-09-23) Tumer Sivri, Talya; Berkol, Ali; Erdem, Hamit
In Human-Computer Interaction, lip reading is essential and still an open research problem. In the last decades, there have been many studies in the field of Automatic Lip-Reading (ALR) in different languages, which is important for societies where the essential applications developed. Similarly to other machine learning and artificial intelligence applications, Deep Learning (DL) based classification algorithms have been applied for ALR in order to improve the performance of ALR. In the field of ALR, few studies have been done on the Turkish language. In this study, we undertook a multifaceted approach to address the challenges inherent to Turkish lip reading research. To begin, we established a foundation by creating an original dataset meticulously curated for the purpose of this investigation. Recognizing the significance of data quality and diversity, we implemented three robust image data augmentation techniques: sigmoidal transform, horizontal flip, and inverse transform. These augmentation methods not only elevated the quality of our dataset but also introduced a rich spectrum of variations, thereby bolstering the dataset's utility. Building upon this augmented dataset, we delved into the application of cutting- edge DL models. Our choice of models encompassed Convolutional Neural Networks (CNN), known for their prowess in extracting intricate visual features, Long-Short Term Memory (LSTM), adept at capturing sequential dependencies, and Bidirectional Gated Recurrent Unit (BGRU), renowned for their effectiveness in handling complex temporal data. These advanced models were selected to leverage the potential of the visual Turkish lip reading dataset, ensuring that our research stands at the forefront of this rapidly evolving field. The dataset utilized in this study was gathered with the primary objective of augmenting the extant corpus of Turkish language datasets, thereby substantively enriching the landscape of Turkish language research while concurrently serving as a benchmark reference. The performance of the applied method has been compared regarding precision, recall, and F1 metrics. According to experiment results, BGRU and LSTM models gave the same results up to the fifth decimal, and BGRU had the fastest training time.
Optimization of Waiting and Journey Time in Group Elevator System Using Genetic Algorithm
(2014) Tartan, Emre Oner; Erdem, Hamit; Berkol, Ali
Efficient elevator group control is an important issue for vertical transportation in high-rise buildings. From the engineering design perspective, regulation of average waiting time and journey time while considering energy consumption is an optimization problem. Alternatively to the conventional algorithms for scheduling and dispatching cars to hall calls, intelligent systems based methods have drawn much attention in the last years. This study aims to improve the elevator group control system's performance by applying genetic algorithm based optimization algorithms considering two systems. Firstly, average passenger waiting time is optimized in the conventional elevator systems in which a hall call is submitted by indicating the travel direction. Secondly, a recent development in elevator industry is considered and it is assumed that instead of direction indicators there are destination button panels at floors that allow passengers to specify their destinations. In this case optimization of average waiting time, journey time and car trip time is investigated. Two proposed algorithms have been applied considering preload conditions in a building with 20 floors and 4 cars. The simulation results have been compared with a previous study and conventional duplex algorithm.
The Turkish lip reading using deep learning method
(Başkent Üniversitesi Fen Bilimleri Enstitüsü, 2023) Berkol, Ali
Automated lip reading is a research problem that has developed considerably in recent years. Lip reading is evaluated both visually and audibly in some cases. Detecting an unwanted word from a security camera is an example of a visual lip-reading problem. Audio-visual datasets are not applicable where such image-only data is involved. Therefore, we may not have audio input in all cases. In certain cases, it is not feasible to obtain the audio input of the spoken word. In this study, we have gathered a novel Turkish dataset consisting solely of images. The dataset was generated using YouTube videos, which constitute an uncontrolled environment. Consequently, the images present challenging parameters with respect to environmental factors such as lighting conditions, angles, colors, and individual facial characteristics. Despite the variations in facial attributes like mustaches, beards, and makeup, the visual speech recognition problem was addressed using Convolutional Neural Networks (CNN) without making any modifications to the data. The problem was formulated with 10 classes, comprising single words and two-word phrases. While developing the study, comparisons were made with LSTM, BGRU, and Dilated CNN. The proposed study using only-visual data obtained a model which is automated visual speech recognition with a deep learning approach. In addition, since this study uses only-visual data, the computational cost and resource usage is less than in multi-modal studies. Also, we introduce introduced a novel approach called Concatenated Frame Images, which involved combining image frames into a single large frame. It is also the first known study to address the lip reading problem with a deep learning algorithm using a new dataset belonging to the Ural-Altaic languages. Otomatik dudak okuma, son yıllarda önemli ölçüde gelişen bir araştırma problemidir. Dudak okuma bazen görsel olarak, bazen de işitsel olarak değerlendirilmektedir. Güvenlik kamerasından istenmeyen bir kelimenin tespiti, görsel dudak okuma problemine bir örnektir. İlgili birimler sadece görüntü verilerinin olduğu durumlarda işitme-görsel veri setlerinden yararlanamazlar. Bu nedenle, tüm durumlarda ses girdisine sahip olmak mümkün değildir. Telaffuz edilen kelimenin ses girişini her zaman elde etmek mümkün değildir. Bu çalışmada yalnızca görüntü kullanılarak yeni bir Türkçe veri seti toplandı. Yeni veri seti, kontrolsüz bir ortam olan Youtube videoları kullanılarak oluşturulmuştur. Bu nedenle, görüntüler ışık, açı, renk ve yüzün kişisel özellikleri gibi çevresel faktörler açısından zor parametrelere sahiptir. Bıyık, sakal ve makyaj gibi farklı yüz özelliklerine rağmen, görsel konuşma tanıma problemi, veri üzerinde herhangi bir müdahale olmadan Konvolüsyonel Sinir Ağları (CNN) kullanılarak tek kelime ve iki kelime öbeklerini içeren 100 sınıfta geliştirilmiştir. Öte yandan çalışma geliştirilirken LSTM, BGRU ve Dilated CNN ile karşılaştırmalar yapılmıştır. Yalnızca görsel veri kullanılarak yapılan önerilen çalışma, derin öğrenme yaklaşımıyla otomatik görsel konuşma tanıma modeli elde etmiştir. Ayrıca, bu çalışma yalnızca görsel veri kullandığından çoklu modalite çalışmalarına göre hesaplama maliyeti ve kaynak kullanımı daha azdır. Ayrıca, Birleşik İmajlar Yönetimiyle, görüntü çerçevelerini tek bir büyük çerçeveye birleştirme işlemine dayandırarak klasik kesik yöntemle karşılaştırma yaptık. Ayrıca, bu çalışma, Ural-Altay dillerine ait yeni bir veri seti kullanarak derin öğrenme algoritmasıyla dudak okuma problemine yönelik yapılan ilk bilinen çalışmadır.