EEE F376 Project · Supervisor: Dr. Rajesh Kumar Tripathi

Lung Sound Classification using Transformer and Convolutional Transformer Models

Rijul Dahiya¹

¹BITS Pilani, Hyderabad Campus

Abstract

Using lung sounds for disease classification has been explored with the advancement of deep learning models. In this project, we explore transformer and convolutional transformer-based models for lung sound classification. We convert all audio recordings to time-frequency images using STFT, CWT, and Stockwell transforms, then feed them to Vision Transformer and CNN architectures to classify normal vs. abnormal lung sounds and identify specific pathologies including pneumonia.

Key Contributions

Comparative study of STFT, CWT, and Stockwell transforms for lung audio to image conversion.
Vision Transformer (ViT) applied to time-frequency lung sound images for classification.
CNN baseline and Android mobile application for real-time lung sound classification.
Evaluation on multiple lung sound datasets across normal/abnormal and normal/pneumonia classification tasks.

Method

Lung audio recordings are converted to 2D time-frequency representations using STFT (Short-Time Fourier Transform), CWT (Continuous Wavelet Transform), and Stockwell Transform. These images are fed into a Vision Transformer model as well as a CNN baseline. An Android application integrates the best-performing model for mobile deployment, enabling real-time classification at the point of care.

Figures

Figure 3.1. Vision Transformer architecture for lung sound classification, STFT/CWT/Stockwell spectrogram patches fed into ViT encoder.

Figure 3.2. CNN baseline architecture, convolutional feature extractor for time-frequency image classification.

Figure 4.1. Train vs. validation loss over epochs for Normal vs. Pneumonia CNN model (Test Accuracy 90%) and confusion matrix.

Additional result visualization, classification performance across lung sound categories.

Full paper