Monitoring equipment and constructing activity data in construction sites are essential to obtain reliable decision-making through simulation models. The audio-based equipment monitoring could provide critical information about the work process and site conditions. Although a large-scale dataset is essential for audio-based activity recognition, it is time consuming and labor intensive to collect data on site. Therefore, this study proposes a framework for constructing an audio dataset of equipment from online sources. The framework involved selecting appropriate audio using machine learning algorithms, audio denoising, and audio separation models. The validity of the constructed dataset was examined with six classifiers and compared with the benchmark models constructed using real-world equipment audio. The classification results provided 64%-93% accuracy, which demonstrates that the constructed dataset using the proposed framework is effective in recognizing real-world sounds. The outcomes are anticipated to improve audio-based activity recognition processes, potentially helping to monitor equipment productivity.