excel 棒球数据游戏_使用librosa的棒球应用的音频发作检测数据准备

excel 棒球数据游戏

介绍 (Introduction)

I wish to build a deep learning project for a baseball application, one of the challenges is the data collection. I need a lot of short clips of batters hitting baseballs. But I could not find this kind of dataset online, therefore, I decided to collect and make the dataset myself.

我希望为棒球应用程序构建一个深度学习项目，其中的挑战之一是数据收集。 我需要击球员打棒球的短片。 但是我无法在线找到这种数据集，因此，我决定自己收集并制作数据集。

Most of the baseball batting video I found contains several times of batting in one long video clip, for example:

我发现的大部分棒球击球视频在一个长视频剪辑中包含几次击球，例如：

演示地址

I need to find the batting moments and cut them down and split them into short clips. One straightforward approach is just to do it manually, but it is very time consuming and tiring.

我需要找到击球时间并将其击倒并切成短片。 一种简单的方法是手动进行操作，但这非常耗时且很累。

I need a tool to do this job automatically.

我需要一种工具来自动完成这项工作。

Inspired by Roger Cheng’s work “Building a Robot Umpire with Deep Learning Video Analysis”, I decided to try “onset detection”.

受到罗杰·郑(Roger Cheng)的工作“ 使用深度学习视频分析构建机器人裁判 ”的启发，我决定尝试“发作检测” 。

After some researches and reading tutorials like this one, I used librosa’s onset detection function for the work.

一些研究和阅读教程等之后这一次 ，我用librosa的发病检测功能的工作。

载入影片 (Load Video)

The first step is to download and load the video from YouTube. I used pafy to do it.

第一步是从YouTube下载并加载视频。 我用pafy来做。

filename='batting_pratice' 
url='https://www.youtube.com/watch?v=dbZy8Q6J7pw' 
video = pafy.new(url)
clip = video.getbest(preftype="mp4")
clip.download()
os.rename(clip.filename,filename+'.mp4')

The video is now downloaded and saved as ‘’batting_pratice.mp4’

视频现在已下载并保存为“ batting_pratice.mp4”

加载音频 (Load Audio)

Then I can load the audio using librosa.

然后，我可以使用librosa加载音频。

x, sr = librosa.load(filename+'.mp4')

and play it if you are interested in it.

如果对它感兴趣的话可以播放。

ipd.Audio(data=x, rate=sr)

and plot the wave plot

并绘制波形图

plt.figure(figsize=(14, 5))
librosa.display.waveplot(x, sr=sr)
plt.show()

Waveplot of the audio (image by author)

音频波形图(作者提供的图像)

You can see there are some spikes, which are the sound of hitting the ball. The job is to identify the time of them.

您会看到有一些尖峰，这是击球的声音。 工作是确定他们的时间。

套用筛选器 (Apply filter)

One additional step is to filter some low-frequency noise using a high-pass filter. It can be done using scipy.

另一步骤是使用高通滤波器来过滤一些低频噪声。 可以使用scipy完成。

from scipy.signal import butter,filtfiltdef butter_highpass(data,cutoff, fs, order=5):
   """
   Design a highpass filter.
   Args:
   - cutoff (float) : the cutoff frequency of the filter.
   - fs     (float) : the sampling rate.
   - order    (int) : order of the filter, by default defined to 5.
   """
   # calculate the Nyquist frequency
   nyq = 0.5 * fs
   # design filter
   high = cutoff / nyq
   b, a = butter(order, high, btype='high', analog=False)
   # returns the filter coefficients: numerator and denominator
   y = filtfilt(b, a, data)
   return yx_f=butter_highpass(x,1000, sr, order=5)

发作检测 (Onset detection)

Then we can start the onset detection, we are using librosa.onset. The details introduction can be found in their doc.

然后，我们可以使用librosa.onset来开始检测发作。 详细介绍可以在他们的文档中找到。

o_env = librosa.onset.onset_strength(x_f, sr=sr)
times = librosa.frames_to_time(np.arange(len(o_env)), sr=sr)
onset_frames = librosa.util.peak_pick(o_env, 3, 3, 3, 5, 0.3, 100)

After this, we should see the peaks stored in onset_frames.

此后，我们应该看到存储在onset_frames中的峰。

We can then plot it and check.

然后我们可以将其绘制并检查。

D = np.abs(librosa.stft(x_f))plt.figure(figsize=(15,10))ax1 = plt.subplot(2, 1, 1)
librosa.display.specshow(librosa.amplitude_to_db(D, ref=np.max),
x_axis='time', y_axis='log')
plt.title('Power spectrogram')plt.subplot(2, 1, 2, sharex=ax1)
plt.plot(times, o_env, label='Onset strength')
plt.vlines(times[onset_frames], 0, o_env.max(), color='r', alpha=0.9,
linestyle='--', label='Onsets')plt.axis('tight')
plt.legend(frameon=True, framealpha=0.75)
plt.show()

Onset detection (image by author)

发病检测(作者提供的图片)

Looks like we identify all of the onsets.

看起来我们已经确定了所有发作。

Now we can convert the onset_frames to time:

现在我们可以将onset_frames转换为时间：

onset_times = librosa.frames_to_time(onset_frames)

This returns an array of times.

这将返回一个时间数组。

Then we can put the data into a dataframe and created the youtube URLs to those specific timing.

然后，我们可以将数据放入数据框中，并按照特定的时间创建youtube URL。

Check one of them for example. It is working!

例如，检查其中之一。 这是工作！

剪切视频 (Cut the video)

Then we can cut the video into short clips using the timings. I want 3-second clips, so I start from the (onset time -1.5) to (onset time +1.5). It can be done using ffmpeg.

然后，我们可以使用时序将视频剪切成短片。 我需要3秒的剪辑，所以我从(开始时间-1.5)到(开始时间+1.5)开始。 可以使用ffmpeg完成。

for hit_time in onset_times:ffmpeg_extract_subclip(filename+'.mp4',round(hit_time)-1.5,    
     round(hit_time)+1.5,  
     targetname=filename+'/'+filename+'_'+str(int(hit_time))+".mp4")

Now I have the tool to detect a batting based on the sound and cut it automatically. The next step will be collecting more data using this tool for further training.

现在，我有了可以根据声音检测打击并自动剪切的工具。 下一步将使用此工具收集更多数据，以进行进一步的培训。

Thanks for reading. Suggestions and comments are welcome.

谢谢阅读。 欢迎提出建议和意见。

翻译自: https://towardsdatascience.com/audio-onset-detection-data-preparation-for-a-baseball-application-using-librosa-7f9735430c17

excel 棒球数据游戏