JavaCV 采集摄像头和麦克风数据推流直播

越来越觉得放弃JavaCV FFmpeg native API，直接使用JavaCV二次封装的API开发是很明智的选择，使用JavaCV二次封装的API开发避免了各种内存操作不当引起的crash。

上一次介绍了 JavaCV 采集摄像头及桌面视频数据，这次介绍一下如何采集摄像头和麦克风数据推送到流媒体服务器。

引入依赖

跟上一次一样，这里使用的还是最新的JavaCV库（1.5.5）.

<properties><javacpp.version>1.5.5</javacpp.version></properties><dependencies><dependency><groupId>org.bytedeco</groupId><artifactId>javacv</artifactId><version>${javacpp.version}</version></dependency><dependency><groupId>org.bytedeco</groupId><artifactId>javacv-platform</artifactId><version>${javacpp.version}</version></dependency></dependencies>

视频采集录制

在 JavaCV 采集摄像头及桌面视频数据介绍了两种采集摄像头数据的方法，这里采用第一种，即使用 OpencvFrameGrabber 采集摄像头数据。

public class VideoRecorder implements Runnable {private static final int VIDEO_DEVICE_INDEX = 0;private FFmpegFrameRecorder recorder;private int width, height;public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {this.recorder = recorder;this.width = width;this.height = height;}@Overridepublic void run() {try {OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);grabber.setImageWidth(width);grabber.setImageHeight(height);grabber.start();long startTS = 0, videoTS = 0;Frame frame = null;while (!Thread.interrupted() && (frame = grabber.grab()) != null) {if (startTS == 0) {startTS = System.currentTimeMillis();}videoTS = 1000 * (System.currentTimeMillis() - startTS);if (videoTS > recorder.getTimestamp()) {recorder.setTimestamp(videoTS);}recorder.record(frame);}grabber.close();} catch (Exception e) {e.printStackTrace();}}}

音频采集录制

音频采集直接使用JDK的API即可，直接读麦克风的数据塞给录制器，采样率为44100，16bit，小端模式。

public class AudioRecoder implements Runnable {private FFmpegFrameRecorder recorder;private int channels;private int sampleRate;public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {this.recorder = recorder;this.sampleRate = sampleRate;this.channels = channels;}@Overridepublic void run() {try {AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));line.open(format);line.start();int sampleRate = (int) format.getSampleRate();int numChannels = format.getChannels();byte[] buffer = new byte[sampleRate * numChannels];while (!Thread.interrupted()) {int nBytesRead = 0;while (nBytesRead == 0) {nBytesRead = line.read(buffer, 0, line.available());}int nSamplesRead = nBytesRead / 2;short[] samples = new short[nSamplesRead];ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);recorder.recordSamples(sampleRate, numChannels, sBuff);}} catch (Exception e) {e.printStackTrace();}}}

录制推流

这里的录制推流涉及音频和视频，音频和视频分开线程采集并录制，这样可以降低延迟，在录制推流方面还可以通过设置录制参数降低延迟：

recorder.setVideoOption("tune", "zerolatency");recorder.setVideoOption("preset", "ultrafast");

启动录制推流器，视频采集录制线程和音频采集录制线程，开始推流：

recorder.start();Thread vt = new Thread(new VideoRecorder(recorder, width, height));Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));vt.start();at.start();vt.join();at.join();