本文初写于 2016 年，部分协议和编码器生态已经变化。下面内容主要用于理解 iOS 直播链路的基础流程，具体选型请结合当前业务和平台能力判断。

1、iOS直播技术的流程

直播技术的流程大致可以分为几个步骤：数据采集、图像处理（实时滤镜）、视频编码、封包、上传、云端（转码、录制、分发）、直播播放器。

  • 数据采集：通过摄像头和麦克风获得实时的音视频数据；
  • 图像处理：将数据采集的输入流进行实时滤镜，得到我们美化之后的视频帧；
  • 视频编码：编码分为软编码和硬编码。H.264 仍是兼容性很好的常用编码格式，H.265/HEVC 在同等画质下通常压缩效率更高，但需要考虑设备、系统版本和业务链路的支持情况。软编码主要利用 CPU，硬编码由系统硬件编码器完成；苹果从 iOS 8 开始开放 VideoToolbox 相关硬编码 API；
  • 封包：传统 RTMP 推流中常见封装格式是 FLV；
  • 上传：传统直播推流常用 RTMP，也可以根据低延迟需求选择 WebRTC、SRT 或平台私有协议；
  • 云端：进行流的转码、分发和录制；
  • 直播播放器：负责拉流、解码、播放。

用一张腾讯云的图来说明上面的流程：
直播技术流程

2、获取系统的授权

直播的第一步就是采集数据，包含视频和音频数据，由于iOS权限的要求，需要先获取访问摄像头和麦克风的权限：

请求获取访问摄像头权限

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
__weak typeof(self) _self = self;
    AVAuthorizationStatus status = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeVideo];
    switch (status) {
        case AVAuthorizationStatusNotDetermined:{
            // 许可对话没有出现，发起授权许可
            [AVCaptureDevice requestAccessForMediaType:AVMediaTypeVideo completionHandler:^(BOOL granted) {
                if (granted) {
                    dispatch_async(dispatch_get_main_queue(), ^{
                        [_self.session setRunning:YES];
                    });
                }
            }];
            break;
        }
        case AVAuthorizationStatusAuthorized:{
            // 已经开启授权，可继续
            [_self.session setRunning:YES];
            break;
        }
        case AVAuthorizationStatusDenied:
        case AVAuthorizationStatusRestricted:
            // 用户明确地拒绝授权，或者相机设备无法访问
            break;
        default:
            break;
    }

请求获取访问麦克风权限

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
AVAuthorizationStatus status = [AVCaptureDevice authorizationStatusForMediaType:AVMediaTypeAudio];
    switch (status) {
        case AVAuthorizationStatusNotDetermined:{
            [AVCaptureDevice requestAccessForMediaType:AVMediaTypeAudio completionHandler:^(BOOL granted) {
            }];
            break;
        }
        case AVAuthorizationStatusAuthorized:{
            break;
        }
        case AVAuthorizationStatusDenied:
        case AVAuthorizationStatusRestricted:
            break;
        default:
            break;
    }

3、配置采样参数

音频：需要配置码率、采样率；
视频：需要配置视频分辨率、视频的帧率、视频的码率。

4、音视频的录制

音频的录制

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
self.taskQueue = dispatch_queue_create("com.1905.live.audioCapture.Queue", NULL);

       AVAudioSession *session = [AVAudioSession sharedInstance];
       [session setActive:YES withOptions:kAudioSessionSetActiveFlag_NotifyOthersOnDeactivation error:nil];

       [[NSNotificationCenter defaultCenter] addObserver: self
                                                selector: @selector(handleRouteChange:)
                                                    name: AVAudioSessionRouteChangeNotification
                                                  object: session];
       [[NSNotificationCenter defaultCenter] addObserver: self
                                                selector: @selector(handleInterruption:)
                                                    name: AVAudioSessionInterruptionNotification
                                                  object: session];

       NSError *error = nil;

       [session setCategory:AVAudioSessionCategoryPlayAndRecord withOptions:AVAudioSessionCategoryOptionDefaultToSpeaker | AVAudioSessionCategoryOptionMixWithOthers error:nil];

       [session setMode:AVAudioSessionModeVideoRecording error:&error];

       if (![session setActive:YES error:&error]) {
           [self handleAudioComponentCreationFailure];
       }

       AudioComponentDescription acd;
       acd.componentType = kAudioUnitType_Output;
       acd.componentSubType = kAudioUnitSubType_RemoteIO;
       acd.componentManufacturer = kAudioUnitManufacturer_Apple;
       acd.componentFlags = 0;
       acd.componentFlagsMask = 0;

       self.component = AudioComponentFindNext(NULL, &acd);

       OSStatus status = noErr;
       status = AudioComponentInstanceNew(self.component, &_componetInstance);

       if (noErr != status) {
           [self handleAudioComponentCreationFailure];
       }

       UInt32 flagOne = 1;

       AudioUnitSetProperty(self.componetInstance, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, 1, &flagOne, sizeof(flagOne));

       AudioStreamBasicDescription desc = {0};
       desc.mSampleRate = _configuration.audioSampleRate;
       desc.mFormatID = kAudioFormatLinearPCM;
       desc.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagsNativeEndian | kAudioFormatFlagIsPacked;
       desc.mChannelsPerFrame = (UInt32)_configuration.numberOfChannels;
       desc.mFramesPerPacket = 1;
       desc.mBitsPerChannel = 16;
       desc.mBytesPerFrame = desc.mBitsPerChannel / 8 * desc.mChannelsPerFrame;
       desc.mBytesPerPacket = desc.mBytesPerFrame * desc.mFramesPerPacket;

       AURenderCallbackStruct cb;
       cb.inputProcRefCon = (__bridge void *)(self);
       cb.inputProc = handleInputBuffer;
       status = AudioUnitSetProperty(self.componetInstance, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 1, &desc, sizeof(desc));
       status = AudioUnitSetProperty(self.componetInstance, kAudioOutputUnitProperty_SetInputCallback, kAudioUnitScope_Global, 1, &cb, sizeof(cb));

       status = AudioUnitInitialize(self.componetInstance);

       if (noErr != status) {
           [self handleAudioComponentCreationFailure];
       }

       [session setPreferredSampleRate:_configuration.audioSampleRate error:nil];


       [session setActive:YES error:nil];

视频的录制：调用GPUImage中的GPUImageVideoCamera

1
2
3
4
5
6
7
8
9
10
_videoCamera = [[GPUImageVideoCamera alloc] initWithSessionPreset:_configuration.avSessionPreset cameraPosition:AVCaptureDevicePositionFront];
_videoCamera.outputImageOrientation = _configuration.orientation;
_videoCamera.horizontallyMirrorFrontFacingCamera = NO;
_videoCamera.horizontallyMirrorRearFacingCamera = NO;
_videoCamera.frameRate = (int32_t)_configuration.videoFrameRate;

_gpuImageView = [[GPUImageView alloc] initWithFrame:[UIScreen mainScreen].bounds];
[_gpuImageView setFillMode:kGPUImageFillModePreserveAspectRatioAndFill];
[_gpuImageView setAutoresizingMask:UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight];
        [_gpuImageView setInputRotation:kGPUImageFlipHorizonal atIndex:0];