在做智慧教室同屏、会议同屏之类的方案时，基于Andriod平台的采集，往往遇到各种各样的问题，以下就几个点，抛砖引玉：
1. 内网环境下，组播还是RTMP？
回答：这个问题，被无数的开发者问到，为此，单独写了篇博客论证：https://blog.csdn.net/renhui1112/article/details/86741428，感兴趣的可以参考下，简单来说，能RTMP的，就RTMP，如果真是内网环境下，没有并发瓶颈的同屏，可以启动内置RTSP服务（走单播），然后，其他终端拉流也不失为一个好的方案。
2. 推送分辨率如何设定或缩放？
回答：一般来说，好多Android设备，特别是高分屏，拿到的视频原始宽高非常大，如果推原始分辨率，编码和上行压力大，所以，一般建议，适当缩放，比如宽高缩放至2/3，缩放一般建议等比例缩放，此外，缩放宽高建议16字节对齐。
废话不多说，上实例代码：
private void createScreenEnvironment() {
sreenWindowWidth = mWindowManager.getDefaultDisplay().getWidth();
screenWindowHeight = mWindowManager.getDefaultDisplay().getHeight();
Log.i(TAG, "screenWindowWidth: " + sreenWindowWidth + ",screenWindowHeight: "
+ screenWindowHeight);
if (sreenWindowWidth > 800)
{
if (screenResolution == SCREEN_RESOLUTION_STANDARD)
{
scale_rate = SCALE_RATE_HALF;
sreenWindowWidth = align(sreenWindowWidth / 2, 16);
screenWindowHeight = align(screenWindowHeight / 2, 16);
}
else if(screenResolution == SCREEN_RESOLUTION_LOW)
{
scale_rate = SCALE_RATE_TWO_FIFTHS;
sreenWindowWidth = align(sreenWindowWidth * 2 / 5, 16);
}
}
Log.i(TAG, "After adjust mWindowWidth: " + sreenWindowWidth + ", mWindowHeight: " + screenWindowHeight);
int pf = mWindowManager.getDefaultDisplay().getPixelFormat();
Log.i(TAG, "display format:" + pf);
DisplayMetrics displayMetrics = new DisplayMetrics();
mWindowManager.getDefaultDisplay().getMetrics(displayMetrics);
mScreenDensity = displayMetrics.densityDpi;
mImageReader = ImageReader.newInstance(sreenWindowWidth,
screenWindowHeight, 0x1, 6);
mMediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);
}
3. 横竖屏自动适配
回答：因为横竖屏状态下，采集的屏幕宽高不一样，如果横竖屏切换，这个时候，需要考虑到横竖屏适配问题，确保比如竖屏状态下，切换到横屏时，推拉流两端可以自动适配，横竖屏自动适配，编码器需要重启，拉流端，需要能自动适配宽高变化，自动播放。
4. 一定的补帧策略
回答：好多人不理解为什么要补帧，实际上，屏幕采集的时候，屏幕不动的话，不会一直有数据下去，这个时候，比较好的做法是，保存最后一帧数据，设定一定的补帧间隔，确保不会因为帧间距太大，导致播放端几秒都收不到数据，当然，如果服务器可以缓存GOP，这个问题迎刃而解。
5. 异常网络处理、事件回调机制
回答：如果是走RTMP，网络抖动或者其他网络异常，需要有好重连机制和状态回馈机制。
class EventHandeV2 implements NTSmartEventCallbackV2 {
@Override
public void onNTSmartEventCallbackV2(long handle, int id, long param1, long param2, String param3, String param4, Object param5) {
Log.i(TAG, "EventHandeV2: handle=" + handle + " id:" + id);
String publisher_event = "";
switch (id) {
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STARTED:
publisher_event = "开始..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTING:
publisher_event = "连接中..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTION_FAILED:
publisher_event = "连接失败..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CONNECTED:
publisher_event = "连接成功..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_DISCONNECTED:
publisher_event = "连接断开..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_STOP:
publisher_event = "关闭..";
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RECORDER_START_NEW_FILE:
publisher_event = "开始一个新的录像文件 : " + param3;
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_ONE_RECORDER_FILE_FINISHED:
publisher_event = "已生成一个录像文件 : " + param3;
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_SEND_DELAY:
publisher_event = "发送时延: " + param1 + " 帧数:" + param2;
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_CAPTURE_IMAGE:
publisher_event = "快照: " + param1 + " 路径：" + param3;
if (param1 == 0) {
publisher_event = publisher_event + "截取快照成功..";
} else {
publisher_event = publisher_event + "截取快照失败..";
}
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUBLISHER_RTSP_URL:
publisher_event = "RTSP服务URL: " + param3;
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_RESPONSE_STATUS_CODE:
publisher_event ="RTSP status code received, codeID: " + param1 + ", RTSP URL: " + param3;
break;
case NTSmartEventID.EVENT_DANIULIVE_ERC_PUSH_RTSP_SERVER_NOT_SUPPORT:
publisher_event ="服务器不支持RTSP推送, 推送的RTSP URL: " + param3;
break;
}
String str = "当前回调状态：" + publisher_event;
Log.i(TAG, str);
Message message = new Message();
message.what = PUBLISHER_EVENT_MSG;
message.obj = publisher_event;
handler.sendMessage(message);
}
}6. 部分屏幕数据采集
回答：我们遇到的好多场景下，教室端，会拿出来3/4的区域用来投递给学生看，1/4的区域，用来做一些指令等操作，这个时候，就需要考虑屏幕区域裁剪，接口可做如下设计：
/**
* 投递裁剪过的RGBA数据
*
* @param data: RGBA data
*
* @param rowStride: stride information
*
* @param width
*
* @param height: height
*
* @param clipedLeft: 左; clipedTop: 上; cliped 裁剪后的宽; clipedHeight: 裁剪后的高; 确保传下去裁剪后的宽、高均为偶数
*
* @return {0} if successful
*/
public native int SmartPublisherOnCaptureVideoClipedRGBAData(long handle, ByteBuffer data, int rowStride, int width, int height, int clipedLeft, int clipedTop, int clipedWidth, int clipedHeight);
//实际裁剪比例，可酌情自行调整
int left = 100;
int cliped_left = 0;
int top = 0;
int cliped_top = 0;
int cliped_width = width_;
int cliped_height = height_;
if(scale_rate == SCALE_RATE_HALF)
{
cliped_left = left / 2;
cliped_top = top / 2;
//宽度裁剪后，展示3/4比例
cliped_width = (width_ *3)/4;
//高度不做裁剪
cliped_height = height_;
}
else if(scale_rate == SCALE_RATE_TWO_FIFTHS)
{
cliped_left = left * 2 / 5;
cliped_top = top * 2 / 5;
//宽度裁剪后，展示3/4比例
cliped_width = (width_ *3)/4;
//高度不做裁剪
cliped_height = height_;
}
if(cliped_width % 2 != 0)
{
cliped_width = cliped_width + 1;
}
if(cliped_height % 2 != 0)
{
cliped_height = cliped_height + 1;
}
if ( (cliped_left + cliped_width) > width_)
{
Log.e(TAG, " invalid cliped region settings, cliped_left: " + cliped_left + " cliped_" + cliped_width + " " + width_);
return;
}
if ( (cliped_top + cliped_height) > height_)
{
Log.e(TAG, "invalid cliped region settings, cliped_top: " + cliped_top + " cliped_height:" + cliped_height + " height:" + height_);
return;
}
//Log.i(TAG, " clipLeft: " + cliped_left + " clipTop: " + cliped_top + " clipWidth: " + cliped_width + " clipHeight: " + cliped_height);
libPublisher.SmartPublisherOnCaptureVideoClipedRGBAData(publisherHandle, last_buffer, row_stride_,
width_, height_, cliped_left, cliped_top, cliped_width, cliped_height );7. 文字、图片水印
回答：好多场景下，同屏者会把公司logo，和一定的文字信息展示在推送端，这个时候，需要考虑到文字和图片水印问题，具体可参考如下接口设置：
/**
* Set Text water-mark(设置文字水印)
*
* @param fontSize: it should be "MEDIUM", "SMALL", "BIG"
*
* @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
*
* @param xPading, yPading: the distance of the original picture.
*
* <pre> The interface is only used for setting font water-mark when publishing stream. </pre>
*
* @return {0} if successful
*/
public native int SmartPublisherSetTextWatermark(long handle, String waterText, int isAppendTime, int fontSize, int waterPostion, int xPading, int yPading);
/**
* Set Text water-mark font file name(设置文字水印字体路径)
*
* @param fontFileName: font full file name, e.g: /system/fonts/DroidSansFallback.ttf
*
* @return {0} if successful
*/
public native int SmartPublisherSetTextWatermarkFontFileName(long handle, String fontFileName);
/**
* Set picture water-mark(设置png图片水印)
*
* @param picPath: the picture working path, e.g: /sdcard/logo.png
*
* @param waterPostion: it should be "TOPLEFT", "TOPRIGHT", "BOTTOMLEFT", "BOTTOMRIGHT".
*
* @param picWidth, picHeight: picture width & height
*
* @param xPading, yPading: the distance of the original picture.
*
* <pre> The interface is only used for setting picture(logo) water-mark when publishing stream, with "*.png" format </pre>
*
* @return {0} if successful
*/
public native int SmartPublisherSetPictureWatermark(long handle, String picPath, int waterPostion, int picWidth, int picHeight, int xPading, int yPading);总结：其实一个好的同屏系统，需要考虑的地方远不止以上几点，比如编码参数策略等，都需要考量，后续有机会再和大家做进一步分享。