【Android】Android Camera原始帧格式转换 —— 获取Camera图像（一）

　  概述：

　　做过Android Camera图像采集和处理的朋友们应该都知道，Android手机相机采集的原始帧（RawFrame）默认是横屏格式的，而官方API有没有提供一个设置Camera采集图像的方向的方法，导致我们拿到原始帧后还需要再次对其进行转换为对应需求的数据，例如YUV的格式，图像的方向等（旋转多少度合适），下面我就粗略的介绍一下大致的流程，理解浅薄，大神请勿喷。

     注意：当前还都是基于API<21的内容，如果压根不用android.hardware.Camera的话可能有区别，还没研究过Camera2是什么原理，这里先不介绍了。

onPreviewFrame的原理：

     1. 原理及一般使用流程：

         大致流程：

         Camera — 取数据(onPreviewFrame(Byte[] rawFrameData, Camera camera)) —> 原始帧处理（Rotate/Scale：使用Libyuv/FFmpeg等工具库）—> 编码器编码得到相应的h24数据 —> 发送给流媒体服务器 

 

原始帧的采集：

     1. 方向问题：

         这里说明一下采集得到的帧的方向问题：Android默认是左横屏状态取景，也就意味着摄像头采集的数据默认就是横屏的，如图1，而且没有对应的方法来设置传感器的采集方向，所以想要实现竖直拍摄取景就得经过Rotate（旋转90°或者270°）得到图2那样的效果。

图 1

 

 

图 2

 

原始帧的分析：类型、YUV格式：

      之前写了一篇博客，关于YUV的区别的，感兴趣的朋友可以看看：http://www.cnblogs.com/raomengyang/p/4924787.html

 

处理原始帧：

　　废话不多说了，上代码吧。

　　MainActivity.java: 将Camera采集的图像显示到屏幕，并且图像经过90度的旋转成竖屏。

package com.dreamy.cameraframedemo;

import android.app.Activity;
import android.graphics.ImageFormat;
import android.hardware.Camera;
import android.os.Bundle;
import android.view.Surface;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.View;

import com.dreamy.cameraframedemo.util.ImageUtils;

import java.io.IOException;

public class MainActivity extends Activity implements SurfaceHolder.Callback,
        Camera.PreviewCallback, View.OnClickListener {
    // raw frame resolution: 1280x720, image format is: YV12
    // you need get all resolution that supported on your devices;
    // my phone is HUAWEI honor 6Plus, most devices can use 1280x720
    private static final int SRC_FRAME_WIDTH = 1280;
    private static final int SRC_FRAME_HEIGHT = 720;
    private static final int IMAGE_FORMAT = ImageFormat.YV12;

    private Camera mCamera;
    private Camera.Parameters mParams;
    private SurfaceView mSurfaceView;
    private SurfaceHolder mSurfaceHolder;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        initView();
        setListener();
    }

    private void initView() {
        mSurfaceView = (SurfaceView) findViewById(R.id.sv_recording);
        mSurfaceHolder = mSurfaceView.getHolder();
        mSurfaceHolder.setFixedSize(SRC_FRAME_WIDTH, SRC_FRAME_HEIGHT);
        mSurfaceHolder.addCallback(this);
        mSurfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    }

    private void setListener() {
        // set Listener if you want, eg: onClickListener
    }

    @Override
    public void onClick(View v) {
    }

    @Override
    public void onPreviewFrame(byte[] data, Camera camera) {
        ImageUtils.saveImageData(data);
        camera.addCallbackBuffer(data);
    }

    private void openCamera(SurfaceHolder holder) {
        releaseCamera(); // release Camera, if not release camera before call camera, it will be locked
        mCamera = Camera.open(Camera.CameraInfo.CAMERA_FACING_BACK);
        mParams = mCamera.getParameters();
        setCameraDisplayOrientation(this, Camera.CameraInfo.CAMERA_FACING_BACK, mCamera);
        mParams.setPreviewSize(SRC_FRAME_WIDTH, SRC_FRAME_HEIGHT);
        mParams.setPreviewFormat(IMAGE_FORMAT); // setting preview format：YV12
        mParams.setFocusMode(Camera.Parameters.FOCUS_MODE_CONTINUOUS_VIDEO);
        mCamera.setParameters(mParams); // setting camera parameters
        try {
            mCamera.setPreviewDisplay(holder);
        } catch (IOException ioe) {
            ioe.printStackTrace();
        }
        mCamera.setPreviewCallback(this);
        mCamera.startPreview();
    }

    private synchronized void releaseCamera() {
        if (mCamera != null) {
            try {
                mCamera.setPreviewCallback(null);
            } catch (Exception e) {
                e.printStackTrace();
            }
            try {
                mCamera.stopPreview();
            } catch (Exception e) {
                e.printStackTrace();
            }
            try {
                mCamera.release();
            } catch (Exception e) {
                e.printStackTrace();
            }
            mCamera = null;
        }
    }

    /**
     * Android API: Display Orientation Setting
     * Just change screen display orientation,
     * the rawFrame data never be changed.
     */
    private void setCameraDisplayOrientation(Activity activity, int cameraId, Camera camera) {
        Camera.CameraInfo info = new Camera.CameraInfo();
        Camera.getCameraInfo(cameraId, info);
        int rotation = activity.getWindowManager().getDefaultDisplay()
                .getRotation();
        int degrees = 0;
        switch (rotation) {
            case Surface.ROTATION_0:
                degrees = 0;
                break;
            case Surface.ROTATION_90:
                degrees = 90;
                break;
            case Surface.ROTATION_180:
                degrees = 180;
                break;
            case Surface.ROTATION_270:
                degrees = 270;
                break;
        }
        int displayDegree;
        if (info.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
            displayDegree = (info.orientation + degrees) % 360;
            displayDegree = (360 - displayDegree) % 360;  // compensate the mirror
        } else {
            displayDegree = (info.orientation - degrees + 360) % 360;
        }
        camera.setDisplayOrientation(displayDegree);
    }

    @Override
    public void surfaceCreated(SurfaceHolder holder) {
        openCamera(holder); // open camera
    }

    @Override
    public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
    }

    @Override
    public void surfaceDestroyed(SurfaceHolder holder) {
    }
}

activity_main.xml:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <SurfaceView
        android:id="@+id/sv_recording"
        android:layout_width="match_parent"
        android:layout_height="match_parent" />
</RelativeLayout>

 别忘了在AndroidManifest中打开Camera的权限:

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera.autofocus" />
<uses-feature android:name="android.hardware.camera" />

保存原始帧为图片格式：

package com.dreamy.cameraframedemo.util;

import android.os.Environment;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.text.SimpleDateFormat;
import java.util.Date;

/**
 * Created by raomengyang on 4/25/16.
 */
public class ImageUtils {
    public static final int MEDIA_TYPE_IMAGE = 1;
    public static final int MEDIA_TYPE_VIDEO = 2;

    // save image to sdcard path: Pictures/MyTestImage/
    public static void saveImageData(byte[] imageData) {
        File imageFile = getOutputMediaFile(MEDIA_TYPE_IMAGE);
        if (imageFile == null) return;
        try {
            FileOutputStream fos = new FileOutputStream(imageFile);
            fos.write(imageData);
            fos.close();
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static File getOutputMediaFile(int type) {
        File imageFileDir =
                new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "MyTestImage");
        if (!imageFileDir.exists()) {
            if (!imageFileDir.mkdirs()) {
                return null;
            }
        }
        // Create a media file name
        String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
        File imageFile;
        if (type == MEDIA_TYPE_IMAGE) {
            imageFile = new File(imageFileDir.getPath() + File.separator +
                    "IMG_" + timeStamp + ".jpg");
        } else if (type == MEDIA_TYPE_VIDEO) {
            imageFile = new File(imageFileDir.getPath() + File.separator +
                    "VID_" + timeStamp + ".mp4");
        } else return null;
        return imageFile;
    }
}   

通过javah -classpath ./ com.dreamy.jni.LibyuvUtils 生成jni的h头文件LibyuvUtils.h

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_dreamy_jni_LibyuvUtils */

#ifndef _Included_com_dreamy_jni_LibyuvUtils
#define _Included_com_dreamy_jni_LibyuvUtils
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_dreamy_jni_LibyuvUtils
 * Method:    initRotateNV21
 * Signature: (II)V
 */
JNIEXPORT void JNICALL Java_com_dreamy_jni_LibyuvUtils_initRotateNV21
  (JNIEnv *, jclass, jint, jint);

/*
 * Class:     com_dreamy_jni_LibyuvUtils
 * Method:    ScaleYV12ToI420
 * Signature: ([B[BIIIII)V
 */
JNIEXPORT void JNICALL Java_com_dreamy_jni_LibyuvUtils_ScaleYV12ToI420
  (JNIEnv *, jclass, jbyteArray, jbyteArray, jint, jint, jint, jint, jint);

/*
 * Class:     com_dreamy_jni_LibyuvUtils
 * Method:    ReleaseRotateNV21
 * Signature: ()V
 */
JNIEXPORT void JNICALL Java_com_dreamy_jni_LibyuvUtils_ReleaseRotateNV21
  (JNIEnv *, jclass);

#ifdef __cplusplus
}
#endif
#endif 

资源下载：

　　YUV格式查看工具：RawViewer：http://download.csdn.net/detail/zxccxzzxz/9508288

     项目地址：https://github.com/eterrao/BlogExamples.git

　  名称：CameraFrameDemo

 

　　接下来会写Libyuv对原始帧进行Rotate和Scale的使用方法。