How to Access Android Camera in Unity (III): Customized camera plugin with preview window and UI on it

Overview

   This article will continue the last article and add functionalities such as the camera preview window and buttons to allow user to adjust camera parameters.
  When Unity call this plugin, instead of take a snapshot and leave, the process will stay in the Camera Setting (unless you press the "Return" button so that it'll return to game),  as shown below. In this moment, Unity is temporarily blocked: You can't hear any game sound (unless you invoke it by the android API here), or running game in the background.

Solution

1. At first we need to add one more function "DisplayCameraSettingActivity()" in AndroidCameraAPI.cs, so that it would like:

/*
 * This Class Call the Android Code to Take Picture.
 * Note: there should be a AndroidCameraAPI.jar in \Assets\Plugins\Android
 */
public class AndroidCameraAPI : MonoBehaviour
{
    private AndroidJavaObject androidCameraActivity;

    public void Awake()
    {
        AndroidJavaClass unity = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
        this.androidCameraActivity = unity.GetStatic<AndroidJavaObject>("currentActivity");
    }

    public void TakePhotoFromAndroidAPI(string _pathToSavePhoto, string _fileNameOfSavedPhoto)
    {
        this.androidCameraActivity.Call("TakePhoto",
                                        _pathToSavePhoto, _fileNameOfSavedPhoto+".jpg");
    }

    public void DisplayCameraSettingActivity()
    {
        this.androidCameraActivity.Call("DisplayCameraSettingActivity");
    }
}

2. In the Entrance of Android API (the MainActivity.java):

import android.content.Intent;
import android.hardware.Camera;
import android.os.Bundle;
import android.util.Log;

import com.unity3d.player.UnityPlayerActivity;

public class MainActivity extends UnityPlayerActivity
{
    private Camera androidCamera = null;
    private Camera.Parameters userDefinedCameraParameters = null;

    @Override
    public void onCreate(Bundle _savedInstanceState)
    {
        super.onCreate(_savedInstanceState);

        openCameraSafely();
        setupCameraWithDefaultParameters();
    }
    private void openCameraSafely()
    {
        Log.i("Unity", "Open Camera Safely...");
        try
        {
            this.androidCamera = Camera.open();
        }
        catch(Exception cameraIOException)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Cannot Connect to Camera,"
                                                + " please Check connection...\n"
                                                + "\tIf this doesn't work, please"
                                                + " Replug the Camera or Reboot this Machine.");

            this.finish();
        }
        Log.i("Unity", "Open Camera Success!");
    }
    private void setupCameraWithDefaultParameters()
    {
        try
        {
            this.userDefinedCameraParameters = this.androidCamera.getParameters();
            this.userDefinedCameraParameters.setExposureCompensation(CameraDataCenter.userDefinedCameraExposureLevel);
            this.userDefinedCameraParameters.setZoom(CameraDataCenter.userDefinedCameraZoomLevel);
            this.userDefinedCameraParameters.setPictureSize(CameraDataCenter.PHOTO_WIDTH,
                                                            CameraDataCenter.PHOTO_HEIGHT);
            this.androidCamera.setParameters(this.userDefinedCameraParameters);
        }
        catch(Exception exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Can't get CameraParameters,"
                                                +" which means the camera might in bad status."
                                                +" Do you access the camera with multithread?");
        }
    }

    public void DisplayCameraSettingActivity()
    {
        Log.i("Unity", "Go to Camera Settings...");
        Intent intentOfCameraSetting = new Intent(this, CameraSettingActivity.class);
        this.androidCamera.release();
        startActivityForResult(intentOfCameraSetting,
                                CameraDataCenter.CAMERA_SETTING_REQUEST_CODE);
    }

    protected void onActivityResult(int _requestCode, int _resultCode, Intent _cameraSettingIntent)
    {
        Log.i("Unity", "MainActivity.onActivityResult().");
        if( (_requestCode == CameraDataCenter.CAMERA_SETTING_REQUEST_CODE)
            &&(_resultCode == RESULT_OK) )
        {
            this.userDefinedCameraParameters.setExposureCompensation(CameraDataCenter.userDefinedCameraExposureLevel);
            this.userDefinedCameraParameters.setZoom(CameraDataCenter.userDefinedCameraZoomLevel);
        }
        this.openCameraSafely();
    }


    public void TakePhoto(final String _targetPathToSavePhoto,
                          final String _targetPhotoFileName)
    {
        Log.i("Unity", "Take Photo...");
        if(this.userDefinedCameraParameters != null)
            this.androidCamera.setParameters(this.userDefinedCameraParameters);
        this.androidCamera.startPreview();

            Camera.PictureCallback savePictureCallBack = new SavePictureCallBack(_targetPathToSavePhoto, _targetPhotoFileName);
            this.androidCamera.takePicture(null, null, savePictureCallBack);

            Log.i("Unity", "Take Photo Success!");
    }

    @Override
    public void onDestroy()
    {
        Log.i("Unity", "CameraActivity is Destroyed!");
        if(this.androidCamera != null)
            this.androidCamera.release();

        super.onDestroy();
    }

}

    You can see that when you call DisplayCameraSettingActivity(), the process will change to CameraSettingActivity which will be demonstrate bellow.
    Note that When you move to another Activity, you should release the lock of android.camera by camera.release() since only one activity can hold the lock of android.camera.
   Also note that when the CameraSettingActivity return (when the user hit the "Return" button on the screen), the function onActivityResult() will be called. That is the time that we require the lock of android.camera back (so that the TakePhoto() can be called latter in Unity). Then the process will return to Unity and continue the game.

3. The CameraSettingActivity.java is shown below:

public class CameraSettingActivity extends Activity
{
    private Camera androidCamera;
    private Camera.Parameters userDefinedCameraParameters = null;
    private CameraPreviewer cameraPreviewer;
    private FrameLayout cameraPreviewFrame;

    private LinearLayout linearLayout;
    private TextView exposureLevelText;
    private TextView zoomLevelText;
    private Button returnButton;
    private final float UI_TEXT_SIZE = 32f;


    @Override
    protected void onCreate(Bundle _savedInstanceState)
    {
        super.onCreate(_savedInstanceState);

        initializeLayoutSetting();

        openCameraSafely();
        setupCameraWithDefaultParameters();
        initializeCameraPreview();

        createReturnButton();
        CreateExposureInspector();
        CreateZoomLevelInspector();
    }
    private void initializeLayoutSetting()
    {
        DisplayMetrics displayedScreen = new DisplayMetrics();
        this.getWindowManager().getDefaultDisplay().getMetrics(displayedScreen);

        this.requestWindowFeature(Window.FEATURE_NO_TITLE);
        this.getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN,
                                  WindowManager.LayoutParams.FLAG_FULLSCREEN);

        this.linearLayout = new LinearLayout(this);
        LayoutParams parametersOfLinearLayout = new LayoutParams(LayoutParams.MATCH_PARENT,
                                                                 LayoutParams.MATCH_PARENT);

        setContentView(this.linearLayout, parametersOfLinearLayout);
    }
    private void openCameraSafely()
    {
        try
        {
            this.androidCamera = Camera.open();
        }
        catch(Exception cameraIOException)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Cannot Connect to Camera,"
                                                + " please Check connection...\n"
                                                + "\tIf this doesn't work, please"
                                                + " Replug the Camera or Reboot this Machine.");

            this.finish();
        }
    }

    /*
     * UI Related...
     */
    private void createReturnButton()
    {
        this.returnButton = new Button(this);
        this.returnButton.setText("Return");
        this.returnButton.setTextSize(UI_TEXT_SIZE);
        this.returnButton.setOnClickListener(
                new Button.OnClickListener()
                {
                    @Override
                    public void onClick(View _view)
                    {
                        OnReturnButtonClick(_view);
                    }
                }
        );
        this.returnButton.setX(100);
        this.returnButton.setY(600);
        addContentView( this.returnButton, new LayoutParams(150, 75) );
    }

    private void CreateExposureInspector()
    {
        createIncreaseExposureButton(350, 75, 75, 65);
        createCurrentExposureText(500, 85, 400, 70);
        createDecreaseExposureButton(850, 75, 75, 65);
    }
    private void createIncreaseExposureButton(int _left, int _top, int _width, int _height)
    {
        Button increaseExposureButton = new Button(this);
        increaseExposureButton.setText("+");
        increaseExposureButton.setTextSize(UI_TEXT_SIZE);
        increaseExposureButton.setOnClickListener(
                new Button.OnClickListener()
                {
                    @Override
                    public void onClick(View _view)
                    {
                        OnIncreaseExposureButtonClick(_view);
                    }
                }
        );
        increaseExposureButton.setX(_left);
        increaseExposureButton.setY(_top);
        addContentView( increaseExposureButton, new LayoutParams(_width, _height) );
    }
    private void createCurrentExposureText(int _left, int _top, int _width, int _height)
    {
        this.exposureLevelText = new TextView(this);
        this.exposureLevelText.setTextSize(UI_TEXT_SIZE);
        updateExposureLevelTexts();
        this.exposureLevelText.setX(_left);
        this.exposureLevelText.setY(_top);
        addContentView(this.exposureLevelText, new LayoutParams(_width, _height));
    }
    private void createDecreaseExposureButton(int _left, int _top, int _width, int _height)
    {
        Button decreaseExposureButton = new Button(this);
        decreaseExposureButton.setText("-");
        decreaseExposureButton.setTextSize(UI_TEXT_SIZE);
        decreaseExposureButton.setOnClickListener(
                new Button.OnClickListener()
                {
                    @Override
                    public void onClick(View _view)
                    {
                        OnDecreaseExposureButtonClick(_view);
                    }
                }
        );
        decreaseExposureButton.setX(_left);
        decreaseExposureButton.setY(_top);
        addContentView( decreaseExposureButton, new LayoutParams(_width, _height) );
    }
    private void updateExposureLevelTexts()
    {
        int currentExposureLevel = CameraDataCenter.userDefinedCameraExposureLevel;

        if( currentExposureLevel > 0)
            this.exposureLevelText.setText("Exposure Level:  +"+currentExposureLevel);
        else
            this.exposureLevelText.setText("Exposure Level:  "+currentExposureLevel);
    }

    private void CreateZoomLevelInspector()
    {
        createIncreaseZoomLevelButton(350, 150, 75, 65);
        createCurrentZoomLevelText(500, 160, 400, 70);
        createDecreaseZoomLevelButton(850, 150, 75, 65);
    }
    private void createIncreaseZoomLevelButton(int _left, int _top, int _width, int _height)
    {
        Button increaseZoomLevelButton = new Button(this);
        increaseZoomLevelButton.setText("+");
        increaseZoomLevelButton.setTextSize(UI_TEXT_SIZE);
        increaseZoomLevelButton.setOnClickListener(
                new Button.OnClickListener()
                {
                    @Override
                    public void onClick(View _view)
                    {
                        OnIncreaseZoomLeveleButtonClick(_view);
                    }
                }
        );
        increaseZoomLevelButton.setX(_left);
        increaseZoomLevelButton.setY(_top);
        addContentView( increaseZoomLevelButton, new LayoutParams(_width, _height) );
    }
    private void createCurrentZoomLevelText(int _left, int _top, int _width, int _height)
    {
        this.zoomLevelText = new TextView(this);
        this.zoomLevelText.setTextSize(UI_TEXT_SIZE);
        updateZoomLevelTexts();
        this.zoomLevelText.setX(_left);
        this.zoomLevelText.setY(_top);
        addContentView(this.zoomLevelText, new LayoutParams(_width, _height));
    }
    private void createDecreaseZoomLevelButton(int _left, int _top, int _width, int _height)
    {
        Button decreaseZoomLevelButton = new Button(this);
        decreaseZoomLevelButton.setText("-");
        decreaseZoomLevelButton.setTextSize(UI_TEXT_SIZE);
        decreaseZoomLevelButton.setOnClickListener(
                new Button.OnClickListener()
                {
                    @Override
                    public void onClick(View _view)
                    {
                        OnDecreaseZoomLevelButtonClick(_view);
                    }
                }
        );
        decreaseZoomLevelButton.setX(_left);
        decreaseZoomLevelButton.setY(_top);
        addContentView( decreaseZoomLevelButton, new LayoutParams(_width, _height) );
    }
    private void updateZoomLevelTexts()
    {
        int currentZoomLevel = CameraDataCenter.userDefinedCameraZoomLevel;

        if( currentZoomLevel > 0)
            this.zoomLevelText.setText("Zoom Level:  +"+currentZoomLevel);
        else
            this.zoomLevelText.setText("Zoom Level:  "+currentZoomLevel);
    }

    /*
     * Camera Settings Related...
     */
    private void initializeCameraPreview()
    {
        this.cameraPreviewer = new CameraPreviewer(this, this.androidCamera);

        this.cameraPreviewFrame = new FrameLayout(this);
        addContentView(this.cameraPreviewFrame,
                       new LayoutParams(ViewGroup.LayoutParams.FILL_PARENT, LayoutParams.FILL_PARENT)
                      );
        this.cameraPreviewFrame.addView(this.cameraPreviewer);
    }
    private void setupCameraWithDefaultParameters()
    {
        try
        {
            this.userDefinedCameraParameters = this.androidCamera.getParameters();
            this.userDefinedCameraParameters.setPictureSize(CameraDataCenter.PHOTO_WIDTH,
                                                            CameraDataCenter.PHOTO_HEIGHT);
            updateCameraExposureSettings(CameraDataCenter.userDefinedCameraExposureLevel);
            updateCameraZoomSettings(CameraDataCenter.userDefinedCameraZoomLevel);
        }
        catch(Exception exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                 "Can't get CameraParameters,"
                                                +" which means the camera might in bad status."
                                                +" Do you access the camera with multithread?");
        }
    }
    private void updateCameraExposureSettings(int _userRequiredExposure)
    {
        try
        {
            CameraDataCenter.userDefinedCameraExposureLevel = _userRequiredExposure;
            this.userDefinedCameraParameters.setExposureCompensation(_userRequiredExposure);
            this.androidCamera.setParameters(userDefinedCameraParameters);
        }
        catch(Exception exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Can't Set CameraParameters,"
                                                +"  Are there Invalid Camera Settings "
                                                +"(such as Invalid Picture Size, Zoom Level"
                                                +" or Exposure Level)?");
        }
    }
    private void updateCameraZoomSettings(int _userDefinedZoomLevel)
    {
        try
        {
            CameraDataCenter.userDefinedCameraZoomLevel = _userDefinedZoomLevel;
            this.userDefinedCameraParameters.setZoom(_userDefinedZoomLevel);
            this.androidCamera.setParameters(userDefinedCameraParameters);
        }
        catch(Exception exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Can't Set CameraParameters,"
                                                +"  Are there Invalid Camera Settings "
                                                +"(such as Invalid Picture Size, Zoom Level"
                                                +" or Exposure Level)?");
        }
    }

    /*
     * Following is the Call Back Functions for Button Clicked.
     */
    public void OnIncreaseExposureButtonClick(View _increaseExposureButton)
    {
        int currentExposure = this.userDefinedCameraParameters.getExposureCompensation();
        ++currentExposure;
        int maxExposure = this.userDefinedCameraParameters.getMaxExposureCompensation();
        if( currentExposure > maxExposure )
            currentExposure = maxExposure;
        updateCameraExposureSettings(currentExposure);
        updateExposureLevelTexts();
    }
    public void OnDecreaseExposureButtonClick(View _decreaseExposureButton)
    {
        int currentExposure = this.userDefinedCameraParameters.getExposureCompensation();
        --currentExposure;
        int minExposure = this.userDefinedCameraParameters.getMinExposureCompensation();
        if( currentExposure < minExposure)
            currentExposure = minExposure;
        updateCameraExposureSettings(currentExposure);
        updateExposureLevelTexts();
    }
    public void OnIncreaseZoomLeveleButtonClick(View _increaseZoomButton)
    {
        int currentZoomLevel = this.userDefinedCameraParameters.getZoom();
        ++currentZoomLevel;
        int maxZoomLevel = this.userDefinedCameraParameters.getMaxZoom();
        if( currentZoomLevel > maxZoomLevel )
            currentZoomLevel = maxZoomLevel;
        updateCameraZoomSettings(currentZoomLevel);
        updateZoomLevelTexts();
    }
    public void OnDecreaseZoomLevelButtonClick(View _decreaseZoomeButton)
    {
        int currentZoomLevel = this.userDefinedCameraParameters.getZoom();
        --currentZoomLevel;
        int minZoomLevel = 0;
        if( currentZoomLevel < minZoomLevel)
            currentZoomLevel = minZoomLevel;
        updateCameraZoomSettings(currentZoomLevel);
        updateZoomLevelTexts();
    }

    public void OnReturnButtonClick(View _returnButton)
    {
        this.androidCamera.stopPreview();

        Intent returnIntent = new Intent();
        setResult(RESULT_OK, returnIntent);

        Log.i("Unity", "Release Camera...");
        this.androidCamera.release();
        this.finish();
    }

    @Override
    public void onDestroy()
    {
        Log.i("Unity", "CameraActivity is Destroyed!");
        if(this.androidCamera != null)
            this.androidCamera.release();

        super.onDestroy();
    }
}

   You can see that I create the buttons dynamically in onCreate(), instead of using the R.layout (which is the normal way to create UI widget in android), because there're some issue about the missing of  R.layout when the Unity use android plugins. I guess the reason might be that Unity will replace the R.layout itself... And this is the reason that I don't call the

    setContentView(R.layout.activity_main);

in the onCreate() as we normally do when writing "pure" android app.

4. Since you have two Activities now, you should also change your AndroidManifest.xml to:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.igs.dinosaur"
      android:versionCode="1"
      android:versionName="1.0">
    <uses-sdk android:minSdkVersion="9" />
 
 <uses-permission android:name="android.permission.CAMERA" />
 <uses-feature android:name="android.hardware.camera" />
 <uses-feature android:name="android.hardware.camera.autofocus" />
 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
 
    <application
  android:icon="@drawable/app_icon"
  android:label="@string/app_name"
  android:debuggable="true">
  
        <activity android:name=".MainActivity"
                  android:label="@string/app_name">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
  
  <activity android:name=".CameraSettingActivity">
  </activity>
  
    </application>
</manifest>

5. Finally, here is the CameraPreviewer that will be called by the CameraSettingActivity:

public class CameraPreviewer extends SurfaceView
                            implements SurfaceHolder.Callback
{
    private SurfaceHolder displayedSurface;
    private Camera currentCamera;

    public CameraPreviewer(Context _cameraContext, Camera _camera)
    {
        super(_cameraContext);
        this.currentCamera = _camera;
        this.displayedSurface = this.getHolder();
        this.displayedSurface.addCallback(this);

        //Following line might be required if you use the early version of android
        this.displayedSurface.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    }

    @Override
    public void surfaceCreated(SurfaceHolder _surfaceHolder)
    {
        try
        {
            this.currentCamera.setPreviewDisplay(_surfaceHolder);
            this.currentCamera.startPreview();
        }
        catch (IOException exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                 "Camera Preview failed, this might due to"
                                                 +" the SurfaceHolder cannot be obtained...");
        }
    }

    @Override
    public void surfaceChanged(SurfaceHolder _surfaceHolder, int _format, int _width, int _height)
    {
        this.Refresh();
    }

    public void Refresh()
    {
        stopPreviewBeforeRefresh();
        restartPreview();
    }
    private void stopPreviewBeforeRefresh()
    {
        try
        {
            this.currentCamera.stopPreview();
        }
        catch(Exception _exception)
        {
            /*
             * This Exception can be Ignore, since it
             * might be no preview before.
             */
        }
    }
    private void restartPreview()
    {
        try
        {
            this.currentCamera.setPreviewDisplay(this.displayedSurface);
            this.currentCamera.startPreview();
        }
        catch (IOException exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                 "Camera Preview failed, this might due to"
                                                 + " the SurfaceHolder cannot be obtained...");
        }
    }

    @Override
    public void surfaceDestroyed(SurfaceHolder _surfaceHolder) {}
}

Result

    Now you can adjust the camera parameters by click the buttons and preview the camera, as shown below.

Conclusion

    Here're the advantages and disadvantages of calling the camera in Android API compare to the Unity API:

Advantages:

     As mention in the last article, you can adjust many camera parameters in android API.

Disadvantages:

    When calling the android API, Unity will temporary stopped. So if you want to play music, display UI or detect IO, you should do it in Android API! You can't rely on the easy-to-use Unity anymore.

2016/6/2 updated:
   We finally solve the Unity temporary stopped problem by using the Android Service. The Service is like running things in another thread, but yet its display can cover the original APP.
   Therefore, we extend the Service that can display camera video. When we want to show the WebCam preview window, we call that Service. And it will display the camera video which will cover the display of our game but yet leave the game still running.
    This solution could also be used to solve the temporary stop problem when you want to apply Unity API to display movie in Android. You can extend the Service to display movie so that the game will still running when the movie is playing.

How to Access Android Camera in Unity (II): Customized android plugin by calling Android API

Overview

     In the last article, I use the Unity default WebCamTexture to read the input from camera device. However, it seems that nearly none of the camera parameters is adjustable.  In this article, I'll show you how to customized the camera by android API and how to invoke it from Unity.

Solution

How to call android plugins in Unity

      If you have no experience of building an android plugin, here is a good tutorial. Note that the "package name" in Android should match the "Bundle ID" in Unity. Also note that if you have upper case in your package name, it would be transferred to lower case automatically when you build the new project. This will result in the mismatch of package name and the Bundle ID. Although there're several websites have mentioned how to change package name, I failed. So it would be better to get it right at the beginning.

How to customize Android Camera

     You can follow the android documents step by step. There're also several examples such as here and here. However, none of them is aimed to build android plugins for Unity. Following will show you how to build customized plugin for Unity.

1. Create a script (AndroidCameraAPI.cs) to call the android plugin in Unity:

/// <summary>
/// This Class Call the Android Code to Take Picture.
/// Note: there should be a AndroidCameraAPI.jar in \Assets\Plugins\Android
/// </summary>
public class AndroidCameraAPI : MonoBehaviour
{
   private AndroidJavaObject androidCameraActivity;

   public void Awake()
   {
      AndroidJavaClass unity = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
      this.androidCameraActivity = unity.GetStatic<AndroidJavaObject>("currentActivity");
   }

   /// <summary>
   /// Note: Only support the "jpg" format, and you should NOT include the File Name Extension. 
   ///   For example, use
   ///   TakePhotoFromAndroidAPI(path, "myImage");
   ///   instead of
   ///   TakePhotoFromAndroidAPI(path, "myImage.jpg");
   /// </summary>
   /// <param name="_pathToSavePhoto"></param>
   /// <param name="_fileNameOfSavedPhoto"></param>
   public void TakePhotoFromAndroidAPI(string _pathToSavePhoto, string _fileNameOfSavedPhoto)
   {
      this.androidCameraActivity.Call("TakePhoto",
                                      _pathToSavePhoto,
                                      _fileNameOfSavedPhoto + ".jpg");
   }
}

2. Now edit the MainActivity.java (which can be thought as the entrance of Android) in AndroidStudio:

import android.content.Intent;
import android.hardware.Camera;
import android.os.Bundle;
import android.util.Log;

import com.unity3d.player.UnityPlayerActivity;

public class MainActivity extends UnityPlayerActivity
{
    private Camera androidCamera = null;
    private Camera.Parameters userDefinedCameraParameters = null;

    @Override
    public void onCreate(Bundle _savedInstanceState)
    {
        super.onCreate(_savedInstanceState);

        openCameraSafely();
        setupCameraWithDefaultParameters();
    }
    private void openCameraSafely()
    {
        Log.i("Unity", "Open Camera Safely...");
        try
        {
            this.androidCamera = Camera.open();
        }
        catch(Exception cameraIOException)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Cannot Connect to Camera,"
                                                + " please Check connection...\n"
                                                + "\tIf this doesn't work, please"
                                                + " Replug the Camera or Reboot this Machine.");

            this.finish();
        }
        Log.i("Unity", "Open Camera Success!");
    }
    private void setupCameraWithDefaultParameters()
    {
        try
        {
            this.userDefinedCameraParameters = this.androidCamera.getParameters();
            this.userDefinedCameraParameters.setExposureCompensation(CameraDataCenter.userDefinedCameraExposureLevel);
            this.userDefinedCameraParameters.setZoom(CameraDataCenter.userDefinedCameraZoomLevel);
            this.userDefinedCameraParameters.setPictureSize(CameraDataCenter.PHOTO_WIDTH,
                                                            CameraDataCenter.PHOTO_HEIGHT);
            this.androidCamera.setParameters(this.userDefinedCameraParameters);
        }
        catch(Exception exception)
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Can't get CameraParameters,"
                                                +" which means the camera might in bad status."
                                                +" Do you access the camera with multithread?");
        }
    }

    public void TakePhoto(final String _targetPathToSavePhoto,
                          final String _targetPhotoFileName)
    {
        Log.i("Unity", "Take Photo...");
        if(this.userDefinedCameraParameters != null)
            this.androidCamera.setParameters(this.userDefinedCameraParameters);
        this.androidCamera.startPreview();

            Camera.PictureCallback savePictureCallBack = new SavePictureCallBack(_targetPathToSavePhoto, _targetPhotoFileName);
            this.androidCamera.takePicture(null, null, savePictureCallBack);

            Log.i("Unity", "Take Photo Success!");
    }

    @Override
    public void onDestroy()
    {
        Log.i("Unity", "CameraActivity is Destroyed!");
        if(this.androidCamera != null)
            this.androidCamera.release();

        super.onDestroy();
    }

}

Note: You should delete the line

    setContentView(R.layout.activity_main);

in onCreate(). The reason will be illustrated in the next article.

3. The CameraDataCenter.java is used to store some camera parameters.

public class CameraDataCenter
{
    public static final int DEFAULT_CAMERA_EXPOSURE_LEVEL = 3;
    public static int userDefinedCameraExposureLevel = DEFAULT_CAMERA_EXPOSURE_LEVEL;

    public static final int DEFAULT_CAMERA_ZOOM_LEVEL = 0;
    public static int userDefinedCameraZoomLevel = DEFAULT_CAMERA_ZOOM_LEVEL;

    public static final int CAMERA_SETTING_REQUEST_CODE = 999;

    public static final int PHOTO_WIDTH = 1280;
    public static final int PHOTO_HEIGHT = 720;
}

4. The SavePictureCallBack.java is a callback for TakePhoto() in MainActivity. And will tell the system how to save the photo you just taken.

import android.hardware.Camera;
import android.util.Log;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;

public class SavePictureCallBack implements Camera.PictureCallback
{
    private String targetPathToSavePhoto;
    private String targetFileNameOfPhoto;

    public SavePictureCallBack(final String _targetPath, final String _targetPhotoFileName)
    {
        this.targetPathToSavePhoto = _targetPath;
        this.targetFileNameOfPhoto = _targetPhotoFileName;
    }

    @Override
    public void onPictureTaken(byte[] _data, Camera _camera)
    {
        File newImageFile = new File(this.targetPathToSavePhoto, this.targetFileNameOfPhoto);
        if( newImageFile != null)
        {
            FileOutputStream fileStream = null;
            try
            {
                fileStream = new FileOutputStream(newImageFile);
                fileStream.write(_data);
                Log.i("Unity", "Save Photo to: " + newImageFile.getPath() + " Success!");
            }
            catch (Exception exception)
            {
                ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                    "Photo is taken, but can't be saved."
                                                    + " If this error keeps happening,"
                                                    + " please Check your Disk Space");
            }
            finally
            {
                CloseFileAndReportExceptionByLog(fileStream);
            }

        }
        else
        {
            ExceptionManager.SaveErrorCodeToFile(ExceptionManager.CAMERA_ERROR_LOG_FILE_NAME,
                                                "Photo is taken, but can't be saved."
                                                + " If this error keeps happening,"
                                                + " please Check your Disk Space");

        }

    }

    public static void CloseFileAndReportExceptionByLog(FileOutputStream _fileToBeClose)
    {
        try
        {
            _fileToBeClose.close();
        }
        catch(IOException exception)
        {
            Log.w("Unity", "Close Image file Failed.");
        }
    }
}

5. Finally, the AndroidManifest.xml:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
      package="com.igs.dinosaur"
      android:versionCode="1"
      android:versionName="1.0">
    <uses-sdk android:minSdkVersion="9" />
 
 <uses-permission android:name="android.permission.CAMERA" />
 <uses-feature android:name="android.hardware.camera" />
 <uses-feature android:name="android.hardware.camera.autofocus" />
 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
 
    <application
  android:icon="@drawable/app_icon"
  android:label="@string/app_name"
  android:debuggable="true">
  
        <activity android:name=".MainActivity"
                  android:label="@string/app_name">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
  
  
    </application>
</manifest>

Result

     If you call the AndroidCameraAPI.TakePhoto() in Unity, it will take a snapshot immediately. The result is shown below. As you can see, after adjust the exposure, the camera  looks better now.

Conclusion

Following list the advantages and disadvantages:

Advantages:

1. You can adjust many parameters provided by android API.
2. The camera will be auto-focus when you take a picture (There'll be no auto-focus if you just take a
    screenshot from the Unity default WebCamTexture).

Disadvantages:

1. Despite the complicated procedure, you can't change the Bundle ID now. Otherwise you should
    also change the package name in AndroidStudio.
2. When you call the Android plugin, the Unity will be temporary paused. In this article, you can't
    feel it because the time for calling android plugin is too soon. In the next article, I will make a
    preview layout for camera and the system will stay in android unless the user press "Return"
    button.  In that situation, the Unity will be paused which means that the Unity can't read any IO
    and stop playing musics when the system go down to android.

How to Access Android Camera in Unity (I): Using Unity WebCamTexture

Overview

    This article will show you how to access Camera in Android device by calling the Unity built-in class: WebCamTexture. This is simply the easiest way to access the native camera.
    However, there're not so much parameters (such as Exposure, Zoom Level) can be adjusted. But if none of those in your concern, WebCamTexture might be the best way to approach. Since you can combine it with other Unity utilities (such as UI, sound) with WebCamTexture easily.
    Moreover, there're many websites have discussed how to use WebCamTexture, but none of them use the new Unity feature UGUI till now, this article will also show you how to combine WebCamTexture with UGUI.

Background

    Our company want to develop a game in a device which use Android as its OS. During the game, the camera in the devices will take some snapshots of the Players' face and print the picture for them as a souvenir.

Solution

Note: This article is original combine WebCamTexture and UI.Image. However, it will create a big array to contain pixels each frame. The game will finally run out of memory. Fortunately, this website suggest the use of UI.RawImage. This article has been updated to use such RawImage.

1. Setup the Unity Editor by following arrangement:
    (1) A parent Game Object "WebCamModule" that has Canvas as its Component.
    (2) A child Game Object "VedioImage" that has RawImage component. The RawImage is used to
          preview the camera.

    (3) Attach the "WebCamModule" Game Object by the following script:

 public class WebCamModule : MonoBehaviour
 {
   private WebCamTexture webCameTexture;
   [SerializeField] private RawImage vedioImage;

   protected void Awake()
   {
    this.webCameTexture = new WebCamTexture(1280, 720);
    this.vedioImage.texture = this.webCameTexture;
   }

   public void Play()
   {
     this.webCameTexture.requestedWidth = 1280;
     this.webCameTexture.requestedHeight = 720;
     this.webCameTexture.Play();
   }
   public void Stop()
   {
     this.webCameTexture.Stop();
   }
   public Texture2D TakePhoto()
   {
    Texture2D photoShot = new Texture2D(this.webCameTexture.width, this.webCameTexture.height);
    photoShot.SetPixels32(this.webCameTexture.GetPixels32());
    photoShot.Apply();

    return photoShot;
   }
 }

    (4) The arrangement of Unity Editor will be:

Result

Build and run in the android device, you will find:

   As you can see, the resulting photo is too dark. This is due to the default exposure of the camera is too low (Unity just use the default camera parameters from devices). This might because we connect the android device with external WebCam. If you use the default camera that has been built-in your android phone, there might be no problem. Since the vendor of your phone may adjust the default parameters of the camera before it has been released...
   So, is there a way to fix this? The answer is no if you want to adjust the exposure by WebCamTexture, since it nearly provide nothing to adjust the camera settings.
    You can however, brighten the photo by some algorithms pixel by pixel. But it might be better if you can adjust the camera parameters directly (by invoke android API). In the next article, I will show you how.

Conclusion

Here are the conclusion of Unity built-in WebCamTexture:

Advantages: 

Easy to setup, and can be combined with UGUI, sounds, or other Unity utilities.

Disadvantages: 

You can not adjust camera parameters such as exposure, zoom level. Moreover, you get the picture just by screen shot, instead of requiring the camera to take picture by some command. This means that the camera will not adjust its focus when you take picture and will finally get a not-so-good photo.

In the next article, I will show you how to access native camera by calling the android camera API.

Serveral Ways to Improve your VIM!

   Since I reinstall Ubuntu several times, it would be better that I write a memo to record some process to set up VIM in Ubuntu. This article reminds me some Plugins of VIM that I usually used.

NERD_Tree

This is a Plugin that can show you the File Architecture.

Set Up

1. Download it from here.

2. In your home directory, create a new ".vim" directory (if you have no VIM plugin yet).
3. Move everything in NERD_Tree you just download to ".vim" directory. And your ".vim"
   directory  would look like:
    doc  nerdtree_plugin  plugin  syntax
4. Add the following line to ".vimrc" file in your $HOME directory, so that when you press F2 in
    your keyboard, the NEARD_Tree will comes out.

nnoremap <silent> <F2> :NERDTree<CR>

Usage

1. When you want to see the file architecture, press F2, and if you want to open a file in current
    tab, just press Enter when your cursor is on that file name.
2. If you want to open a file in a new tab, just hit "Shift" + "t" when your cursor is on that file name.
3. You can use "Ctrl" + "w" + "w" to switch between windows in VIM.
4. Here is some commands (which already shiped with VIM) that can  control the window or tabs:
     (a) ":sp" + "FileName": Open a new file in the horizontal split window (so that you have two
           window (the top and the bottom window). If you don't specified FileName, VIM would open
           the same file.
           ":vs" + "FileName": Likewise, open a new file in the vertical split windows.
     (b) "Ctrl" + "w" + "T": If you have splitted window, you can use this command to move the split
          window to a new tab.
     (c) "Ctrl" + "w" + "+": If you have more than one horizontal split window, it would expand your
          activate window (in which your cursor in) expanded by one more line (and shrink the others
          by one line in the same time).
           "Ctrl" + "w" + "-": The opposite of above.
     (d) "Ctrl" + "w" + "r": If you have more than one split windows, this will change the two
          windows upside down.
     (e) ":tabm +1": If you have more than one tabs, you can move the current tab to right by this
           command.
          ":tabm -1": Likewise, you can use this command to move current tab to left.

TagList

You can use this plugin to see the class, functions, enums in the file.

Set Up

1. Download TagList from here.
2. Like NERD_Tree, Unzip the file you just download, and move them to the ".vim" directory. But
    be careful not to replace the file that belong to NERD_Tree. For example, after you set up the
   NEARD_Tree or other plugins, you would have a "doc" and "plugin" directory in your ".vim"
   directory. Now move files inside "plugin" (which is in the TagList directory you just Unzip) to
   the ".vim/plugin". Other plugins of VIM can be installed in the similar way.

Share Clipboard between VIM and Linux

   Have you ever want to search all files to find a keyword (e.g. a function) been referenced?
One way to do this is to copy that keyword in VIM and move to the other terminal and use the "grep" command. This can be done by use the mouse right-click and select "copy" in the VIM and move to the other terminal and type "grep '" and use the mouse right-click and select "paste" to paste the keyword. And finally you search that keyword by "grep" command.
    Isn't that kind of silly? And the reason you use VIM is to avoid your hand switching between keyboard and the mouse...
   Here is the solution: Make VIM share clipboard with the system! Do do so, you must install
the vim-gnome. The normal VIM fail to share its clipboard with system (if you want to see the detail, please see this article).
Fortunately, change from normal VIM to VIM-Gnome is easy. You just need to download it by:

sudo apt-get install vim-gnome

And it will read your original .vimrc setting without any further adjustment and must of the VIM plugins can be used directly. Now, add the following line in your .vimrc:

vnoremap <C-c> "+y

And when you in the command mode of VIM, select keyword by "v" and copy it by "Ctrl" + "c".  Now move to the other terminal in the other tab by "Ctrl" + "Page Down" (or, use "Ctrl" + "Shift" + "t" to create a new terminal in the new tab), and now you can type "Ctrl" + "Shift" + "v" to paste the keyword!

Gtags

    Do you want to find a plugin that can lead you to the definition of a function or has the ability to "find all reference of a variable or function"? Although the Ctags and Cscope support part of this, they don't support some C++ features (such as function overloading, class... etc). Therefore, that's why I recommend using Gtags.

Set Up

1. Download it from here.

2. You should also download the ncurses that will be used by Gtags.

3. Build ncurses in the command line:

        ./configure
        make
        sudo make install

4. Build Gtags:

        ./configure
        make
        sudo make install

5. Copy Plugin to Vim:

        cp /usr/local/share/gtags/gtags.vim $HOME/.vim/plugin

6. Add following line to your .vimrc:

        nnoremap gd :GtagsCursor<CR>
        map <C-n> :cn<CR>
        map <C-p> :cp<CR>
        nnoremap <C-f> :Gtags -r<CR><CR>

Thus, you can type "g" + "d" on a function, the VIM will go to its definition, and "Ctrl" + "f" to find all references (in this case, you can type "Ctrl" + "p" to go to the previous reference, and type "Ctrl" + "n" to go to the next reference).
 

QuickFix

    QuickFix is built in VIM already, you can type ":copen" in the VIM command mode to open it (the QuickFix will reside in the bottom of the window). And when you want to compile your code, you can type ":make" in the VIM command mode. Then the code will start compiling, and show its compile info in the QuickFix window (as the first figure in this article).

Finally, here is my final .vimrc:

" Record the last preview line
au BufReadPost * if line("'\"") > 0|if line("'\"") <= line("$")|exe("norm '\"")|else|exe "norm $"|endif|endif

" highlight search
set hlsearch

" Quick leave by F12
nnoremap <silent> <F12> :q<CR>

" Refresh by F5
nnoremap <silent> <F5> :edit<CR>

" Plugins
nnoremap <silent> <F2> :NERDTree<CR>
nnoremap <silent> <F3> : Tlist<CR>
nnoremap <silent> <F4> :copen<CR>

" Copy
vnoremap <C-c> "+y

" Gtags
nnoremap gd :GtagsCursor<CR>
map <C-n> :cn<CR>
map <C-p> :cp<CR>
nnoremap <C-f> :Gtags -r<CR><CR>

2015.07.12
Joshua P. R. Pan