java – Problems with Simple OCR app using Firebase ML Kit-ThrowExceptions

Exception or error:

I am new to Android development but I have been learning on the way as I create my first OCR app using Firebase in Java. I essentially followed a youtube video to create the app but I had the following problems that I needed help with:

1) If I take the picture in landscape, the app can detect the text. However, when I take the picture in portrait, the captured image is rotated 90 degrees and the app cannot detect the text in the image. Whats the simplest way for me to resolve this?

2) Currently I take the picture with the phone’s camera and this image is displayed in the app. I click my detect text button and the text appears. But I would like to see some bounding boxes on the images that shows what Firebase ML kit is seeing.

3) Also when I take a simple screenshot of a smartphone pin screen, the app can detect most of the numbers, but it always seems to miss one. I assume this is because I am using the local on phone version of Firebase ML kit, but is it possible to make it more accurate without running on cloud. I am currently using:

implementation 'com.google.firebase:firebase-core:15.0.2'

implementation 'com.google.firebase:firebase-ml-vision:16.0.0'

Thanks

The following is the code in my main activity (It pretty much the same thing on Firebase):

'''public class MainActivity extends AppCompatActivity {
Button captureImageBtn, detectTextBtn;
ImageView imageView;
TextView textView, outputText;
Bitmap imageBitmap;
static final int REQUEST_IMAGE_CAPTURE = 1;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    ActionBar actionBar = getSupportActionBar();
    actionBar.setTitle("Image Reader");
    actionBar.setDisplayUseLogoEnabled(true);
    actionBar.setDisplayShowHomeEnabled(true);

    captureImageBtn = findViewById(R.id.capture_image_btn);
    detectTextBtn = findViewById(R.id.detect_text_image_btn);
    imageView = findViewById(R.id.image_view);
    textView = findViewById(R.id.text_display);

    outputText = findViewById(R.id.outputText);
    outputText.setVisibility(View.INVISIBLE);

    imageView.setImageResource(R.mipmap.mi2_foreground);

    captureImageBtn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            dispatchTakePictureIntent();
            textView.setText("");
        }
    });

    detectTextBtn.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            detectTextFromImage();
        }
    });
}

public boolean onCreateOptionsMenu(Menu menu){

    getMenuInflater().inflate(R.menu.main, menu);
    return super.onCreateOptionsMenu(menu);
}

private void dispatchTakePictureIntent() {
    Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
    if (takePictureIntent.resolveActivity(getPackageManager()) != null) {
        startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE);
    }
}

@Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    super.onActivityResult(requestCode, resultCode, data);

    if (requestCode == REQUEST_IMAGE_CAPTURE && resultCode == RESULT_OK) {
        Bundle extras = data.getExtras();
        imageBitmap = (Bitmap) extras.get("data");
        imageView.setImageBitmap(imageBitmap);
    }
}

private void detectTextFromImage()
{
    FirebaseVisionImage firebaseVisionImage = FirebaseVisionImage.fromBitmap(imageBitmap);
    FirebaseVisionTextDetector firebaseVisionTextDetector = FirebaseVision.getInstance().getVisionTextDetector();
    firebaseVisionTextDetector.detectInImage(firebaseVisionImage).addOnSuccessListener(new OnSuccessListener<FirebaseVisionText>() {
        @Override
        public void onSuccess(FirebaseVisionText firebaseVisionText) {
            displayTextFromImage(firebaseVisionText);
        }
    }).addOnFailureListener(new OnFailureListener() {
        @Override
        public void onFailure(@NonNull Exception e) {
            Toast.makeText(MainActivity.this, "Error: " + e.getMessage(), Toast.LENGTH_SHORT).show();
        }
    });
}

private void displayTextFromImage(FirebaseVisionText firebaseVisionText) {
    List<FirebaseVisionText.Block> blockList = firebaseVisionText.getBlocks();
    if (blockList.size() == 0) {
        Toast.makeText(MainActivity.this, "No Text Found in Image.", Toast.LENGTH_SHORT).show();
    } else {
        int i = 0;
        String complete ="";
        for (FirebaseVisionText.Block block : firebaseVisionText.getBlocks()) {
            String text = block.getText();
            complete = complete.concat(text+" ");
        outputText.setVisibility(View.VISIBLE);
        outputText.setText(complete);
    }
}

}

How to solve:

Leave a Reply

Your email address will not be published. Required fields are marked *