How to use google_ml_kit package in flutter for text recognition

Hello everyone, this is my very first blogpost. I decided to write this blog because i found very little blogs about explain how google_ml_kit package works in flutter, There are loads of blogs and videos explaing about firebase ml kit. But there very few blogs explaining google’s standalone on-device machine learning sdk for google_ml_kit. So I hope this will be useful for someone who wants to integrate google_ml_kitto flutter app.

In this blog we will discuss about how to integrate google’s ml kit flutter plugin with flutter for machine learning services. This is an on-device standalone package for android and iOS and requires no firebase setup.

So lets start by adding the package to pubspec.yaml

dependencies:
  google_ml_kit: ^0.7.2

Follow below steps to configure ML kit to the flutter project

For Android

Go to project/andorid/app/main/AndroidManifest.xml and add below code in <application> tag.

WARNING: DO NOT ADD INSIDE ACTIVITY TAG

<application ....>
<meta-data
        android:name="com.google.mlkit.vision.DEPENDENCIES"
        android:value="ica,ocr,face" />
</application>

Also check the minSdkVersion of your project. google_ml_kit sdk requires minSdkVersion of 21 to perform.

Check the minSdkVersion by going to android>app>build.gradle

defaultConfig {minSdkVersion:21}

For iOS

Go to Project > Runner > Building Settings > Excluded Architectures > Any SDK > armv7

# add this line:
$iOSVersion = '10.0'

post_install do |installer|
  # add these lines:
  installer.pods_project.build_configurations.each do |config|
    config.build_settings["EXCLUDED_ARCHS[sdk=*]"] = "armv7"
    config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = $iOSVersion
  end
  
  installer.pods_project.targets.each do |target|
    flutter_additional_ios_build_settings(target)
    
    # add these lines:
    target.build_configurations.each do |config|
      if Gem::Version.new($iOSVersion) > Gem::Version.new(config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'])
        config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = $iOSVersion
      end
    end
    
  end
end

Enough setup. Now you are all set to use the package.

Using the Text recognition feature

For the sake of simplicity I will just show how to use text recognition service in this article.

First we need an image for analyzing. So we choose a image from gallery or can use camera. For this purpose we can use image_picker package.

The ImageModel is a custom model I created for images to abstract the image class and only get the image path from the image.

The above code will pick image from gallery and will return to image provider which in return will provide the image to our UI. In case if you are not aware of provider and how it works, it is a simple state management technique that separates our UI from our business logic. Whenever we have an image or update the image it will notify the UI about the change.

After getting the image we want to analyze the image for texts.

For analyzing the image we use TextDetector class from Google ML kit. TextDetector has processImage method which analyzes the image and returns the text.

processImage method returns text in the form of RecognisedText model. RecognisedText model has three main properties for texts block, line and element. Block specifies the blocks in the image. Each block can have multiple lines and each line can have multiple element. Elements are simple words present in the image. For more understanding refer to the below image.

Other than block, element and line we can also get the coordinates of the text in the image. We can use these properties to highlight the text in the image itself.

And this is how text recognition of google_ml_kit package works in flutter. Other than text recognition we can also do vision based machine learning tasks such as face detection, pose detection, barcode scanning, image labelling, object detection and tracking, digital ink recognition.

google_ml_kit also provides natural language based machine learning features such as language identification, on-device translation, smart reply and entity extraction.

I will also cover the other google_ml_kit features in upcoming blogs.

The GitHub repo for complete code for text recognition is given below.

https://github.com/Aaq007/Text-Recognition-App