Android Script OCR Tutorial: How On-Device Offline PPOCR Text Recognition Works, and How to Use It

Why local OCR: the idea, and two routes

An automation script has to look at the screen, make a decision, and tap a button. Most of the time that is the control tree: the system accessibility service parses the current UI into a node tree, and each button, input, and text node has an id, text, description, and position. Finding a control by text and then tapping it is less sensitive to resolution. One class of text the control tree cannot read:

Scene	Why it cannot be read	What OCR does
Game dialogs / announcements	The game engine draws the frame, with no standard control nodes	Read the text in the dialog image directly
Custom-drawn text (Canvas)	The text is painted, and it is not in the control tree	Read the text on the canvas
Image copy / promo images	The text is part of the image	Run OCR on the image region
Captcha-like content	Designed so a machine cannot read it, and usually with no text interface	OCR reads it (whether you may use it depends on whether the scene is compliant)

This content has to be read by treating the screen as a picture. OCR (Optical Character Recognition) does that job: a picture goes in, and the text in the picture comes out with its position. In Bluetooth HID / OTG HID mode, the script does not depend on the accessibility service, the control tree is not available, and the main way to read text is OCR (that mode only supports image matching, OCR, taps, swipes, and other basic operations). EasyClick OCR is no-root, supports Android 5.0 through the latest system, and the full PPOCR-V4/V5/V6 family is free and recognized locally offline (OCR docs).

The other approach uploads the picture to a cloud OCR API and waits for the server to send the result back. Inside an automation script, the two differ like this:

Dimension	Cloud OCR	On-device OCR (on the phone)
Latency	Screenshot, upload, queue on the cloud, send back, affected by the network round trip	Computed locally, no network step
Privacy	Screen content is uploaded to a third-party server	Data does not leave the device
Cost	Per call / by volume	The whole family is free
Offline	Unusable when the network is down	Recognition continues offline
Stability	Depends on the server and network quality	Does not depend on any external service

Scripts often recognize once per screenshot, thousands to tens of thousands of times a day. A cloud setup waits on the network and stops when the network drops. On-device OCR puts the recognition engine and the model on the phone and computes locally. The docs also offer online recognition types (such as Baidu online recognition, baiduOnline, and the EC PC-side paddleOcrOnline service), plus a free PC-side PPOCR-ONNX program (it runs on Windows and is called over HTTP by Android / iOS / HarmonyOS Next, default port 9022, PPOCR notes). Use those when a computer is right there and you need more throughput. The default PPOCR usage in EasyClick is offline recognition on the phone.

Recognition has two stages. Detection finds where the text is in the picture and boxes each line or block. Recognition turns the image inside each box into characters. Each stage is one neural network model, or a group of them. The model files ship with the app (the path is configurable; if you do not set it, the built-in files are used). The whole pass finishes on the phone: no network, no uploaded picture, no per-call fee. PaddleOCR is the open-source OCR project in Baidu’s PaddlePaddle ecosystem, and PPOCR-V4 / V5 / V6 are its model versions. EasyClick ships this family. The docs confirm PPOCR-V4, V5, and V6, and V6 needs Android 8.0 / API 26 or later (OCR docs). When tuning, “detection box” and “the text box did not enclose the text” refer to the first stage. The padding below is what you adjust for that stage.

How to pick a model, and how to call it from a script

Look at the system version first. Try the rest against real results. The docs do not say which one is definitely faster.

initOcr type	Model	Notes	Doc example version
paddleOcrOnnxV4	PPOCR-V4	ONNX implementation, full parameters (padding, maxSideLen, text-direction detection, and more)	11.26+
paddleOcrOnnxV5	PPOCR-V5	ONNX implementation, same parameter family as V4	11.26+
paddleOcrNcnnV5	PPOCR-V5	ncnn implementation: ncnn is a neural-network inference framework optimized for phones	11.28+
paddleOcrOnnxV6	PPOCR-V6	Official ppocr-sdk plus ORT, default modelTier=small	12.4.0+, needs Android 8.0+

On Android 8.0 and later, prefer paddleOcrOnnxV6. It is the latest model (official ppocr-sdk plus ORT), the default tier is modelTier=small, and the current latest EC version 12.4 already supports it. Below Android 8.0, use paddleOcrNcnnV5 or paddleOcrOnnxV5. The docs say older systems cannot run V6. The docs do not give an accuracy or speed comparison among V4, V5, and V6. Get it running on the default parameters first, then tune for speed, missed characters, and misreads. From EC 9.17 onward, OCR changed from a singleton to multiple instances. One script can initialize several OCR instances at once and switch as needed, for example one V5 and one V6, each covering a class of scenes.

The call is four fixed steps: create an instance, initialize, recognize, release resources.

Step	Function	Role
Create an instance	ocr.newOcr()	Create an OCR instance (multi-instance mode from 9.17 onward)
Initialize	initOcr({type: ...})	Set the recognition engine and parameters, and return whether it succeeded
Recognize	ocrImage(img, timeout, extra)	Recognize a screenshot or image, and return a result array
Release	releaseAll()	Release the resources OCR is holding

Before recognition, call startEnv() to start the automation environment and image.requestScreenCapture() to request screenshot permission (image docs). The example below omits that part and focuses on OCR itself. PPOCR-V6 as the example (needs Android 8.0+):

// 1. Create an OCR instance (once for the whole script is enough)
let ocrEngine = ocr.newOcr();

// 2. Initialize: type selects the engine, here paddleOcrOnnxV6 (PPOCR-V6)
let config = {
    "type": "paddleOcrOnnxV6",
    "modelTier": "small",  // model tier, default small (PP-OCRv6_small)
    "numThread": 2,        // CPU threads to use
    "padding": 32,         // white border around the image, default 32
    "maxSideLen": 640      // scale by the long edge, default 640
};
if (!ocrEngine.initOcr(config)) {
    loge("OCR 初始化失败: " + ocrEngine.getErrorMsg());
    exit();
}

// 3. Screenshot and recognize (call many times on the same instance)
function ocrScreen() {
    let img = image.captureFullScreenEx();
    if (!img) {
        loge("截图失败");
        return;
    }
    // Recognize the whole screen, timeout 20 seconds; extra can also pass recognition parameters dynamically, such as {"padding":32}
    let result = ocrEngine.ocrImage(img, 20 * 1000, {});
    if (result) {
        for (let i = 0; i < result.length; i++) {
            let item = result[i];
            logd("文字: " + item.label +
                 " 置信度: " + item.confidence +
                 " 位置: " + item.x + "," + item.y +
                 " 大小: " + item.width + "x" + item.height);
        }
    } else {
        logw("未识别到文字");
    }
    // Recycle the image when you are done with it
    image.recycle(img);
}

ocrScreen();
ocrScreen();

// 4. Release OCR resources before exit (can also go in the stop callback setStopCallback)
ocrEngine.releaseAll();

ocrImage returns a JSON array. Each item looks like this (the structure follows the official docs):

[
  {
    "label": "领取奖励",
    "confidence": 0.93,
    "x": 11,
    "y": 25,
    "width": 100,
    "height": 40
  }
]

label is the recognized text, confidence is the confidence (higher means more trustworthy), and x / y / width / height are the position and size of the text. With the coordinates, the script can read a word and tap it. For example, only handle text that contains “领取” and whose confidence is high enough, and tap its center:

for (let i = 0; i < result.length; i++) {
    let item = result[i];
    if (item.label.indexOf("领取") >= 0 && item.confidence > 0.8) {
        let cx = item.x + Math.floor(item.width / 2);
        let cy = item.y + Math.floor(item.height / 2);
        clickPoint(cx, cy);
        break;
    }
}

Combined location and parameter tuning

OCR reads the words. Image and color search recognize the look. OCR can misread similar glyphs, and that shows more on low-resolution images and decorative type. Image search can miss because resolution, theme, or scale does not match. You can locate the text region with OCR first, then confirm again with image search inside that region, and tap only when both hit.

// 1. Use OCR to find text that contains "确认"
let target = null;
for (let i = 0; i < result.length; i++) {
    if (result[i].label.indexOf("确认") >= 0) {
        target = result[i];
        break;
    }
}

if (target) {
    // 2. Confirm again near the text with a template image (put the template in the project res directory)
    let template = readResAutoImage("confirm_btn.png");
    // Search region: expand outward from the text box, and stay inside the screen
    let sx = Math.max(0, target.x - 50);
    let sy = Math.max(0, target.y - 50);
    let rect = image.findImage(
        img, template,
        sx, sy,
        target.x + target.width + 50, target.y + target.height + 50,
        0.7, 0.9, 1, 5   // weak threshold, similarity, result count, match method
    );
    if (rect && rect.length > 0) {
        let cx = parseInt((rect[0].left + rect[0].right) / 2);
        let cy = parseInt((rect[0].top + rect[0].bottom) / 2);
        clickPoint(cx, cy);
    }
}

The reverse is common too: confirm the button exists with image search first, then OCR the button region and branch on the text, such as reading a balance, a status, or a countdown. Words go to OCR. Appearance goes to image and color search.

A few high-frequency parameters, with their roles taken from the official docs:

Parameter	Role	How to tune it
padding	White border around the image, used to raise the recognition rate. Increase it when the text box does not enclose all of the text	Raise it when the box is incomplete or characters are missed
maxSideLen	Scale the image by its long edge. Larger is slower and more accurate, smaller is faster and less accurate (in V4/V5, 0 means do not scale)	Lower it for speed, raise it for accuracy
numThread	CPU threads to use (paddleOcrNcnnV5 defaults to 0, and -1 means the maximum CPU)	More threads recognize faster, and CPU use is higher
modelTier	V6 model tier, default small (PP-OCRv6_small)	Pick from the tiers the docs provide

Slow recognition is usually the whole screen being computed. Crop the screen to the text region first, lower maxSideLen, then raise numThread a bit. If small text is not recognized, or characters fall outside the box, raise maxSideLen. Accuracy goes up, and time goes up with it. padding is for text boxes that do not enclose all of the text. Creating the instance and loading the model happen once. The docs say once at the start of the script is enough, and the loop reuses the same instance. Call image.recycle(img) when a screenshot is done, or memory keeps growing. If initialization fails, read getErrorMsg() first. If an old device fails to initialize V6, the system is usually below Android 8.0. Switch to the V5 series. The engine type is fixed in initOcr, and some recognition parameters can be passed again in the extra argument of ocrImage.

---

About EasyClick: A mobile automation script development platform covering three phone ecosystems: Android without root, iOS without jailbreak, and HarmonyOS. It also offers Apple script writing, Apple cluster control, Android cluster control, remote cloud control, and AI phone cluster control. → Explore all products