Can python read text on screen?

I want to build project that reads texts continuously from part of my pc screen and shows them on the console of pycharm

I am using python 3, all modules are installed using pip from pycharm console.

I used this code:

import time
import cv2
import mss
import numpy
import pytesseract


mon = {'top': 0, 'left': 0, 'width': 150, 'height': 150}

with mss.mss() as sct:
    while True:
        im = numpy.asarray(sct.grab(mon))
        # im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)

        text = pytesseract.image_to_string(im)
        print(text)

        cv2.imshow('Image', im)

        # Press "q" to quit
        if cv2.waitKey(25) & 0xFF == ord('q'):
            cv2.destroyAllWindows()
            break

        # One screenshot per second
        time.sleep(1)

got this error which I couldn't deal with :

C:\Users\ali91\PycharmProjects\OCR\venv\Scripts\python.exe
"C:/Users/ali91/PycharmProjects/OCR/lesson 1.py" Traceback (most
recent call last):   File
"C:\Users\ali91\PycharmProjects\OCR\venv\lib\site-packages\pytesseract\pytesseract.py",
line 252, in run_tesseract
    proc = subprocess.Popen(cmd_args, **subprocess_args())   File "C:\Program Files\Python39\lib\subprocess.py", line 947, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,   File "C:\Program Files\Python39\lib\subprocess.py", line 1416, in
_execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file
specified

During handling of the above exception, another exception occurred:

Traceback (most recent call last):   File
"C:\Users\ali91\PycharmProjects\OCR\lesson 1.py", line 15, in 
    text = pytesseract.image_to_string(im)   File "C:\Users\ali91\PycharmProjects\OCR\venv\lib\site-packages\pytesseract\pytesseract.py",
line 413, in image_to_string
    return {   File "C:\Users\ali91\PycharmProjects\OCR\venv\lib\site-packages\pytesseract\pytesseract.py",
line 416, in 
    Output.STRING: lambda: run_and_get_output(*args),   File "C:\Users\ali91\PycharmProjects\OCR\venv\lib\site-packages\pytesseract\pytesseract.py",
line 284, in run_and_get_output
    run_tesseract(**kwargs)   File "C:\Users\ali91\PycharmProjects\OCR\venv\lib\site-packages\pytesseract\pytesseract.py",
line 256, in run_tesseract
    raise TesseractNotFoundError() pytesseract.pytesseract.TesseractNotFoundError: tesseract is not
installed or it's not in your PATH. See README file for more
information.

Dealing with images is not a trivial task. To you, as a human, it’s easy to look at something and immediately know what is it you’re looking at. But computers don’t work that way.

Tasks that are too hard for you, like complex arithmetics, and math in general, is something that a computer chews without breaking a sweat. But here the exact opposite applies — tasks that are trivial to you, like recognizing is it cat or dog in an image are really hard for a computer. In a way, we are a perfect match. For now at least.

While image classification and tasks that involve some level of computer vision might require a good bit of code and a solid understanding, reading text from a somewhat well-formatted image turns out to be a one-liner in Python —and can be applied to so many real-life problems.

And in today’s post, I want to prove that claim. There will be some installation to go though, but it shouldn’t take much time. These are the libraries you’ll need:

• OpenCV
• PyTesseract

I don’t want to prolonge this intro part anymore, so why don’t we jump into the good stuff now.

OpenCV

Now, this library will only be used to load the images(s), you don’t actually need to have a solid understanding of it beforehand (although it might be helpful, you’ll see why).

According to the official documentation:

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. Being a BSD-licensed product, OpenCV makes it easy for businesses to utilize and modify the code.[1]

In a nutshell, you can use OpenCV to do any kind of image transformations, it’s fairly straightforward library.

If you don’t already have it installed, it’ll be just a single line in terminal:

pip install opencv-python

And that’s pretty much it. It was easy up until this point, but that’s about to change.

PyTesseract

What the heck is this library? Well, according to Wikipedia:

Tesseract is an optical character recognition engine for various operating systems. It is free software, released under the Apache License, Version 2.0, and development has been sponsored by Google since 2006.[2]

I’m sure there are more sophisticated libraries available now, but I’ve found this one working out pretty well. Based on my own experience, this library should be able to read text from any image, provided that the font isn’t some bulls*** that even you aren’t able to read.

If it can’t read from your image, spend more time playing around with OpenCV, applying various filters to make the text stand out.

Now the installation is a bit of a pain in the bottom. If you are on Linux it all boils down to a couple of sudo-apt get commands:

sudo apt-get update
sudo apt-get install tesseract-ocr
sudo apt-get install libtesseract-dev

I’m on Windows, so the process is a bit more tedious.

First, open up THIS URL, and download 32bit or 64bit installer:

The installation by itself is straightforward, boils down to clicking Next a couple of times. And yeah, you also need to do a pip installation:

pip install pytesseract

Is that all? Well, no. You still need to tell Python where Tesseract is installed. On Linux machines, I didn’t have to do so, but it’s required on Windows. By default, it’s installed in Program Files.

If you did everything correctly, executing this cell should not yield any error:

Is everything good? You may proceed.

Before you leave

Reading text from an image is a pretty difficult task for a computer to perform. Think about it, the computer doesn’t know what a letter is, it only works only with numbers. What happens behind the hood might seem like a black box at first, but I encourage you to investigate further if this is your area of interest.

I’m not saying that PyTesseract will work perfectly every time, but I’ve found it good enough even on some trickier images. But not straight out of the box. Some image manipulation is required to make the text stand out.

It’s a complex topic, I know. Take it one day at a time. One day it will be second nature to you.

Loved the article? Become a Medium member to continue learning without limits. I’ll receive a portion of your membership fee if you use the following link, with no extra cost to you.

How do you display text on screen in Python?

How does Python recognize text in an image?

Which Python function puts text on the screen?

How do I open a text file window in Python?