How to Get Text inside a Canvas using Webdriver or Protractor

https://stackoverflow.com/questions/43609429/how-to-get-text-inside-a-canvas-using-webdriver-or-protractor

 https://sqa.stackexchange.com/questions/3253/how-to-automate-the-action-on-a-canvas-object-when-the-canvas-element-has-no-na

0

To work inside a canvas you can use the Kantu web automation software or Sikuli. Both solutions are based on screenshots, image comparison and OCR, and can get the text inside the canvas.

You mark the data that you want to extract ("scrape") by drawing pink frames around it. Kantu then retrieves the data by using OCR. With Sikuli, you would need to find out the coordinates first.

Or, as JeffC suggested, use your current software to take a screenshot, and then OCR it. For OCR, there is the well-known Tesseract lib or use an online service like the free ocr.space.

1

The basics are that you can't with Selenium. the CANVAS tag is like an applet in the page. It doesn't actually contain any HTML. There are a few options:

1. If you have access to the devs, you can have them expose an API for you so that you can access text, etc. using Javascript from your Selenium script. If it's part of some library, etc. the library itself may provide an API that you can use. This is the most reliable option.

2. For executing actions, you can use coordinates. You can base all click, etc. actions off coordinates but this is highly dependent on the browser rendering, screen resolution, etc. This will not help you get text out of the CANVAS though.

3. For the text, you really don't have any options to get the text directly. You could take screenshots and verify the text after the run is complete but that's about your best option. If you wanted to get really fancy, depending on the text, etc. you may be able to find an OCR library that will be able to extract the text from the screenshot that you took.

Canvas does not separate the graphics it drew included text. You need OCR. You may use a tool to cut the canvas image to clipboard and invoke a program to paste the image to preprocess, and perform OCR. Image preprocessing will extract the region of interest which is the text to improve OCR accuracy.