Move mouse to text on screen (OCR) Action(Power Automate Desktop)
The action is to find the specified text and move the mouse to that location.
You can optionally make it click.
This action is useful when the "Mouse move" action does not work well.
- 1. How to use
- 1.1. Parameter
- 1.1.1. OCR engine type
- 1.1.2. Text to find
- 1.1.3. Is regular expression
- 1.1.4. Occurrence
- 1.1.5. Search for text on
- 1.1.6. Search mode
- 1.1.7. Moves mouse from previous position
- 1.1.8. Windows OCR language
- 1.1.9. Use other language
- 1.1.10. Language abbreviation.
- 1.1.11. Language data path
- 1.1.12. Tesseract language
- 1.1.13. Image width multiplier/Image height multiplier
- 1.1.14. Wait for text to appear
- 1.1.15. Send a click after moving mouse
- 1.1.16. Wait before clicking for
- 1.1.17. Mouse position relative to text
- 1.1.18. Offset X, Y
- 1.2. Variables produced
- 1.1. Parameter
- 2. For those who want to learn Power Automate Desktop effectively
- 3. Links
How to use
Drag "Move mouse to text on screen (OCR)" from "Mouse and Keyboard" under "Actions".
Set parameters.
Parameter
OCR engine type
The "Windows OCR engine" has more languages that can be used, so this is basically recommended.
- Windows OCR engine
- Tesseract engine
Text to find
Specify the text that will be used as the mouse move position.
Is regular expression
When turned ON, "Text to find" is treated as a regular expression.
There are a wide variety of regular expressions, but the following are typical ones that are used frequently.
As an example, the string "aB1C2dE" is parsed.
| Regular expression | Description | Result |
|---|---|---|
| [0-9] | Numerical value | 12 |
| [a-zA-Z] | Alphabet | aBCdE |
| B.*2 | Starts with a "B" and ends with a "2". | B1C2 |
Occurrence
Specifies how many times to move to the found text.
Adjust this item if the target text occurs more than once.
If it is not found for that number of times, an error is generated.
Search for text on
Specify whether to search the entire image or only the active window.
The larger the search area, the higher the probability of a match, but the slower the operation.
- Entire screen
- Foreground window only
Search mode
When a sub-area is specified, the position is also specified in X and Y coordinates.
- Whole of specified source
- Specific sbregion only
- Subregion relative to image
A narrower search is possible.
Moves mouse from previous position
Select the speed of mouse movement.
Windows OCR language
This is the language setting when the "OCR engine type" is "Windows OCR engine".
Use other language
"Tesseract Engine" setting.
Turn ON if you want to use languages other than English, German, Spanish, French, or Italian.
Language abbreviation.
"Tesseract Engine" setting.
Set this if "Use other language" is ON.
Select the code from "Lang Code" here.
Language data path
"Tesseract Engine" setting.
Set this if "Use other language" is ON.
Download the "<Lang Code>.traineddata" and "<Lang Code>_vert.traineddata" files.
Specify the directory where the two files are stored.
For example, "jpn.traineddata" and "jpn_vert.traineddata" in Japanese.
Tesseract language
This is the setting for the "Tesseract engine".
If "Use other language" is ON, this parameter is not displayed.
- English
- German
- Spanish
- French
- Italian
Image width multiplier/Image height multiplier
Adjust for high-resolution images and screens.
Wait for text to appear
If ON, waits a specified number of seconds for no text to be found.
If OFF, it will not wait and an error will occur.
Fail if text doesn't appear within
Displayed when "Wait for text to appear" is ON.
The number of seconds to wait for the text to be found.
Send a click after moving mouse
Change to ON if you want the click to occur after the move.
Click type
Displayed when "Send a click after moving mouse" is ON.
Wait before clicking for
Displayed when "Send a click after moving mouse" is ON.
Mouse position relative to text
Select the destination position from upper left corner, upper center, upper right corner, left center, center, right center, lower left corner, lower center, lower right corner.
Offset X, Y
Adjusts the destination in pixels.
Variables produced
LocationOfTextFoundX, LocationOfTextFoundY
The X and Y locations of the search result are stored.
WidthOfTextFound, HeightOfTextFound
The width and height of the search result are stored.
For those who want to learn Power Automate Desktop effectively
The information on this site is now available in an easy-to-read e-book format.
Or Kindle Unlimited (unlimited reading).
You willl discover how to about basic operations.
By the end of this book, you will be equipped with the knowledge you need to use Power Automate Desktop to streamline your workflow.