Automation through scripting, as you may know, is a process of making a computer perform repetitive tasks by itself without human intervention. Normally this is achieved by coding scripts in languages such as Java, or Objective C or Python.
An exciting new project called Sikuli has been developed by the folks at User Interface Design Group, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) that takes the pain away from learning a scripting language or creating an automation script. All you have to do is take screenshots and put them together using some basic commands.
Sikuli uses simple mouse commands like click, double click, right click, hover and drag-and-drop and keyboard commands like type and paste. You tell the program what action to perform by selecting the appropriate command from the list on the left or by typing it, and also tell on which part of the screen to act on by including a screenshot of the area.
Sikuli includes a screenshot taking tool, or you can use any third party program to capture screenshots. Sometimes the built in screen capture tool is not enough, for example, when you have to take screenshot of open menus. In such cases it becomes necessary to use another screen-grabbing tool. It doesn’t matter how you grab a screenshot, just make sure there is enough context in the image for the program to identify it on the screen.
To demonstrate how Sikuli works, let me start with an example.
The first line of the script instructs the program to click on the arrow that hides the Quick Launch bar icons. The second line tells it to click on the Google Chrome icon. The third line makes it click on the address bar to gain focus and type ‘imdb.com’. The final line tells it to press Enter. When the script is run, it performs exactly as instructed and IMDB website is opened in Chrome.
Okay, that was easy. Let’s try one a little more complicated.
This time I’m going to create a script that performs the following steps:
- Launch VLC media player from the Start menu
- Click on Media> Services Discovery>Shoutcast Radio listings
- Check to see if Playlist is enabled. If not, enable Playlist by clicking on the Playlist button.
- Click on ‘Shoutcast Radio Listings’
- Scroll the window down to the precise location
- Double click on ‘Soft Rock’ to start the radio
This is the script.
As you can see, using commands like exists and conditional statements such as if-else, we can create pretty complex decisions making scripts. The commands you see on Sikuli’s editor is only a small subset of it. The program has a lot more to offer. Interested users who wish to delve more into it can do so by going through the documentation on the website.
Sikuli is amazingly fun to play with. Give it a try, you will like it.
Comments
Post a Comment