Skip to main content

Sikuli: Create simple automation scripts using screenshots

Automation through scripting, as you may know, is a process of making a computer perform repetitive tasks by itself without human intervention. Normally this is achieved by coding scripts in languages such as Java, or Objective C or Python.

An exciting new project called Sikuli has been developed by the folks at User Interface Design Group, MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) that takes the pain away from learning a scripting language or creating an automation script. All you have to do is take screenshots and put them together using some basic commands.

Sikuli uses simple mouse commands like click, double click, right click, hover and drag-and-drop and keyboard commands like type and paste. You tell the program what action to perform by selecting the appropriate command from the list on the left or by typing it, and also tell on which part of the screen to act on by including a screenshot of the area.


Sikuli includes a screenshot taking tool, or you can use any third party program to capture screenshots. Sometimes the built in screen capture tool is not enough, for example, when you have to take screenshot of open menus. In such cases it becomes necessary to use another screen-grabbing tool. It doesn’t matter how you grab a screenshot, just make sure there is enough context in the image for the program to identify it on the screen.

To demonstrate how Sikuli works, let me start with an example.


The first line of the script instructs the program to click on the arrow that hides the Quick Launch bar icons. The second line tells it to click on the Google Chrome icon. The third line makes it click on the address bar to gain focus and type ‘’. The final line tells it to press Enter. When the script is run, it performs exactly as instructed and IMDB website is opened in Chrome.

Okay, that was easy. Let’s try one a little more complicated.

This time I’m going to create a script that performs the following steps:

  • Launch VLC media player from the Start menu
  • Click on Media> Services Discovery>Shoutcast Radio listings
  • Check to see if Playlist is enabled. If not, enable Playlist by clicking on the Playlist button.
  • Click on ‘Shoutcast Radio Listings’
  • Scroll the window down to the precise location
  • Double click on ‘Soft Rock’ to start the radio

This is the script.


As you can see, using commands like exists and conditional statements such as if-else, we can create pretty complex decisions making scripts. The commands you see on Sikuli’s editor is only a small subset of it. The program has a lot more to offer. Interested users who wish to delve more into it can do so by going through the documentation on the website.

Sikuli is amazingly fun to play with. Give it a try, you will like it.


Popular posts from this blog

How to Record CPU and Memory Usage Over Time in Windows?

Whenever the computer is lagging or some application is taking too long to respond, we usually fire up task manager and look under the Performance tab or under Processes to check on processor utilization or the amount of free memory available. The task manager is ideal for real-time analysis of CPU and memory utilization. It even displays a short history of CPU utilization in the form of a graph. You get a small time-window, about 30 seconds or so, depending on how large the viewing area is.

How to Schedule Changes to Your Facebook Page Cover Photo

Facebook’s current layout, the so called Timeline, features a prominent, large cover photo that some people are using in a lot of different creative ways. Timeline is also available for Facebook Pages that people can use to promote their website or business or event. Although you can change the cover photo as often as you like, it’s meant to be static – something which you design and leave it for at least a few weeks or months like a redesigned website. However, there are times when you may want to change the cover photo frequently and periodically to match event dates or some special promotion that you are running or plan to run. So, here is how you can do that.

Diagram 101: Different Types of Diagrams and When To Use Them

Diagrams are a great way to visualize information and convey meaning. The problem is that there’s too many different types of diagrams, so it can be hard to know which ones you should use in any given situation. To help you out, we’ve created this diagram that lays out the 7 most common types of diagrams and when they’re best used: