Skip to main content

Microsoft Develops Algorithm to Depixelate Pixel Art

Over the last 20 years, some tremendous products have come out of Microsoft Research many of which you are already familiar with such as Photosynth, Image Composite Editor, WorldWide Telescope, Surface, Songsmith, Kinnect, Street Slide, Touch Mouse, PowerPivot and so many others.

In a new research paper, Johannes Kopf of Microsoft Research and Dani Lischinski of The Hebrew University describe a new algorithm for converting pixel art (those old school 8-bit video games) into super smooth vectors which can be magnified by an arbitrary amount without image degradation.

Our algorithm resolves pixel-scale features in the input and converts them into regions with smoothly varying shading that are crisply separated by piecewise-smooth contour curves. In the original image, pixels are represented on a square pixel lattice, where diagonal neighbors are only connected through a single point. This causes thin features to become visually disconnected under magnification by conventional means, and it causes connectedness and separation of diagonal neighbors to be ambiguous. The key to our algorithm is in resolving these ambiguities. This enables us to reshape the pixel cells so that neighboring pixels belonging to the same feature are connected through edges, thereby preserving the feature connectivity under magnification. We reduce pixel aliasing artifacts and improve smoothness by fitting spline curves to contours in the image and optimizing their control points.

image

In the examples shown above, each of the images in the middle show the results of Kopf’s and Lischinski’s method as compared to the pixel input on the left and the results of other depixelating algorithms on the right. As can be seen, the new algorithm produces superior results.

Johannes Kopf and Dani Lischinski says that they’ve applied their algorithm to a wide variety of inputs from video games and other software. While they did not invest much time into optimizing their algorithm, it already performs quite well.

Here is a video of the algorithm being applied to Super Mario. Since the algorithm does not run in real-time speed, they dumped frames from a video game emulator to disk and applied their technique to the full frames.

There are many avenues for future work. Obviously, it would be nice to optimize the performance of the algorithm so that it can be applied in real-time in an emulator. Some of the ideas presented in this work can potentially beneļ¬t general image vectorization techniques.

Another interesting direction would be to improve the handling of anti-aliased input images. This might by done by rendering some curves as soft edges rather than sharp contours. A whole new interesting topic would be to look into temporal upsampling of animated pixel art images. If we magnify from a tiny input to HD resolution, locations are quite quantized which might result in “jumpy” animations. Since many modern display devices are operating at a much higher refresh rate than earlier hardware, one could improve the animations by generating intermediate frames.

Comments

  1. It looks very nice, make SMB look awesome.

    ReplyDelete
  2. Can I use this! Just make a little tool so I can use it on my pixel art!

    ReplyDelete
  3. Not yet. They have not released any software based on the algorithm.

    ReplyDelete
  4. Use BenVista PhotoZoom Pro 4

    ReplyDelete

Post a Comment

Popular posts from this blog

How to Record CPU and Memory Usage Over Time in Windows?

Whenever the computer is lagging or some application is taking too long to respond, we usually fire up task manager and look under the Performance tab or under Processes to check on processor utilization or the amount of free memory available. The task manager is ideal for real-time analysis of CPU and memory utilization. It even displays a short history of CPU utilization in the form of a graph. You get a small time-window, about 30 seconds or so, depending on how large the viewing area is.

Diagram 101: Different Types of Diagrams and When To Use Them

Diagrams are a great way to visualize information and convey meaning. The problem is that there’s too many different types of diagrams, so it can be hard to know which ones you should use in any given situation. To help you out, we’ve created this diagram that lays out the 7 most common types of diagrams and when they’re best used:

How to Schedule Changes to Your Facebook Page Cover Photo

Facebook’s current layout, the so called Timeline, features a prominent, large cover photo that some people are using in a lot of different creative ways. Timeline is also available for Facebook Pages that people can use to promote their website or business or event. Although you can change the cover photo as often as you like, it’s meant to be static – something which you design and leave it for at least a few weeks or months like a redesigned website. However, there are times when you may want to change the cover photo frequently and periodically to match event dates or some special promotion that you are running or plan to run. So, here is how you can do that.