Turn photo into painting: Using image blending for foreground and background

In the previous post we discussed how to use different mixing level for foreground and background, and the example worked really nicely. However, when the foreground and background has a huge contrast, mixing foreground and background can be challenging. Let's take a look at an example. This is a wedding photo taken at sunset.

We want to make it more artistic, and we chose this style from photopaint.us:

With low mixing level, the result looks like this. It looks quite nice at first sight, but if you look more carefully, you will see that there are some noisy patterns in the people and in the dark ground. Those noisy patterns don't look nice.

We actually want to use the people and the dark ground in the original photo, so we created the following mask:

By simply overlaying the original photo and the stylized photo using the mask above, we get the following image:

Unfortunately because the contrast and color in the original image and the stylized image look too different, you can see that there is a clear "cut line" near the dark ground, which looks awkward. Fortunately, we can use the image pyramid technique to create a smooth transition between the original photo and the stylized image. We are using this script to make the image-pyramid based blending. The result looks like this. This actually looks quite nice and smooth!

Turn photo into painting: Using different mixing levels for foreground and background

Most of the pictures are taken with people in some kind of background scene. We usually want to make the background scene artistic, but we may not always want to stylize the people that much because we still want the people in the picture to look like the original people. In this post we demonstrate the technique of using two different mixing levels in photopaint.us to create two different stylized images, and use grabCut to separate the foreground and background and then combine the low mixing foreground and the high mixing background. Here we start from the raw picture:

To stylize this wedding photo, we used the following style:

We created two stylized images with a high mixing level and a low mixing level:

 

high mixing

low mixing

We can see that the image with high mixing level looks quite artistic, but the people in the picture is more distorted than we want them to be. The people in the low mixing level image look nice and real, but the background is not too different from the raw picture. To combine the two images, we use grabCut algorithm to create a mask like below. The exact code which created this mask can be found here.

 Then combing the high mixing and low mixing image using the mask, we get the following image. See how the background is very artistic while the foreground (the people) are still nice and real.

Below is another example using similar technique. The details of the process is omitted due to similarity.

Original photo

\

 Style

High Mixing

Low Mixing

 

 Mask

 

 

Combined

 

 

Turn photo into painting: How to choose style

There are more than 90 styles in photopaint.us. How should we choose the most appropriate style for a photo. Honestly, I think the answer is just try as many different ones as possible. It is hard to guess what a stylized image would look like for a photo with a given style. Here is an example. The original photo is just a portrait of a girl. There are a few portrait styles in photopaint.us, but there is no such a rule that you are supposed to use a portrait style for a portrait photo. Below shows 5 different stylized images using 5 different styles that has completely nothing to do with portrait, but yet the results looks very good. So the conclusion is: just try it out!

Turn photo into painting: create a Panorama with photopaint.us

We have previously introduced photopaint.us. One advantage of this site is that it allows users to create high resolution stylized images, and this is particularly good for creating panorama images. Panorama photos are very popular nowadays due to the popularity of smart phones, and here is an example of stylized panorama image created with photopaint.us. Below is the original photo that I took in Hawaii:

After stylizing the photo with this style image:

The result look like this. How pretty! prisma has a similar style, but it cuts the photo into a square shape and it is difficult to make panorama photo with it.

Below is another example of Panorama photo. The original picture looks like this:

Using the following style:

The result image look like this:

Turn photo into painting: introduction to photopaint.us

Ever since the publication of neural style, there are a few different websites and smart phone apps leveraging this technology to stylize photo into artistic paints. deepart.io is based on the optimization approach, which is takes tens of minutes or even hours to render high resolution pictures. The advantage is that user can upload any style image they want and create a painting with the style they upload. The popular app prisma, on the other hand, is based on the fast neural style approach. It can render stylized images in seconds, but the style model can take 6 hours or more to train on a high performance GPU computer.

photopaint.us, however, is based on a technology that combines the advantage of both approaches. It can render high resolution stylized images in seconds, but it can also create new style models in a reasonably short time, around 1 hour. This technology allows photopaint.us to provide a large number of style models (around 90 style models for now), and it also allow users to submit their new style images to photopaint.us@gmail.com, and the site administrator will train and add the new style in a day or so.

Another feature of photopaint.us is that it allows users to render high resolution images with the largest side of the image up to 2500px. High resolution is particularly important when there are people in the photo. Consider the following example. The original photo look like this:

When using the non-premium service of photopaint.us, which result in a image with 768px on the largest side, the stylized images look like this:

We can clearly see that the face of the mother and the child is blurred due to the low resolution. When using premium service, which allows image with largest side up to 2500px, the photo rendered with the same style look like this. We can see how clearly the details of the faces, and the people in the image look much nicer comparing with the low resolution image.

In the following few blog post, I am going to demonstrate the capabilities of photopaint.us, and some useful tips on how to choose appropriate styles and resolutions when stylizing your photos into paintings.