To extract hair or fur in Photoshop is probably the most challenging job. The selection tools work fine for geometric objects with plain background but with images like this one they can be irritating and time-consuming.
Here is my photo:
I want to cut out the dog and place it on a new background:
The First step is to look at the Channels panel.
Now lets inspect each channel separately. The goal is to choose the one with highest contrast.
But there’s a problem. None of the three channels has a contrast good enough to create a channel mask. Well, we could try using the red channel but there’s a better approach and I’m going to show it to you.
Go to Image>Duplicate and create a copy of your image. Now Image>Mode and change the color mode to CMYK. Lets look at the Channels panel again
As you can see the black channel represents the lightest foreground and the yellow channel represents the darkest background. Now we’ll use Calculations to mix these two channels and form a new Alpha channel with the highest possible contrast between the foreground and the background.
Choose Image>Calculations to open the Calculations dialog box. Set the first channel option to the Yellow channel, the second one to the Black channel, blending mode to Multiply and the Result to New Channel.
Now grab the Dodge tool, set the Range to Midtones, exposure to around 50% and using a soft brush, start painting over the edges to brighten them up even further.
Now switch over to the Burn tool, set the Range to Midtones, Exposure to around 65% and again, using a soft brush, paint over the area where the foreground overlaps the background.
When you’ve done with this part choose Image>Adjustments>Levels (CMD/Ctrl+L) to bring up the Levels dialog box. Adjust settings to increase the contrast between the dog and the background.
Be careful not to go too far. Keeping some grey tones at the edges will result in smooth transition between the dog and its new background.
Now select the Brush tool (B), set the foreground color to white and paint over the remaining areas of the dog’s body. Then change the color to black and finish off the background.
CMD/Ctrl+Click on the Layer’s thumbnail to load the selection. Go to Select>Save Selection and in the dialog box set the options as follows: set the Document option to the original image (not the current one which is a copy), Channel – New, Name – Dog (or whatever you like) and hit OK.
Now switch back to the original image. Load the alternative background and place it below the “dog” layer.
Click on the “dog” layer to make it active. Go to Select>Load Selection and choose “dog” from the drop-down menu.
When the selection is loaded click on the Add layer mask button in the Layers panel to convert the selection into a layer mask
As you can see, it’s not a bad mask but still needs some fine-tuning.
Make sure that the layer mask is selected (click on its thumbnail) and click on the Mask edge button to open the Refine mask dialog box.
Inside the dialog box select the On Layers View. Now select the refine edge tool and paint over the outer edges of the dog. Then play around with sliders until you’re satisfied with the result. Here are my settings:
Now click on the image thumbnail (not the layer mask) to make it active. Grab the Sponge tool; set Mode to Desaturate, Flow to 50% and using a soft brush start painting over the edges to remove the blue color cast.
Then choose Edit>Free Transform and adjust the size and position of the dog. Use Burn/Dodge tool if needed to fine-tune the edges. Also you could run Refine Mask again and shift the edges a little bit more (set the Feather to 1 px and the other sliders to 0).
Here is my result:
Of course this technique won’t work so smoothly for every image. Sometimes you may need to use it in combination with other tools to create more complex selections.