HDR Cookbook – Complex Selections
For most images, you will need to apply different adjustments to different parts of the image. For example, you may want to increase the saturation of the blue tones on the sky, the yellows and greens on the forest, and the saturation of the red tones on a building’s roof. Making all of these adjustments globally (on the entire image) will not lead to good results. In Photoshop (and most other editing tools) selective editing can be done using layer masks. A layer mask is a grey-scale map that reveals or hides the layer it is applied to depending on whether the respective area in the mask is white (revealing the layer’s bits), grey (partially hiding the layer’s bits or black (completely hiding the layer’s bits).
The simplest way of creating a mask is to create a selection and then use the “Add layer mask” button at the bottom of the Layers palette. There are cases where making such a selection is very simple, e.g. when you have clear edges in your image separating those parts you want to select from those parts you want to exclude from your selection. (e.g. a skyscraper on a blue sky). However, the more interesting shots can make selections very challenging. Take, for instance, a shot with trees in front of a cloudy sky. Furthermore, assume that you would like to treat the blue sky, the clouds and the trees separately. This is a situation that you will find frequently. But how can you select the sky without the clouds and the trees?
Photoshop offers different tools for doing this. We will take a look at each of them in turn and explain how they can be combined.
Related Video
In this episode of Hands-on Photo Tips, I am explaining how you can save time and effort in Photoshop by combining layer masks. If you are applying selective adjustments to your images, you may end up with a number of layers, each with their own mask. Sometimes, combining existing masks into a new one helps you to quickly isolate an area that you need to work on next.
Quick Selection Tool
Learn how to mask like a pro!
The complete guide to layer masking in Photoshop
6-hour video course, 21 lessons, 20+ resources
The quick selection tool is perfect for selecting clearly separated areas (like the skyscraper and the sky). However, selecting the branches of a tree with this tool is extremely difficult or even impossible. I use it for simple selection tasks.
Magic Wand
The Magic Wand selects areas of an image that are similar in color. Pressing the Shift key on your keyboard, you can apply it multiple times to extend a selection. You may select a continuous area or all similar areas in the image with one click. For complex selections, you need to Shift-click repeatedly. The big disadvantage of the Magic Wand (in my opinion) is that the selection is displayed as “marching ants” (animated dashed lines around selected areas). When you click once, it is not hard to identify your selection in the image. When you click multiple times, however, your screen is full of marching ants and identifying what you actually selected is close to impossible. You can press “q” on your keyboard to check the selection using the “Quick Mask Mode” but this is very cumbersome. For this reason, I rarely use it at all.
Color Range
Color range dialog
In the “Select” menu, you will find the “Color Range” item. This tool is similar to the Magic Wand. However, it is perfect for selecting areas with a similar color, which is what I mostly need. Clicking on the menu item brings up a dialog with some sliders and a small preview. Leave everything at its default values. Your main image view will turn black (or almost black). Start by clicking on an area that belongs to the region that you wish to select (e.g. the blue sky) – we will call this area the “target area”. This will turn a certain area around the position you clicked white. Now press the Shift key on your keyboard and click on a another position that belongs the target area. This will extend the selected area. The tool will select every pixel that has the color of the pixel you clicked on or a similar color. The tolerance can be adjusted in the Color Range dialog. By holding the Shift key and successively clicking inside the region that you wish to select, more and more of that region will turn white (or grey) which means that this region is (more or less) selected. When you select the sky in this way, you will notice that your tree branches stay black since their color is quite different from the color of the blue sky. You can also press the Alt key on your keyboard and click on a selected area to remove the selection of this color and similar colors. Be careful not to click on anything that is outside your target area. Once you are content with the selection, click on “Ok” in the dialog. The selection will be generated and you can turn it into a mask by clicking the “Add layer mask” button in the Layers palette. The big advantage of the Color Range tool is that you can see very clearly what you selected at any time. This gives you optimal control over your selection.
Example for a complex selection
Depending on the image, this technique may be used in combination with other selection tools: Use the Color Range tool to select those part of the target area that are at the very border of that area. These are most difficult to select. Then, after turning your selection into a mask, use the brush tool to brush over the remaining black parts of your target area. As you can see on the right side, you can create very complex masks using this technique. Note how delicate the selection of the tree on the right and the windows of the castle is.
Color Channels
If you go to the Channels palette, you can see the different color channels of your image. E.g. in RGB mode, you see the Red, Green and Blue channels and the RGB channel. You can turn any of these channels into a selection simply by pressing the Ctrl key on your keyboard and clicking on the respective channel. This can be used, e.g. for images with a blue sky: Create a selection and then a layer mask from the blue channel (use the channel in which your target area is well separated from the remaining image – either very dark or very bright). Invert the mask if necessary to turn the target area white. In most cases, your target area will not be completely white in the original mask. In this case, go to the Layers palette, activate the mask (click on it) and press Ctrl-L. This will bring up the Levels dialog. Use the controls in this dialog to change the white and black points and the gamma in order to turn your target areas completely white.
Using the Brush Tool on a Layer Mask
You can also create a mask manually using the brush tool (graphics tablet recommended!): Activate the layer you wish to mask and add a new mask (“Add layer mask” button). Invert the mask (Ctrl-I). The mask will now turn black, and the layer will be completely masked (not visible). Choose the Brush from the tools palette (left side of the screen) and press Ctrl-D to reset the foreground and background colors to the default values (white and black). Choose the brush size and the hardness as appropriate. Now, start brushing with the white color to reveal the layer. Make sure the layer mask is selected before you start brushing.
Topaz ReMask
Topaz ReMask is part of the Topaz Photoshop Bundle. It allows you to draw a rough boundary around your target area and to mark you target area as well as the areas outside you target area. Then the tool automatically selects the target area. It also allows you to manually refine the edges after the automatic computation. This is a very important tool that comes in handy when many other methods fail to work.
Combining Different Methods
You can also combine a number of selection methods to get the desired result. I have already hinted to this in the “Color Range” section. This is a very common case. The final selection, even if it merely separates the sky from the foreground is almost never the result of using a single one of the introduced methods. I usually use the Color Range tool to select the very edge of the target area, in order to separate those fine tree branches from the blue sky, for example. In most cases, this leaves some parts of the sky (e.g. clouds) unselected. When the very border is properly selected, I use the brush tool to simply brush away the remaining parts of the target area. This is quick and easy as there are no delicate fine details here. You just brush over those parts with white colors. Sometimes I even use the “Rectangle Tool” to do this as it is even quicker.
Often, even the most sophisticated selection methods are not perfect: At the edges of the target area, some additional pixels or too few pixels are selected. Sometimes, this can be easily fixed by using the “Refine Edge…” dialog in the Select menu. In most cases, however, I use the brush to fix this: Zoom into the image (usually 200-300%) and refine the edges of the mask with the brush. This is a lot of work. But it can really pay off. You have to decide whether it is worth the effort.
Good explanation of selection, didn’t knew about Color range selection, will try it next time
Great description of different methods that I was not aware of – thanks for sharing!
Is Topaz ReMask available with PS Elements? I’m just beginning to understand these techniques and hoping to avoid spending a great deal of money right now.
I found it nevermind. Topaz 3. Using the 30 day trial now.
I’m not sure what you mean by the “brush” tool: all Photoshop tools are “brushes” in some sense. For myself, I use the quick selection tool, then refine the edge using the lassoo, then, when it is as precise as I can get it, I feather the edge by 2 or 3 pixels and adjust. I don’t often put these selections on separate layers. I have used the refine edge dialogue on occasion, which does produce a layer mask, but I have yet to find an ideal combination of variables that allows me to adjust the refine edge selection without getting, when layers are combined, a zone of highlight between the selection and the rest of the image. And if you use the auto-blend to combine the layers, often the result is blown contrast values on the layer you just spent 10 or 15 minutes adjusting.
Long comment, apologies: what do you mean by “brush” in the above description?
Hi Jon Paul,
I simply mean that you can create a layer mask (equivalent to a selection) by brushing manually on a layer mask.