To give final modification to our selection we need refine edge tool. Press Q to get out from quick mask tool. Once you press Q Photoshop the non red area will automatically be selected.
Now press Ctrl+Alt+R/Cmd+Opt+R to open refine edge box. In my case the below shown settings has really done a cool job. You can also read my whole article on Refine Edge tool.
Your selection should look like this.