Sounds like firescale to me, too. If it is, then polishing will just shine it up. You'll have to sand or file it down and then polish. The black silicone rubber wheel works pretty well for this, but keep it moving or you'll end up with a groove in your metal.
One other comment...You said you're working with a small butane torch. I've used butane in my home studio for over 20 years and for about 70% of my work, including large and heavy pieces. The downside is that you cannot control the heat or flame as well as with propane or acetylene and a jeweler's torch, so firescale and overheating (burning or melting) can be a problem. You might want to get a larger butane torch if you are going to work with heavy wire or sheet, or consider getting a jeweler's torch and another type of fuel for heavy and/or delicate pieces. Either way, as Loopy said, make sure you heat the whole piece and don't be in a hurry to get to the solder seam.
Hope this helps.