If you find their nest, you can kill them off with a co2 fire extinguisher. A blast form a co2 will freeze them solid.

Another warning about bees and wasps: Don't ever leave a can of a soft drink unattended after it has been opened. The sugar content of the drink will attract bees and they will enter the can to get to the sweetness. If you should drink from the can and and the bee stings your throat, you would probably sufficate before help would arrive. This happened to a friend. His throat and windpipe swelled closed. He died right there from suffication. He was with his family on a picnic outing.