Yes the answer is always reuse reuse reuse, and buy items that are used, or if it's new choose items that will last and not have to be thrown out. Or better yet, there is an abundance of natural organic materials that bags are made of now that once they are finally finished being useful, are 100% completely eco-friendly and compostable. (ie. You can use that bag as soil for your flowers and plants.) A lot of the reusable shopping bags are nylon, which is plastic, which is not good. Make sure the material is sustainable and natural. Or renewable... Bamboo bags?
I wouldn't suggest buying another cloth bag unless you needed a cloth bag, and knew that it was made of sustainable natural materials (ie. Not Nylon or Poly.. ) Part of being green is also the interest in reducing the amount we are cosuming. (Even organic bags take energy to be produced, transported, and packaged) instead I would suggest that, if you didnt bring a bag and it's also not possible to carry the veggies and fruit you need, to take the paper bag, as it is also compostable, and can be used for other purposes. Places like Capers offer plastic bags made from corn, which is also able to be composted. But I can't say many other stores offer this. Paper is always the better way to go.
Something to consider: An item that is 99% Recyclable/Reusable is still wasting a lot when you consider how many people are using this item. 1% multiplied by the population of Vancouver alone is a lot.
The question to ask yourself is: Do I buy garbage bags for my home? That is even worse than bags for groceries, as once the garbage is *in* the brand new bags, they are tossed. Regular plastic is a scary thing to place in a landfill. And if it's biodegradable plastic from corn or something, then fabulous, but there is still energy and production and transporation and packaging involved with creating these 'eco-friendly' disposable objects.
I guess I'm playing a bit of the Devil's advocate here, and rambling a bit, but what it comes down to, is if you take that paper or plastic bag for your groceries, you suddenly have a bag for multiple purposes and is being reused. That bag can become a bag for garbage, or recyclables (in which case, would be reused again & again) Does anyone use cloth bags for garbage, and then wash it afterwards? And do you need another cloth bag? It would probably just sit in your shelf and collect dust.
The real answer is always bring a bag with you, even if you don't plan to. I've made that mistake a few times. But I never feel terrible about taking the plastic or paper bag, as i know I will reuse it. I never buy garbage bags, I just use ones that I've collected from various sources. It's very easy to find bags, I'm sure your friends have some around they aren't using.
Keep it natural and eco-friendly and limit your waste and consumption.

The solution is to bring your own reusuable cloth shopping bag (or knapsack, or pannier). Forgot to bring your own? Then buy another cloth bag or quit shopping for the day. 



The environmental cost difference between plastic and paper bags is insignificant. 



The production of plastic bags generates less water pollution, less air pollution, and less solid waste. Plastic bags burn better in sold waste incinerators, and take up less space in landfills.



The production of paper bags uses more material, albeit renewable, and generates more pollution. But paper bags biodegrade much more quickly, both in a landfill and alongside the road. Also, paper bags are accepted by more local recycling programs.