I just read an article from a norwegian sociology professor a couple of days ago who claims most academics don't realize the full extent of what it is that makes cannabis dangerous. He says there's more to it than cannabis' effect on biology and that the associated cannabis culture can also be harmful to society. In Norway, cannabis smokers largely make up a subculture. You may think subcultures are prone to change, but appearantly the "cannabis culture" has been quite similar since the 60s. People (in Norway) who smoke often adopt part of that culture into their own identity and he argues that it's the culture which is bad because the values that it holds dear are counterproductive to society and could turn people into takers rather than givers.
This is more relevant to a leftist country like Norway than it is to the US as takers are more taxing to the givers here, but I thought it might be an interesting point. Although it's a very general comment about a subculture in a small country, maybe someone could recognize if the concern has validity elsewhere.
Source (norwegian) :
http://www.uio.no/forskning/aktuelt/.../cannabis.html