Sounds like a pollinization problem. The action you describe is normal for the male flowers (if, when it falls off you're left a straight stem)... plants normally have both male and female flowers and if you've only planted a few seeds, you could have predominatly male flowers. Some growers find they need to pollinate by hand with a Q-tip. You can alleviate the problem next year by planting more seeds as well as planting flowers like Bee Balm (monarda) to attract bees to do the pollinization for you...