A much more productive way to garden in this situation is by gardening for flavours that you like to use for cooking. A few pots are a good start. A pot is certainly the most appropriate place for herbs such as mint, which has such a vigorous habit that in time it can overtake your other plants.
But I suggest that once you start cooking with really fresh herbs, you'll find a larger herb garden might be in order. Many herbs do well in marginal ground. Thyme and fennel are two that spring to mind. Drying your herbs is also a wonderful way to flavour food over winter, and ground fennel or coriander seeds are the base of some tasty rubs for meat and fish while tea tree or manuka provides lovely smoking material.
Everybody loves basil which can be grown in conjunction with tomatoes and in many different forms and flavours.
I am a big French tarragon fan. I grow mine in a large pot because it prefers a free draining soil and the one I had in the ground died.
There are pseudo-herbs like rocket which add a peppery bite to any summer salad or stir fry.
Herbs also have an aesthetic component - that is, they look good in the garden. Chives for example make wonderful borders with their distinctive purple flower heads and grassy foliage, as well as having a lovely refreshing taste. Lavender can be used as a herb but its taste can be overbearing. As a rule I don't fancy it but I'm yet to taste lavender ice cream and I'm told lavender shortbread is to die for.
Actually, now that I think about it, my lack of success with lavender in the kitchen probably says more about my cooking than it does about lavender as a herb.
Rosemary, like lavender, makes a strong hedge. The bay laurel provides good garden structure and the opportunity to enjoy a bit of topiary action, but be aware that the bay tree can grow quite big if you let it get away on you. Its vigorous growth and attractive limbs make it a versatile landscaping tree and the dried leaves are a must for many sauces and hearty soups through the winter.
I really can't get enough herbs. Everything we plant, I use, so I've invested a bit of time constructing comparatively big herb bed. With the menagerie of herbal delights that spring up around vegetable garden each year I'm looking good for a very tasty summer indeed.