As a gardener one wants a varied and attractive garden and with interesting combinations of plants that add aesthetic value as well as serve their function as crops. In addition, one would also like to garner more productive use of the space available.
Like we have relationships with each other and different organisms, plants too tend to have different symbiotic relationships with each other. Companion plants do not tend to compete for space, sun, or nutrients or more than one of these. The plants that compete with those that we want to grow have gained the notoriety of being weeds!
Tall plants can provide shade for sun-sensitive shorter plants. Some plants act as decoys in the sense that they attract all the pests et al towards themselves. Some plants have greater affinity to attract bees which aid in pollination while others attract beneficial insects like bees, predators of pests etc. Other plants act as strong repellants for insects, pests, weeds, fungi et al.
Some can fix nitrogen and in doing so make it available to other plants as well. Herbs are known to subtly change the flavor in plants around them. Some plants also cause the yield of companion plants to increase substantially. Some plants may also serve as wind shelters and provide wind breaks.
However it has taken many forms and has been revived currently with the advent of organic gardening, in which the use of man-made fertilizers and pesticides is kept to a minimum. Moreover, square foot gardening, which packs plants as close together as possible, also promotes this concept since compatibility of plants has to be high for them to be in close proximity.
This concept has been around for centuries. Actually this concept is more prevalent in nature than is apparent. After all forests have companion plants growing in close proximity creating actual ecosystems. Monoculture is a very artificial form of growth for crops and I have always felt that some disruption in monoculture is needed to have sustainable cropping. In a way, companion planting is a way of emulating the same so that your plants can thrive without outside help.