Inbreeding is generally deleterious, even in flowering plants. Since inbreeding raises the risk that bad copies of a gene will be expressed, inbred progeny suffer from reduced viability.

Many flowering plants are able to recognize and reject their own pollen, thereby preventing inbreeding despite the plants' hermaphroditic nature. This mechanism is a complex trait that involves the interaction of a gene that tags the pollen with an identifier molecule, and a gene that produces a molecule capable of detecting pollen produced by the same plant.

Evolutionary biologists have often argued that once complex traits are lost, they are seldom regained. But a new study, led by biologists at McGill University and published in the journal PLOS Biology, suggests that this may not be the case for self-pollen recognition.

In the evolutionary lineage leading to the genus Leavenworthia (a plant group related to canola and cole crops such as broccoli and cabbage), the ancestral genes that code for self-pollen recognition were lost. But the self-pollen recognition function in Leavenworthia appears to have been taken up by two other genes that originally may have had a different role -- for example, in pathogen recognition.

"Self-incompatibility," the pollen-recognition system that enables plants to avoid the inbreeding caused by self-pollination, involves a pair of tightly linked genes known as the S locus. In this study, the researchers analyzed the gene sequence, genome organization, and gene evolutionary history of S loci in members of the Brassicaceae family, which includes plants of the genus Leavenworthia.

"We conclude that both genes that comprise the ancestral S locus in the Brassicaceae were lost in Leavenworthia," says McGill researcher Sier-Ching Chantha, lead author of the study. Our analyses show, however, that plants of this genus have two other linked genes that exhibit patterns characteristic of an S locus.

These genes occupy the same genomic position in Leavenworthia as do two non-S-locus genes in a related species. We suggest that these genes have evolved to assume the function of the pollen recognition system of self-incompatibility in Leavenworthia."

How plants avoid inbreeding, and the related topic of S locus evolution have been important research subjects for plant biologists. There can be hundreds of variants of a single S-locus in individual plant populations-a very unusual situation. In the animal world, a similar pheomenon is the many variations in immune-system genes.

Immune system genes in animals, like the S locus in plants, are also involved in recognition, though in the case of immune genes it is foreign antigens rather than pollen types that are recognized. It seems that the recognition function can act in both systems to allow the evolution of large amounts of genetic diversity.

"Francois Jacob, the famous French biologist, once compared the action of natural selection to that of a tinkerer who uses the materials around him to produce a working object," notes McGill biology professor Daniel Schoen, the corresponding author of the study.

"The evolution of the genes involved in self-pollen recognition in Leavenworthia provides a compelling example of this idea, and lends credence to the notion that the loss of complex traits may not always be irreversible."

The study's other co-authors are Adam C. Herman and Adrian E. Platts of McGill's Department of Biology and Xavier Vekemans of Universite Lille 1 in France.