Mapping the Melatonin Receptor. 5. Melatonin Agonists and Antagonists Derived from Tetrahydrocyclopent[b]indoles, Tetrahydrocarbazoles and Hexahydrocyclohept[b]indoles

• Published in: Journal of medicinal chemistry 1998-02, Vol.41 (4), p.451-467
• Main Authors: Davies, David J, Garratt, Peter J, Tocher, Derek A, Vonhoff, Stefan, Davies, John, Teh, Muy-Teck, Sugden, David
• Format: Article
• Language: English
• Subjects: Animals; Binding, Competitive; Biological and medical sciences; Brain - metabolism; Carbazoles - chemical synthesis; Carbazoles - chemistry; Carbazoles - pharmacology; Cell Membrane - metabolism; Chickens; Crystallography, X-Ray; Hormones. Endocrine system; Indicators and Reagents; Indoles - chemical synthesis; Indoles - chemistry; Indoles - pharmacology; Medical sciences; Melanophores - cytology; Melanophores - drug effects; Melanophores - physiology; Melatonin - metabolism; Models, Molecular; Molecular Conformation; Pharmacology. Drug treatments; Receptors, Cell Surface - agonists; Receptors, Cell Surface - antagonists & inhibitors; Receptors, Cytoplasmic and Nuclear - agonists; Receptors, Cytoplasmic and Nuclear - antagonists & inhibitors; Receptors, Melatonin; Structure-Activity Relationship; Xenopus laevis
• ISSN: 0022-2623; 1520-4804
• DOI: 10.1021/jm970246n

Tetrahydrocyclopent[b]indoles, tetrahydrocarbazoles, and hexahydrocyclohept[b]indoles have been prepared as melatonin analogues to investigate the nature of the binding site of the melatonin receptor. The affinity of analogues was compared in a radioligand binding assay using chicken brain membranes and agonist and antagonist potency measured in clonal Xenopus laevis melanophore cells. Comparison of the N-acyl-3-amino-6-methoxytetrahydrocarbazoles (2) with N-acyl-4-(aminomethyl)-6-methoxy-9-methyltetrahydrocarbazoles (9) showed that the latter have much higher binding affinities for the chicken brain receptor. Comparison of N-acyl-1-(aminomethyl)-7-methoxy-4-methyltetrahydrocyclopent[b]indoles (10), 6-methoxytetrahydrocarbazoles (9), and N-acyl-10-(aminomethyl)-2-methoxy-5-methylhexahydrocyclohept[b]indoles (11) showed that the tetrahydrocarbazoles had the highest binding affinity with the cyclohept[b]indoles and the cyclopent[b]indoles having rather lower affinities. All of these observations are in agreement with our postulated model of melatonin orientation at the binding pocket in which the 3-amidoethane side chain is in a conformation close to the 5-methoxyl group, as is shown in the X-ray crystallographic structure of 9m and in the energy-minimized computed structures. Separation of the enantiomers of members from each of these three systems was accomplished by chiral HPLC. It was found that in all cases the (−)-enantiomer had a higher binding affinity than the (+)-enantiomer. An X-ray crystallographic analysis of the two enantiomers of 9a showed that the (+)-enantiomer had the (R) absolute stereochemistry. Since the sign of the Cotton curves, determined from circular dichroism studies, was the same for all (+)-enantiomers, it is assumed that the absolute stereochemistry at these centers is identical. In the Xenopus melanophore assay, the tetrahydrocarbazoles 2 (R = H) were mainly weak antagonists, while those with R = OMe were agonists. The biological behavior of the tetrahydrocarbazoles 9 (R = H) depended on R, some being agonists and some antagonists, whereas those with R = OMe were generally agonists. Variation of the R and R1 groups in compounds of type 9 produced both agonists and antagonists. The tetrahydrocylopentaindoles 10 had similar biological properties to the corresponding analogues of 9, but the hexahydrocycloheptaindoles 11 showed a much greater propensity to be antagonists. In all cases the (S)-enantiomers were found to be more poten