Unconventional pairing and electronic dimerization instabilities in the doped Kitaev-Heisenberg model

• Published in: Physical review. B, Condensed matter and materials physics Condensed matter and materials physics, 2014-07, Vol.90 (4), Article 045135
• Main Authors: Scherer, Daniel D, Scherer, Michael M, Khaliullin, Giniyat, Honerkamp, Carsten, Rosenow, Bernd
• Format: Article
• Language: English
• Subjects: Exchange; Flow equations; Honeycomb; Honeycomb construction; Instability; Lattices; Mathematical models; Stability
• ISSN: 1098-0121; 1550-235X
• DOI: 10.1103/physrevb.90.045135

Using the functional renormalization group approach this paper obtains the rich phase diagram of the Kitaev-Heisenberg model on the honeycomb lattice which describes Na sub(2) IrO sub(3). We study the quantum many-body instabilities of the t-J sub(K)-J sub(H) Kitaev-Heisenberg Hamiltonian on the honeycomb lattice as a minimal model for a doped spin-orbit Mott insulator. This spin-1/2 model is believed to describe the magnetic properties of the layered transition-metal oxide Na sub(2) IrO sub(3). We determine the ground state of the system with finite charge-carrier density from the functional renormalization group (fRG) for correlated fermionic systems. To this end, we derive fRG flow equations adapted to the lack of full spin-rotational invariance in the fermionic interactions, here represented by the highly frustrated and anisotropic Kitaev exchange term. Additionally employing a set of the Ward identities for the Kitaev-Heisenberg model, the numerical solution of the flow equations suggests a rich phase diagram emerging upon doping charge carriers into the ground-state manifold (Z sub(2) quantum spin liquids and magnetically ordered phases). We corroborate superconducting triplet p-wave instabilities driven by ferromagnetic exchange and various singlet pairing phases. For filling delta > 1/4, the p-wave pairing gives rise to a topological state with protected Majorana edge modes. For antiferromagnetic Kitaev and ferromagnetic Heisenberg exchanges, we obtain bond-order instabilities at van Hove filling supported by nesting and density-of-states enhancement, yielding dimerization patterns of the electronic degrees of freedom on the honeycomb lattice. Further, our flow equations are applicable to a wider class of model Hamiltonians.