In the fourteenth year of the reign of Emperor Tonghe of the Liao Dynasty (996), an Arabic Shaykh named Naruding came to Nanjing and built a temple in the Liuhe Village in the southern suburbs, namely the Niujie Mosque. Main buildings of the mosque are the Worship Hall (also known as the Kiln Hall), Bangge Building, Stele Pavilion, Wangyue Building, and Living Room (also known as Seven Rooms). The Worship Hall faces east and can accommodate thousands of people to worship at the same time. To the west of the hall is a hexagonal pavilion building with niches full of carved Arabic script and flowers. There is a mission desk on the northwest side of the hall, which is used by the masters to give lectures on Jumu'ah or festivals. The Bangge Building is used to report the time and call the people to worship before the service, which is why it is also called the Xuanli Building or the Wake-up Building. On both sides of the platform of the Worship Hall are two symmetrical square pavilions. The stele in the pavilion was erected in the ninth year of the reign of Emperor Hongzhi of the Ming Dynasty (1496). Wangyue Building is located between the two gates in front of the mosque. It is hexagonal and parishioners would watch the moon before the Festival of Fast-breaking on the building. Wangyue Building is a typical building in a mosque.

With well-organized architectural layout, the mosque boasts simple and magnificent halls. All of the main architectural decorations are of Arabic style, which is extremely exquisite. After the founding of the People's Republic of China, the mosque received many dignitaries and guests from Arab countries.