Two explosions struck the al-Balili mosque during prayers for the Eid al-Adha holiday.
The attack comes two days after Yemen's President Abdrabbuh Mansour Hadi returned to the southern city of Aden from exile in Saudi Arabia.
He had fled in March following gains by Houthi rebels, who have since been targeted by a Saudi-led coalition.
The coalition, along with loyalist forces, have pushed back the Houthis from some areas, including Aden.
However the Houthis - Shia Muslim rebels from the north of the country - still control the capital.
The UN says almost 4,900 people, including more than 2,200 civilians, have been killed in Yemen in fighting on the ground and air strikes since 26 March.