It is reported that Isḥāq b. Rāhuway – Allāh have mercy on him – said:

If you ask the ignorant: ‘Who are Al-Sawād Al-Aʿẓam (Main Body of the Muslims)?’ they will say, ‘All the people (together).’ But they do not know that The Jamā’ah (Main Body) is a scholar who follows the footsteps of the Prophet – Allāh’s praise and peace be upon him – and his way. So whoever is with him and follows him is [one of] The Jamā’ah, but whoever contradicts him in it leaves The Jamā’ah.

Abū Nu’aym, Ḥilyatu Al-Awliyā' 9:238,239 et al