Giving preference to Salaah with Jamaat

A few days back I was just leaving home to go to masjid for salaah and my uncle (who is an Aalim) came unexpectedly with some friends to drop off a parcel. I asked him if he is coming to the masjid and he said no (They were musafir). My mother was alone in the house and none of them are her mahrams so they obviously could not be in the house while I went to masjid, so they had to drop off the parcel and leave immediately. This Aalim seemed to be upset because I went to masjid and did not accommodate them. Did I do the right thing or was I supposed to miss salaah with jamaat to accommodate them?