William Rudolph Smith was the son of Thomas Duncan Smith (1812-1880) and his wife, and a descendant of Joseph Anthony. Joseph Anthony (1738-1798) ran a shipping firm in Philadelphia which traded in the West Indies, New York, and Boston. Anthony's sister, Elizabeth, was the mother of the artist Gilbert Stuart, whom Joseph Anthony patronized. Anthony commissioned Stuart to make portraits of his family; a portrait of Anthony by Stuart is now in the NGA (1942.8.11).