Rubin began his career as an art dealer in 1956 at a gallery in Rome. In 1959 he opened his the Galerie Neufville in Paris. He returned to New York where in 1969 he opened a gallery under his own name on 57th Street. In 1973 he became affiliated with Knoedler's, whose President and Director he was named in 1977.