In 1960, he founded the Graff Diamonds company. In 1966 he commissioned the English jewellery-designer Robert Thomas to design a Diamond jewel to enter into the De Beers Diamond International Awards competition. The ribbon bracelet created won the competition. By 1974, he had begun specialising in selling to newly rich buyers from the Middle East. In particular, he supplied many jewels for Hassanal Bolkiah, the 29th Sultan of Brunei, who became a lifelong client and friend. One day, Prince Turki II bin Abdulaziz Al Saud walked into the shop and bought everything including a 14 carat Diamond. Graff has expanded his company, with over 35 stores in US, Europe, Asia, Africa and the Middle East.
In 2008, Graff purchased the Wittelsbach Diamond for £16.4 million, a considerable premium over the £9 million guide price. Almost two years later, Graff revealed he had had three Diamond cutters repolish the stone to eliminate the chips and improve the clarity, reducing the Diamond from 35.52 carats (7.104 g) to 31 carats (6.2 g). This action has been compared by critics to making the Mona Lisa prettier. However, according to gemologist Richard W. Wise, "At a cost of only 4.45 carats the recut and renamed Wittelsbach-Graff Diamond has been raised from a GIA grade of Fancy Deep Grayish Blue to a Fancy Deep Blue. Its clarity grade has been likewise elevated from VS2 to Internally Flawless (IF). This is a substantial upgrade." Further, the "Graff recut retained the original double stellate brilliant facet pattern thus retaining the overall look of the original stone."
Graff was appointed Officer of the Order of the British Empire (OBE) in the 2013 Birthday Honours for services to the jewellery industry.