Olofsson, who is in charge of Nestle's strategic business units, marketing and sales, will join Carrefour after Duran leaves at the end of this year, the retailer said.
Duran, who took office in late July, has been in the spotlight for several months because of his friction with two of Carrefour's largest investors, Bernard Arnault and Colony Capital.
The value of Carrefour has fallen by 45 percent since Jan. 1 due to poor performance, particularly in France, its traditional market.
With a revenue of 82 billion euros (104 billion U.S. dollars) in 2007, Carrefour ranks as the world's No. 2 retailer after U.S.-based Wal-Mart Stores.
Carrefour has sought to expand its markets in countries such as China, Brazil and Romania in recent years.