Mr Trump tweeted that Mr Pompeo, "will do a fantastic job!" and thanked Mr Tillerson for his service.
Trump also thanked Tillerson for his service.
CIA Deputy Director Gina Haspel will replace Pompeo as the head of the CIA.
Trump's tweet Tuesday came the day after Tillerson blamed Russian Federation for an assassination in London - something the White House had specifically declined to do just hours before.
The breaking point appears to be Trump's surprise announcement last week that he was willing to meet with North Korean leader Kim Jong-un, according to CBS News.
Hamas condemns Gaza attack targeting Palestinian Prime Minister Hamdallah
The ties between Palestine and the United States have been witnessing rising tensions since last October, and peaked when U.S. He was also met with senior Hamas officials to discuss reconciliation talks between Hamas and the PA.
Trump's statement did not mention Tillerson, with whom he has been at odds on issues ranging from North Korea to steel and aluminum tariffs throughout the first 14 months of his administration.
A report by the Washington Post said that Tillerson was asked on Friday to step down from his post, which he assumed just over a year ago. Last week, Trump's chief economic advisor Gary Cohn resigned after the president announced he would impose tariffs on steel and aluminum imports.
More on this as it develops.
Tillerson is also deeply unpopular within the State Department for his imperious management style and disregard for the traditions of diplomacy, which has led to a steady level of attrition throughout his tenure.
Rex Tillerson is out as secretary of state.