Mr Trump tweeted that Mr Pompeo, "will do a fantastic job!" and thanked Mr Tillerson for his service.
Trump also thanked Tillerson for his service.
CIA Deputy Director Gina Haspel will replace Pompeo as the head of the CIA.
Trump's tweet Tuesday came the day after Tillerson blamed Russian Federation for an assassination in London - something the White House had specifically declined to do just hours before.
The breaking point appears to be Trump's surprise announcement last week that he was willing to meet with North Korean leader Kim Jong-un, according to CBS News.
Florida residents band together to stop suspected hit and run driver
But the driver would not pull over, so the cop backed off because of the Miami PD's no-chase policy for non-violent crimes. He attempts to flee the scene of the crash several times but each attempt is blocked by witnesses in the area.
Trump's statement did not mention Tillerson, with whom he has been at odds on issues ranging from North Korea to steel and aluminum tariffs throughout the first 14 months of his administration.
A report by the Washington Post said that Tillerson was asked on Friday to step down from his post, which he assumed just over a year ago. Last week, Trump's chief economic advisor Gary Cohn resigned after the president announced he would impose tariffs on steel and aluminum imports.
More on this as it develops.
Tillerson is also deeply unpopular within the State Department for his imperious management style and disregard for the traditions of diplomacy, which has led to a steady level of attrition throughout his tenure.
Rex Tillerson is out as secretary of state.