The sources,who have knowledge of the matter,said Mori would be replaced by former Japan Football Association president and mayor of the Olympic village,Saburo Kawabuchi.
Kawabuchi,84,represented Japan in football at the 1964 Tokyo Olympics and helped Japan co-host the 2002 FIFA World Cup with South Korea.
Mori made the sexist remark –that women talk too much – at a Japanese Olympic Committee board meeting early this month,setting off a storm of criticism at home and abroad.
Mori,83,told Nippon TV he would “explain his thoughts” at a meeting on Friday,but he had to deal with the issue. He did not confirm the reports that he would step down.
“I cannot let this problem prolong any longer,” Mori said,and he again apologised for the remark.
A spokesman for the organising committee declined to comment.
Mori firstretracted the comment about women at a hastily called news conference on February 4,acknowledging it was inappropriate and against the Olympic spirit. But he declined,at that time,to resign.